Processing analysis system and method based on information data storage

By combining HDD and SSD storage in NAS servers and using the free capacity of computer memory and hard disks to form a distributed storage cluster, the performance bottleneck problem of NAS servers in high concurrent read and write situations is solved, achieving more efficient read and write performance and lower hardware upgrade costs.

CN120215832AInactive Publication Date: 2025-06-27GUANGDONG QILI TECHNOLOGY CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202510343818.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-03-22
Publication Date
2025-06-27
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When existing NAS servers handle a large number of concurrent read and write operations, they are prone to reach the performance bottleneck of SSD cache, resulting in system performance degradation, and hardware upgrades will increase costs and bring about data transfer risks.

Method used

By introducing HDD hard disk arrays, SSD read caches and SSD write caches into NAS servers, and using the free capacity of computer memory and hard disks to form a distributed storage cluster to coordinate storage resources to meet continuous read and write needs.

Benefits of technology

It improves the upper limit of data read and write redundancy performance of NAS servers and computer groups, optimizes read and write performance response speed, and reduces the cost of hardware upgrades and data transfer risks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120215832A_ABST
    Figure CN120215832A_ABST
Patent Text Reader

Abstract

The invention discloses a processing analysis system and method based on information data storage, and relates to the technical field of information processing. Comprising an NAS server, a network switch and a computer group, the computer group is formed by connecting a plurality of computers with the network switch, the NAS server comprises an HDD hard disk array, a processor, an SSD read cache and an SSD write-in cache, the data storage mode of the HDD hard disk array is RAID10, the HDD hard disk array exchanges data with the computers in the computer group through the network switch, and the processor is connected with the SSD read cache and the SSD write-in cache. According to the method, the idle computer memory and the idle computer hard disk are used for balancing the continuous read-write requirement which cannot be met by the NAS server, the upper limit of the data read-write redundancy performance of the NAS server is improved, the computer memory and the computer hard disk are evaluated and graded, the data read-write redundancy performance of the NAS server is improved, and the storage capacity of the SSD read cache and the storage capacity of the SSD write cache are consistent. The read-write performance response speed outside the NAS server can be improved, and the read-write experience of the computer group is optimized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of information processing, and specifically provides a processing and analysis system and method based on information data storage. Background Art

[0002] NAS, i.e., Network Attached Storage, is a device or system dedicated to centralized data storage and sharing. It is connected to multiple users or devices via a network, providing a convenient way to store, manage, and access data. By centralizing data storage in one place, it offers data backup and recovery functions to protect data security and integrity. Among them, installing an SSD cache in the NAS server can provide faster data reading speed, especially for frequently accessed files or data, and the acceleration effect is more obvious. This can improve the overall performance of the system and make data access more rapid. Generally speaking, the SSD cache in NAS can significantly enhance the performance and response ability of the system, especially in scenarios involving a large number of concurrent read and write operations or with requirements for speed.

[0003] However, the current NAS servers are weak in performance and can only meet the read and write requirements of small and medium-sized computer workgroups. When the number of devices in the computer workgroup increases, continuous data read and write can easily reach the performance bottleneck of the SSD cache. Upgrading either the NAS server or the SSD cache will increase hardware investment and also require taking the risk of data transfer. Therefore, optimizing the storage systems of NAS servers and computer groups to improve read and write redundancy performance without upgrading hardware resources is a technical problem that those skilled in the art need to solve. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a processing and analysis system and method based on information data storage, which solves the problems raised in the above background art.

[0005] To achieve the above objectives, the present invention is realized through the following technical solutions: A processing and analysis system based on information data storage, including a NAS server, a network switch, and a computer group, where the computer group is composed of several computers connected to the network switch;

[0006] The NAS server includes an HDD hard disk array, a processor, an SSD read cache, and an SSD write cache. The data storage method of the HDD hard disk array is RAID10. The HDD hard disk array exchanges data with the internal computers of the computer group through a network switch. The storage capacities of the SSD read cache and the SSD write cache are the same. The SSD read cache reads data from the HDD hard disk array and transmits the data to the internal computers of the computer group through the network switch. The SSD write cache writes the data generated by the internal computers of the computer group into the HDD hard disk array through the network switch for storage. The processor collects the read performance of the SSD read cache and the write performance of the SSD write cache in the NAS server. The processor collects the read and write performance of the HDD hard disk array in the NAS server;

[0007] The computer includes a computer processor, computer memory, and a computer hard disk. The computer processor allocates a cache spare space from the free capacity of the computer memory. When the computer hard disk is not being read or written, the computer processor allocates a storage backup space from the free capacity of the computer hard disk.

[0008] Further, the processor in the NAS server exchanges data with the computer processors of each computer in the computer group through a network switch. The processor in the NAS server and the computer processors in the computers connected to the network switch jointly form a heterogeneous asymmetric distributed cluster based on the local area network. The NAS server processor and the computer processors cooperate to process and call the storage resources in the entire local area network. The storage resources are all the hard disks and memories within the local area network. The NAS server has two or more network ports. The NAS server is connected to the network switch through the multi-network port link aggregation method to improve the transmission bandwidth of the network switch.

[0009] Further, the storage backup spaces inside the two computers adopt the RAID1 data storage method. When the capacity of the computer cache backup space is saturated, the computer processor sends a request message to another idle computer processor. After receiving the request message, the other computer creates a new cache backup space from the idle capacity of the computer memory. The computer processor uses the new cache backup space to ensure the requirements for continuous reading and writing. The computer processor transfers the internal data of the saturated cache backup space to the storage backup space under the same computer. After the internal data of the cache backup space is transferred, the computer processor clears the data in the cache backup space and releases the capacity of the computer memory again. When there is a new reading and writing requirement next time, the computer processor creates a new cache backup space from the idle capacity of the computer memory again. When the capacity of the computer cache backup space is saturated and there is no storage backup space or the storage backup space does not meet the transfer requirements under the same computer, the computer processor transfers the internal data of the saturated cache backup space to the idle storage backup space in another computer through a network switch.

[0010] Further, when the SSD read cache is in an idle state, the processor in the NAS server clears the data in the SSD read cache and releases the capacity space of the SSD read cache. When the SSD write cache is in an idle state, the processor in the NAS server transfers the data in the SSD write cache to the HDD hard disk array and then releases the capacity space of the SSD write cache. The data to be written in the internal cache backup space of the computer is transferred to the SSD write cache when the SSD write cache is idle. The data to be read in the internal cache backup space of the computer is transferred to the SSD read cache after the SSD read cache is cleared and released. When the HDD hard disk array is in an idle state, the data to be written in the storage backup space is transferred to the HDD hard disk array through a network switch. The data to be read in the storage backup space is transferred to the SSD read cache.

[0011] A method based on information data storage includes an analysis process and a processing process. The analysis process includes the following steps:

[0012] Step 1: The computer processor evaluates the performance of the computer memory.

[0013] Step 2: The computer processor evaluates the performance of the computer hard disk.

[0014] Step 3: The NAS server processor and the computer processor execute a processing flow according to the evaluation results. The processing flow is a distributed storage method of data. A heterogeneous asymmetric distributed storage cluster of the local area network is used to coordinate storage resources to ensure read and write requirements. Heterogeneous asymmetric means that the hardware specifications of the storage resources within the local area network are inconsistent. The local area network is used for unified management of distributed storage.

[0015] The processing flow includes the following steps:

[0016] Step 4: The computer reads data from the SSD read cache in the NAS server, and the computer writes the generated data into the SSD write cache in the NAS server;

[0017] Step 5: The processor determines whether the performance of the SSD read cache and the SSD write cache in the NAS server is saturated. If the cache capacity is saturated, execute Step 6; otherwise, if the cache performance is not saturated, execute Step 4;

[0018] Step 6: The computer processor creates a cache spare space from the free capacity of the computer memory. The cache spare space is used to replace the SSD read cache and the SSD write cache for read and write caching;

[0019] Step 7: The computer processor determines whether the storage capacity of the cache spare space is saturated. If the cache capacity is saturated, execute Step 8; otherwise, if the cache performance is not saturated, execute Step 6;

[0020] Step 8: The computer processor creates a storage backup space from the free capacity of the computer hard disk. The storage backup space is used to replace the cache spare space for read and write caching to meet continuous read and write requirements.

[0021] Further, the evaluation of the computer memory performance by the computer processor includes the following steps:

[0022] 101: Each computer processor in the computer group obtains the capacity and frequency of the computer memory. The computer processor classifies according to the capacity and frequency of the computer memory. The computer memory is divided into two levels, A and B;

[0023] 102: The computer memory with a capacity greater than or equal to 4GB and a frequency higher than 1600Mhz is Class A memory, and the rest is Class B memory. When using the computer memory to create a cache spare space, Class A memory is preferentially used, and when the Class A memory resources are insufficient, Class B memory is used to create a cache spare space, which can ensure the timely response speed of read and write within the system and optimize the experience of continuous read and write.

[0024] Further, the evaluation of the computer hard disk performance by the computer processor includes the following steps:

[0025] 201: The computer processor performs read and write tests on the computer hard disk using three test files of 1GB, 5GB, and 20GB. The computer processor collects the transfer rate and response time of the computer hard disk at each time node, and the computer processor transmits the transfer rate and response time of the computer hard disk to the NAS server processor;

[0026] 202: The NAS server processor inputs the transfer rate and response time of the computer hard disk into the coordinate system in sequence according to the corresponding time nodes, obtains the transfer rate curve and response time curve of the computer hard disk, calculates the slopes of the transfer rate curve and response time curve respectively, and records them as transfer rate slope y c and response time slope y x , and uses the set formula group:

[0027]

[0028] Calculate to obtain the transfer rate slope y c and response time slope y x , where w1, w2, w3, and w4 are proportionality coefficients, k i is the slope value at point i, k i-1 represents the slope value at point i - 1, k i+1 represents the slope value at point i + 1, and m is the number of curve time nodes;

[0029] 203: When the transfer rate slope y c value is less than or equal to 0.15 and the response time slope y x value is less than or equal to 0.25, it is a Class A hard disk, and the rest are Class B hard disks. The smaller the transfer rate slope y c value, the more stable the read and write rate of the computer hard disk, and it is not easy to drop the speed during continuous read and write. The smaller the value of the response time slope y x , the shorter and more stable the response time of the computer hard disk, and the superior performance in continuous read and write.

[0030] Furthermore, when the computer memory performs continuous read and write, it preferentially uses the classified Class A memory. After the capacity of the Class A memory is saturated, it then uses the Class B memory for read and write operations; when the capacity of the Class B memory is not fully occupied and the Class A memory is emptied and released, the data to be written in the Class B memory is transferred to the Class A memory, and after the Class B memory completes the data transfer, it releases its own capacity space; because the computer memory occupancy rate changes in real time and the capacity is limited, the cache backup space opened up by the computer memory can only store data for a short time. When the capacities of the Class A memory and the Class B memory are saturated, they are immediately transferred and their own capacities are released, which can ensure the response flexibility and read and write redundancy performance of the cache backup space.

[0031] Further, when the computer hard disk performs continuous read and write operations, the classified level-A hard disk is preferentially used. After the capacity of the level-A hard disk is saturated, the level-B hard disk is used for read and write operations. When the capacity of the level-B hard disk is not fully occupied and the level-A hard disk is emptied and released, the data to be written on the level-B hard disk is transferred to the level-A hard disk. After the data transfer of the level-B hard disk is completed, its own capacity space is released. By classifying the hard disks inside the computer group, the read and write response time of the storage backup space can be indirectly improved, avoiding the overall read and write response speed of the system being slowed down by individual hard disks with poor read and write performance, and ensuring the usage experience of the computer group for reading and writing.

[0032] Compared with computer memory, the computer hard disk has a larger capacity space and can save temporarily read and written data when powered off. Therefore, in the processing flow, if the data in the cache backup space cannot be stored in the HDD hard disk array in time, it will be transferred to the storage backup space. The storage backup space also has the data redundancy ability of RAID1, that is, the storage backup space is the data buffer space of the HDD hard disk array, and is used to store temporary data in the extreme case of continuous large-scale data read and write. The network switch transfers the temporary data back to the NAS server HDD hard disk array bit by bit, indirectly improving the data redundancy ability of the NAS server.

[0033] The present invention has the following beneficial effects:

[0034] 1. The processing and analysis system and method based on information data storage utilize the idle computer memory and idle computer hard disk to balance the continuous read and write requirements that cannot be met by the NAS server, and transfer the redundant data stored outside the NAS server to the HDD hard disk array when idle, improving the upper limit of the data read and write redundancy performance of the computer group and the NAS server.

[0035] 2. The processing and analysis system and method based on information data storage can improve the read and write performance response speed outside the NAS server and optimize the read and write experience of the computer group by evaluating and classifying the computer memory and computer hard disk.

[0036] Of course, it is not necessary for any product implementing the present invention to achieve all the above advantages simultaneously. Description of the Drawings

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0038] Figure 1Block diagram of a processing and analysis system based on information data storage according to the present invention;

[0039] Figure 2 Analysis flow chart of the present invention;

[0040] Figure 3 Processing flow chart of the present invention. Specific embodiments

[0041] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts belong to the scope of protection of the present invention.

[0042] Please refer to Figures 1 - 3 , the present invention provides a technical solution: a processing and analysis system based on information data storage, including a NAS server, a network switch, and a computer group. The computer group is composed of computers 1, 2,..., n connected through a network switch;

[0043] The NAS server includes an HDD hard disk array, a processor, an SSD read cache, and an SSD write cache. The data storage method of the HDD hard disk array is RAID10. The redundant array method of RAID10 can ensure data security while maintaining the hard disk read and write performance. The HDD hard disk array serves as a network disk mapped by the computer group. When the data read and write volume of the computer group is less than 1 GB, the read and write speed of the HDD hard disk array can fully meet the read and write requirements of the computer group. At this time, the HDD hard disk array directly exchanges data with the internal computers of the computer group through the network switch, and the SSD read cache and SSD write cache are in an idle state;

[0044] The SSD read cache and the SSD write cache have the same storage capacity, and the reading and writing are separated independently to facilitate the NAS server processor to call storage resources and balance the read and write load of the HDD hard disk array. The SSD read cache reads data from the HDD hard disk array and transmits the data to the internal computers of the computer group through the network switch. The SSD write cache writes the data generated by the internal computers of the computer group into the HDD hard disk array through the network switch for storage;

[0045] The processor in the NAS server collects the read performance of the SSD read cache and the write performance of the SSD write cache. By collecting the performance metrics of the SSD read cache and the SSD write cache, the saturation point of the read and write performance can be determined. The processor can promptly transfer the read and write targets to the cache spare space. The processor also collects the read and write performance of the HDD hard disk array in the NAS server. By collecting the read and write performance of the HDD hard disk array, it is convenient to determine the saturation point of the read and write performance of the HDD hard disk array. The processor can promptly transfer the read and write targets to the SSD read cache and the SSD write cache;

[0046] The computer includes a computer processor, computer memory, and a computer hard disk. The computer processor allocates a cache spare space from the free capacity of the computer memory to temporarily handle the continuous read and write requirements that cannot be met by the SSD read cache and the SSD write cache. When the computer hard disk is not being read or written, the computer processor allocates a storage backup space from the free capacity of the computer hard disk to temporarily handle the continuous read and write requirements that cannot be met by the cache spare space.

[0047] Among them, the processor in the NAS server exchanges data with the computer processors of each computer in the computer group through a network switch. The processor in the NAS server and the computer processors in the computers connected to the network switch together form a heterogeneous asymmetric distributed cluster based on the local area network. The NAS server processor and the computer processors cooperate to process and call the storage resources in the entire local area network. The storage resources are all the hard disks and memories within the local area network. By utilizing the free computer memory and free computer hard disks, the continuous read and write requirements that cannot be met by the NAS server are balanced. When there is free time, the excess data stored outside the NAS server is transferred to the HDD hard disk array to increase the upper limit of the data read and write redundancy performance of the computer group and the NAS server;

[0048] The NAS server has two or more network ports. The NAS server is connected to the network switch through the method of multi-network port link aggregation to increase the transmission bandwidth of the network switch, increase the data throughput of the NAS server in a short time, and make the read and write data transmission more rapid.

[0049] Among them, the storage backup spaces inside the two computers adopt the RAID1 data storage method, that is, the hard disk capacities and read / write performances of the two computers are similar. After the data read / write is completed, the same data is stored, which can ensure the security of the data outside the NAS server's HDD hard disk array. Since the computer memory occupancy rate changes in real time and the data will be lost when the power is off, the cache backup space opened up in the computer memory can only store limited data for a short time. The computer memory can shorten the response speed of data read / write outside the NAS server. Once the continuous read / write stops, it means that the cache backup space is in an idle state, and the data stored in the cache backup space will be immediately transferred to the SSD write cache. If the SSD write cache is not in an idle state, the data stored in the cache backup space will be transferred to the storage backup space;

[0050] When the capacity of the computer cache spare space is saturated, the computer processor sends a request message to another idle computer processor. After receiving the request message, the other computer opens up a new cache spare space with the idle capacity of the computer memory. The computer processor uses the new cache spare space to ensure the needs of continuous read / write. The computer processor transfers the data inside the saturated cache spare space to the storage backup space under the same computer. After the data inside the cache spare space is transferred, the computer processor clears the data in the cache spare space and releases the capacity of the computer memory again. When there is a new read / write requirement next time, the computer processor will re-open a new cache spare space from the idle capacity in the computer memory. When the capacity of the computer cache spare space is saturated and there is no storage backup space or the storage backup space does not meet the transfer requirement under the same computer, the computer processor transfers the data inside the saturated cache spare space to the idle storage backup space in another computer through the network switch.

[0051] Among them, when the SSD read cache is in an idle state, the processor in the NAS server clears the data in the SSD read cache and releases the capacity space of the SSD read cache. When the SSD write cache is in an idle state, the processor in the NAS server transfers the data in the SSD write cache to the HDD hard disk array and then releases the SSD write cache capacity space. The data that needs to be written in the computer internal cache spare space is transferred to the SSD write cache when the SSD write cache is idle. The data that needs to be read in the computer internal cache spare space is transferred to the SSD read cache after the SSD read cache is cleared and released. When the HDD hard disk array is in an idle state, the data that needs to be written in the storage spare space is transferred to the HDD hard disk array through the network switch, and the data that needs to be read in the storage spare space is transferred to the SSD read cache.

[0052] A method based on information data storage, including an analysis process and a processing process. The analysis process is used to evaluate the read and write performance of computer memory and computer hard disks within a computer group. The analysis process includes the following steps:

[0053] Step 1: The computer processor evaluates the performance of the computer memory;

[0054] Step 2: The computer processor evaluates the performance of the computer hard disk;

[0055] Step 3: The NAS server processor and the computer processor execute the processing process according to the evaluation results. The processing process is a data distributed storage method, using a heterogeneous asymmetric distributed storage cluster in the local area network to coordinate storage resources to ensure read and write requirements. Heterogeneous asymmetric means that the hardware specifications of the storage resources within the local area network are inconsistent, and the local area network is used for unified management of distributed storage.

[0056] The processing process includes the following steps:

[0057] Step 4: The computer reads data from the SSD read cache in the NAS server, and the computer writes the generated data into the SSD write cache in the NAS server;

[0058] Step 5: The processor determines whether the performance of the SSD read cache and the SSD write cache in the NAS server is saturated. If the cache capacity is saturated, execute Step 6; otherwise, if the cache performance is not saturated, execute Step 4;

[0059] Step 6: The computer processor creates a cache backup space from the free capacity of the computer memory. The cache backup space is used to replace the SSD read cache and the SSD write cache for read and write caching;

[0060] Step 7: The computer processor determines whether the storage capacity of the cache backup space is saturated. If the cache capacity is saturated, execute Step 8; otherwise, if the cache performance is not saturated, execute Step 6;

[0061] Step 8: The computer processor creates a storage backup space from the free capacity of the computer hard disk. The storage backup space is used to replace the cache backup space for read and write caching to meet continuous read and write requirements.

[0062] Among them, the computer processor's evaluation of the computer memory performance includes the following steps:

[0063] 101: Each computer processor in the computer group obtains the capacity and frequency of the computer memory. The computer processor classifies according to the capacity and frequency of the computer memory. The computer memory is divided into two levels, A and B;

[0064] 102: When the computer memory capacity is greater than or equal to 4GB and the frequency is higher than 1600Mhz, it is Class A memory, and the rest is Class B memory. When using the computer memory to create a cache backup space, Class A memory is preferentially used, and when the Class A memory resources are insufficient, Class B memory is used to create a cache backup space, which can ensure the timely response speed of reading and writing in the system and optimize the continuous reading and writing experience;

[0065] Among them, the computer processor evaluates the performance of the computer hard disk through the following steps:

[0066] 201: The computer processor uses three test files of 1GB, 5GB, and 20GB to perform read and write tests on the computer hard disk. Using files of different sizes for batch reading and writing is to test the cache capacity design of the computer hard disk itself. A hard disk with a built-in cache capacity design has better continuous read and write performance than a hard disk without a cache capacity design; the computer processor collects the transfer rate and response time of the computer hard disk at each time node, and the computer processor transfers the transfer rate and response time of the computer hard disk to the NAS server processor;

[0067] 202: The NAS server processor inputs the transfer rate and response time of the computer hard disk into the coordinate system in sequence according to the corresponding time nodes, obtains the transfer rate curve and response time curve of the computer hard disk, calculates the slopes of the transfer rate curve and response time curve respectively, and records them as the transfer rate slope y c and the response time slope y x , using the set formula group:

[0068]

[0069] Calculate to obtain the transfer rate slope y c and the response time slope y x , where w1, w2, w3, and w4 are proportionality coefficients, and the specific appropriate values of the proportionality coefficients need to be determined according to actual application analysis. k i is the slope value at point i, k i-1 represents the slope value at point i - 1, k i+1 represents the slope value at point i + 1, and m is the number of curve time nodes;

[0070] 203: When the value of the computer hard disk transfer rate slope y c is less than or equal to 0.15 and the value of the response time slope y x is less than or equal to 0.25, it is a Class A hard disk, and the rest is a Class B hard disk; the smaller the value of the transfer rate slope y c , the more stable the read and write rate of the computer hard disk, and it is not easy to drop speed during continuous read and write. The response time slope y xThe smaller the value, the shorter and more stable the response time of the computer hard disk, and the superior performance of continuous reading and writing.

[0071] Among them, when the computer memory performs continuous reading and writing, it preferentially uses the classified Class A memory. After the capacity of the Class A memory is saturated, the Class B memory is used for reading and writing operations; when the capacity of the Class B memory is not fully occupied and the Class A memory is emptied and released, the data to be written in the Class B memory is transferred to the Class A memory. After the Class B memory completes the data transfer, it releases its own capacity space; because the memory occupancy rate of the computer is changing in real time and the capacity is limited, the cache backup space opened up by the computer memory can only store data for a short time. When the capacities of the Class A memory and the Class B memory are saturated, they are immediately transferred and their own capacities are released, which can ensure the response flexibility and read-write redundancy performance of the cache backup space.

[0072] When the computer hard disk performs continuous reading and writing, it preferentially uses the classified Class a hard disk. After the capacity of the Class a hard disk is saturated, the Class b hard disk is used for reading and writing operations; when the capacity of the Class b hard disk is not fully occupied and the Class a hard disk is emptied and released, the data to be written in the Class b hard disk is transferred to the Class a hard disk. After the Class b hard disk completes the data transfer, it releases its own capacity space; by classifying the hard disks inside the computer group, the read-write response time of the storage backup space can be indirectly improved, avoiding the overall read-write response speed of the system being slowed down by individual hard disks with poor read-write performance, and ensuring the usage experience of the computer group for reading and writing.

[0073] Compared with the computer memory, the computer hard disk has a larger capacity space and can save temporarily read and written data when powered off. Therefore, in the processing process, if the data in the cache backup space cannot be stored in the HDD hard disk array in time, it will be transferred to the storage backup space. The storage backup space also has the data redundancy ability of RAID1, that is, the storage backup space is the data buffer space of the HDD hard disk array, which is used to store temporary data in the extreme case of continuous large-scale data reading and writing, and the temporary data is transferred back to the HDD hard disk array of the NAS server little by little by the network switch, indirectly improving the data redundancy ability of the NAS server.

[0074] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution of the present invention and its inventive concept, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

Claims

1. A processing and analysis system based on information data storage, comprising a NAS server, a network switch and a computer group, wherein the computer group is composed of a number of computers connected by a network switch, characterized in that: The NAS server includes a HDD hard disk array, a processor, an SSD read cache and an SSD write cache. The HDD hard disk array exchanges data with the internal computers of the computer group through a network switch. The SSD read cache reads data from the HDD hard disk array and transmits the data to the internal computers of the computer group through the network switch. The SSD write cache writes the data generated by the internal computers of the computer group into the HDD hard disk array through the network switch for storage. The processor collects the read performance of the SSD read cache and the write performance of the SSD write cache in the NAS server. The processor collects the read and write performance of the HDD hard disk array in the NAS server. The computer includes a computer processor, a computer memory and a computer hard disk. The computer processor uses the idle capacity of the computer memory to create a cache backup space. When the computer hard disk is not reading or writing, the computer processor uses the idle capacity of the computer hard disk to create a storage backup space.

2. A processing and analysis system based on information data storage according to claim 1, characterized in that: The processor in the NAS server exchanges data with the computer processor of each computer in the computer group through a network switch, together forming a heterogeneous asymmetric distributed cluster based on a local area network, and collaboratively processing and calling storage resources in the entire local area network. The storage resources are all hard disks and memories within the local area network. The NAS server is connected to the network switch by means of multi-network port link aggregation.

3. The processing and analysis system based on information data storage according to claim 1 is characterized in that: The storage backup space inside the two computers adopts RAID1 data storage method. When the capacity of the computer cache backup space is saturated, a request message is sent to another idle computer processor. After receiving the request message, the other computer uses the idle capacity of the computer memory to open up a new cache backup space, and uses the new cache backup space to meet the needs of continuous reading and writing. The computer processor transfers the internal data of the saturated cache backup space to the storage backup space under the same computer. After the internal data is transferred, the computer processor clears the cache backup space data and releases the capacity of the computer memory again. When there is a new reading and writing demand next time, the computer processor will re-open a new cache backup space from the idle capacity in the computer memory. When the capacity of the computer cache backup space is saturated, there is no storage backup space under the same computer or the storage backup space does not meet the transfer demand, the data inside the saturated cache backup space is transmitted to the idle storage backup space in another computer through a network switch.

4. The processing and analysis system based on information data storage according to claim 1 is characterized in that: When the SSD read cache is in an idle state, the processor in the NAS server clears the data in the SSD read cache and releases the capacity space of the SSD read cache again. When the SSD write cache is in an idle state, the processor in the NAS server transfers the data in the SSD write cache to the HDD hard disk array and releases the SSD write cache capacity space. The data that needs to be written in the computer's internal cache standby space is transferred to the SSD write cache when the SSD write cache is idle. The data that needs to be read in the computer's internal cache standby space is transferred to the SSD read cache after the SSD read cache is cleared and released; When the HDD hard disk array is in an idle state, the data to be written in the storage standby space is transferred to the HDD hard disk array through the network switch, and the data to be read in the storage standby space is transferred to the SSD read cache.

5. A method based on information data storage, characterized in that: A processing and analysis system based on information data storage as described in any one of claims 1 to 4, comprising the following steps: Step 1: the computer processor evaluates the computer memory performance; Step 2: the computer processor evaluates the performance of the computer hard disk; Step 3: The NAS server processor and the computer processor execute a processing flow according to the evaluation result, wherein the processing flow is a distributed storage method of data; The process flow includes the following steps: Step 4: the computer reads data from the SSD read cache in the NAS server, and the computer writes the generated data into the SSD write cache in the NAS server; Step 5: The processor determines whether the SSD read cache and SSD write cache performance in the NAS server are saturated. If the cache capacity is saturated, step 6 is executed; otherwise, step 4 is executed if the cache performance is not saturated. Step 6: The computer processor uses the idle capacity of the computer memory to create a cache standby space, and the cache standby space is used to replace the SSD read cache and the SSD write cache for read and write cache; Step 7: The computer processor determines whether the storage capacity of the cache spare space is saturated. If the cache capacity is saturated, step 8 is executed. Otherwise, step 6 is executed if the cache capacity is not saturated. Step 8: The computer processor uses the free capacity of the computer hard disk to create a storage backup space, and the storage backup space is used to replace the cache standby space for read and write caching.

6. A method based on information data storage according to claim 5, characterized in that: The computer processor evaluating the computer memory performance comprises the following steps: 101: Each computer processor in the computer group obtains the capacity and frequency of the computer memory, and the computer processor is graded according to the capacity and frequency of the computer memory, and the computer memory is divided into two grades, A and B; 102: The computer memory with a capacity greater than or equal to 4GB and a frequency higher than 1600Mhz is Class A memory, and the rest is Class B memory.

7. The method based on information data storage according to claim 5, characterized in that: The computer processor evaluating the computer hard disk performance comprises the following steps: 201: the computer processor performs a read and write test on the computer hard disk using three test files of 1 GB, 5 GB and 20 GB, collects the transfer rate and response time of the computer hard disk at each time node, and transmits the transfer rate and response time of the computer hard disk to the NAS server processor; 202: The NAS server processor inputs the transmission rate and response time of the computer hard disk into the coordinate system according to the corresponding time nodes, obtains the transmission rate curve and response time curve of the computer hard disk, calculates the slopes of the transmission rate curve and the response time curve respectively, and records them as the transmission rate slope y c and the response time slope y x , using the set formula group: Calculate the transmission rate slope y c and the response time slope y x , where w1, w2, w3 and w4 are proportional coefficients, k i is the slope value of point i, k i-1 represents the slope value of point i-1, k i+1 It represents the slope value of point i+1, and m is the number of time nodes of the curve; 203: The computer hard disk transmission rate slope y c The value is less than or equal to 0.15 and the response time slope y x When the value is less than or equal to 0.25, it is a Class A hard disk, and the rest are Class B hard disks.

8. The method based on information data storage according to claim 6, characterized in that: When the computer memory is continuously read and written, the classified Class A memory is used first. After the Class A memory capacity is saturated, the Class B memory is used for reading and writing operations. When the Class B memory capacity is not fully occupied and the Class A memory is cleared and released, the data that needs to be written to the Class B memory is transferred to the Class A memory, and the Class B memory releases its own capacity space after completing the data transfer.

9. The method based on information data storage according to claim 7, characterized in that: When the computer hard disk is continuously reading and writing, the graded A-level hard disk is used first. After the capacity of the A-level hard disk is saturated, the B-level hard disk is used for reading and writing operations. When the capacity of the B-level hard disk is not fully occupied and the A-level hard disk is cleared and released, the data to be written will be transferred to the A-level hard disk. After the B-level hard disk completes the data transfer, its own capacity space will be released.

Citation Information

Cited By

  • Cache management method, storage network and electronic equipment

    CN120743808A

  • Cache management method, storage network and electronic device

    CN120743808B