Integrated circuit big data storage optimization system
By designing an integrated circuit big data storage optimization system and using multi-module coordination and optimization technology, the problems of limited storage capacity and slow reading and writing speed in the traditional integrated circuit storage system are solved, and efficient and reliable big data storage is achieved and low power system performance.
Patent Information
- Application Number
- CN202411472226.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-22
- Publication Date
- 2025-02-14
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When facing big data storage, traditional integrated circuit storage systems have problems such as limited storage capacity and slow reading and writing speed.
An integrated circuit big data storage optimization system is designed, including a data control module, a storage unit module, a data cache module, an error detection and correction module, a data compression module, an address mapping module and a power management module. These modules improve storage capacity and read and write speed and reduce power consumption by coordinating work, optimizing storage unit structure and materials, adopting intelligent cache strategies, error detection and correction algorithms, data compression, address mapping and dynamic power management technologies.
It significantly improves storage capacity and data reading and writing speed, enhances data reliability and system energy efficiency performance, and meets the efficiency and reliability requirements of big data storage.
Smart Images
Figure CN119440405A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data storage, and in particular to an integrated circuit big data storage optimization system. Background Art
[0002] With the rapid development of information technology, the scale of big data continues to expand, and the requirements for storage systems are becoming higher and higher. Compared with traditional storage methods, integrated circuit storage can integrate a large number of storage units in a very small space to achieve efficient data storage, greatly saving physical space.
[0003] At present, traditional integrated circuit storage systems often have problems such as limited storage capacity and slow reading and writing speeds when facing big data storage. In view of this, we propose an integrated circuit big data storage optimization system. Summary of the invention
[0004] The main purpose of the present invention is to provide an integrated circuit big data storage optimization system that can solve the problems raised in the above background technology.
[0005] To achieve the above-mentioned purpose, the integrated circuit big data storage optimization system proposed by the present invention comprises: A data control module, which is responsible for coordinating the work of various modules and controlling and managing the reading and writing operations of data; A storage unit module, the storage unit module is used to determine the structural design of the storage unit and monitor the status of the storage unit; A data cache module, which is used to temporarily store data to be written into the storage unit module or data read from the storage unit module to increase the reading and writing speed of the data; An error detection and correction module, the error detection and correction module is used to perform error detection and correction on the data stored in the storage unit module to improve the reliability of the data; A data compression module, which is used to compress the data to be written into the storage unit module to reduce the storage space occupied by the data; An address mapping module, the address mapping module is used to map the logical address provided by the external device to the physical address in the storage unit module to achieve fast location and access of data; The power management module is responsible for managing the power supply of the system to reduce the power consumption of the system.
[0006] Preferably, the data control module includes a request processing unit, an address management unit, and a data scheduling unit.
[0007] Preferably, the request processing unit is responsible for receiving storage requests from external devices, parsing and classifying the requests, and sending the requests to corresponding processing units for processing according to the type of request (such as read request, write request, erase request, etc.). The address management unit is responsible for mapping the logical address provided by the external device to the physical address in the storage unit module. The address management unit can adopt a variety of address mapping algorithms, such as direct mapping, fully associative mapping, group associative mapping, etc., and select according to the needs and performance requirements of the system. The data scheduling unit is responsible for coordinating the transmission of data between various modules. When the storage controller receives a storage request, the data scheduling unit transfers the data from the external device to the data cache module for temporary storage, and then writes the data to the storage unit module according to the status of the storage unit and the physical address provided by the address management unit. When the storage controller receives a read request, the data scheduling unit transfers the data read from the storage unit module to the data cache module for temporary storage, and then returns the data to the external device.
[0008] Preferably, the storage unit module includes a storage structure unit, a storage material unit, and a storage unit management unit. The storage structure unit is responsible for selecting different structural designs, such as a planar structure, a three-dimensional structure, etc. The three-dimensional structure storage unit can accommodate more storage units on the same chip area, thereby increasing the storage capacity. The storage material unit load material selects different materials, such as ferroelectric materials, phase change materials, etc., to improve the read and write speed, durability and data reliability of the storage unit. The storage unit management unit is responsible for managing and maintaining the storage unit module, and the storage unit management unit can monitor the status of the storage unit.
[0009] Preferably, the data cache module includes a cache structure unit, a cache replacement strategy unit, and a cache prefetch strategy unit. The cache structure unit can adopt different cache structures, such as direct-mapped cache, fully associative cache, set associative cache, etc. Different cache structures have different performance characteristics and can be selected according to system needs and performance requirements. In order to ensure that the data in the cache is always the most commonly used data, the cache replacement strategy unit needs to adopt a reasonable cache replacement strategy. Common cache replacement strategies include the least recently used (LRU) algorithm, the least frequently used (LFU) algorithm, etc. The cache prefetch strategy unit can adopt a cache prefetch strategy to improve the reading and writing speed of data. The cache prefetch strategy can prefetch data that may be accessed into the cache in advance to reduce data access delay.
[0010] Preferably, the error detection and correction module includes an error detection algorithm, an error correction algorithm, and an error management unit. The error detection algorithm is used to detect whether there are errors in the data stored in the storage unit module. Common error detection algorithms include Hamming code, cyclic redundancy check (CRC), etc. The error correction algorithm is used to correct the detected erroneous data. Common error correction algorithms include Hamming code, BCH code, RS code, etc. The error management unit is responsible for managing and controlling the error detection and correction process. The error management unit can monitor the results of error detection and correction, record the type and location of the error, and take corresponding measures according to the severity of the error, such as data rewriting, storage unit replacement, etc.
[0011] Preferably, the data compression module includes a compression algorithm selection unit, a compression engine, and a compressed data management unit. The compression algorithm selection unit is responsible for selecting a suitable data compression algorithm. Different data compression algorithms have different compression ratios and compression speeds, and can be selected according to the characteristics of the data and the performance requirements of the system. The compression engine is responsible for compressing and decompressing the data. The compression engine can use hardware acceleration technology to improve the compression and decompression speed. The compressed data management unit is responsible for managing and storing the compressed data. The compressed data management unit can store the compressed data in the storage unit module and record information such as the address and length of the compressed data so as to perform decompression operations when reading the data.
[0012] Preferably, the address mapping module includes an address mapping algorithm selection unit, an address mapping table, and an address mapping update unit. The address mapping algorithm selection unit is responsible for selecting a suitable address mapping algorithm. Different address mapping algorithms have different performance characteristics and can be selected according to system needs and performance requirements. The address mapping table is used to record the mapping relationship between logical addresses and physical addresses. The address mapping table can be implemented in hardware to increase the speed of address mapping. The address mapping update unit is responsible for updating and maintaining the address mapping table. When the data in the storage unit module changes, the address mapping update unit needs to update the address mapping table in time to ensure the accuracy of the address mapping.
[0013] Preferably, the power management module includes a power monitoring unit, a power adjustment unit, and a power protection unit.
[0014] Preferably, the power monitoring unit is responsible for monitoring the power status of the system, such as power voltage, current, power, etc. The power monitoring unit can monitor the power status of the system in real time and feed back the monitoring results to the power management module so that the power management module can take corresponding measures. The power adjustment unit is responsible for automatically adjusting the power voltage and operating frequency according to the load condition of the system. The power adjustment unit can adopt dynamic voltage and frequency adjustment (DVFS) technology to automatically adjust the power voltage and operating frequency according to the load condition of the system to achieve the purpose of energy saving and consumption reduction. The power protection unit is responsible for protecting the power supply of the system to prevent power failure from damaging the system. The power protection unit can adopt overvoltage protection, overcurrent protection, short circuit protection and other technologies to protect the power supply of the system.
[0015] The present invention provides an integrated circuit big data storage optimization system. It has the following beneficial effects: (1) The integrated circuit big data storage optimization system greatly improves the storage capacity by adopting new non-volatile memory technology and optimized storage unit structure and materials. At the same time, the optimization of the data cache module, including the use of intelligent cache replacement strategy, multi-level cache structure and cache pre-fetch technology, significantly improves the data reading and writing speed. This makes it possible to quickly store and read large amounts of data in big data storage scenarios, meeting the high requirements of modern information technology for data processing.
[0016] (2) The integrated circuit big data storage optimization system adopts efficient algorithms such as low-density parity check code and Reed-Solomon code through the error detection and correction module, and improves the algorithm execution speed through hardware acceleration technology. In addition, the storage unit management unit monitors and maintains the status of the storage unit module to ensure the reliability of the data. In big data storage, reliable data storage is crucial to ensuring the integrity and availability of information.
[0017] (3) The integrated circuit big data storage optimization system adopts dynamic voltage and frequency adjustment technology through the power management module to automatically adjust the power supply voltage and operating frequency according to the system load conditions to achieve energy saving and consumption reduction. At the same time, low-power consumption technology and strategies are used for modules with high power consumption such as storage unit modules and data cache modules to reduce the power consumption of the entire system. This not only saves energy, but also reduces operating costs, which meets the requirements of sustainable development. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.
[0019] Figure 1 is a system block diagram of the present invention; Figure 2 This is a block diagram of the data control module of the present invention; Figure 3 This is a block diagram of the storage unit module of the present invention; Figure 4 This is a block diagram of a data buffer module of the present invention; Figure 5 This is the block diagram of the error detection and correction module; Figure 6 This is a block diagram of a data compression module of the present invention; Figure 7 This is a block diagram of the address mapping module of the present invention; Figure 8 This is a block diagram of the power management module of the present invention.
[0020] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0022] See also Figure 1-8 The present invention proposes an integrated circuit big data storage optimization system, including a data control module, a storage unit module, a data cache module, an error detection and correction module, a data compression module, an address mapping module, and a power management module. The data control module is responsible for coordinating the work of each module, controlling and managing the reading and writing operations of data, the storage unit module is used to determine the structural design of the storage unit, and monitor the status of the storage unit, the data cache module is used to temporarily store data to be written to the storage unit module or data read from the storage unit module to improve the reading and writing speed of the data, the error detection and correction module is used to perform error detection and correction on the data stored in the storage unit module to improve the reliability of the data, the data compression module is used to compress the data to be written to the storage unit module to reduce the storage space occupied by the data, the address mapping module is used to map the logical address provided by the external device to the physical address in the storage unit module to achieve rapid positioning and access to the data, and is responsible for managing the power supply of the system to reduce the power consumption of the system.
[0023] In the present invention, the data control module includes a request processing unit, an address management unit, and a data scheduling unit; the request processing unit is responsible for receiving storage requests from external devices, parsing and classifying the requests, and sending the requests to the corresponding processing unit for processing according to the type of request (such as read request, write request, erase request, etc.); the address management unit is responsible for mapping the logical address provided by the external device to the physical address in the storage unit module. The address management unit can adopt a variety of address mapping algorithms, such as direct mapping, fully associative mapping, group associative mapping, etc., and select according to the needs and performance requirements of the system; the data scheduling unit is responsible for coordinating the transmission of data between various modules; when the storage controller receives a storage request, the data scheduling unit transfers the data from the external device to the data cache module for temporary storage, and then writes the data to the storage unit module according to the state of the storage unit and the physical address provided by the address management unit; when the storage controller receives a read request, the data scheduling unit transfers the data read from the storage unit module to the data cache module for temporary storage, and then returns the data to the external device.
[0024] Furthermore, the storage unit module includes a storage structure unit, a storage material unit, and a storage unit management unit. The storage structure unit is responsible for selecting different structural designs and can adopt new non-volatile memory technologies, such as three-dimensional flash memory (3DFlash), vertical magnetic tunnel junction random access memory (VMRAM), etc., to improve the storage density and reliability of the storage unit. If large-capacity storage and cost advantages are more important, then three-dimensional flash memory is more suitable; if there are extremely high requirements for read and write speed, power consumption and durability, and are relatively less sensitive to price, vertical magnetic tunnel junction random access memory may be a better choice. The storage material unit load material uses different materials, such as ferroelectric materials, phase change materials, etc., to improve the read and write speed, durability and data reliability of the storage unit. The storage unit management unit is responsible for managing and maintaining the storage unit module, and the storage unit management unit can monitor the status of the storage unit.
[0025] Furthermore, the data cache module includes a cache structure unit, a cache replacement strategy unit, and a cache prefetch strategy unit. The cache structure unit can adopt different cache structures, such as direct-mapped cache, fully associative cache, set associative cache, etc. Different cache structures have different performance characteristics and can be selected according to system needs and performance requirements. A multi-level cache structure is adopted to store data with different access frequencies in caches of different levels. In order to ensure that the data in the cache is always the most commonly used data, the cache replacement strategy unit needs to adopt a reasonable cache replacement strategy. An intelligent cache replacement strategy is adopted to automatically select the most appropriate cache replacement algorithm according to factors such as data access frequency and access time. For example, a least recently used (LRU) algorithm, a least frequently used (LFU) algorithm, etc. are adopted to ensure that the data in the cache is always the most commonly used data. In order to improve the reading and writing speed of data, the cache prefetch strategy unit can adopt a cache prefetch strategy. The cache prefetch strategy can prefetch data that may be accessed into the cache in advance to reduce data access delay. The cache prefetch technology is adopted to prefetch data that may be accessed into the cache in advance to improve the cache hit rate and reading and writing speed.
[0026] Furthermore, the error detection and correction module includes an error detection algorithm, an error correction algorithm, and an error management unit. The error detection algorithm is used to detect whether there are errors in the data stored in the storage unit module. Common error detection algorithms include Hamming code, cyclic redundancy check (CRC), etc. The error correction algorithm is used to correct the detected erroneous data. Common error correction algorithms include Hamming code, BCH code, RS code, etc., but we use low-density parity check code (LDPC) and Reed-Solomon code (RS code) to improve the reliability of data and perform error detection and correction on data. Hardware acceleration technology is used to implement the calculation process of the error detection and correction algorithm in hardware, improve the execution speed of the algorithm, optimize the implementation method of the error detection and correction algorithm, and reduce the complexity and calculation amount of the algorithm. The error management unit is responsible for managing and controlling the error detection and correction process. The error management unit can monitor the results of error detection and correction, record the type and location of the error, and take corresponding measures according to the severity of the error, such as data rewriting and storage unit replacement.
[0027] Furthermore, the data compression module includes a compression algorithm selection unit, a compression engine, and a compressed data management unit. The compression algorithm selection unit is responsible for selecting a suitable data compression algorithm. Different data compression algorithms have different compression ratios and compression speeds, which can be selected according to the characteristics of the data and the performance requirements of the system. We use an efficient lossless compression algorithm to compress the data and reduce the storage space occupied by the data. For example, Huffman coding, LZ77 algorithm, etc. are used to efficiently compress the data. The compression engine is responsible for compressing and decompressing the data. The compression engine can use hardware acceleration technology to improve the compression and decompression speed, and use parallel compression technology to divide the data into multiple blocks and compress them simultaneously to improve the compression speed; use adaptive compression technology to automatically select the most suitable compression algorithm according to the characteristics of the data, improve the compression ratio, optimize the implementation method of the data compression algorithm, and improve the compression ratio and compression speed of the algorithm. The compressed data management unit is responsible for managing and storing the compressed data. The compressed data management unit can store the compressed data in the storage unit module and record information such as the address and length of the compressed data so as to perform decompression operations when reading the data.
[0028] Furthermore, the address mapping module includes an address mapping algorithm selection unit, an address mapping table, and an address mapping update unit. The address mapping algorithm selection unit is responsible for selecting a suitable address mapping algorithm. Different address mapping algorithms have different performance characteristics and can be selected according to system needs and performance requirements. Address mapping algorithms include direct mapping, fully associative mapping, group associative mapping, etc. An efficient address mapping algorithm is used to map the logical address provided by an external device to a physical address in a storage unit module to achieve rapid data positioning and access. The address mapping table is used to record the mapping relationship between the logical address and the physical address. The address mapping table can be implemented in hardware to increase the speed of address mapping. By adopting hardware acceleration technology, the calculation process of the address mapping algorithm is implemented in hardware to increase the execution speed of the algorithm, optimize the implementation method of the address mapping algorithm, and reduce the complexity and calculation amount of the algorithm. The address mapping update unit is responsible for updating and maintaining the address mapping table. When the data in the storage unit module changes, the address mapping update unit needs to update the address mapping table in time to ensure the accuracy of the address mapping.
[0029] Furthermore, the power management module includes a power monitoring unit, a power adjustment unit, and a power protection unit; the power monitoring unit is responsible for monitoring the power status of the system, such as power voltage, current, power, etc. The power monitoring unit can monitor the power status of the system in real time and feed back the monitoring results to the power management module so that the power management module can take corresponding measures. The power adjustment unit is responsible for automatically adjusting the power voltage and operating frequency according to the load of the system. The power adjustment unit can adopt dynamic voltage and frequency adjustment (DVFS) technology to automatically adjust the power voltage and operating frequency according to the load of the system to achieve the purpose of energy saving and consumption reduction. For modules with high power consumption such as storage unit modules and data cache modules, low-power storage technology and caching strategies can be adopted to reduce power consumption. The power protection unit is responsible for protecting the power supply of the system to prevent power failure from damaging the system. The power protection unit can adopt overvoltage protection, overcurrent protection, short circuit protection and other technologies to protect the power supply of the system.
[0030] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.
Claims
1. An integrated circuit big data storage optimization system, characterized in that: include: A data control module, which is responsible for coordinating the work of various modules and controlling and managing the reading and writing operations of data; A storage unit module, the storage unit module is used to determine the structural design of the storage unit and monitor the status of the storage unit; A data cache module, which is used to temporarily store data to be written into the storage cell array or data to be read from the storage cell array to increase the reading and writing speed of the data; An error detection and correction module, the error detection and correction module is used to perform error detection and correction on data stored in the storage cell array to improve the reliability of the data; A data compression module, the data compression module is used to compress the data to be written into the storage cell array to reduce the storage space occupied by the data; An address mapping module, the address mapping module is used to map the logical address provided by the external device into the physical address in the storage unit array to achieve fast location and access of data; The power management module is responsible for managing the power supply of the system to reduce the power consumption of the system.
2. The integrated circuit big data storage optimization system according to claim 1, characterized in that: The data control module includes a request processing unit, an address management unit, and a data scheduling unit.
3. The integrated circuit big data storage optimization system according to claim 2, characterized in that: The request processing unit is responsible for receiving storage requests from external devices and parsing and classifying the requests. The address management unit is responsible for mapping the logical addresses provided by the external devices to physical addresses in the storage unit module. The data scheduling unit is responsible for coordinating the transmission of data between various modules.
4. The integrated circuit big data storage optimization system according to claim 1, characterized in that: The storage unit module includes a storage structure unit, a storage material unit, and a storage unit management unit. The storage structure unit is responsible for selecting different structural designs, the storage material unit load material selects different materials, the storage unit management unit is responsible for managing and maintaining the storage unit module, and the storage unit management unit can monitor the status of the storage unit.
5. The integrated circuit big data storage optimization system according to claim 1, characterized in that: The data cache module includes a cache structure unit, a cache replacement strategy unit, and a cache prefetch strategy unit. The cache structure unit can adopt different cache structures. In order to ensure that the data in the cache is always the most commonly used data, the cache replacement strategy unit needs to adopt a reasonable cache replacement strategy. In order to improve the reading and writing speed of data, the cache prefetch strategy unit can adopt a cache prefetch strategy. The cache prefetch strategy can prefetch data that may be accessed into the cache in advance to reduce data access delay.
6. The integrated circuit big data storage optimization system according to claim 1, characterized in that: The error detection and correction module includes an error detection algorithm, an error correction algorithm, and an error management unit. The error detection algorithm is used to detect whether there are errors in the data stored in the storage unit module. The error correction algorithm is used to correct the detected erroneous data. The error management unit is responsible for managing and controlling the error detection and correction process.
7. The integrated circuit big data storage optimization system according to claim 1, characterized in that: The data compression module includes a compression algorithm selection unit, a compression engine, and a compressed data management unit. The compression algorithm selection unit is responsible for selecting a suitable data compression algorithm, the compression engine is responsible for compressing and decompressing data, and the compressed data management unit is responsible for managing and storing compressed data.
8. The integrated circuit big data storage optimization system according to claim 1, characterized in that: The address mapping module includes an address mapping algorithm selection unit, an address mapping table, and an address mapping update unit. The address mapping algorithm selection unit is responsible for selecting a suitable address mapping algorithm. The address mapping table is used to record the mapping relationship between the logical address and the physical address. The address mapping update unit is responsible for updating and maintaining the address mapping table.
9. The integrated circuit big data storage optimization system according to claim 1, characterized in that: The power management module includes a power monitoring unit, a power adjustment unit, and a power protection unit.
10. The integrated circuit big data storage optimization system according to claim 9, characterized in that: The power monitoring unit is responsible for monitoring the power status of the system, such as power voltage, current, power, etc. The power adjustment unit is responsible for automatically adjusting the power voltage and operating frequency according to the load conditions of the system. The power protection unit is responsible for protecting the power supply of the system to prevent power failure from damaging the system.