Data processing method, system, device and computer equipment
By introducing cache management hardware into the RAID card, the problem of low performance caused by limited CPU resources is solved, and efficient data processing and performance improvement are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DAWNING INFORMATION IND (BEIJING) CO LTD
- Filing Date
- 2022-09-07
- Publication Date
- 2026-07-31
AI Technical Summary
Using the CPU for data processing in a RAID card results in poor performance and affects the overall performance of the storage system.
By introducing dedicated cache management hardware devices into the RAID card, including request processing circuits and cache management circuits, CPU resources can be avoided by directly requesting cache resources, thereby improving data processing efficiency.
Without consuming CPU resources, it improves the performance of the RAID card, processes data requests in a timely manner, and enhances the overall performance of the storage system.
Smart Images
Figure CN116303133B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of storage technology, and in particular to a data processing method, system, apparatus and computer device. Background Technology
[0002] With the development of the storage field, a caching management scheme has emerged in storage system management. This caching management scheme is an important component of the data path in RAID (Redundant Arrays of Independent Disks) cards. It can act as a buffer for low-speed hard drives, thereby improving the service quality and overall performance of the storage system.
[0003] RAID cards in related technologies use a CPU (central processing unit) to implement data processing. This process includes command reception, command parsing, cache allocation, hard drive access, RAID calculation, data migration, command response, and handling of exceptions that occur during this process. Because RAID cards are embedded devices, the performance of the CPU within them is limited. If CPU resources are used for data processing, CPU performance will decrease, resulting in lower RAID card performance. Summary of the Invention
[0004] Therefore, it is necessary to provide a data processing method, system, device, and computer equipment that can improve RAID performance in response to the above-mentioned technical problems.
[0005] Firstly, this application provides a data processing method. Applied to a data processing system, the data processing system includes: a request processing circuit and a cache management circuit, the request processing circuit being connected to the cache management circuit; the method includes:
[0006] The request processing circuit processes the data request, generates a cache request, and sends the cache request to the cache management circuit through the calling interface;
[0007] The cache management circuit parses the cache request to obtain the hard disk address, determines the target cache index corresponding to the hard disk address, and returns the target cache index to the request processing circuit.
[0008] The request processing circuit performs data processing based on the target cache index and the processing type of the data request.
[0009] In the above data processing method, the request processing circuit processes the data request, generates a cache request, and sends the cache request to the cache management circuit through an interface call. The cache management circuit parses the cache request, obtains the hard disk address, determines the target cache index corresponding to the hard disk address, and returns the target cache index to the request processing circuit. The request processing circuit performs data processing based on the target cache index and the processing type of the data request. By adopting the method provided in this embodiment of the invention, dedicated hardware devices for cache management can be provided, avoiding the consumption of CPU resources. Cache resources can be requested without the CPU's awareness, improving the performance of the RAID card and enabling timely processing of data requests.
[0010] In one embodiment, the data processing system further includes a state machine, and the request processing circuit is connected to the cache management circuit through the state machine; the method includes:
[0011] The state machine determines the time sequence of each cache request corresponding to each data request, and sends each cache request to the cache management circuit through the calling interface according to the time sequence.
[0012] Based on the above scheme, the state machine in the data processing system can process the cache request requests corresponding to each data request in chronological order, ensuring the timeliness of processing each data request.
[0013] In one embodiment, the cache management circuit includes a sub-state machine, a cache lookup unit, a tag management circuit, and a cache allocation circuit. The cache management circuit parses the cache request to obtain a hard disk address, determines the target cache index corresponding to the hard disk address, and returns the target cache index to the request processing circuit, including:
[0014] The sub-state machine parses the cache request, obtains the hard disk address, and transmits the hard disk address to the cache lookup unit;
[0015] The cache lookup unit searches for the index corresponding to the hard disk address in a preset lookup table;
[0016] If an index corresponding to the hard disk address is found in the preset lookup table, the found index is used as the target cache index, and the target cache index is sent to the tag management circuit.
[0017] The tag management circuit determines the locking status of the target cache index. If the locking status of the target cache index is unlocked, the target cache index and its unlocked status are returned to the request processing circuit.
[0018] Based on the above scheme, the data processing system can efficiently and accurately determine the locking status of the target cache index based on the tag management circuit.
[0019] In one embodiment, the method further includes:
[0020] If the cache lookup unit does not find the index corresponding to the hard disk address in the preset lookup table, it generates an invalid index flag and returns the invalid index flag to the sub-state machine.
[0021] The sub-state machine determines the processing type of the data request. If the processing type of the data request is the target type, it generates a cache allocation request in response to the invalid index flag and sends the cache allocation request to the cache allocation circuit.
[0022] After receiving the cache allocation request, the cache allocation circuit extracts the free cache index from the preset free list and returns the free cache index and its status to the request processing circuit.
[0023] Based on the above scheme, cache allocation can allocate free cache to the cache request corresponding to the data request when an invalid index flag is received, thus ensuring the comprehensiveness of cache allocation.
[0024] In one embodiment, the method further includes:
[0025] The cache allocation circuit adds the free cache index and the hard disk address corresponding to the free cache index to the preset lookup table.
[0026] Based on the above scheme, the preset lookup table in the cache lookup unit can be continuously updated to ensure the accuracy, real-time performance, and integrity of the data in the preset lookup table.
[0027] In one embodiment, the method further includes:
[0028] If the target cache index is in a locked state, the tag management circuit determines that the target cache index is an invalid index and returns the target cache index and its locked state to the request processing circuit.
[0029] Based on the above scheme, it is possible to accurately identify invalid cache indexes.
[0030] In one embodiment, the method further includes:
[0031] The request processing circuit calculates the memory address based on the target cache index, and performs data processing based on the memory address and the processing type of the data request.
[0032] Based on the above scheme, the memory address can be calculated based on the calculated target cache index, and data requests can be processed in a timely and efficient manner based on the memory address.
[0033] Secondly, this application also provides a data processing system, the data processing system comprising: a request processing circuit and a cache management circuit, wherein the request processing circuit is connected to the cache management circuit, and wherein:
[0034] The request processing circuit is used to process data requests, generate cache request requests, and send the cache request requests to the cache management circuit through the calling interface;
[0035] The cache management circuit is used to parse the cache request, obtain the hard disk address, determine the target cache index corresponding to the hard disk address, and return the target cache index to the request processing circuit.
[0036] The request processing circuit is also used to perform data processing based on the target cache index and the processing type of the data request.
[0037] Thirdly, this application also provides a data processing apparatus, which is applied to a data processing system, the data processing system comprising: a request processing circuit and a cache management circuit, wherein the request processing circuit is connected to the cache management circuit; the apparatus comprises:
[0038] The sending unit, which operates in the request processing circuit, is used to process data requests, generate cache request requests, and send the cache request requests to the cache management circuit through the calling interface.
[0039] The parsing unit, which operates in the cache management circuit, is used to parse the cache request, obtain the hard disk address, determine the target cache index corresponding to the hard disk address, and return the target cache index to the request processing circuit.
[0040] The data processing unit, which operates within the request processing circuit, performs data processing based on the target cache index and the processing type of the data request.
[0041] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising:
[0042] processor;
[0043] Memory used to store the processor's executable instructions;
[0044] The processor is configured to execute the instructions to implement the data processing method as described in any one of the first aspects.
[0045] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform a data processing method as described in any one of the first aspects.
[0046] According to a sixth aspect of the present disclosure, a computer program product is provided, the computer program product including instructions that, when executed by a processor of an electronic device, enable the electronic device to perform a data processing method as described in any one of the first aspects.
[0047] The aforementioned data processing method, system, apparatus, and computer equipment are applied to a data processing system. The data processing system includes a request processing circuit and a cache management circuit, with the request processing circuit connected to the cache management circuit. The method includes: the request processing circuit processes data requests, generates cache request requests, and sends the cache request requests to the cache management circuit via an interface call; the cache management circuit parses the cache request requests, obtains the hard disk address, determines the target cache index corresponding to the hard disk address, and returns the target cache index to the request processing circuit; the request processing circuit performs data processing based on the target cache index and the processing type of the data request. By adopting the method provided in this embodiment of the invention, dedicated hardware devices for cache management can be provided, avoiding CPU resource consumption. Cache resources can be requested without the CPU's awareness, improving the performance of the RAID card and enabling timely processing of data requests. Attached Figure Description
[0048] Figure 1 This is a diagram illustrating the application environment of a data processing method in one embodiment.
[0049] Figure 2 This is a flowchart of a data processing method in one embodiment;
[0050] Figure 3 This is a schematic diagram of the structure of a data processing system in one embodiment;
[0051] Figure 4A This is a schematic diagram of the cache management circuit in one embodiment;
[0052] Figure 4B This is a schematic diagram of the cache management circuit in one embodiment;
[0053] Figure 4CThis is a schematic diagram illustrating the interaction between the linked lists in the elimination management circuit in one embodiment;
[0054] Figure 5 This is a schematic diagram of the structure of a data processing system in one embodiment;
[0055] Figure 6 This is a flowchart illustrating the step of determining the target cache index in one embodiment;
[0056] Figure 7 This is a flowchart illustrating the step of allocating an idle cache in one embodiment;
[0057] Figure 8 This is a schematic diagram of the data processing system in another embodiment;
[0058] Figure 9A This is a flowchart illustrating the write request processing of a data processing method in one embodiment.
[0059] Figure 9B This is a flowchart illustrating the degree request processing of a data processing method in one embodiment.
[0060] Figure 10 This is a structural block diagram of a data processing device in one embodiment;
[0061] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0063] The data processing method provided in this application embodiment can be applied to, for example... Figure 1 In the data processing system shown, the data processing system 100 can be a data processing system 100 within a RAID card, or any data processing system 100 used to process data requests. Specifically, the data processing system 100 can be an I / O (input / output) request processing system. The data processing system 100 includes a request processing circuit 10 and a cache management circuit 20. The request processing circuit 10 can be a Quick-Path Module circuit, and the cache management circuit 20 can be a cache management hardening module (CMHarden Module), specifically a logic module implemented by an ASIC (Application Specific Integrated Circuit).
[0064] The data processing method provided in this embodiment can process a data processing request through a request processing circuit 10 upon receiving the request. During the data processing process, the request processing circuit 10 generates a cache request to allocate cache resources. The request processing circuit 10 can invoke a cache management circuit 20 via an interface. The cache management circuit 20 then processes the cache request and returns corresponding information to the request processing circuit 10, allowing the request processing circuit 10 to continue executing the data processing request based on this information.
[0065] Figure 2 This is a flowchart illustrating a data processing method according to an exemplary embodiment, such as... Figure 2 As shown, the method includes the following steps.
[0066] Step 202: The request processing circuit processes the data request, generates a cache request, and sends the cache request to the cache management circuit by calling the interface.
[0067] This data request can be an I / O request. The API call is used to implement the call from the request processing circuit 10 to the cache management circuit 20.
[0068] In this embodiment, the user can perform data processing operations on the business system based on the needs of the actual application. In response to the user's data processing operation, the business system can generate a data request and send it to the corresponding data processing system 100. After receiving the data request, the request processing circuit 10 in the data processing system 100 can process the data request. Specifically, the request processing circuit 10 can process the data processing request in response to the data request. This processing flow may include command reception, command parsing, cache allocation, hard disk access, RAID calculation, data migration, and command response. It may also include exception handling processes generated during the above processing. Thus, when the request processing circuit 10 is performing a cache allocation process, it can generate a cache allocation request and send it to the cache management circuit 20 through a preset calling interface.
[0069] Optionally, the data request may include the disk ID and storage unit address (LBA).
[0070] Step 204: The cache management circuit parses the cache request, obtains the hard disk address, determines the target cache index corresponding to the hard disk address, and returns the target cache index to the request processing circuit.
[0071] In this embodiment, after the cache management circuit 20 receives the cache request, it can parse the request to determine the hard disk address, i.e., the hard disk number and the storage unit address, carried in the request. Based on this hard disk address, the cache management circuit 20 can determine the target cache index in memory corresponding to that address and return the determined target cache index to the request processing circuit 10.
[0072] Step 206: The request processing circuit performs data processing based on the target cache index and the processing type of the data request.
[0073] In this embodiment, the request processing circuit 10 can process the data request based on the target cache index, the status data of the target cache index, and the processing type of the data request.
[0074] Optionally, the data request processing type can be write type or read type. In one example, the data request can be a write request. In this way, the request processing circuit 10 can move the target data into memory and then write the target data in memory into the hard disk based on the hard disk address, thus completing the processing of the data request.
[0075] In the above data processing method, the request processing circuit processes the data request, generates a cache request, and sends the cache request to the cache management circuit through an interface call. The cache management circuit parses the cache request, obtains the hard disk address, determines the target cache index corresponding to the hard disk address, and returns the target cache index to the request processing circuit. The request processing circuit performs data processing based on the target cache index and the processing type of the data request. By adopting the method provided in this embodiment of the invention, dedicated hardware devices for cache management can be provided, avoiding the consumption of CPU resources. Cache resources can be requested without the CPU's awareness, improving the performance of the RAID card and enabling timely processing of data requests.
[0076] In one exemplary embodiment, the data processing system 100 further includes a state machine. Specifically, the state machine may be a multiplexing module, such as... Figure 3 As shown, the request processing circuit 10 can be connected to the cache management circuit 20 via the state machine 30. Accordingly, the data processing method includes:
[0077] The state machine determines the time sequence of each cache request corresponding to each data request, and sends each cache request to the cache management circuit through the API call according to the time sequence.
[0078] In this embodiment, the calling interface can be a pre-configured interface. State machine 30 receives multiple cache request requests corresponding to various data requests, each with a specific time sequence. Upon receiving these cache request requests, state machine 30 can determine their temporal order. Based on this temporal order, state machine 30 can then send each cache request to the cache management circuit 20 via the pre-defined calling interface, proceeding from first to last.
[0079] Based on the above scheme, the state machine in the data processing system can process the cache request requests corresponding to each data request in chronological order, ensuring the timeliness of processing each data request.
[0080] In one exemplary embodiment, such as Figure 4A As shown, the cache management circuit 20 includes a sub-state machine 400 (Finite-state machine Fsm, FSM), a cache lookup unit 410 (Hash Tbl), a tag management circuit 420 (Tag_Mgr, tag manager), and a cache allocation circuit 440 (Cache Allocator, cache allocation manager). The sub-state machine 400 is connected to the cache lookup unit 410, the cache lookup unit 410 is connected to the tag management circuit 420, the tag management circuit 420 is connected to the cache allocation circuit 440, and the sub-state machine 400 is also connected to the cache allocation circuit 440.
[0081] like Figure 4B As shown, the cache management circuit 20 also includes an eviction management circuit 450, which is connected to the sub-state machine 400.
[0082] Specifically, such as Figure 4C As shown, this eviction management circuit is used to move caches that meet preset conditions to a preset free list. In one example, the eviction management circuit can scan the LRU list (the cache list in the eviction management circuit) to determine whether the caches in the LRU list have been repeatedly accessed in a recent period, identify hotspots, and reduce the likelihood of caches in the LRU list being evicted. The eviction management circuit can implement cache eviction priority by maintaining two linked lists: Active-List and Inactive-List. A preset Two-Lists algorithm can be used to enable the two linked lists to jointly implement the LRU function, prioritizing the removal of caches at the head of the Inactive-List. The LRU list includes both the Active-List and the Inactive-List.
[0083] Caches that have been used once are first moved to the Inactive-List and marked as 'Referenced'. Caches that have not been accessed between two LRU cache scans are marked as 'Non-Referenced'. Caches in the linked list are first swapped out according to the FIFO (First-In-First-Out) principle. If a cache in the Inactive-List is accessed again before the next LRU scan, then this cache will be moved to the Active-List during this scan and marked as 'Non-Reference'.
[0084] The caches entering the Active-List will change between 'Non-Reference' and 'Reference' states depending on whether they are accessed in two scans. When the number of caches in the Active-List reaches a certain amount, caches marked as 'Non-Reference' will be moved to the Inactive-List first, and then eventually evicted to the Free List (a pre-defined free list) according to this principle. In this way, the cache allocation circuit can allocate new caches for data requests from the pre-defined free list.
[0085] like Figure 5 As shown, the data processing system 100 may further include a first memory 40 and a second memory 50. The management structure of each module in the cache management circuit 20 can be stored in the first memory 40, and the cache entity corresponding to the data processing system 100 can be stored in the second memory 50. The first memory may be on-a-chip memory (OCM), and the second memory may be double data rate synchronous dynamic random access memory (DDR).
[0086] Specifically, such as Figure 6 As shown, in step 230, the cache management circuit 20 parses the cache request, obtains the disk address, determines the target cache index corresponding to the disk address, and returns the target cache index to the request processing circuit 10. The specific execution process includes:
[0087] Step 2301: The sub-state machine parses the cache request, obtains the hard disk address, and transmits the hard disk address to the cache lookup unit.
[0088] In this embodiment, after the cache management circuit 20 receives a cache request, the sub-state machine 400 in the cache management circuit 20 parses the cache request to determine the hard disk address carried in the cache request. This hard disk address can be, for example, an LBA address. Then, the sub-state machine 400 can transmit the parsed LBA address to the cache lookup unit 410.
[0089] The cache lookup unit 410 may include a hash lookup table.
[0090] Step 2302: The cache lookup unit searches for the index corresponding to the hard disk address in the preset lookup table.
[0091] In this embodiment, the cache lookup unit 410 may be configured with a mapping table between hard disk addresses and cache indexes, i.e., a preset lookup table. Thus, after receiving the hard disk address transmitted by the sub-state machine 400, the cache lookup unit 410 can perform a lookup operation based on this preset lookup table to determine the index corresponding to the hard disk address.
[0092] Step 2303: If an index corresponding to the hard disk address is found in the preset lookup table, the found index is used as the target cache index and sent to the tag management circuit.
[0093] In this embodiment, if the cache lookup unit 410 finds the index corresponding to the hard disk address in the preset lookup table, that is, if the cache lookup unit 410 determines that the preset lookup table pre-stores the hard disk address and the cache index corresponding to the hard disk address, the cache lookup unit 410 can determine that the index corresponding to the hard disk address found is a valid index. Thus, the cache lookup unit 410 can use the found index as the target cache index and transmit the target cache index to the tag management circuit 420.
[0094] Step 2304: The tag management circuit determines the lock status of the target cache index. If the lock status of the target cache index is unlocked, the target cache index and the unlocked status of the target cache index are returned to the request processing circuit.
[0095] In this embodiment, after receiving the target cache index, the tag management circuit 420 can determine the locking status of the target cache index. Since the cache is mutually exclusive for multiple data requests, meaning that the cache can only receive and process one data request within a preset time period, the tag management circuit 420 can search for the locking status of the target cache index in the management data stored in the first memory under the current circumstances. If the tag management circuit 420 determines that the target cache index is unlocked under the current circumstances, it can determine that no other data requests are currently processing the cache corresponding to the target cache index. Therefore, the tag management circuit 420 can return the target cache index and the unlocked status of the cache corresponding to the target cache index to the data processing circuit.
[0096] Optionally, the tag management circuit 420 can transmit the target cache index and the unlocked state of the cache corresponding to the target cache index to the state machine 30 through the sub-state machine 400. In this way, the state machine 30 can return the target cache index and the unlocked state of the cache corresponding to the target cache index to the data request circuit so that the data request circuit can process the data request.
[0097] Based on the above scheme, the data processing system can efficiently and accurately determine the locking status of the target cache index based on the tag management circuit.
[0098] In one embodiment, such as Figure 7 As shown, the data processing method also includes:
[0099] Step 710: If the cache lookup unit does not find the index corresponding to the hard disk address in the preset lookup table, it generates an invalid index flag and returns the invalid index flag to the sub-state machine.
[0100] In this embodiment, if the cache lookup unit 410 does not find the index corresponding to the hard disk address in the preset lookup table, the cache lookup unit 410 can determine that the hard disk address and the index corresponding to the hard disk address are not pre-stored in the preset lookup table. Thus, the cache lookup can determine that the index corresponding to the hard disk address is an invalid index, generate an invalid index flag, and return the invalid index flag to the sub-state machine 400. In one example, the invalid index flag could be ffffffff.
[0101] Step 720: The sub-state machine determines the processing type of the data request. If the processing type of the data request is the target type, a cache allocation request is generated in response to the invalid index flag, and the cache allocation request is sent to the cache allocation circuit.
[0102] In this embodiment, the sub-state machine 400 can pre-determine the processing type of the data request. If it is determined that the processing type of the data request corresponding to the cache request is the target processing type, the sub-state machine 400 can respond to the invalid index flag, generate a cache allocation request, and send the cache allocation request to the cache allocation circuit 440.
[0103] Optionally, the processing type of the data request may include both read and write types, and the target type may be write type. If the sub-state machine 400 determines that the data request corresponding to the cache request is a write type, in response to the invalid index flag, the sub-state machine 400 generates a cache allocation request. This cache allocation request instructs the cache allocation circuit 440 to allocate an available cache for the disk address.
[0104] Step 730: After receiving a cache allocation request, the cache allocation circuit extracts the free cache index from the preset free list and returns the free cache index and its status to the request processing circuit.
[0105] In this embodiment, the cache allocation circuit 440 can pre-store the multiple free caches and generate a preset free list based on the cache indices corresponding to each of the multiple free caches. Thus, after receiving a cache allocation request, the cache allocation circuit 440 can extract the target free cache index from the preset free list and determine the state of the target free cache index. This state can include the locking state of the target free cache index, the state of the data in the target free cache index, and the state of the target free cache index itself. Based on this, the cache allocation circuit 440 can return the free cache index and its state to the request processing circuit 10, so that the request processing circuit 10 can process the data request based on the free cache index and its state.
[0106] The target free cache index can be any free cache index in a preset free list. The preset free list can be a first-in-first-out queue. Thus, the target free cache index can be the free cache index that first enters the preset free list in the time dimension.
[0107] In one example, the request processing circuit 10 can obtain entity data from the hard disk based on the hard disk address and write the entity data into the free cache corresponding to the target free cache index.
[0108] Based on the above scheme, cache allocation can allocate free cache to the cache request corresponding to the data request when an invalid index flag is received, thus ensuring the comprehensiveness of cache allocation.
[0109] In one embodiment, the data processing method further includes:
[0110] The cache allocation circuit 440 adds the free cache index and the corresponding hard disk address to the preset lookup table.
[0111] In this embodiment, after the cache allocation circuit 440 returns the free cache index and its status to the request processing circuit 10, it can determine that there is a mapping relationship between the free cache index determined in the above steps and the hard disk address carried in the cache request. Thus, the cache allocation circuit 440 can send the free cache index and the hard disk address corresponding to the free cache index to the cache lookup unit 410. In this way, the cache lookup unit 410 can add the free cache index and the hard disk address corresponding to the free cache index to the preset lookup table.
[0112] Based on the above scheme, the preset lookup table in the cache lookup unit can be continuously updated to ensure the accuracy, real-time performance, and integrity of the data in the preset lookup table.
[0113] In one embodiment, the data processing method further includes:
[0114] If the target cache index is in a locked state, the flag management circuit determines that the target cache index is invalid and returns the target cache index and its locked state to the request processing circuit.
[0115] In this embodiment, when the tag management circuit 420 determines that the target cache index is locked, it can determine that the cache corresponding to the target cache index is being processed by other data requests in the current situation. Since the cache is mutually exclusive for multiple data requests, it is not possible to process the data request at this time, and the target cache index is determined to be an invalid index. In this way, the tag management circuit 420 can return the target cache index and the locked state (invalid state) of the target cache index to the request processing circuit 10 through the sub-state machine 400.
[0116] Based on the above scheme, it is possible to accurately identify invalid cache indexes.
[0117] In one embodiment, the request processing circuit 10 performs data processing based on the target cache index and the processing type of the data request, including:
[0118] The request processing circuit calculates the memory address based on the target cache index, and performs data processing based on the memory address and the processing type of the data request.
[0119] In this embodiment of the present disclosure, the state machine 30 sends the received target cache index and the state of the target cache index to the request processing circuit 10. Thus, after the request processing circuit 10 receives the above data, if the request processing circuit 10 determines that the target cache index is a valid index, it can calculate the memory address based on the target cache index, and perform DMA calculation or XOR calculation based on the memory address to obtain the calculation result. Based on the calculation result, the processing of the data request is realized.
[0120] Based on the above scheme, the memory address can be calculated based on the calculated target cache index, and data requests can be processed in a timely and efficient manner based on the memory address.
[0121] In one embodiment, such as Figure 8 The diagram shown illustrates the internal structure of a data processing system 100. This system includes a request processing circuit 10 (QuickPath), a state machine 30, a cache management circuit 20, a first memory 40, and a second memory 50. The cache management circuit 20 includes a sub-state machine 400, a cache lookup unit 410, an evictation management circuit 450, a cache allocation circuit 440, and a tag management circuit 420. The data processing system 100 may also include a CPU for collaboratively processing data requests.
[0122] State machine 30 (FSM) is used for state management of various circuits and modules within cache management circuit 20, and can transmit state and control signals to other modules in cache management circuit 20. Cache lookup unit 410 (HashTbl) can internally store a cache lookup table, i.e., a hash table. Cache lookup unit 410 can hash a preset buffer area using a preset hash function, thus improving the lookup efficiency of cache lookup unit 410. Cache lookup unit 410 can internally store the mapping relationship between the starting address of the cache and the address (LBA) corresponding to the virtual disk. The depth of the hash table buckets and collision table is determined by resources. Cache allocation circuit 440 (Cache Allocator) is used to maintain free caches in the cache, i.e., the free linked list corresponding to the free caches. Marking management circuit 420 can mark the status of the cache index corresponding to each cache.
[0123] Thus, when the FSM receives a cache request (cache request operation) initiated by the QuickPath module, the FSM parses the cache request, obtains the LBA address carried in the cache request, and transmits the LBA address to the cache lookup unit 410. The cache lookup unit 410 determines the cache index corresponding to the LBA address based on a preset lookup table. If the cache lookup unit 410 finds the cache index corresponding to the LBA address in the preset lookup table, it determines that the cache index is a valid index; if the cache lookup unit 410 does not find the cache index corresponding to the LBA address in the preset lookup table, it determines that the cache index is an invalid index.
[0124] If the cache lookup unit 410 determines that the cache index is a valid index, it can send the valid index to the tag management circuit 420. The tag management circuit 420 (Tag_Mgr) will determine the locking status of the valid index, that is, whether there are other data requests processing the cache corresponding to the valid index. If there are no other data requests processing the cache corresponding to the valid index, the tag management circuit 420 can determine that the index status of the valid index is unlocked. If there are other data requests processing the cache corresponding to the valid index, the tag management circuit 420 can determine that the index status of the valid index is locked. In this way, when the tag management circuit 420 receives a valid index, it can query the lock status identification information corresponding to the valid index to determine the locking status of the valid index.
[0125] If the valid index is unlocked, the tag management circuit 420 can return the index and its unlocked state to the request processing circuit 10. If the valid index is locked, the locked state of the index and the cached index is returned to the request processing circuit 10.
[0126] If the cache lookup unit 410 does not find the cache index corresponding to the LBA address in the preset lookup table, it determines that the cache index is invalid. Then, a cache allocation request can be generated through the sub-state machine 400 and sent to the Cache Allocator. Unused cache is retrieved from the free list and added to the preset hash lookup table in the cache lookup unit 410.
[0127] In one embodiment, a data processing request can send a cache request to state machine 30. State machine 30 can determine the size of the data request corresponding to the cache request. If the data request is less than a preset threshold, state machine 30 can send the cache request to cache management circuit 20. If the data request is greater than or equal to the preset threshold, state machine 30 can send the cache request to CPU. In this way, by implementing a mutual exclusion mechanism, both hardware and software can have the ability to use the cache, making it possible to fully harden the processing of some small I / O requests. The preset threshold can be 4kb.
[0128] Optionally, the cache management circuit 20 is a hardware processing circuit for cache resources, which can be used to handle smaller IO requests, while the CPU can handle larger IO requests. In this way, by configuring the same interface for the CPU and the data processing system 100, and realizing the application and configuration of cache through the interface, the cache management circuit 20 and the firmware on the CPU can work together to achieve the effect of using cache resources together.
[0129] The following describes in detail the process of the data processing system in processing data requests, using a specific embodiment as an example. Figure 9A As shown, when the data processing system determines that the processing type of the data request can be write, it obtains the IO request of the write type through the Quick-Path write entry of the Quick-Path Module. Based on the diskID and LBA carried in the IO request as input parameters, it performs a cache allocation request. The cache management circuit can process this cache allocation request and determine whether an available cache can be allocated. If an available cache can be allocated, DMA can be initiated to move the host data to the cache, thus releasing the cache and then executing the steps to respond to the host. If no available cache can be allocated, the Quick Path process can be exited, and the IO request (IO command) can be forwarded to the firmware, i.e., forwarded to the CPU.
[0130] The following describes in detail the process of the data processing system in processing data requests, using a specific embodiment as an example. Figure 9BAs shown, when the data processing system determines that the processing type of the data request can be read, it obtains the IO request of the read type through the Quick-Path read entry of the Quick-Path Module. Based on the diskID and LBA carried in the IO request as input parameters, it performs a cache allocation request. The cache management circuit can process this cache allocation request and determine whether an available cache can be allocated. If an available cache can be allocated, it can determine whether the data stored in the cache is valid. If valid, it can initiate DMA to move the host data to the cache, thus releasing the cache and then executing the steps to respond to the host. If no available cache can be allocated, the Quick Path process can be exited, and the IO request (IO command) can be forwarded to the firmware, i.e., forwarded to the CPU.
[0131] If the data corresponding to the cache is invalid, a pass-through disk read command can be generated by bypassing the cache. Based on this pass-through disk read command, the data in the hard drive can be read. In this way, the cache can be released, and then the steps of responding to the host can be executed.
[0132] It should be understood that, although Figure 1 The steps in the flowchart in Figure 9 are shown sequentially as indicated by the arrows; however, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order requirement for the execution of these steps, and they can be performed in other orders. Furthermore, Figure 1 - At least some of the steps in Figure 9 may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.
[0133] It is understood that the same / similar parts between the various embodiments of the methods described above in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the description of other method embodiments.
[0134] Figure 1 This is a system block diagram illustrating a data processing system according to an exemplary embodiment. The data processing system includes: a request processing circuit and a cache management circuit, wherein the request processing circuit and the cache management circuit are connected, and:
[0135] The request processing circuit is used to process data requests, generate cache request requests, and send the cache request requests to the cache management circuit by calling the interface.
[0136] The cache management circuit is used to parse cache request requests, obtain the disk address, determine the target cache index corresponding to the disk address, and return the target cache index to the request processing circuit.
[0137] The request processing circuit is also used to process data based on the target cache index and the processing type of the data request.
[0138] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.
[0139] Figure 10 This is a block diagram illustrating a data processing apparatus 1000 according to an exemplary embodiment. (Refer to...) Figure 10 The device includes a sending unit 1001, a parsing unit 1002, and a data processing unit 1003.
[0140] The sending unit 1001 operates in the request processing circuit and is used to process data requests, generate cache request requests, and send the cache request requests to the cache management circuit by calling the interface.
[0141] The parsing unit 1002 operates in the cache management circuit and is used to parse cache request requests, obtain the hard disk address, determine the target cache index corresponding to the hard disk address, and return the target cache index to the request processing circuit.
[0142] The data processing unit 1003 operates in the request processing circuit and is used to process data based on the target cache index and the processing type of the data request.
[0143] Each module in the aforementioned data processing device 1000 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0144] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to data requests. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a data processing method.
[0145] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0146] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.
[0147] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.
[0148] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0149] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0150] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0151] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0152] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A data processing method, characterized by, The method is applied to a data processing system, the data processing system including: a request processing circuit and a cache management circuit, the request processing circuit being connected to the cache management circuit; the method includes: In the case of the request processing circuit performing a cache request process for data requests, a cache request is generated and sent to the cache management circuit through an interface call; the data request carries the hard disk number and storage unit address; The cache management circuit parses the cache request to obtain the hard disk address, determines the target cache index corresponding to the hard disk address, and returns the target cache index to the request processing circuit. When the request processing circuit determines that the target cache index is a valid index, it calculates the memory address based on the target cache index, and performs direct memory access (DMA) calculation or XOR calculation based on the memory address to obtain the calculation result. The calculation result is used to process the data request.
2. The method of claim 1, wherein, The data processing system further includes a state machine, and the request processing circuit is connected to the cache management circuit through the state machine; the method includes: The state machine determines the time sequence of each cache request corresponding to each data request, and sends each cache request to the cache management circuit through the calling interface according to the time sequence.
3. The method of claim 1, wherein, The cache management circuit includes a sub-state machine, a buffer lookup unit, a tag management circuit, and a cache allocation circuit. The cache management circuit parses the cache request to obtain the hard disk address, determines the target cache index corresponding to the hard disk address, and returns the target cache index to the request processing circuit, including: The sub-state machine parses the cache request, obtains the hard disk address, and transmits the hard disk address to the cache lookup unit; The cache lookup unit searches for the index corresponding to the hard disk address in a preset lookup table; If an index corresponding to the hard disk address is found in the preset lookup table, the found index is used as the target cache index, and the target cache index is sent to the tag management circuit. The tag management circuit determines the locking status of the target cache index. If the locking status of the target cache index is unlocked, the target cache index and its unlocked status are returned to the request processing circuit.
4. The method of claim 3, wherein, The method further includes: If the cache lookup unit does not find the index corresponding to the hard disk address in the preset lookup table, it generates an invalid index flag and returns the invalid index flag to the sub-state machine. The sub-state machine determines the processing type of the data request. If the processing type of the data request is the target type, it generates a cache allocation request in response to the invalid index flag and sends the cache allocation request to the cache allocation circuit. After receiving the cache allocation request, the cache allocation circuit extracts the free cache index from the preset free list and returns the free cache index and its status to the request processing circuit.
5. The method of claim 4, wherein, The method further includes: The cache allocation circuit adds the free cache index and the hard disk address corresponding to the free cache index to the preset lookup table.
6. The method of claim 3, wherein, The method further includes: If the target cache index is in a locked state, the tag management circuit determines that the target cache index is an invalid index and returns the target cache index and its locked state to the request processing circuit.
7. A data processing system, characterized by The data processing system includes: a request processing circuit and a cache management circuit, wherein the request processing circuit is connected to the cache management circuit, and: The request processing circuit is used to generate a cache request in the case of a data request cache request process, and send the cache request to the cache management circuit through an interface call; the data request carries the hard disk number and storage unit address; The cache management circuit is used to parse the cache request, obtain the hard disk address, determine the target cache index corresponding to the hard disk address, and return the target cache index to the request processing circuit. The request processing circuit is further configured to, when the target cache index is determined to be a valid index, calculate a memory address based on the target cache index, and perform direct memory access (DMA) calculation or XOR calculation based on the memory address to obtain a calculation result, which is used to process the data request.
8. The system according to claim 7, characterized in that, The data processing system further includes a state machine, and the request processing circuit is connected to the cache management circuit through the state machine; wherein: The state machine is used to determine the time sequence of each cache request corresponding to each data request, and to send each cache request to the cache management circuit through the calling interface according to the time sequence.
9. A data processing apparatus, characterized by, The device is applied to a data processing system, the data processing system including: a request processing circuit and a cache management circuit, the request processing circuit being connected to the cache management circuit; the device includes: The sending unit, operating within the request processing circuit, generates a cache request in the case of a data request cache allocation process, and sends the cache request to the cache management circuit via an interface call; the data request carries the hard disk number and storage unit address; The parsing unit, which operates in the cache management circuit, is used to parse the cache request, obtain the hard disk address, determine the target cache index corresponding to the hard disk address, and return the target cache index to the request processing circuit. The data processing unit, running in the request processing circuit, is used to calculate a memory address based on the target cache index when the target cache index is determined to be a valid index, and to perform direct memory access (DMA) calculation or XOR calculation based on the memory address to obtain a calculation result. The calculation result is used to process the data request.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.