Cold and hot data distribution management method and device for nonvolatile storage device

Abstract

The invention provides a cold and hot data distribution management method and device for a nonvolatile storage device. Aiming at the mixed storage requirement of real-time data and historical data in an industrial control scene, a dynamic shunting method based on logic space popularity evaluation is provided. A 512GB SSD user address space is divided into 64000 logic spaces, an exponential weighting algorithm is adopted to collect operation frequency data in real time, an access sequence Order array is generated in combination with a DRAM historical double linked list, and a comprehensive heat value is calculated. And dynamically generating a threshold value based on the quantile coefficient, distributing the logic space with the heat value greater than the threshold value to the low-wear flash memory block with the erasure times less than or equal to 5000, and distributing the logic space with the heat value less than the threshold value to the high-wear block. According to the scheme, the storage life is greatly prolonged, the real-time instruction response delay is reduced, the hybrid load characteristic of industrial control high-frequency real-time and low-frequency archiving is adapted, and the balance between data access efficiency and storage reliability is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to a non-volatile storage device cold and hot data shunting management method and device, belonging to the technical field of non-volatile storage. BACKGROUND

[0002] With the development of cloud computing, big data and Internet of Things technology, non-volatile storage devices (such as SSD) are widely used due to their high read / write speed, low power consumption and other advantages. However, flash memory devices have physical property limitations: the erase count of their storage unit flash memory blocks is limited, usually 10,000-100,000 times, and data errors or even failures will occur after exceeding the threshold. Therefore, how to achieve wear leveling through cold and hot data shunting has become a core technology for prolonging the service life of storage devices.

[0003] In the prior art, cold and hot data shunting mainly has the following problems: Traditional methods mostly use fixed thresholds, such as determining hot data when the access frequency is >10 times, which cannot adapt to dynamically changing access patterns, resulting in misjudgment of hot data as cold data or vice versa, accelerating the aging of high-wear blocks. Only based on access frequency or access time to determine data hotness ignores the correlation between access frequency and time, for example, recently high-frequency accessed data should be more "hot" than historically high-frequency but recently unaccessed data, which often leads to distorted hotness evaluation. In addition, direct shunting by physical pages or entire address spaces without logical division of user data results in mixed storage of small-grained hot and cold data, and low wear leveling efficiency. For example, in an industrial control scenario, device log data and real-time control instructions are randomly written to flash memory blocks, and traditional static thresholds cannot distinguish between the two access characteristics, resulting in repeated writing of hot data to already worn-out blocks, shortening the service life of the storage device.

[0004] Therefore, the prior art has the following technical difficulties: 1. There is no suitable solution for the storage device to dynamically adapt to data access patterns, avoid the problem that static thresholds cannot respond to changes in access frequency and order, and realize real-time adjustment of cold and hot data division; 2. The storage device cannot quantify data hotness; 3. How to realize the optimization of logical space management, separate and store cold and hot data in fine granularity by reasonably dividing the user address space, and ultimately achieve wear leveling. SUMMARY

[0005] In view of the above deficiencies of the prior art, the present application aims to provide a non-volatile storage device cold and hot data shunting management method and device, which is particularly suitable for use in flash-based storage devices, and realizes intelligent shunting of cold and hot data by dynamically identifying data access hotness, optimizes storage block wear leveling, and prolongs the service life of the device.

[0006] According to the embodiment of the present application, a first scheme is provided, which is a cold and hot data shunting management method for a non-volatile storage device, comprising the steps of: A user address space in the storage device for receiving user data is uniquely identified by a logical block address, the user address space being a continuous logical address range, and the logical block address being applied to a minimum data block in the address space; Based on the continuous logical address range, the user address space is divided into N logical spaces, each logical space containing a preset number of continuous logical block addresses, and N being a positive integer; When the storage device receives an I / O operation for the user address space, access behavior collection of a target logical space to which the logical block address belongs is triggered, and an operation frequency data of the target logical space is updated by an exponential weighting algorithm; A historical bidirectional linked list is maintained in the DRAM, and each time an I / O operation accesses a target logical space to which a logical block address belongs, the position of the target logical space in the linked list is adjusted to a head position, and an Order array is generated according to the position of each node in the linked list, wherein the smaller the value in the Order array is, the newer the access order of the corresponding logical space is; The heat value of the target logical space is calculated according to the operation frequency data of the target logical space and the Order array reflecting the operation order; A dynamic threshold value is generated based on the heat value of the target logical space, and user data in the target logical space is shunted into hot data and cold data according to the dynamic threshold value, and the hot data is allocated to a low-wear flash memory block, and the cold data is allocated to a high-wear flash memory block.

[0007] Further, the user address space is a logical address region in the storage device for storing user data and does not contain a metadata region and a system reserved region of the storage device, the logical block address uniquely identifies the user address space by a continuous integer, and the capacity of the minimum data block is 512 bytes, 1 KB, 2 KB or 4 KB.

[0008] Further, the step of dividing the user address space into N logical spaces comprises: presetting the number M of continuous logical block addresses contained in each logical space, the value range of M being 128≤M≤4096, and M being an integer power of 2; the total number N of logical spaces = the total number of logical block addresses of the user address space / M, if not divisible, rounding up and adjusting the number of logical block addresses of the last logical space to the remaining value; and sequentially dividing into N continuous logical spaces in the order of logical block addresses from small to large.

[0009] Further, the step of triggering access behavior collection of the target logical space comprises: the I / O operation request comprises a read request and a write request, and the collection of access behavior is triggered only when the target address of the I / O operation falls within the user address space.

[0010] Further, the step of updating the operation frequency data of the target logical space by the exponential weighting algorithm comprises: initializing the operation frequency data T[x] of the target logical space as 0; each time the I / O operation is triggered, updating the operation frequency data by an exponential weighting formula: T[x]=β×T[x]+(1-β)×1, wherein β is a historical weight coefficient, representing the decay proportion of historical operation frequency, 0.5≤β<1, and (1-β) represents the weight of the current operation.

[0011] Further, the step of maintaining the historical double-linked list in the DRAM comprises: traversing the historical double-linked list to find the node corresponding to the target logical space; if the node exists, removing the node from the current position and inserting the node into the head position of the linked list; if the node does not exist, creating a new node and inserting the node into the head position of the linked list; the head node of the historical double-linked list is the most recently accessed logical space, and the tail node is the least recently accessed logical space.

[0012] Further, the step of generating the Order array comprises: initializing the Order array, each element of the Order array corresponding to a logical space; traversing the historical double-linked list in the order from the head to the tail, assigning a sequence value to each logical space node, the head node having a sequence value of 0, and then sequentially increasing by 1; writing the sequence value of each logical space into the position corresponding to the index of the logical space in the Order array, to obtain the Order array reflecting the access sequence of all logical spaces.

[0013] Further, based on the sequence value S_order of the target logical space in the Order array, the access sequence score S[x] is calculated according to the formula S[x]=1 / (S_order+1), wherein the value range of S[x] is (0, 1]; the heat value H[x] is H[x]=α×T[x]+(1-α)×S[x], wherein α is a weight coefficient, 0<α<1, and T[x] is the operation frequency data of the target logical space.

[0014] Further, the step of shunting user data according to the dynamic threshold value comprises: collecting the heat values H[x] of all N logical spaces, arranging the heat values in ascending order to obtain H_sorted=[h_0, h_1, h_2,..., h_{N-1}]; calculating the dynamic threshold value Q=H_sorted[floor(γ×N)], wherein γ is a quantile coefficient, 0.5<γ<1, and floor() is a floor function; determining the logical space with a heat value H[x]>Q as hot data and allocating the hot data to a low-wear flash memory block with an erase count ≤ a preset threshold value, and determining the logical space with a heat value H[x]≤Q as cold data and allocating the cold data to a high-wear flash memory block with an erase count > the preset threshold value, wherein the preset threshold value is a positive integer configured according to the life characteristics of the flash memory block.

[0015] According to an embodiment of the present application, the non-volatile storage device cold and hot data shunting management method in the first scheme provided by the present application is used, and a second scheme is provided. A non-volatile storage device cold and hot data shunting management system comprises: A user address space identification module is configured to uniquely identify a user address space in a storage device for receiving user data by a logical block address, wherein the user address space is a continuous logical address range, and the logical block address is used for a minimum data block in the address space. A logical space division module is configured to divide the user address space into N logical spaces based on the continuous logical address range, wherein each logical space contains a preset number of continuous logical block addresses, and N is a positive integer. An operation frequency collection module is configured to trigger collection of access behavior of a target logical space to which a logical block address belongs when the storage device receives an I / O operation for the user address space, and update operation frequency data of the target logical space by an exponential weighting algorithm. An access order collection module is configured to maintain a historical bidirectional linked list in DRAM, adjust a position of a target logical space to which a logical block address belongs in the linked list to a head position when the target logical space is accessed each time an I / O operation is performed, and generate an Order array according to positions of nodes in the linked list, wherein a smaller value in the Order array indicates a newer access order of a corresponding logical space. A heat value calculation module is configured to calculate a heat value of a target logical space according to operation frequency data of the target logical space and the Order array reflecting the operation order. A dynamic threshold value shunting module is configured to generate a dynamic threshold value based on the heat value of the target logical space, shunt user data in the target logical space into hot data and cold data according to the dynamic threshold value, and allocate the hot data to a low-wear flash memory block and the cold data to a high-wear flash memory block.

[0016] Compared with the prior art, the technical scheme provided by the application has the beneficial effects that the dynamic characteristics of data access are captured in real time through the operation frequency collection and access sequence collection, the historical frequency is attenuated by an exponential weighting algorithm to reflect that the recent operation weight is higher, the access sequence is quantified by an Order array, and the smaller the value is, the newer the access is. The heat value H[x] generated by the combination of the two can accurately reflect the current heat of the data and avoid the rigid problem of the static threshold. The heat value H[x] = a x T[x] + (1-a) x S[x], wherein T[x] is the frequency data, S[x] is the sequence score, and the two-dimensional characteristics of frequency and time are fused. For data with high frequency but not accessed recently, the sequence score S[x] is reduced to make the heat value decrease; for data with low frequency but continuously accessed, the sequence score S[x] is improved to make the heat value increase. The model improves the shunting accuracy of single-dimensional evaluation and reduces invalid data migration. The user address space is divided into N continuous logical spaces, each logical space contains a preset number of LBAs, the block-level management of hot and cold data is realized, the hot data is centrally allocated to the low-wear flash memory block, and the cold data is centrally allocated to the high-wear flash memory block. The historical bidirectional linked list and the Order array are maintained through the DRAM, and the time complexity of the linked list adjustment and the array update of a single I / O operation is O(1), so that the influence on the read-write performance of the storage device is small, and the high real-time scene demand can be met.

[0017] The application solves the problems of inaccurate hot and cold data shunting and uneven wear in the prior art through dynamic and multi-dimensional heat evaluation and fine logical space management, significantly prolongs the service life of the non-volatile storage device, and guarantees the efficient read-write performance. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only show some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0019] Among them: Figure 1 It is a flow chart of a hot and cold data shunting management method of a non-volatile storage device in an embodiment; Figure 2 It is a structural block diagram of a hot and cold data shunting management system of a non-volatile storage device in an embodiment; Figure 3 It is a structural block diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0020] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Example 1 The technical problem addressed in this embodiment is how to improve the lifespan of non-volatile storage devices in special scenarios. For example, in the field of industrial control, PLC and DCS systems need to store two types of data simultaneously: real-time control instructions and historical operation logs. Real-time control instructions are usually defined as hot data, and historical operation logs are defined as cold data. Real-time control instructions have the characteristics of high-frequency read and write, with up to 100 read and write operations per second, but their lifespan is relatively short, being valid only within the control cycle, and requiring low-latency access. Historical operation logs are rarely read after being written, and are only statistically analyzed once at the end of a statistical cycle, but the retention time is required to be very long, such as more than 3 years, which places high demands on storage capacity.

[0022] Existing hot and cold data separation management methods, when applied to industrial control, fail to distinguish between hot and cold data, resulting in real-time instructions and log data being co-stored in the same flash memory block. Due to the high-frequency erasure and writing of real-time instructions, the flash memory block wears out quickly, while the block containing log data remains unupdated for a long time, leading to excessive cooling, reduced overall storage lifespan, and unreasonable storage logic.

[0023] Similarly, when dealing with industrial control storage issues, the storage logic of other existing flash memory blocks does not dynamically adapt to data access patterns, does not quantify the popularity of user data, and cannot reasonably divide the user address space to achieve fine-grained separation of hot and cold data and targeted, efficient storage, thereby improving storage lifespan through wear leveling.

[0024] To address the aforementioned technical problems, this embodiment provides a method for managing hot and cold data splitting in a non-volatile storage device, such as... Figure 1 As shown, the steps include: S101: The user address space in the storage device used to receive user data is uniquely identified by a logical block address. The user address space is a continuous range of logical addresses, and the logical block address corresponds to the smallest data block in the user address space. Specifically, the user address space is a logical address area in the storage device used to store user data and does not include the storage device's metadata area or system reserved area. The logical block address is used to uniquely identify the user address space through consecutive integers, and the capacity of the smallest data block is 512 bytes, 1KB, 2KB, or 4KB.

[0025] Specifically, the steps for dividing the user address space into N logical spaces include: pre-setting the number of consecutive logical block addresses M contained in each logical space, where M ranges from 128 to M to 4096, and M is an integer power of 2; the total number of logical spaces N = the total number of logical block addresses in the user address space / M, and if the division is not exact, rounding up and adjusting the number of logical block addresses in the last logical space to the remaining value; dividing the space into N consecutive logical spaces in ascending order of logical block addresses.

[0026] For example, in the field of industrial control, the system stores user data including real-time control commands, such as motor start and stop signals and sensor threshold parameters, with a single command size ranging from 512 bytes to 2KB. It also stores historical operation logs, including equipment temperature records and fault code records, with a single log size of 4KB. After being written, the logs are only read during monthly maintenance, with an annual access frequency of 10-12 times. The storage device uses a 512GB Nand Flash SSD.

[0027] The total storage capacity is 512GB. The metadata area stores the FTL mapping table and bad block management information, occupying 8GB of space. The system reserved area stores the firmware program, occupying 4GB of space. The remaining 500GB is the user address space, used to store real-time control commands and historical logs. The logical block address (LBA) in the user address space is uniquely identified by consecutive integers. The starting LBA is 0, and the ending LBA is calculated as: total user address space capacity ÷ minimum data block size - 1.

[0028] In the field of industrial control, if the minimum data block size is selected as 4KB, then the terminating LBA = 500GB = 524288000KB, and the total number of LBAs = 524288000KB ÷ 4KB = 131072000. Therefore, the LBA range of the user address space is 0-131072000, totaling 131072000 consecutive LBAs.

[0029] S102: Based on a continuous logical address range, the user address space is divided into N logical spaces, each logical space containing a preset number of continuous logical block addresses, where N is a positive integer; Specifically, the steps for dividing the user address space into N logical spaces include: pre-setting the number of consecutive logical block addresses M contained in each logical space, where M ranges from 128 to M to 4096, and M is an integer power of 2; the total number of logical spaces N = the total number of logical block addresses in the user address space / M, and if the division is not exact, rounding up and adjusting the number of logical block addresses in the last logical space to the remaining value; dividing the space into N consecutive logical spaces in ascending order of logical block addresses.

[0030] Similarly, in the industrial control field, the access characteristics of industrial control data are: high-frequency, small-batch real-time control commands, requiring a smaller logical space to reduce invalid data migration; and low-frequency, large-batch historical logs, requiring a larger logical space to reduce management overhead. Taking M=2048 LBAs as a whole, satisfying 128≤M≤4096, 2048=2 11 Each logical space has a capacity of 2048 × 4KB = 8MB. Since the total number of LBAs is 131072000, N = 131072000 ÷ 2048 = 64000. Therefore, the user address space is divided into 64000 contiguous logical spaces, each containing 2048 LBAs.

[0031] The logical space is divided in ascending order of LBA, numbered 0-63999. Specifically: Logical space 0: LBA is 0-2047; Logical space 1: LBA is 2048-4095... The first 1,000 logical spaces are allocated to real-time control data, with a total capacity of 8MB × 1,000 = 8GB, and the remaining 63,000 logical spaces are allocated to historical log data, with a total capacity of 504GB, which matches the storage needs of the two types of data well.

[0032] S103: When the storage device receives an I / O operation for the user address space, it triggers the collection of access behavior to the target logical space to which the logical block address belongs, and updates the operation frequency data of the target logical space through an exponential weighted algorithm; Specifically, the steps for triggering the collection of access behavior to the target logical space include: the I / O operation request includes read request and write request, and the collection of access behavior is triggered only when the target address of the I / O operation falls within the user address space.

[0033] Specifically, the steps for updating the operation frequency data of the target logic space using the exponential weighting algorithm include: initializing the operation frequency data T[x] of the target logic space to 0; updating the operation frequency data using the exponential weighting formula each time an I / O operation is triggered: T[x] = β × T[x] + (1-β) × 1, where β is the historical weight coefficient, representing the attenuation ratio of the historical operation frequency, 0.5 ≤ β < 1, and (1-β) represents the weight of the current operation.

[0034] S104: Maintain a historical doubly linked list in DRAM. When accessing the target logical space of the logical block address during each I / O operation, adjust the position of the target logical space in the linked list to the head position. Generate an Order array based on the position of each node in the linked list. The smaller the value in the Order array, the newer the access order of the corresponding logical space. Specifically, the steps for maintaining a historical doubly linked list in DRAM include: traversing the historical doubly linked list to find the node corresponding to the target logical space; if the node exists, removing the node from its current position and inserting it at the head of the list; if the node does not exist, creating a new node and inserting it at the head of the list; the head node of the historical doubly linked list is the most recently accessed logical space, and the tail node is the least recently accessed logical space.

[0035] Specifically, the Order array generation steps include: initializing the Order array, where each element of the Order array corresponds to a logical space; traversing the historical doubly linked list from head to tail, assigning an order value to each logical space node, with the head node having an order value of 0, and then incrementing by 1; writing the order value of each logical space into the position corresponding to the logical space index in the Order array, thereby obtaining an Order array that reflects the access order of all logical spaces.

[0036] S105: Calculate the heat value of the target logic space based on the operation frequency data of the target logic space and the Order array that reflects the operation order; Specifically, based on the order value S_order of the target logical space in the Order array, the access order score S[x] is calculated according to the formula S[x]=1 / (S_order+1), where the value range of S[x] is (0,1]; the heat value H[x] is: H[x]=α×T[x]+(1-α)×S[x], where α is the weighting coefficient, 0<α<1, and T[x] is the operation frequency data of the target logical space.

[0037] For example, in the field of industrial control, the I / O operation types and triggering conditions are as follows: When the PLC system receives an I / O request, it first determines whether the target LBA falls within the user address space (0-131071999). For example, a real-time control instruction write request with a target LBA of 1024, belonging to logical space 0, triggers data acquisition; a system firmware update request with a target LBA of 131072000, belonging to the system reserved area, does not trigger data acquisition. For I / O operation types, both read and write requests are acquired simultaneously. Logical space 0 receives 2000 write requests daily for real-time instruction updates and 500 read requests daily for instruction readback verification. Logical space 50000 receives 1 write request monthly for log archiving and 0 read requests monthly.

[0038] Specifically, in the exponential weighted algorithm, the operation frequency data of all logic spaces are initialized with T[x] being 0. The operation frequency of each I / O operation is updated by T[x] = β × T[x] + (1-β) × 1. The historical weight coefficient β can be 0.8 based on the characteristics of short-term bursts and long-term stability of industrial control data access. This means that the historical operation attenuation ratio is 80% and the current operation weight is 20%.

[0039] For example: At time t0, the system starts, T[x] is initialized to 0, the real-time instruction T[0] = 0 in logical space 0, and the historical log T

[50000] = 0 in logical space 50000; At time t1, logical space 0 receives one write request, and T[0] of logical space 0 is 0.8×0+(1-0.8)×1=0.2. Logical space 50000 has no operation, so T[0] is kept at 0. At time t2, the logical space receives the second write request, and the T[0] of logical space 0 is 0.8 × 0.2 + 0.2 × 1 = 0.36; At time t3, logical space 50000 receives one write request, logical space 0 has no operation, and the T

[50000] of logical space 50000 is 0.8×0+(1-0.8)×1=0.2.

[0040] When performing calculations and derivations using the scheme of this embodiment, it can be seen from the examples above that the real-time instruction logic space can accumulate T[x] quickly due to high-frequency operations, while the historical log logic space grows slowly due to low operation frequency.

[0041] In DRAM, each node of a doubly linked list contains a logical space number, a predecessor pointer, and a successor pointer. The maximum capacity of the linked list is 64,000, which is consistent with the total number of logical spaces. After each I / O operation, the target logical space node is moved to the head of the linked list, indicating the most recent access. In the industrial control field, assume the following access event occurs after the system starts up: t1, access logical space 0, the linked list is empty, create a new node and insert it at the head of the list, the linked list is [0]; t2, access logical space 1, create a new node and insert it at the head of the list, the linked list is [1,0]; t3, visit logical space 0 again, remove node 0, insert the head of the list, and the linked list is [0,1]; t4, access logical space 50000, create a new node and insert it at the head of the list, the linked list is [50000,0,1]; t5, access logical space 2, create a new node and insert it at the head of the list, the linked list is [2, 50000, 0, 1]... Continue generating the Order array: The Order array is an array of length 64000, with all elements initialized to -1, indicating that they have not been accessed.

[0042] Sequential value assignment is performed by traversing the linked list from head to tail, assigning sequential values ​​one by one. The head node is 0, and subsequent nodes are incremented by 1. For example, at time t5, when accessing logical space 2, the linked list is [2, 5000, 0, 1]. The order[2] of the head node 2 is 0; the order

[50000] of node 50000 is 1; the order[0] of node 0 is 2; the order[1] of node 1 is 3; and the order[x] of the unvisited node is -1, waiting to be updated during subsequent visits. It can be seen that the smaller the order[x], the closer the logical space access time.

[0043] Continue calculating the heat value H[x]: Based on the requirements for frequency stability in industrial control scenarios, i.e., real-time instructions prioritize continuous high-frequency access, the weighting coefficient α is set to 0.6, which means that the frequency weight is 60% and the order weight is 40%.

[0044] Based on the Order arrays accessing logical space 0 again at time t3 and logical space 50000 at time t5, the heat value of the logical space is: Logical space 0 is a real-time instruction, a high-frequency recent access, T[x] is 0.36, that is, 2 write requests were accumulated at t3; S_order is 2, Order[0]=2, the third recent access, access order score S[x]=1 / (2+1) ≈0.333; heat value H[x]=0.6×0.36+0.4×0.33≈0.349.

[0045] Logical space 50000 is the historical log, low-frequency recent access, T[x] is 0.2, and 1 write request was accumulated at t3; S_order is 1, Order

[50000] =1, the second most recent access, S[x]=1 / (1+1)=0.5, and the heat value H[x]=0.6×0.2+0.4×0.5=0.32.

[0046] If logical space 1000 is not accessed, then the heat value H[x] = 0 is calculated.

[0047] In summary, logical space 0 (H=0.349): real-time instructions have the highest heat value due to high-frequency access and will be identified as hot data; logical space 50000 (H=0.32): although recently accessed, the frequency is extremely low, resulting in the next highest heat value; logical space 1000 (H=0): no access records, the lowest heat value, and is identified as cold data. Therefore, the heat value in this scheme can objectively express the degree of data popularity.

[0048] S106: Generate a dynamic threshold based on the heat value of the target logical space, and divide the user data in the target logical space into hot data and cold data according to the dynamic threshold. Allocate the hot data to low-wear flash memory blocks and the cold data to high-wear flash memory blocks.

[0049] Specifically, collect the heat values ​​H[x] of all N logical spaces and sort them in ascending order to obtain: H_sorted=[h_0,h_1,h_2,...,h_{N-1}]; Calculate the dynamic threshold Q = H_sorted[floor(γ×N)], where γ is the quantile coefficient, 0.5 < γ < 1, and floor() is the floor function. Logical spaces with heat values ​​H[x] > Q are identified as hot data and allocated to low-wear flash memory blocks with erase counts ≤ a preset threshold. Logical spaces with heat values ​​H[x] ≤ Q are identified as cold data and allocated to high-wear flash memory blocks with erase counts > a preset threshold. The preset threshold is a positive integer configured based on the lifespan characteristics of the flash memory blocks.

[0050] Continue by separating hot and cold data based on the heat value: Based on the calculated heat values, the heat values ​​are sorted, that is, traversing 64,000 logical spaces and collecting their current heat values ​​H[x]. In industrial control scenarios, the heat values ​​have obvious hot and cold separation characteristics. The hot data candidate area is the 0-999 logical space. Due to the high frequency of real-time access to the data, H[x] is concentrated in 0.3-0.5. The cold data candidate area is the 1000-63999 logical space. Due to the low frequency of archiving access, H[x] is concentrated in 0-0.2. In particular, the H[x] of the unaccessed logical space is 0.

[0051] Arrange all H[x] in ascending order to get H_sorted=[h_0,h_1,h_2,...,h_63999], where h_0 is the unaccessed log space and h_63999=0.5 is the most frequently accessed real-time instruction space.

[0052] In industrial control scenarios, it is necessary to ensure that 99% of the real-time instruction space is identified as hot data. Therefore, the quantile coefficient γ is set to 0.99. At this time, the target is divided into positions: floor(γ×N)=floor(0.99×64000)=63360. The dynamic threshold Q=H_sorted[floor(γ×N)]=H_sorted

[63360] is calculated. Since the first 63000 values ​​of H_sorted are concentrated in 0-0.2, the 63360th value is located at the 360th position of the last 1000 values ​​of H_sorted, which corresponds to the part with lower heat value in the hot data candidate area. After testing, Q=0.3 is selected.

[0053] When splitting hot and cold data, if the logical space H[x] > 0.3, it is allocated to a low-wear block; if the logical space H[x] ≤ 0.3, it is allocated to a high-wear block.

[0054] Furthermore, the threshold Q needs to be dynamically adjusted. The threshold Q is updated every 24 hours by recalculating H_sorted and updating the Q value. The block wear status can also be refreshed; for example, the number of erases for each block is recounted after every 1000 erases to ensure the real-time accuracy of high and low wear block division.

[0055] Through the above steps, refined logical management of user data in industrial control scenarios is achieved: user data is strictly separated from metadata and system data to avoid interference with real-time instruction reading and writing due to metadata updates; the 8MB logical space satisfies both small-batch, high-frequency access to real-time instructions and avoids the management overhead of log data.

[0056] Although the Order value of logical space 0 is greater than that of logical space 50000, the final heat value surpasses it because T[x] is higher, reflecting that high-frequency continuous access takes precedence over single recent access; the heat value of the unaccessed log area logical space is 0, achieving a significant distinction from hot data; the exponential weighting algorithm and linked list operation are both O(1) complexity. On the PLC system processor, the calculation time of a single heat value is very short, which is sufficient to meet the real-time requirements. In the embodiment, the parameter selection of β=0.8 and α=0.6, as well as the dynamic update mechanism of the Order array, are designed for the mixed load characteristics of high-frequency real-time data and low-frequency archived data in industrial control, ensuring that the heat evaluation results are highly matched with actual business needs.

[0057] Hot data is centrally written to low-wear blocks, while cold data utilizes the remaining lifespan of high-wear blocks, preventing rapid aging of individual blocks. Low-wear blocks, due to fewer erase / write cycles, have faster read / write response speeds than high-wear blocks, meeting the stringent real-time requirements of industrial control. In this embodiment, only 400 low-wear blocks are used for the 640 hot data logic spaces, with the remaining low-wear blocks reserved for backup, avoiding resource waste. By linking dynamic thresholds with wear status, precise separation of hot and cold data in industrial control scenarios is achieved, ensuring low-latency access to real-time commands while maximizing the utilization of flash memory block lifespan.

[0058] Example 2 This embodiment provides a hot and cold data splitting management system for non-volatile storage devices, such as... Figure 2 As shown, it includes: The user address space identification module 100 is used to uniquely identify the user address space in the storage device used to receive user data by means of a logical block address. The user address space is a continuous range of logical addresses, and the logical block address corresponds to the smallest data block in the user address space. The logical space partitioning module 200 is used to divide the user address space into N logical spaces based on a continuous logical address range. Each logical space contains a preset number of continuous logical block addresses, where N is a positive integer. The operation frequency acquisition module 300 is used to trigger the acquisition of access behavior of the target logical space to which the logical block address belongs when the storage device receives an I / O operation for the user address space, and update the operation frequency data of the target logical space through an exponential weighting algorithm. The access order acquisition module 400 is used to maintain a historical doubly linked list in DRAM. When each I / O operation accesses the target logical space to which the logical block address belongs, the module adjusts the position of the target logical space in the linked list to the head position. The module generates an Order array based on the position of each node in the linked list. The smaller the value in the Order array, the newer the access order of the corresponding logical space. The heat value calculation module 500 is used to calculate the heat value of the target logical space based on the operation frequency data of the target logical space and the Order array that reflects the operation order; The dynamic threshold splitting module 600 is used to generate a dynamic threshold based on the heat value of the target logical space. According to the dynamic threshold, the user data in the target logical space is split into hot data and cold data. The hot data is allocated to the low-wear flash memory block, and the cold data is allocated to the high-wear flash memory block.

[0059] Example 3 Figure 3 An internal structural diagram of a computer device in one embodiment is shown. This computer device can specifically be a terminal or a server. Figure 3As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and may also store a computer program. When executed by the processor, this computer program enables the processor to implement a hot and cold data splitting management method. The memory may also store a computer program, which, when executed by the processor, enables the processor to implement the hot and cold data splitting management method. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0060] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the following steps: A method for managing hot and cold data splitting in a non-volatile storage device includes the following steps: The user address space in the storage device used to receive user data is uniquely identified by logical block addresses. The user address space is a continuous range of logical addresses, and the logical block address is applied to the smallest data block within the address space. Based on a continuous range of logical addresses, the user address space is divided into N logical spaces, each containing a preset number of consecutive logical block addresses, where N is a positive integer. When the storage device receives an I / O operation for the user address space, it triggers the collection of access behavior to the target logical space to which the logical block address belongs, and updates the operation frequency data of the target logical space through an exponential weighted algorithm. A historical doubly linked list is maintained in DRAM. When the target logical space to which the logical block address belongs is accessed during each I / O operation, the position of the target logical space in the linked list is adjusted to the head position. An Order array is generated according to the position of each node in the linked list. The smaller the value in the Order array, the newer the access order of the corresponding logical space. The heat value of the target logical space is calculated based on the operation frequency data of the target logical space and the Order array that reflects the operation order; A dynamic threshold is generated based on the heat value of the target logical space. User data in the target logical space is divided into hot data and cold data according to the dynamic threshold. Hot data is allocated to low-wear flash memory blocks, and cold data is allocated to high-wear flash memory blocks.

[0061] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, causes the processor to perform the following steps: A method for managing hot and cold data splitting in a non-volatile storage device includes the following steps: The user address space in the storage device used to receive user data is uniquely identified by logical block addresses. The user address space is a continuous range of logical addresses, and the logical block address is applied to the smallest data block within the address space. Based on a continuous range of logical addresses, the user address space is divided into N logical spaces, each containing a preset number of consecutive logical block addresses, where N is a positive integer. When the storage device receives an I / O operation for the user address space, it triggers the collection of access behavior to the target logical space to which the logical block address belongs, and updates the operation frequency data of the target logical space through an exponential weighted algorithm. A historical doubly linked list is maintained in DRAM. When the target logical space to which the logical block address belongs is accessed during each I / O operation, the position of the target logical space in the linked list is adjusted to the head position. An Order array is generated according to the position of each node in the linked list. The smaller the value in the Order array, the newer the access order of the corresponding logical space. The heat value of the target logical space is calculated based on the operation frequency data of the target logical space and the Order array that reflects the operation order; A dynamic threshold is generated based on the heat value of the target logical space. User data in the target logical space is divided into hot data and cold data according to the dynamic threshold. Hot data is allocated to low-wear flash memory blocks, and cold data is allocated to high-wear flash memory blocks.

[0062] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0063] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0064] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for managing hot and cold data splitting in a non-volatile storage device, characterized in that, Including the following steps: The user address space in the storage device used to receive user data is uniquely identified by logical block addresses. The user address space is a continuous range of logical addresses, and the logical block address is applied to the smallest data block within the address space. Based on a continuous range of logical addresses, the user address space is divided into N logical spaces, each containing a preset number of consecutive logical block addresses, where N is a positive integer. When the storage device receives an I / O operation for the user address space, it triggers the collection of access behavior to the target logical space to which the logical block address belongs, and updates the operation frequency data of the target logical space through an exponential weighted algorithm. A historical doubly linked list is maintained in DRAM. When the target logical space to which the logical block address belongs is accessed during each I / O operation, the position of the target logical space in the linked list is adjusted to the head position. An Order array is generated according to the position of each node in the linked list. The smaller the value in the Order array, the newer the access order of the corresponding logical space. The heat value of the target logical space is calculated based on the operation frequency data of the target logical space and the Order array that reflects the operation order; A dynamic threshold is generated based on the heat value of the target logical space. User data in the target logical space is divided into hot data and cold data according to the dynamic threshold. Hot data is allocated to low-wear flash memory blocks, and cold data is allocated to high-wear flash memory blocks.

2. The method for managing hot and cold data splitting in a non-volatile storage device according to claim 1, characterized in that, The user address space is a logical address area in the storage device used to store user data, and does not include the metadata area of ​​the storage device or the system reserved area. The logical block address is used to uniquely identify the user address space through consecutive integers. The capacity of the smallest data block is 512 bytes, 1KB, 2KB or 4KB.

3. The method for managing hot and cold data splitting in a non-volatile storage device according to claim 1, characterized in that, The steps to divide the user address space into N logical spaces include: The number of consecutive logical block addresses M contained in each logical space is preset. The value of M is in the range of 128≤M≤4096, and M is an integer power of 2. The total number of logical spaces N = the total number of logical block addresses in the user address space / M. If the division is not exact, round up and adjust the number of logical block addresses in the last logical space to the remaining value. The logical block addresses are divided into N consecutive logical spaces in ascending order.

4. The method for managing hot and cold data splitting in a non-volatile storage device according to claim 1, characterized in that, The steps to trigger the collection of access behavior to the target logical space include: The I / O operation requests include read requests and write requests, and the collection of access behavior is triggered only when the target address of the I / O operation falls within the user address space.

5. The method for managing hot and cold data splitting in a non-volatile storage device according to claim 1, characterized in that, The steps for updating the operation frequency data of the target logical space using an exponentially weighted algorithm include: Initialize the operation frequency data T[x] of the target logical space to 0; Each time an I / O operation is triggered, the operation frequency data is updated using an exponentially weighted formula: T[x]=β×T[x]+(1-β)×1 Where β is the historical weight coefficient, representing the attenuation ratio of historical operation frequency, 0.5≤β<1, and (1-β) represents the weight of the current operation.

6. The method for managing hot and cold data splitting in a non-volatile storage device according to claim 1, characterized in that, The steps for maintaining the historical doubly linked list in DRAM include: Traverse the historical doubly linked list to find the node corresponding to the target logical space; If the node exists, remove it from its current position and insert it at the head of the linked list; If the node does not exist, create the node and insert it at the head of the linked list; The head node of the historical doubly linked list represents the most recently accessed logical space, and the tail node represents the least recently accessed logical space.

7. The method for managing hot and cold data splitting in a non-volatile storage device according to claim 1, characterized in that, The Order array generation step includes: Initialize the Order array, where each element of the Order array corresponds to a logical space; Traverse the historical doubly linked list from head to tail, assigning an order value to each logical space node, with the head node having an order value of 0, and then incrementing by 1 for each subsequent node; Write the order value of each logical space into the position corresponding to the index of that logical space in the Order array to obtain the Order array that reflects the access order of all logical spaces.

8. The method for managing hot and cold data splitting in a non-volatile storage device according to claim 5, characterized in that, Based on the order value S_order of the target logical space in the Order array, the access order score S[x]=1 / (S_order+1) is calculated according to the formula, where the value range of S[x] is (0,1]. The heat value H[x] is: H[x]=α×T[x]+(1-α)×S[x], where α is the weighting coefficient, 0<α<1, and T[x] is the operation frequency data of the target logic space.

9. The method for managing hot and cold data splitting in a non-volatile storage device according to claim 8, characterized in that, The steps for user data triage based on dynamic thresholds include: Collect the heat values ​​H[x] of all N logical spaces, and sort them in ascending order to get H_sorted=[h_0,h_1,h_2,...,h_{N-1}]; Calculate the dynamic threshold Q = H_sorted[floor(γ×N)], where γ is the quantile coefficient, 0.5 < γ < 1, and floor() is the floor function. Logical spaces with heat values ​​H[x] > Q are identified as hot data and allocated to low-wear flash memory blocks with erase counts ≤ a preset threshold. Logical spaces with heat values ​​H[x] ≤ Q are identified as cold data and allocated to high-wear flash memory blocks with erase counts > a preset threshold. The preset threshold is a positive integer configured based on the lifespan characteristics of the flash memory blocks.

10. A hot and cold data distribution management system for a non-volatile storage device, characterized in that, include: The user address space identification module is used to uniquely identify the user address space in the storage device used to receive user data by means of logical block addresses. The user address space is a continuous range of logical addresses, and the logical block address is applied to the smallest data block within the address space. The logical space partitioning module is used to divide the user address space into N logical spaces based on a continuous range of logical addresses. Each logical space contains a preset number of continuous logical block addresses, where N is a positive integer. The operation frequency acquisition module is used to trigger the acquisition of access behavior of the target logical space to which the logical block address belongs when the storage device receives an I / O operation for the user address space, and update the operation frequency data of the target logical space through an exponential weighting algorithm. The access order acquisition module is used to maintain a historical doubly linked list in DRAM. When each I / O operation accesses the target logical space to which the logical block address belongs, the position of the target logical space in the linked list is adjusted to the head position. An Order array is generated according to the position of each node in the linked list. The smaller the value in the Order array, the newer the access order of the corresponding logical space. The heat value calculation module is used to calculate the heat value of the target logical space based on the operation frequency data of the target logical space and the Order array that reflects the operation order; The dynamic threshold splitting module is used to generate dynamic thresholds based on the heat value of the target logical space. According to the dynamic thresholds, user data in the target logical space is split into hot data and cold data. Hot data is allocated to low-wear flash memory blocks, and cold data is allocated to high-wear flash memory blocks.

Images