A non-volatile storage device hot and cold data shunting management method and device

Abstract

The application provides a non-volatile storage device cold and hot data shunt management method and device. In view of the mixed storage demand of real-time data and historical data in the industrial control scene, a dynamic shunt method based on logical space heat evaluation is proposed. By dividing the 512GB SSD user address space into 64000 logical spaces, using the exponential weighting algorithm to collect operation frequency data in real time, combining the DRAM history bidirectional list to generate the access order array, and calculating the comprehensive heat value. Based on the quantile coefficient, the threshold value is dynamically generated, the logical space with a heat value greater than the threshold value is allocated to the low-wear flash block with an erase number ≤5000, and the logical space with a heat value less than the threshold value is allocated to the high-wear block. The scheme realizes the substantial prolongation of the storage life, the reduction of the real-time instruction response delay, the adaptation to the mixed load characteristics of the high-frequency real-time and low-frequency archiving of industrial control, and ensures the balance between the data access efficiency and the storage reliability.

Description

TECHNICAL FIELD

[0001] The application relates to a non-volatile storage device cold and hot data shunting management method and device. BACKGROUND

[0002] With the development of cloud computing, big data and Internet of Things technology, non-volatile storage devices (such as SSDs) are widely used due to their high read / write speed, low power consumption and other advantages. However, flash memory devices have physical property limitations: the erase times of their storage unit flash memory blocks are limited, usually 10,000-100,000 times, and data errors or even failures will occur after exceeding the threshold. Therefore, how to achieve wear leveling through cold and hot data shunting has become a core technology for prolonging the service life of storage devices.

[0003] In the prior art, cold and hot data shunting mainly has the following problems:

[0004] Traditional methods mostly use fixed thresholds, for example, access frequency > 10 times is determined as hot data, which cannot adapt to dynamically changing access patterns, resulting in misjudgment of hot data as cold data or vice versa, accelerating the aging of high-wear blocks. Only based on access frequency or access time to judge data hotness, the correlation between access frequency and time is ignored, for example, recently high-frequency accessed data should be more "hot" than historically high-frequency but recently unaccessed data, which often leads to distorted hotness evaluation. In addition, direct shunting in units of physical pages or entire address spaces does not logically divide user data, resulting in mixed storage of small-grained hot and cold data and low wear leveling efficiency. For example, in a certain industrial control scenario, device log data and real-time control instructions are randomly written to flash memory blocks, and traditional static thresholds cannot distinguish the access characteristics of the two, resulting in repeated writing of hot data to already worn blocks, shortening the service life of the storage device.

[0005] Therefore, the prior art has the following technical difficulties: 1. There is no suitable solution for the storage device to dynamically adapt to the data access pattern, avoid the problem that the static threshold cannot respond to the change of access frequency and order, and realize real-time adjustment of cold and hot data division; 2. The storage device cannot realize the quantification of data hotness; 3. How to realize the optimization of logical space management, separate and store cold and hot data in fine granularity through reasonable division of user address space, and finally realize wear leveling. SUMMARY

[0006] In view of the deficiencies of the prior art, the purpose of the present application is to provide a non-volatile storage device cold and hot data shunting management method and device, which is particularly suitable for use in flash-based storage devices, and realizes intelligent shunting of cold and hot data by dynamically identifying data access hotness, optimizes storage block wear leveling, and prolongs the service life of the device.

[0007] According to the embodiments of the present application, a first aspect provides a non-volatile storage device hot and cold data shunting management method, comprising the steps of:

[0008] A user address space in the storage device for receiving user data is uniquely identified by a logical block address, the user address space being a continuous logical address range, and the logical block address being applied to a minimum data block in the address space;

[0009] Based on the continuous logical address range, the user address space is divided into N logical spaces, each logical space containing a preset number of continuous logical block addresses, and N being a positive integer;

[0010] When the storage device receives an I / O operation for the user address space, a target logical space to which the logical block address belongs is triggered to collect access behavior, and an operation frequency data of the target logical space is updated by an exponential weighting algorithm;

[0011] A historical bidirectional linked list is maintained in the DRAM, and each time the target logical space to which the logical block address belongs is accessed in the I / O operation, the position of the target logical space in the linked list is adjusted to a head position, and an Order array is generated according to the position of each node in the linked list, and the smaller the value in the Order array is, the newer the access order of the corresponding logical space is;

[0012] The heat value of the target logical space is calculated according to the operation frequency data of the target logical space and the Order array reflecting the operation order;

[0013] A dynamic threshold value is generated based on the heat value of the target logical space, and user data in the target logical space is shunted into hot data and cold data according to the dynamic threshold value, and the hot data is allocated to a low-wear flash block, and the cold data is allocated to a high-wear flash block.

[0014] Further, the user address space is a logical address region in the storage device for storing user data and does not contain a metadata region and a system reserved region of the storage device, the logical block address uniquely identifies the user address space by a continuous integer, and the capacity of the minimum data block is 512 bytes, 1 KB, 2 KB or 4 KB.

[0015] Further, the step of dividing the user address space into N logical spaces comprises: presetting the number M of continuous logical block addresses contained in each logical space, the value range of M being 128≤M≤4096, and M being an integer power of 2; the total number N of logical spaces being N=total number of logical block addresses of the user address space / M, if not divisible, rounding up and adjusting the number of logical block addresses of the last logical space to the remaining value; and sequentially dividing into N continuous logical spaces in the order of logical block addresses from small to large.

[0016] Further, the step of triggering the collection of the access behavior of the target logical space comprises: the I / O operation request comprises a read request and a write request, and the collection of the access behavior is triggered only when the target address of the I / O operation falls within the user address space.

[0017] Further, the step of updating the operation frequency data of the target logical space by the exponential weighting algorithm comprises: initializing the operation frequency data T[x] of the target logical space as 0; and updating the operation frequency data by an exponential weighting formula each time the I / O operation is triggered: T[x]=β×T[x]+(1-β)×1, wherein β is a historical weight coefficient, representing a decay ratio of the historical operation frequency, 0.5≤β<1, and (1-β) represents a weight of the current operation.

[0018] Further, the step of maintaining the historical double-linked list in the DRAM comprises: traversing the historical double-linked list to find a node corresponding to the target logical space; if the node exists, removing the node from the current position and inserting the node into a head position of the linked list; if the node does not exist, creating a new node and inserting the node into the head position of the linked list; and the head node of the historical double-linked list is a recently accessed logical space, and the tail node is a least recently accessed logical space.

[0019] Further, the step of generating the Order array comprises: initializing the Order array, each element of the Order array corresponding to a logical space; traversing the historical double-linked list in a sequence from the head to the tail, and assigning a sequence value to each logical space node, the head node having a sequence value of 0, and the sequence value being sequentially incremented by 1 thereafter; and writing the sequence value of each logical space into a position corresponding to the index of the logical space in the Order array, to obtain the Order array reflecting the access sequence of all logical spaces.

[0020] Further, based on the sequence value S_order of the target logical space in the Order array, the access sequence score S[x] is calculated according to the formula S[x]=1 / (S_order+1), wherein the value range of S[x] is (0, 1]; and the heat value H[x] is H[x]=α×T[x]+(1-α)×S[x], wherein α is a weight coefficient, 0<α<1, and T[x] is the operation frequency data of the target logical space.

[0021] Further, the step of shunting user data according to the dynamic threshold value comprises: collecting the heat values H[x] of all N logical spaces, arranging the heat values in ascending order to obtain H_sorted=[h_0, h_1, h_2,..., h_{N-1}]; calculating the dynamic threshold value Q=H_sorted[floor(γ×N)], wherein γ is a quantile coefficient, 0.5<γ<1, and floor() is a floor function; determining the logical space with a heat value H[x]>Q as hot data and allocating the hot data to a low-wear flash memory block with an erase count ≤ a preset threshold value, and determining the logical space with a heat value H[x]≤Q as cold data and allocating the cold data to a high-wear flash memory block with an erase count > the preset threshold value, wherein the preset threshold value is a positive integer configured according to the life characteristics of the flash memory block.

[0022] According to an embodiment of the present application, the non-volatile storage device cold-hot data shunting management method in the first scheme provided by the present application is used, and a second scheme is provided.

[0023] A non-volatile storage device cold-hot data shunting management system comprises:

[0024] A user address space identification module is configured to uniquely identify a user address space in a storage device for receiving user data by a logical block address, wherein the user address space is a continuous logical address range, and the logical block address is used for a minimum data block in the address space.

[0025] A logical space division module is configured to divide the user address space into N logical spaces based on the continuous logical address range, wherein each logical space contains a preset number of continuous logical block addresses, and N is a positive integer.

[0026] An operation frequency acquisition module is configured to trigger acquisition of access behavior of a target logical space to which a logical block address belongs when the storage device receives an I / O operation for the user address space, and update operation frequency data of the target logical space by an exponential weighting algorithm.

[0027] An access order acquisition module is configured to maintain a historical bidirectional linked list in DRAM, adjust a position of a target logical space to which a logical block address belongs in the linked list to a head position when the target logical space is accessed by an I / O operation, and generate an Order array according to positions of nodes in the linked list, wherein a smaller value in the Order array indicates a newer access order of a corresponding logical space.

[0028] A heat value calculation module is configured to calculate a heat value of a target logical space according to operation frequency data of the target logical space and an Order array reflecting an operation order.

[0029] A dynamic threshold shunting module is configured to generate a dynamic threshold based on a hotness value of the target logical space, and shunt user data in the target logical space into hot data and cold data according to the dynamic threshold, and allocate the hot data to the low-wear flash memory block and the cold data to the high-wear flash memory block.

[0030] Compared with the prior art, the technical scheme provided by the application has the beneficial effects that the dynamic characteristics of data access are captured in real time through the operation frequency collection and access sequence collection, the historical frequency is attenuated by an exponential weighting algorithm to reflect that the recent operation weight is higher, the access sequence is quantified by an Order array, and the smaller the numerical value is, the newer the access is. The hotness value H[x] generated by the combination of the two can accurately reflect the current hotness of the data and avoid the rigid problem of the static threshold. The hotness value H[x] = a x T[x] + (1-a) x S[x], wherein T[x] is the frequency data, and S[x] is the sequence score, which combines the two-dimensional characteristics of frequency and time. For data with high frequency but not accessed recently, the sequence score S[x] is reduced to make the hotness value decrease; for data with low frequency but continuously accessed, the sequence score S[x] is improved to make the hotness value increase. The model improves the shunting accuracy of single-dimensional evaluation and reduces invalid data migration. The user address space is divided into N continuous logical spaces, each of which contains a preset number of LBAs, to realize block-level management of hot and cold data, and hot data is centrally allocated to the low-wear flash memory block and cold data is centrally allocated to the high-wear flash memory block. The history bidirectional linked list and the Order array are maintained by DRAM, and the time complexity of the linked list adjustment and the array update of a single I / O operation is O(1), which has little effect on the read-write performance of the storage device and can meet the demand of high real-time scenarios.

[0031] The application effectively solves the problems of inaccurate hot and cold data shunting and uneven wear in the prior art through dynamic and multi-dimensional hotness evaluation and fine logical space management, significantly prolongs the service life of the non-volatile storage device, and guarantees efficient read-write performance. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only show some embodiments of the application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0033] Among them:

[0034] Figure 1 It is a flow chart of the hot and cold data shunting management method of the non-volatile storage device in one embodiment;

[0035] Figure 2A structural block diagram of a non-volatile storage device cold and hot data shunting management system in an embodiment;

[0036] Figure 3 A structural block diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0037] In order for those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0038] Embodiment one

[0039] The technical problem solved by the present embodiment is how to improve the service life of non-volatile storage devices in special scenarios. For example, in the industrial control field, such as PLC and DCS systems, two types of data need to be stored simultaneously: real-time control instructions and historical operation logs. Real-time control instructions are defined as hot data, and historical operation logs are defined as cold data. Real-time control instructions have the characteristics of high-frequency reading and writing, with up to 100 times per second, but their life cycle is relatively short, only valid within the control period, and require low-latency access. Historical operation logs are rarely read after being written, and only once for statistical analysis at the end of a statistical period, but require long-term storage, such as more than 3 years, which requires a high storage capacity.

[0040] The existing cold and hot data shunting management method does not distinguish between cold and hot data when applied to the industrial control field, resulting in real-time instructions and log data being mixed in the same flash memory block. Due to the high-frequency erasing and writing of real-time instructions, the flash memory block wears out quickly, and the blocks where the log data is located are not updated for a long time, causing overcooling, reducing the overall storage life and making the storage logic unreasonable.

[0041] Similarly, when dealing with industrial control storage problems, the existing storage logic of flash memory blocks also does not dynamically adapt to the data access mode, does not quantify the heat of user data, and cannot reasonably divide the user address space to achieve fine-grained separation of hot and cold data and targeted efficient storage, thereby improving the storage life through balanced wear.

[0042] To solve the above technical problems, the present embodiment provides a non-volatile storage device cold and hot data shunting management method, as shown in Figure 1 The method comprises the following steps:

[0043] S101: uniquely identify a user address space for receiving user data in the storage device by a logical block address, the user address space being a continuous logical address range, and the logical block address corresponding to a minimum data block in the user address space;

[0044] Specifically, the user address space is a logical address region in the storage device for storing user data and does not include a metadata region and a system reserved region of the storage device, the logical block address uniquely identifies the user address space by a continuous integer, and the capacity of the minimum data block is 512 bytes, 1 KB, 2 KB or 4 KB.

[0045] Specifically, the step of dividing the user address space into N logical spaces includes: presetting the number M of continuous logical block addresses contained in each logical space, the value range of M being 128≤M≤4096, and M being an integer power of 2; the total number N of logical spaces = the total number of logical block addresses of the user address space / M, if not divisible, rounding up and adjusting the number of logical block addresses of the last logical space to the remaining value; in the order of logical block addresses from small to large, sequentially dividing into N continuous logical spaces.

[0046] For example, in the industrial control field, the system stores user data including real-time control instructions, including motor start-stop signals, sensor threshold parameters, a single instruction size being 512 bytes to 2 KB, historical operation logs, including device temperature records, fault code records, a single log size being 4 KB, written and only read at monthly maintenance, an annual access frequency being 10-12 times, and the storage device using a Nand Flash SSD with a capacity of 512 GB.

[0047] The total capacity of the storage device is 512 GB, the metadata region stores an FTL mapping table and bad block management information, occupying a space of 8 GB, the system reserved region stores a firmware program, occupying a space of 4 GB, and the remaining 500 GB is a user address space for storing real-time control instructions and historical logs. The logical block address (LBA) of the user address space is uniquely identified by a continuous integer, which is actually LBA=0, and the terminal LBA=total capacity of the user address space÷capacity of the minimum data block-1.

[0048] In the industrial control field, the capacity of the minimum data block is selected to be 4 KB, so the terminal LBA=500 GB=524288000 KB, the total number of LBAs=524288000 KB÷4 KB=131072000, and therefore the LBA range of the user address space is 0-131072000, a total of 131072000 continuous LBAs.

[0049] S102: Divide the user address space into N logical spaces based on continuous logical address ranges, each logical space containing a preset number of continuous logical block addresses, N being a positive integer;

[0050] Specifically, the step of dividing the user address space into N logical spaces includes: presetting the number of continuous logical block addresses M contained in each logical space, the value range of M being 128≤M≤4096, and M being an integer power of 2; the total number of logical spaces N = the total number of logical block addresses of the user address space / M, if not divisible, rounding up and adjusting the number of logical block addresses of the last logical space to the remaining value; in the order of logical block addresses from small to large, sequentially dividing into N continuous logical spaces.

[0051] Similarly, in the industrial control field, the access characteristics of industrial control data are high-frequency small batch range of real-time control instructions, which requires smaller logical spaces to reduce invalid data migration, and low-frequency large batch storage of historical logs, which requires larger logical spaces to reduce management overhead. In combination, M = 2048 LBA, 128≤M≤4096, 2048 = 2 11 Each logical space capacity = 2048 x 4KB = 8MB, since the total LBA number is 131072000, N = 131072000 ÷ 2048 = 64000, therefore, the user address space is divided into 64000 continuous logical spaces, each containing 2048 LBAs.

[0052] In the order of LBA from small to large, the number of logical spaces is 0-63999, specifically:

[0053] Logical space 0: LBA is 0-2047; logical space 1: LBA is 2048-4095...

[0054] The first 1000 logical spaces are allocated to real-time control data, with a total capacity of 8MB x 1000 = 8GB, and the last 63000 logical spaces are allocated to historical log data, with a total capacity of 504GB, which is a good match with the storage requirements of the two types of data.

[0055] S103: When the storage device receives an I / O operation for the user address space, trigger the collection of access behavior of the target logical space to which the logical block address belongs, and update the operation frequency data of the target logical space by an exponential weighting algorithm;

[0056] Specifically, the step of triggering the collection of access behavior of the target logical space includes: the I / O operation request includes read request and write request, and the collection of access behavior is triggered only when the target address of the I / O operation falls within the user address space.

[0057] Specifically, the step of updating the operation frequency data of the target logical space by the exponential weighting algorithm comprises: initializing the operation frequency data T[x] of the target logical space as 0; each time the I / O operation is triggered, the operation frequency data is updated by an exponential weighting formula: T[x]=β×T[x]+(1-β)×1,=, wherein β is a historical weight coefficient, representing a decay ratio of historical operation frequency, 0.5≤β<1, and (1-β) represents a weight of the current operation.

[0058] S104: maintaining a historical double-direction linked list in the DRAM, adjusting a position of a target logical space in the linked list as a head position each time an I / O operation accesses the target logical space to which a logical block address belongs, and generating an Order array according to positions of nodes in the linked list, wherein a smaller value in the Order array represents a newer access order of a corresponding logical space;

[0059] Specifically, the step of maintaining the historical double-direction linked list in the DRAM comprises: traversing the historical double-direction linked list to find a node corresponding to the target logical space; if the node exists, removing the node from a current position and inserting the node into a head position of the linked list; if the node does not exist, creating a new node and inserting the node into the head position of the linked list; the head node of the historical double-direction linked list is a recently accessed logical space, and the tail node is a longest unaccessed logical space.

[0060] Specifically, the step of generating the Order array comprises: initializing the Order array, each element of the Order array corresponding to a logical space; traversing the historical double-direction linked list in a sequence from the head to the tail, and assigning a sequence value to each logical space node, the head node having a sequence value of 0 and the tail node having a sequence value of n-1, wherein n is a total number of logical spaces; and writing the sequence value of each logical space into a position corresponding to an index of the logical space in the Order array, to obtain the Order array reflecting access orders of all the logical spaces.

[0061] S105: calculating a heat value of the target logical space according to the operation frequency data of the target logical space and the Order array reflecting the operation order;

[0062] Specifically, based on a sequence value S_order of the target logical space in the Order array, an access order score S[x]=1 / (S_order+1) is calculated according to a formula, wherein a value range of S[x] is (0, 1]; and a heat value H[x] is H[x]=α×T[x]+(1-α)×S[x], wherein α is a weight coefficient, 0<α<1, and T[x] is the operation frequency data of the target logical space.

[0063] For example, in the industrial control field, the I / O operation type and the triggering condition are as follows:

[0064] When the PLC system receives an I / O request, it first determines whether the target LBA falls within the user address space (0-131071999), for example, a real-time control instruction write request: target LBA = 1024, belonging to logical space 0, triggering collection; a system firmware update request: target LBA = 131072000, belonging to the system reserved area, not triggering collection. The I / O operation type, at the same time, collects read requests and write requests, and logical space 0 receives 2000 write requests per day for real-time instruction updates and 500 read requests for instruction readback verification, and logical space 50000 receives 1 write request per month for log archiving and 0 read requests.

[0065] Specifically, the exponential weighting algorithm, the operation frequency data of all logical spaces is 0 at initialization, and T[x] is updated for each I / O operation through T[x] = β × T[x] + (1-β) × 1, wherein the historical weight coefficient β is 0.8 according to the characteristics of short-term burst and long-term stability of industrial control data access, which means that the historical operation decay ratio is 80% and the current operation weight is 20%.

[0066] For example:

[0067] The time point is t0, the system is started, T[x] is initialized to 0, the real-time instruction of logical space 0 is T[0]=0, and the historical log of logical space 50000 is T

[50000] =0.

[0068] The time point is t1, logical space 0 receives 1 write request, T[0] of logical space 0 is 0.8 × 0 + (1-0.8) × 1 = 0.2, logical space 50000 has no operation, and T[0] remains 0.

[0069] The time point is t2, the logical space receives the second write request, T[0] of the logical space 0 is 0.8 × 0.2 + 0.2 × 1 = 0.36.

[0070] The time point is t3, logical space 50000 receives 1 write request, logical space 0 has no operation, and T

[50000] of logical space 50000 is 0.8 × 0 + (1-0.8) × 1 = 0.2.

[0071] When the calculation and derivation are performed by the embodiment, it can be seen from the above example that the real-time instruction logical space can quickly accumulate T[x] due to high-frequency operation, and the historical log logical space grows slowly due to low operation frequency.

[0072] Each node of the double linked list in the DRAM contains the number of logical space, the predecessor pointer and the successor pointer, the upper limit of the capacity of the linked list is 64000, which is consistent with the total number of logical spaces. After each I / O operation, the target logical space node is moved to the head of the linked list, indicating the most recent access. In the industrial control field, it is assumed that the following access events occur after the system is started:

[0073] t1, access logical space 0, the linked list is empty, a new node is inserted into the head, and the linked list is [0];

[0074] t2, access logical space 1, a new node is inserted into the head, and the linked list is [1, 0];

[0075] t3, access logical space 0 again, remove node 0, insert into the head, and the linked list is [0, 1];

[0076] t4, access logical space 50000, insert a new node into the head, and the linked list is [50000, 0, 1];

[0077] t5, access logical space 2, insert a new node into the head, and the linked list is [2, 50000, 0, 1]...

[0078] Continue to generate the Order array:

[0079] The Order array is an array with a length of 64000, and the initial value is -1, indicating that it has not been accessed.

[0080] Assign the order value, traverse from the head to the tail of the linked list, and assign the order value in turn. The head is 0, and the subsequent increments by 1. For example, at t5, access logical space 2, the linked list is [2, 50000, 0, 1], the Order[2] of the head node 2 is 0; the Order

[50000] of the node 50000 is 1; the Order[0] of the node 0 is 2; the Order[1] of the node 1 is 3; the Order[x] of the unvisited node is -1, waiting for the subsequent access to update. It can be seen that the smaller the Order[x], the more recent the access time of the logical space.

[0081] Continue to calculate the heat value H[x]:

[0082] According to the requirement of frequency stability in the industrial control scene, that is, the real-time instruction gives priority to continuous high-frequency access, and the weighting coefficient α is taken as 0.6, which means that the frequency weight is 60% and the order weight is 40%.

[0083] Based on the Order array of the access logical space 0 at t3 and the access logical space 50000 at t5, the heat value of the logical space is:

[0084] Logical space 0 is real-time instruction, high frequency recent access, T[x] is 0.36, i.e. 2 write requests are accumulated at t3; S_order is 2, Order[0]=2, third recent access, access order score S[x]=1 / (2+1)≈0.333; heat value H[x]=0.6×0.36+0.4×0.33≈0.349.

[0085] Logical space 50000 is historical log, low frequency recent access, T[x] is 0.2, i.e. 1 write request is accumulated at t3; S_order is 1, Order

[50000] =1, second recent access, S[x]=1 / (1+1)=0.5, heat value H[x]=0.6×0.2+0.4×0.5=0.32.

[0086] Logical space 1000 is not accessed, and the heat value H[x] is calculated as 0.

[0087] In summary, logical space 0 (H=0.349): real-time instruction has the highest heat value due to high frequency access, and will be determined as hot data subsequently; logical space 50000 (H=0.32): although recent access, the frequency is extremely low, and the heat value is second; logical space 1000 (H=0): no access record, and the heat value is the lowest, and is determined as cold data. It can be seen that the heat value of the present scheme can objectively express the cold and hot degree of data.

[0088] S106: generating a dynamic threshold value based on the heat value of the target logical space, and according to the dynamic threshold value, the user data in the target logical space is divided into hot data and cold data, the hot data is allocated to a low-wear flash memory block, and the cold data is allocated to a high-wear flash memory block.

[0089] Specifically, the heat values H[x] of all N logical spaces are collected, and are arranged in ascending order to obtain:

[0090] H_sorted=[h_0,h_1,h_2,...,h_{N-1}];

[0091] The dynamic threshold value Q=H_sorted[floor(γ×N)] is calculated, wherein γ is a quantile coefficient, 0.5<γ<1, and floor() is a floor function;

[0092] The logical space with the heat value H[x]>Q is determined as hot data and is allocated to a low-wear flash memory block with an erasing number ≤ a preset threshold value, and the logical space with the heat value H[x]≤Q is determined as cold data and is allocated to a high-wear flash memory block with an erasing number > the preset threshold value, and the preset threshold value is a positive integer configured according to the life characteristics of the flash memory block.

[0093] Continuing, the cold and hot data are divided according to the heat value:

[0094] The heat values are sorted according to the heat values calculated above, that is, 64,000 logical spaces are traversed, the current heat values H[x] thereof are collected, in the industrial control scenario, the heat values have the obvious cold-hot separation feature, the hot data candidate area is 0-999 logical spaces, due to the high-frequency real-time access of data, H[x] is concentrated in 0.3-0.5; the cold data candidate area is 1000-63999 logical spaces, due to the low-frequency archiving access, H[x] is concentrated in 0-0.2, and the H[x] of the unvisited logical space is 0.

[0095] All H[x] are arranged in ascending order to obtain H_sorted=[h_0, h_1, h_2,..., h_63999], wherein h_0 is an unvisited logical space, and h_63999=0.5 is the highest-frequency real-time instruction space.

[0096] In the industrial control scenario, it is necessary to ensure that 99% of the real-time instruction spaces are determined as hot data, therefore, the quantile coefficient γ is taken as 0.99, at this time, the target part is the position floor(γ×N)=floor(0.99×64000)=63360; the dynamic threshold Q is calculated as H_sorted[floor(γ×N)]=H_sorted

[63360] , since the first 63000 values of H_sorted are concentrated in 0-0.2, the 63360th value is in the last 1000 values of H_sorted, and corresponds to the lower part of the heat value in the hot data candidate area, and Q=0.3 is selected through testing.

[0097] When the cold and hot data are shunted, if the H[x] of the logical space is greater than 0.3, the logical space is allocated to the low-wear block, and if the H[x] of the logical space is less than or equal to 0.3, the logical space is allocated to the high-wear block.

[0098] Further, the threshold Q needs to be dynamically adjusted, the update period of the threshold Q is to recalculate H_sorted and update the Q value every 24 hours. The state of the block wear can also be refreshed, for example, the number of erasures of each block is re-counted after every 1000 erasures, to ensure the real-time accuracy of the high-low wear block division.

[0099] Through the above steps, the user data in the industrial control scenario is realized. The user data is strictly separated from the metadata and system data, avoiding interference of metadata update on real-time instruction read and write; the 8MB logical space meets the small-batch high-frequency access of real-time instructions and avoids the management overhead of log data.

[0100] Although the Order value of logical space 0 is greater than that of logical space 50000, the final heat value surpasses it because T[x] is higher, reflecting that high-frequency continuous access takes precedence over single recent access; the heat value of the unaccessed log area logical space is 0, achieving a significant distinction from hot data; the exponential weighting algorithm and linked list operation are both O(1) complexity. On the PLC system processor, the calculation time of a single heat value is very short, which is sufficient to meet the real-time requirements. In the embodiment, the parameter selection of β=0.8 and α=0.6, as well as the dynamic update mechanism of the Order array, are designed for the mixed load characteristics of high-frequency real-time data and low-frequency archived data in industrial control, ensuring that the heat evaluation results are highly matched with actual business needs.

[0101] Hot data is centrally written to low-wear blocks, while cold data utilizes the remaining lifespan of high-wear blocks, preventing rapid aging of individual blocks. Low-wear blocks, due to fewer erase / write cycles, have faster read / write response speeds than high-wear blocks, meeting the stringent real-time requirements of industrial control. In this embodiment, only 400 low-wear blocks are used for the 640 hot data logic spaces, with the remaining low-wear blocks reserved for backup, avoiding resource waste. By linking dynamic thresholds with wear status, precise separation of hot and cold data in industrial control scenarios is achieved, ensuring low-latency access to real-time commands while maximizing the utilization of flash memory block lifespan.

[0102] Example 2

[0103] This embodiment provides a hot and cold data splitting management system for non-volatile storage devices, such as... Figure 2 As shown, it includes:

[0104] The user address space identification module 100 is used to uniquely identify the user address space in the storage device used to receive user data by means of a logical block address. The user address space is a continuous range of logical addresses, and the logical block address corresponds to the smallest data block in the user address space.

[0105] The logical space partitioning module 200 is used to divide the user address space into N logical spaces based on a continuous logical address range. Each logical space contains a preset number of continuous logical block addresses, where N is a positive integer.

[0106] The operation frequency acquisition module 300 is used to trigger the acquisition of access behavior of the target logical space to which the logical block address belongs when the storage device receives an I / O operation for the user address space, and update the operation frequency data of the target logical space through an exponential weighting algorithm.

[0107] The access order acquisition module 400 is used to maintain a historical doubly linked list in DRAM. When each I / O operation accesses the target logical space to which the logical block address belongs, the module adjusts the position of the target logical space in the linked list to the head position. The module generates an Order array based on the position of each node in the linked list. The smaller the value in the Order array, the newer the access order of the corresponding logical space.

[0108] The heat value calculation module 500 is used to calculate the heat value of the target logical space based on the operation frequency data of the target logical space and the Order array that reflects the operation order;

[0109] The dynamic threshold splitting module 600 is used to generate a dynamic threshold based on the heat value of the target logical space. According to the dynamic threshold, the user data in the target logical space is split into hot data and cold data. The hot data is allocated to the low-wear flash memory block, and the cold data is allocated to the high-wear flash memory block.

[0110] Example 3

[0111] Figure 3 An internal structural diagram of a computer device in one embodiment is shown. This computer device can specifically be a terminal or a server. Figure 3 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and may also store a computer program. When executed by the processor, this computer program enables the processor to implement a hot and cold data splitting management method. The memory may also store a computer program, which, when executed by the processor, enables the processor to implement the hot and cold data splitting management method. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0112] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the following steps:

[0113] A method for managing hot and cold data splitting in a non-volatile storage device includes the following steps:

[0114] The user address space in the storage device used to receive user data is uniquely identified by logical block addresses. The user address space is a continuous range of logical addresses, and the logical block address is applied to the smallest data block within the address space.

[0115] based on the continuous logical address range, the user address space is divided into N logical spaces, each logical space containing a preset number of continuous logical block addresses, and N is a positive integer;

[0116] When the storage device receives an I / O operation for the user address space, the access behavior collection of the target logical space to which the logical block address belongs is triggered, and the operation frequency data of the target logical space is updated by an exponential weighting algorithm;

[0117] A historical bidirectional linked list is maintained in the DRAM, and the position of the target logical space in the linked list is adjusted to the head position each time the target logical space to which the logical block address belongs is accessed by an I / O operation, and an Order array is generated according to the positions of the nodes in the linked list, wherein the smaller the value in the Order array is, the newer the access order of the corresponding logical space is;

[0118] The heat value of the target logical space is calculated according to the operation frequency data of the target logical space and the Order array reflecting the operation order;

[0119] A dynamic threshold is generated based on the heat value of the target logical space, and the user data in the target logical space is shunted into hot data and cold data according to the dynamic threshold, the hot data is allocated to a low-wear flash block, and the cold data is allocated to a high-wear flash block.

[0120] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to make the processor execute the following steps:

[0121] A non-volatile storage device hot and cold data shunting management method, comprising the steps of:

[0122] The user address space in the storage device for receiving user data is uniquely identified by a logical block address, and the user address space is a continuous logical address range, and the logical block address is applied to the smallest data block in the address space;

[0123] based on the continuous logical address range, the user address space is divided into N logical spaces, each logical space containing a preset number of continuous logical block addresses, and N is a positive integer;

[0124] When the storage device receives an I / O operation for the user address space, the access behavior collection of the target logical space to which the logical block address belongs is triggered, and the operation frequency data of the target logical space is updated by an exponential weighting algorithm;

[0125] A history bidirectional linked list is maintained in the DRAM, and a position of a target logical space to which a logical block address belongs in the linked list is adjusted to a head position each time an I / O operation accesses the target logical space, and an Order array is generated according to positions of nodes in the linked list, wherein a smaller value in the Order array indicates a newer access order of a corresponding logical space;

[0126] A hotness value of the target logical space is calculated according to operation frequency data of the target logical space and the Order array reflecting the operation order;

[0127] A dynamic threshold is generated based on the hotness value of the target logical space, and user data in the target logical space is shunted into hot data and cold data according to the dynamic threshold, the hot data is allocated to a low-wear flash block, and the cold data is allocated to a high-wear flash block.

[0128] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the program can be stored in a non-volatile computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0129] Any combination of the technical features of the above embodiments can be made, and in order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0130] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the patent of the present application. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for managing hot and cold data splitting in a non-volatile storage device, characterized in that, The method comprises the steps of: a user address space in a storage device for receiving user data is uniquely identified by a logical block address, the user address space is a continuous logical address range, and the logical block address is applied to a minimum data block in the address space; based on the continuous logical address range, the user address space is divided into N logical spaces, each of which contains a preset number of continuous logical block addresses, and N is a positive integer; when the storage device receives an I / O operation for the user address space, access behavior collection of a target logical space to which the logical block address belongs is triggered, and the operation frequency data T[x] of the target logical space is initialized to 0; each time the I / O operation is triggered, the operation frequency data is updated by an exponential weighting formula: T[x] = β × T[x] + (1-β) × 1; wherein β is a historical weight coefficient, representing the decay proportion of historical operation frequency, 0.5 ≤ β < 1, and (1-β) represents the weight of the current operation; a historical double-linked list is maintained in the DRAM, and each time the I / O operation accesses the target logical space to which the logical block address belongs, the position of the target logical space in the linked list is adjusted to the head position, and an Order array is generated according to the positions of the nodes in the linked list, wherein the smaller the value in the Order array is, the newer the access order of the corresponding logical space is; wherein the Order array generation step comprises: initializing the Order array, each element of the Order array corresponding to a logical space; traversing the historical double-linked list in the order from the head to the tail, and assigning each logical space node with an order value, the head node having an order value of 0, and then the order value is sequentially increased by 1; the order value of each logical space is written into the position corresponding to the index of the logical space in the Order array, and the Order array reflecting the access order of all logical spaces is obtained; calculating the hotness value of the target logical space according to the operation frequency data of the target logical space and the Order array reflecting the operation order, wherein based on the order value S_order of the target logical space in the Order array, the access order score S[x] = 1 / (S_order+1) is calculated according to the formula, wherein the value range of S[x] is (0, 1]; the hotness value H[x] is H[x] = α × T[x] + (1-α) × S[x], wherein α is a weight coefficient, 0 < α < 1, and T[x] is the operation frequency data of the target logical space; generating a dynamic threshold value based on the hotness value of the target logical space, and according to the dynamic threshold value, the user data in the target logical space is divided into hot data and cold data, the hot data is allocated to a low-wear flash block, and the cold data is allocated to a high-wear flash block.

2. The method according to claim 1, wherein The user address space is a logical address region in the storage device for storing user data and does not include the metadata region and the system reserved region of the storage device, the logical block address uniquely identifies the user address space by a continuous integer, and the capacity of the minimum data block is 512 bytes, 1 KB, 2 KB or 4 KB.

3. The method according to claim 1, wherein The step of dividing the user address space into N logical spaces comprises: A preset number of continuous logical block addresses M contained in each logical space, the value range of M is 128≤M≤4096, and M is an integer power of 2; The total number of logical spaces N = the total number of logical block addresses of the user address space / M, if not divisible, rounding up and adjusting the number of logical block addresses of the last logical space to the remaining value; In the order of logical block addresses from small to large, N continuous logical spaces are divided in turn.

4. The method according to claim 1, wherein The step of triggering the collection of the access behavior of the target logical space includes: The I / O operation request includes a read request and a write request, and the collection of the access behavior is triggered only when the target address of the I / O operation falls within the user address space.

5. The method according to claim 1, wherein The step of maintaining the historical double-linked list in the DRAM includes: Traverse the historical double-linked list to find the node corresponding to the target logical space; If the node exists, remove the node from the current position and insert the node into the head position of the linked list; If the node does not exist, create a new node and insert the node into the head position of the linked list; The head node of the historical double-linked list is the most recently accessed logical space, and the tail node is the least recently accessed logical space.

6. The method according to claim 1, wherein The step of shunting user data according to the dynamic threshold value includes: Collect the heat values H[x] of all N logical spaces, and arrange them in ascending order to obtain H_sorted=[h_0, h_1, h_2, …, h_{N-1}]; Calculate the dynamic threshold value Q=H_sorted[floor(γ×N)], where γ is a quantile coefficient, 0.5<γ<1, and floor() is a floor function; Determine the logical space with a heat value H[x]>Q as hot data and assign it to a low-wear flash memory block with an erase count ≤ a preset threshold value, and determine the logical space with a heat value H[x]≤Q as cold data and assign it to a high-wear flash memory block with an erase count > a preset threshold value, the preset threshold value being a positive integer configured according to the lifetime characteristics of the flash memory block.

7. A nonvolatile storage apparatus cold-hot data shunting management system, characterized by, The system for applying the hot and cold data shunting management method of the non-volatile storage device of any one of claims 1 to 6 includes: A user address space identification module for uniquely identifying a user address space in a storage device for receiving user data by a logical block address, the user address space being a continuous logical address range, and the logical block address being applied to the smallest data block in the address space; A logical space division module for dividing the user address space into N logical spaces based on the continuous logical address range, each logical space containing a preset number of continuous logical block addresses, and N being a positive integer; An operation frequency collection module for triggering the collection of the access behavior of a target logical space to which the logical block address belongs when the storage device receives an I / O operation for the user address space, and updating the operation frequency data of the target logical space by an exponential weighting algorithm; The access order collection module is configured to maintain a history bidirectional linked list in the DRAM, adjust a position of a target logical space to which a logical block address belongs in the linked list to a head position each time an I / O operation accesses the target logical space, and generate an Order array according to positions of nodes in the linked list, wherein a smaller value in the Order array indicates a newer access order of a corresponding logical space. The hotness value calculation module is configured to calculate a hotness value of the target logical space according to operation frequency data of the target logical space and the Order array reflecting the operation order. The dynamic threshold shunting module is configured to generate a dynamic threshold based on the hotness value of the target logical space, shunt user data in the target logical space into hot data and cold data according to the dynamic threshold, and allocate the hot data to the low-wear flash memory block and the cold data to the high-wear flash memory block.

Images