AI-driven adaptive storage tiering and cache prefetching system

Abstract

The present invention discloses an AI-driven adaptive storage tiering and cache prefetching system, which relates to the field of data storage technology, including: dividing storage space based on historical storage data and obtaining occupancy evaluation criteria; performing multi-layer storage processing on data stored in the storage system; performing cache prefetching processing based on occupancy evaluation criteria; and updating the allocation of storage space in the storage system in real time based on artificial intelligence. The present invention is used to solve the problems in existing storage tiering and cache prefetching methods, in which the storage space in the storage system is not effectively divided, resulting in the inability to adaptively adjust the cache storage space, and the inability to reasonably divide the location of stored data, resulting in slow efficiency and abnormal calls during data search and data call.

Description

Technical Field

[0001] The present invention relates to the field of data storage technology, and in particular to an AI-driven adaptive storage tiering and cache prefetching system. Background Art

[0002] Storage tiering refers to storing data in different levels of storage media and automatically or manually migrating and replicating data between these levels to achieve a balance between performance and cost. Cache prefetching refers to fetching data blocks into the cache before they are actually used to avoid processor stalls caused by cache failures. The role of storage tiering and cache prefetching in storage systems is primarily reflected in improving data access speed, reducing access latency, and optimizing storage resource utilization.

[0003] Existing methods for storage tiering and cache prefetching generally obtain an instruction group address and verify the instruction group address requested for access in a preset cache, thereby transferring the content requested for access to the preset cache and updating the content name in the preset cache to prevent repeated reading due to instruction loops. Although this improved method can improve the reading efficiency of continuous instructions and loop instructions, it does not effectively divide the storage space in the storage system, resulting in the inability to adaptively adjust the storage space of the cache and the inability to reasonably divide the location of stored data, resulting in slow efficiency and abnormal calls during data search and data call. For example, in the patent application with publication number CN110442382A, a prefetch cache control method is disclosed. , devices, chips and computer-readable storage media. This solution is to update the second content in the prefetch cache to the record cache, and update the second content in the prefetch cache to the first content, thereby avoiding the need to re-read due to the overwriting of loop instructions, and improving the reading efficiency of continuous instructions and loop instructions. Other improvements for storage tiering and cache prefetching are usually improvements in the establishment of cache memory, which still do not effectively divide the storage space in the storage system, resulting in the inability to adaptively adjust the cache storage space, and the inability to reasonably divide the location of stored data, resulting in slow efficiency and call exceptions during data search and data call. In view of this, it is necessary to improve the existing storage tiering and cache prefetching methods. Summary of the Invention

[0004] The present invention aims to solve, at least to a certain extent, one of the technical problems in the prior art. By proposing an AI-driven adaptive storage tiering and cache prefetching system, it is used to solve the problem that the storage space in the storage system is not effectively divided in the existing storage tiering and cache prefetching methods, resulting in the inability to adaptively adjust the storage space of the cache, and the inability to reasonably divide the locations where data is stored, resulting in slow efficiency and abnormal calls during data search and data calls.

[0005] To achieve the above objectives, this application provides an AI-driven adaptive storage tiering and cache prefetching system, including a storage system storage analysis module, a storage tiering and cache setting module, and an intelligent optimization module;

[0006] The storage system storage analysis module is used to obtain historical storage data and historical call data of the storage system, analyze the storage space of the storage system based on the historical storage data, and divide the storage space into buffer space, fluctuating storage space and stable storage space based on the analysis results, and obtain occupancy evaluation criteria for the buffer space and stable storage space, where the occupancy evaluation criteria are the standards for the size of the occupied storage space;

[0007] The storage tiering and cache setting module is used to perform multi-layer storage processing on the data stored in the storage system based on historical storage data and historical call data; and to perform cache pre-fetch processing on the data retrieved from the storage system based on the buffer space and the buffer space occupancy evaluation criteria;

[0008] The intelligent optimization module is used to update the allocation of storage space in the storage system in real time based on artificial intelligence, and use the real-time updated storage system to perform multi-layer storage processing and cache pre-fetching processing on the data stored in the storage system and the data retrieved from the storage system.

[0009] Furthermore, the storage system storage analysis module includes a storage analysis unit, and the storage analysis unit is configured with a storage analysis strategy, which includes:

[0010] Obtain the historical storage data and historical call data of the storage system; record the time corresponding to the earliest data recorded in the historical storage data as the earliest record time; establish a plane rectangular coordinate system, recorded as the storage analysis coordinate system, where the unit of the X axis of the storage space coordinate system is h and the unit of the Y axis is MB;

[0011] Obtain the change data of the storage space of the storage system from the earliest recorded time to the current time based on the historical storage data, draw the corresponding curve between X=0 and X=X1 on the X-axis, and record it as the space change curve, where X1 is the time difference between the earliest recorded time and the current time. The point with the horizontal coordinate of 0 in the space change curve is the storage space corresponding to the earliest recorded time, and the point with the horizontal coordinate of X1 is the storage space corresponding to the current time.

[0012] Furthermore, the storage analysis strategy also includes:

[0013] Based on the historical storage data, the time corresponding to when data is stored in the storage system and when data is retrieved from the storage system is obtained and recorded as the storage change time; in the spatial change curve, all points whose horizontal axes are storage change times are recorded as storage change points, and the space occupied by the data stored in the storage system or the data retrieved from the storage system corresponding to the storage change points is recorded as the storage occupancy value;

[0014] All peaks and all troughs in the spatial variation curve are obtained, and are recorded as spatial variation peaks and spatial variation troughs respectively; the curves between adjacent spatial variation peaks and spatial variation troughs in the spatial variation curve are recorded as monotonic subcurves, and the points with the largest slope in the monotonic subcurves are recorded as storage variation poles; all monotonic subcurves and all storage variation poles in the spatial variation curve are obtained.

[0015] Furthermore, the storage analysis strategy also includes:

[0016] The storage change extreme point with the smallest vertical coordinate is recorded as the stable peak point, and the vertical coordinate of the stable peak point is marked as the stable space threshold; the size of the stable storage space is set as the stable space threshold, and the occupancy evaluation standard of the stable storage space is set to be greater than or equal to the stable space threshold;

[0017] The storage change point with the largest vertical coordinate is recorded as the fluctuation peak point. When there is a storage change point with a vertical coordinate greater than the fluctuation peak point, the maximum value of all storage occupancy values ​​corresponding to the storage change points with a vertical coordinate greater than the fluctuation peak point is recorded as the minimum buffer value; when there is no storage change point with a vertical coordinate greater than the fluctuation peak point, the size of the storage space of the storage system minus the value of the vertical coordinate of the fluctuation peak point is recorded as the minimum buffer value.

[0018] Furthermore, the storage analysis strategy also includes:

[0019] The value obtained by subtracting the vertical coordinate of the stable peak point from the vertical coordinate of the fluctuation peak point is recorded as the fluctuation storage value, and the size of the fluctuation storage space is set to the fluctuation storage value; the value obtained by subtracting the fluctuation storage value from the size of the storage space of the storage system and then subtracting the stable space threshold is recorded as the size of the buffer space, and the buffer space occupancy judgment standard is set to be greater than or equal to the minimum buffer value.

[0020] Furthermore, the storage analysis strategy also includes:

[0021] When the storage space required by the stable storage space is greater than the stable space threshold, the buffer space allocates space to the stable storage space under the buffer space occupancy judgment standard; when the storage space required by the buffer space is greater than the minimum buffer value, the stable storage space allocates space to the buffer space under the stable storage space occupancy judgment standard.

[0022] Furthermore, the storage tiering and cache setting module includes a storage tiering setting unit, which is configured with a storage tiering setting strategy. The storage tiering setting strategy includes:

[0023] Based on historical storage data and historical call data, the maximum time of temporary storage in the storage system is obtained and recorded as the maximum access time;

[0024] When any data A is stored in the storage system, data A is processed by multi-layer storage;

[0025] The multi-layer storage process is as follows: data A is stored in the buffer space first; when the remaining space in the fluctuating storage space is larger than the space occupied by data A, data A is transferred to the fluctuating storage space;

[0026] When the remaining space of the fluctuating storage space is less than or equal to the space occupied by data A, the data with the longest fluctuating occupancy time in the fluctuating storage space is transferred to the stable storage space in sequence, until the remaining space of the fluctuating storage space is greater than the space occupied by data A, then data A is transferred to the fluctuating storage space;

[0027] The storage time of data A in the fluctuation storage space is recorded in real time and recorded as the fluctuation occupancy time;

[0028] When the fluctuation occupancy time of data A is greater than the maximum access time, data A is transferred to the stable storage space.

[0029] Furthermore, the storage tiering and cache setting module further includes a cache prefetch unit, which is configured with a cache prefetch strategy. The cache prefetch strategy includes:

[0030] When any data A in the storage system is retrieved, cache prefetching is performed based on data A;

[0031] The cache prefetch process includes: recording the size of the space occupied by data A as G, and when the size of the unoccupied space in the buffer space is greater than G, transferring data A to the buffer space;

[0032] When the size of the unoccupied space in the buffer space is less than or equal to G and the sum of the size of the unoccupied space in the buffer space and the size of the unoccupied space in the stable storage space is greater than G, the stable storage space allocates space to the buffer space according to the occupancy evaluation criteria of the stable storage space until the size of the unoccupied space in the buffer space is greater than G, and the data A is transferred to the buffer space.

[0033] Furthermore, the cache prefetching strategy also includes:

[0034] When the size of the unoccupied space in the buffer space is less than or equal to G and the sum of the size of the unoccupied space in the buffer space and the size of the unoccupied space in the stable storage space is less than or equal to G, the stable storage space allocates space to the buffer space under the stable storage space occupancy evaluation criteria until the size of the stable storage space reaches the stable space threshold. The size of the unoccupied space in the buffer space at this time is recorded as F, and the data occupying the space F in data A is transferred to the buffer space.

[0035] Furthermore, the intelligent optimization module includes an intelligent optimization unit, and the intelligent optimization unit is configured with an intelligent optimization strategy, which includes:

[0036] Based on artificial intelligence, the system acquires the latest historical storage data and historical call data of the storage system in real time, and updates the space occupied by the buffer space, fluctuating storage space, and stable storage space, as well as the evaluation criteria for the buffer space and stable storage space in real time;

[0037] Based on the latest buffer space, fluctuating storage space and stable storage space, multi-layer storage processing and cache pre-fetching processing are performed on the data stored in the storage system and the data retrieved from the storage system.

[0038] Beneficial effects of the present invention: The present application first obtains historical storage data and historical call data of a storage system, analyzes the storage space of the storage system based on the historical storage data, and divides the storage space into a buffer space, a fluctuating storage space, and a stable storage space based on the analysis results, and obtains occupancy evaluation criteria for the buffer space and the stable storage space. The advantage of this is that, by dividing the storage space and obtaining the occupancy evaluation criteria for the buffer space and the stable storage space, the data stored in the storage space can be effectively divided, so that more efficient data acquisition can be achieved through the divided storage space when searching for data. In addition, the space allocation of the buffer space and the stable storage space can be flexibly changed using the occupancy evaluation criteria, so that the cached storage space can be adaptively adjusted when caching data, so as to prevent the problem of call abnormality caused by insufficient cache space when calling data;

[0039] The present application also performs multi-layer storage processing on the data stored in the storage system based on historical storage data and historical call data; performs cache pre-fetching processing on the data retrieved from the storage system based on the buffer space and the buffer space occupancy evaluation criteria; finally, based on artificial intelligence, the allocation of storage space in the storage system is updated in real time, and the real-time updated storage system is used to perform multi-layer storage processing and cache pre-fetching processing on the data stored in the storage system and the data retrieved from the storage system. Through multi-layer storage processing and cache pre-fetching processing, more efficient data storage and smoother cache pre-fetching can be achieved based on the layered storage space. At the same time, by using artificial intelligence to update data in real time, the timeliness of space allocation in the storage space can be improved while saving manpower, so as to ensure that data storage and cache pre-fetching can be performed in the optimal storage state when the storage space is accessed in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a functional block diagram of the system of the present invention;

[0041] Figure 2 A schematic diagram of the storage analysis coordinate system of the present invention;

[0042] Figure 3 Schematic diagram of the storage space of the storage system of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] See also Figure 1 As shown, the present application provides an AI-driven adaptive storage tiering and cache prefetching system, including a storage system storage analysis module, a storage tiering and cache setting module, and an intelligent optimization module;

[0045] The storage system storage analysis module is used to obtain historical storage data and historical call data of the storage system, analyze the storage space of the storage system based on the historical storage data, and divide the storage space into buffer space, fluctuating storage space and stable storage space based on the analysis results, and obtain occupancy evaluation criteria for the buffer space and stable storage space, where the occupancy evaluation criteria are the standards for the size of the occupied storage space;

[0046] The storage system storage analysis module includes a storage analysis unit, which is configured with a storage analysis policy. The storage analysis policy includes:

[0047] Obtain the historical storage data and historical call data of the storage system; record the time corresponding to the earliest data recorded in the historical storage data as the earliest record time; establish a plane rectangular coordinate system, recorded as the storage analysis coordinate system, where the unit of the X axis of the storage space coordinate system is h and the unit of the Y axis is MB;

[0048] Obtain the change data of the storage space of the storage system from the earliest recorded time to the current time based on the historical storage data, draw the corresponding curve between X=0 and X=X1 on the X-axis, and record it as the space change curve, where X1 is the time difference between the earliest recorded time and the current time. The point with the horizontal coordinate of 0 in the space change curve is the storage space corresponding to the earliest recorded time, and the point with the horizontal coordinate of X1 is the storage space corresponding to the current time;

[0049] In the specific implementation process, for example, during a data analysis, the storage analysis coordinate system obtained is as follows Figure 2 As shown, curve KB is the spatial variation curve, points CB1 to CB4 are storage variation points, points BF1 to BF3 are storage variation peaks, points BG1 and BG2 are storage variation troughs, and points CJ1 to CJ4 are storage variation extremes. Analysis shows that the curve between points BF1 and BG1 is a monotonic subcurve, point CJ1 is a stable peak point, and CJ3 is a fluctuation peak point. The stable space threshold is 2000MB. At the same time, there is a storage variation point whose ordinate is greater than the fluctuation peak point. Therefore, the maximum value of all storage occupancy values ​​corresponding to points CB1, CB2, and CB4 can be recorded as the minimum buffer value.

[0050] Based on the historical storage data, the time corresponding to when data is stored in the storage system and when data is retrieved from the storage system is obtained and recorded as the storage change time; in the spatial change curve, all points whose horizontal axes are storage change times are recorded as storage change points, and the space occupied by the data stored in the storage system or the data retrieved from the storage system corresponding to the storage change points is recorded as the storage occupancy value;

[0051] Obtain all peaks and all troughs in the spatial variation curve, and record them as spatial variation peaks and spatial variation troughs respectively; record the curve between adjacent spatial variation peaks and spatial variation troughs in the spatial variation curve as a monotonic subcurve, and record the point with the largest slope in the monotonic subcurve as a storage variation pole; obtain all monotonic subcurves and all storage variation poles in the spatial variation curve;

[0052] The storage change extreme point with the smallest vertical coordinate is recorded as the stable peak point, and the vertical coordinate of the stable peak point is marked as the stable space threshold; the size of the stable storage space is set as the stable space threshold, and the occupancy evaluation standard of the stable storage space is set to be greater than or equal to the stable space threshold;

[0053] The storage change point with the largest vertical coordinate is recorded as the fluctuation peak point. When there is a storage change point with a vertical coordinate greater than the fluctuation peak point, the maximum value of all storage occupancy values ​​corresponding to the storage change points with a vertical coordinate greater than the fluctuation peak point is recorded as the minimum buffer value. When there is no storage change point with a vertical coordinate greater than the fluctuation peak point, the value of the storage system's storage space minus the vertical coordinate of the fluctuation peak point is recorded as the minimum buffer value.

[0054] The value obtained by subtracting the vertical coordinate of the stable peak point from the vertical coordinate of the fluctuation peak point is recorded as the fluctuation storage value, and the size of the fluctuation storage space is set as the fluctuation storage value; the value obtained by subtracting the fluctuation storage value from the storage space of the storage system and then subtracting the stable space threshold is recorded as the buffer space size, and the buffer space occupancy evaluation standard is set to be greater than or equal to the minimum buffer value;

[0055] In the specific implementation process, for example, during a data analysis, the storage space size is 20000MB, the fluctuating storage value is 2000MB, the stable space threshold is 4000MB, and the minimum buffer value is 2000MB. The buffer space size is 4000MB. Based on the above, the storage space distribution of the storage system is as follows: Figure 3 As shown, CC is the storage space of the storage system, area Q1 and area Q2 are areas occupied by the buffer space, and their sizes are 2000MB and 2000MB respectively, and the size corresponding to area Q1 is the minimum buffer value; area Q3 is the area occupied by the fluctuating storage space, and its size is 2000MB; area Q4 is the stable storage space, and its size is 4000MB. When the storage space required by the stable storage space is greater than 4000MB, the space in area Q2 can be allocated to the stable storage space, thereby flexibly changing the space allocation of the buffer space and the stable storage space, and being able to realize adaptive adjustment of the storage space required for each during data caching or data storage, so as to prevent call exceptions due to insufficient cache space during data call and storage failures due to insufficient storage space during data storage;

[0056] When the storage space required by the stable storage space is greater than the stable space threshold, the buffer space allocates space to the stable storage space under the buffer space occupancy judgment standard; when the storage space required by the buffer space is greater than the minimum buffer value, the stable storage space allocates space to the buffer space under the stable storage space occupancy judgment standard.

[0057] The storage tiering and cache setting module is used to perform multi-layer storage processing on the data stored in the storage system based on historical storage data and historical call data; and to perform cache pre-fetch processing on the data retrieved from the storage system based on the buffer space and the buffer space occupancy evaluation criteria;

[0058] The storage tiering and cache setting module includes a storage tiering setting unit, which is configured with a storage tiering setting strategy. The storage tiering setting strategy includes:

[0059] Based on historical storage data and historical call data, the maximum time of temporary storage in the storage system is obtained and recorded as the maximum access time;

[0060] When any data A is stored in the storage system, data A is processed by multi-layer storage;

[0061] The multi-layer storage process is as follows: data A is stored in the buffer space first; when the remaining space in the fluctuating storage space is larger than the space occupied by data A, data A is transferred to the fluctuating storage space;

[0062] When the remaining space of the fluctuating storage space is less than or equal to the space occupied by data A, the data with the longest fluctuating occupancy time in the fluctuating storage space is transferred to the stable storage space in sequence, until the remaining space of the fluctuating storage space is greater than the space occupied by data A, then data A is transferred to the fluctuating storage space;

[0063] In the specific implementation process, the specific implementation of "sequentially transferring the data with the longest fluctuation occupancy time in the fluctuation storage space to the stable storage space" is as follows: for example, during a data analysis, the spaces occupied by two data with longer fluctuation occupancy time in the fluctuation storage space are 1000MB and 800MB respectively, and are named data B1 and data B2 respectively, wherein the fluctuation occupancy time of data B1 is longer than that of data B2; when the remaining space of the fluctuation storage space is 500MB and the space occupied by data A is 1700MB, the data B1 with the longest fluctuation occupancy time in the fluctuation storage space can be transferred to the stable storage space first, and at this time the remaining space of the fluctuation storage space is 1500MB, which is still less than 1700MB, then the data B2 with the longest fluctuation occupancy time in the fluctuation storage space can be transferred to the stable storage space to realize the storage of data A;

[0064] The storage time of data A in the fluctuation storage space is recorded in real time and recorded as the fluctuation occupancy time;

[0065] When the fluctuation occupancy time of data A is greater than the maximum access time, data A is transferred to the stable storage space.

[0066] The storage tiering and cache setting module further includes a cache prefetch unit, which is configured with a cache prefetch strategy. The cache prefetch strategy includes:

[0067] When any data A in the storage system is retrieved, cache prefetching is performed based on data A;

[0068] The cache prefetch process includes: recording the size of the space occupied by data A as G, and when the size of the unoccupied space in the buffer space is greater than G, transferring data A to the buffer space;

[0069] When the size of the unoccupied space in the buffer space is less than or equal to G and the sum of the size of the unoccupied space in the buffer space and the size of the unoccupied space in the stable storage space is greater than G, the stable storage space allocates space to the buffer space according to the occupancy evaluation criteria of the stable storage space until the size of the unoccupied space in the buffer space is greater than G, and then the data A is transferred to the buffer space;

[0070] In a specific implementation process, for example, the size of the unoccupied space in the buffer space is 200MB, and G is 500MB. At this time, data A cannot be transferred to the buffer space for cache prefetching. At this time, the size of the unoccupied space in the stable storage space is 200MB, which means that even if all the unoccupied space in the stable storage space is transferred to the buffer space, cache prefetching of data A cannot be achieved. In this case, only a portion of data A can be cache prefetched. When the stable storage space allocates space to the buffer space under the stable storage space occupancy evaluation standard until the size of the stable storage space reaches the stable space threshold, the size of the unoccupied space in the buffer space is 344MB. In this case, the data A occupying 344MB can be transferred to the buffer space to achieve maximum cache prefetching of data A.

[0071] The cache prefetch strategy also includes: when the size of the unoccupied space in the buffer space is less than or equal to G and the sum of the size of the unoccupied space in the buffer space and the size of the unoccupied space in the stable storage space is less than or equal to G, the stable storage space allocates space to the buffer space under the stable storage space occupancy evaluation criteria until the size of the stable storage space reaches the stable space threshold, and the size of the unoccupied space in the buffer space at this time is recorded as F, and the data occupying the space F in data A is transferred to the buffer space.

[0072] The intelligent optimization module is used to update the allocation of storage space in the storage system in real time based on artificial intelligence, and use the updated storage system to perform multi-layer storage processing and cache pre-fetching on data stored in and retrieved from the storage system. The intelligent optimization module includes an intelligent optimization unit, which is configured with intelligent optimization strategies. The intelligent optimization strategies include:

[0073] Based on artificial intelligence, the system acquires the latest historical storage data and historical call data of the storage system in real time, and updates the space occupied by the buffer space, fluctuating storage space, and stable storage space, as well as the evaluation criteria for the buffer space and stable storage space in real time;

[0074] In the specific implementation process, by using artificial intelligence to update the space occupied by buffer space, fluctuating storage space, and stable storage space, as well as the occupancy evaluation criteria of buffer space and stable storage space in real time, it is possible to improve the timeliness of space allocation in storage space while saving manpower, so as to ensure that data storage and cache pre-fetching can be carried out in the optimal storage state when the storage space is accessed in real time;

[0075] Based on the latest buffer space, fluctuating storage space and stable storage space, multi-layer storage processing and cache pre-fetching processing are performed on the data stored in the storage system and the data retrieved from the storage system.

[0076] Working principle: First, obtain the historical storage data and historical call data of the storage system, analyze the storage space of the storage system based on the historical storage data, and divide the storage space into buffer space, fluctuating storage space and stable storage space based on the analysis results, and obtain the occupancy evaluation criteria of the buffer space and the stable storage space, wherein the occupancy evaluation criteria are the standards for the size of the occupied storage space; then, based on the historical storage data and historical call data, perform multi-layer storage processing on the data stored in the storage system; based on the buffer space and the buffer space occupancy evaluation criteria, perform cache pre-fetching processing on the data retrieved from the storage system; finally, based on artificial intelligence, update the allocation of storage space in the storage system in real time, and use the real-time updated storage system to perform multi-layer storage processing and cache pre-fetching on the data stored in the storage system and the data retrieved from the storage system.

[0077] Through the description of the above embodiments, the embodiments of the present invention can be provided as methods, systems, or computer program products. Based on this understanding, the essence of the above technical solutions or the portion that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for enabling a computer device (such as a personal computer, server, or network device) to execute the methods described in various embodiments or certain portions of the embodiments.

[0078] In the embodiments provided in this application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of systems, modules and units can be electrical, mechanical or other forms.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. AI-driven adaptive storage tiering and cache prefetching system, characterized by: Includes storage system storage analysis module, storage tiering and cache setting module, and intelligent optimization module; The storage system storage analysis module is used to obtain historical storage data and historical call data of the storage system, analyze the storage space of the storage system based on the historical storage data, and divide the storage space into buffer space, fluctuating storage space and stable storage space based on the analysis results, and obtain occupancy evaluation criteria for the buffer space and stable storage space, where the occupancy evaluation criteria are the standards for the size of the occupied storage space; The storage tiering and cache setting module is used to perform multi-layer storage processing on the data stored in the storage system based on historical storage data and historical call data; and to perform cache pre-fetch processing on the data retrieved from the storage system based on the buffer space and the buffer space occupancy evaluation criteria; The intelligent optimization module is used to update the allocation of storage space in the storage system in real time based on artificial intelligence, and use the real-time updated storage system to perform multi-layer storage processing and cache pre-fetching processing on data stored in the storage system and data retrieved from the storage system; The storage tiering and cache setting module further includes a cache prefetch unit, which is configured with a cache prefetch strategy. The cache prefetch strategy includes: When any data A in the storage system is retrieved, cache prefetching is performed based on data A; The cache prefetch process includes: recording the size of the space occupied by data A as G, and when the size of the unoccupied space in the buffer space is greater than G, transferring data A to the buffer space; When the size of the unoccupied space in the buffer space is less than or equal to G and the sum of the size of the unoccupied space in the buffer space and the size of the unoccupied space in the stable storage space is greater than G, the stable storage space allocates space to the buffer space according to the occupancy evaluation criteria of the stable storage space until the size of the unoccupied space in the buffer space is greater than G, and then the data A is transferred to the buffer space; Cache prefetching strategies also include: When the size of the unoccupied space in the buffer space is less than or equal to G and the sum of the size of the unoccupied space in the buffer space and the size of the unoccupied space in the stable storage space is less than or equal to G, the stable storage space allocates space to the buffer space under the stable storage space occupancy evaluation criteria until the size of the stable storage space reaches the stable space threshold. The size of the unoccupied space in the buffer space at this time is recorded as F, and the data occupying the space F in data A is transferred to the buffer space.

2. The AI-driven adaptive storage tiering and cache prefetching system according to claim 1, characterized in that: The storage system storage analysis module includes a storage analysis unit, which is configured with a storage analysis policy. The storage analysis policy includes: Obtain the historical storage data and historical call data of the storage system; record the time corresponding to the earliest data recorded in the historical storage data as the earliest record time; establish a plane rectangular coordinate system, recorded as the storage analysis coordinate system, where the unit of the X axis of the storage space coordinate system is h and the unit of the Y axis is MB; Obtain the change data of the storage space of the storage system from the earliest recorded time to the current time based on the historical storage data, draw the corresponding curve between X=0 and X=X1 on the X-axis, and record it as the space change curve, where X1 is the time difference between the earliest recorded time and the current time. The point with the horizontal coordinate of 0 in the space change curve is the storage space corresponding to the earliest recorded time, and the point with the horizontal coordinate of X1 is the storage space corresponding to the current time.

3. The AI-driven adaptive storage tiering and cache prefetching system according to claim 2, characterized in that: Storage analytics strategies also include: Based on the historical storage data, the time corresponding to when data is stored in the storage system and when data is retrieved from the storage system is obtained and recorded as the storage change time; in the spatial change curve, all points whose horizontal axes are storage change times are recorded as storage change points, and the space occupied by the data stored in the storage system or the data retrieved from the storage system corresponding to the storage change points is recorded as the storage occupancy value; All peaks and all troughs in the spatial variation curve are obtained, and are recorded as spatial variation peaks and spatial variation troughs respectively; the curves between adjacent spatial variation peaks and spatial variation troughs in the spatial variation curve are recorded as monotonic subcurves, and the points with the largest slope in the monotonic subcurves are recorded as storage variation poles; all monotonic subcurves and all storage variation poles in the spatial variation curve are obtained.

4. The AI-driven adaptive storage tiering and cache prefetching system according to claim 3, characterized in that: Storage analytics strategies also include: The storage change extreme point with the smallest vertical coordinate is recorded as the stable peak point, and the vertical coordinate of the stable peak point is marked as the stable space threshold; the size of the stable storage space is set as the stable space threshold, and the occupancy evaluation standard of the stable storage space is set to be greater than or equal to the stable space threshold; The storage change point with the largest vertical coordinate is recorded as the fluctuation peak point. When there is a storage change point with a vertical coordinate greater than the fluctuation peak point, the maximum value of all storage occupancy values ​​corresponding to the storage change points with a vertical coordinate greater than the fluctuation peak point is recorded as the minimum buffer value; when there is no storage change point with a vertical coordinate greater than the fluctuation peak point, the size of the storage space of the storage system minus the value of the vertical coordinate of the fluctuation peak point is recorded as the minimum buffer value.

5. The AI-driven adaptive storage tiering and cache prefetching system according to claim 4, characterized in that: Storage analytics strategies also include: The value obtained by subtracting the vertical coordinate of the stable peak point from the vertical coordinate of the fluctuation peak point is recorded as the fluctuation storage value, and the size of the fluctuation storage space is set to the fluctuation storage value; the value obtained by subtracting the fluctuation storage value from the size of the storage space of the storage system and then subtracting the stable space threshold is recorded as the size of the buffer space, and the buffer space occupancy judgment standard is set to be greater than or equal to the minimum buffer value.

6. The AI-driven adaptive storage tiering and cache prefetching system according to claim 5, characterized in that: Storage analytics strategies also include: When the storage space required by the stable storage space is greater than the stable space threshold, the buffer space allocates space to the stable storage space under the buffer space occupancy judgment standard; when the storage space required by the buffer space is greater than the minimum buffer value, the stable storage space allocates space to the buffer space under the stable storage space occupancy judgment standard.

7. The AI-driven adaptive storage tiering and cache prefetching system according to claim 6, characterized in that: The storage tiering and cache setting module includes a storage tiering setting unit, which is configured with a storage tiering setting strategy. The storage tiering setting strategy includes: Based on historical storage data and historical call data, the maximum time of temporary storage in the storage system is obtained and recorded as the maximum access time; When any data A is stored in the storage system, data A is processed by multi-layer storage; The multi-layer storage process is as follows: data A is stored in the buffer space first; when the remaining space in the fluctuating storage space is larger than the space occupied by data A, data A is transferred to the fluctuating storage space; When the remaining space of the fluctuating storage space is less than or equal to the space occupied by data A, the data with the longest fluctuating occupancy time in the fluctuating storage space is transferred to the stable storage space in sequence, until the remaining space of the fluctuating storage space is greater than the space occupied by data A, then data A is transferred to the fluctuating storage space; The storage time of data A in the fluctuation storage space is recorded in real time and recorded as the fluctuation occupancy time; When the fluctuation occupancy time of data A is greater than the maximum access time, data A is transferred to the stable storage space.

8. The AI-driven adaptive storage tiering and cache prefetching system according to claim 7, characterized in that: The intelligent optimization module includes an intelligent optimization unit, which is configured with an intelligent optimization strategy. The intelligent optimization strategy includes: Based on artificial intelligence, the system acquires the latest historical storage data and historical call data of the storage system in real time, and updates the space occupied by the buffer space, fluctuating storage space, and stable storage space, as well as the evaluation criteria for the buffer space and stable storage space in real time; Based on the latest buffer space, fluctuating storage space and stable storage space, multi-layer storage processing and cache pre-fetching processing are performed on the data stored in the storage system and the data retrieved from the storage system.

Images