A data storage management system and method based on big data model

Abstract

The present invention relates to the technical field of data storage management, and specifically to a data storage management system and method based on a big data model, comprising: a historical storage information acquisition module, a spatial change trend analysis module, a data screening reference module and a data screening management module, wherein the historical storage information of data of a data storage platform is collected by the historical storage information acquisition module, the reception index of data of the data storage platform after different remaining storage spaces in the past is analyzed by the spatial change trend analysis module, training data is composed according to the historical storage information of the data and the data reception index, a space occupancy analysis model is established by the data screening reference module, the time for screening the platform data is selected by referring to the space occupancy analysis model, the remaining storage space of the platform is checked at the selected time by the data screening management module, and data screening is performed when the remaining storage space of the platform is less than the remaining space threshold, thereby reducing the probability of the data storage platform being fully loaded.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage management, and in particular to a data storage management system and method based on a big data model. Background Art

[0002] A large amount of data is stored on the data storage platform. Space management of the data stored on the platform is conducive to stabilizing the storage platform in a state of not being fully loaded, that is, there is still remaining storage space, so as to ensure the smooth transmission and storage of subsequent data;

[0003] However, in order to stabilize the data storage platform and keep it in a state with remaining storage space, the existing technology generally analyzes the stored data, waits until the system is fully loaded, and then selectively filters and deletes the data stored on the data storage platform. The data is filtered out only after the platform is fully loaded. At this time, the full load abnormality has already occurred, which has affected the smooth transmission of some subsequent data. The above-mentioned method for data filtering is not conducive to reducing the probability of the data storage platform being fully loaded.

[0004] Therefore, people need a data storage management system and method based on a big data model to solve the above problems. Summary of the invention

[0005] The object of the present invention is to provide a data storage management system and method based on a big data model to solve the problems raised in the above background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a data storage management system based on a big data model, the system comprising: a historical storage information acquisition module, a spatial change trend analysis module, a data screening reference module and a data screening management module;

[0007] The output end of the historical storage information acquisition module is connected to the input end of the spatial change trend analysis module, the output end of the spatial change trend analysis module is connected to the input end of the data screening reference module, and the output end of the data screening reference module is connected to the input end of the data screening management module;

[0008] The historical storage information collection module is used to collect the historical storage information of the data storage platform; the spatial change trend analysis module is used to analyze the data reception index of the data storage platform after the remaining different storage spaces in the past, and training data is composed according to the historical storage information of the data and the data reception index; the data screening reference module is used to establish a space occupancy analysis model, and the time for screening the platform data is selected by referring to the space occupancy analysis model; the data screening management module is used to check the remaining storage space of the platform at the selected time, and perform data screening when the remaining storage space of the platform is less than the remaining space threshold.

[0009] Further, the historical storage information collection module includes a storage space data collection unit, a historical data upload information collection unit and a full load time collection unit;

[0010] The storage space data collection unit is used to collect the previous storage space data of the data storage platform used to store equipment monitoring data; the historical data upload information collection unit is used to collect the amount of data uploaded to the platform each time after the platform had different remaining storage spaces in the past, as well as the interval duration between two adjacent data uploads; the full load time collection unit is used to collect the time information taken for the platform to be fully loaded after the platform had different remaining storage spaces in the past.

[0011] Further, the spatial change trend analysis module includes a historical reception index analysis unit and a training data generation unit;

[0012] The input end of the historical reception index analysis unit is connected to the output end of the historical data upload information collection unit, and the input end of the training data generation unit is connected to the output ends of the historical reception index analysis unit, the storage space data collection unit and the full load time collection unit;

[0013] The historical reception index analysis unit analyzes the data reception index of the platform after different remaining storage spaces in the past according to the amount of data uploaded to the platform each time after the platform had different remaining storage spaces in the past and the interval time information between two adjacent data uploads; the training data generation unit combines the historical remaining storage space, the data reception index and the time it takes for the platform to be fully loaded after different remaining storage spaces to form training data.

[0014] Further, the data screening reference module includes a space occupancy analysis model building unit, a current reception index prediction unit and a system full load time prediction unit;

[0015] The input end of the space occupancy analysis model establishment unit is connected to the output end of the training data generation unit, the input end of the current reception index prediction unit is connected to the output end of the historical reception index analysis unit, and the input end of the system full load time prediction unit is connected to the output ends of the space occupancy analysis model establishment unit and the current reception index prediction unit;

[0016] The space occupancy analysis model establishment unit establishes a space occupancy analysis model based on the training data; the current reception index prediction unit predicts the current reception index of the platform based on the changes in the reception index of historical data; the system full load time prediction unit inputs the current remaining storage space of the platform and the current reception index into the space occupancy analysis model, and outputs and predicts the time required for the platform to be fully loaded under the current circumstances.

[0017] Further, the data screening management module includes an importance analysis unit and a stored data screening unit;

[0018] The input end of the importance analysis unit is connected to the output end of the system full load time prediction unit, and the output end of the importance analysis unit is connected to the input end of the storage data screening unit;

[0019] The importance analysis unit selects the time for screening the data stored on the platform based on the prediction results. When screening, the upload time and the number of times the data is retrieved for viewing are collected, and the importance of the data is analyzed based on the upload time and the number of times the data is retrieved for viewing; the storage data screening unit sorts the data stored on the platform from small to large in order of importance, and screens the data in the order of arrangement until the remaining storage space of the platform is greater than the remaining space threshold.

[0020] A data storage management method based on a big data model comprises the following steps:

[0021] S100: Collecting historical data storage information from a data storage platform;

[0022] S200: Analyze the data receiving index of the data storage platform after the remaining different storage spaces in the past, and form training data according to the historical data storage information and the data receiving index;

[0023] S300: Establishing a space occupancy analysis model, and selecting a time for filtering platform data by referring to the space occupancy analysis model;

[0024] S400: Check the remaining storage space of the platform at the selected time and perform data filtering.

[0025] Further, in S100: the previous storage space data of the data storage platform for storing device monitoring data is collected, and the historical remaining storage space set is obtained as V = {V1, V2, ..., V i , …, V f}, where the elements in set V are arranged from first to last according to the time when the corresponding remaining storage space appears, V f It represents the remaining storage space obtained the closest to the current time before the current time. f represents the number of elements in the set V. The remaining storage space collected on the platform in the past is V i The amount of data uploaded to the platform each time is B = {B1, B2, ..., B n}, where n represents the remaining storage space V i The number of data uploads from the platform to the full load time is collected, and the remaining storage space of the platform is V i The interval length set between two consecutive data uploads is T = {T2, T3, ..., T n}, where T2 indicates that the remaining storage space on the platform is V i The interval between the second data upload time and the first data upload time, the amount of data uploaded to the platform each time after the platform has different remaining storage spaces in the past, and the interval between two adjacent data uploads, the time set spent on the platform being fully loaded after the platform has different remaining storage spaces in the past is collected as R = {R1, R2, ..., R f}.

[0026] Furthermore, in S200: according to the formula The remaining storage space of the computing platform is V i After the data is received index Q i , the data receiving index set after the platform has different remaining storage spaces is obtained as Q = {Q1,

[0027] Q2, ..., Q i , …, Q f}, where B e Indicates that the remaining storage space is V i The amount of data uploaded to the platform for the eth time, T e Indicates that the remaining storage space is V i The interval between the time of the e-th data upload and the time of the e-1-th data upload is used to form a training data set {(V1, Q1, R1), (V2, Q2, R2), …, (V f , Q f , R f )}.

[0028] Further, in S300: the training data is fitted to establish a space occupancy analysis model: z = α1*x+α2*y+α0, where α1, α2 and α0 represent fitting coefficients, x represents an independent variable referring to the remaining storage space, y represents an independent variable referring to the data reception index, and z represents a dependent variable referring to the time it takes for the platform to be fully loaded. The data reception index Q of the current platform is predicted according to the following formula: f+1 :

[0029] Q f+1 =Q f *β+ρ f *(1-β);

[0030] Among them, β represents the exponential smoothing coefficient, 0<β<1, ρ f represents the exponential smoothing value of the f-th data receiving index, ρ f The solution is: according to Calculate the exponential smoothing value ρ1 of the first data reception index, calculate the exponential smoothing value ρ2 of the second data reception index according to ρ2=Q1*β+ρ1*(1-β), calculate the exponential smoothing value ρ3 of the third data reception index according to ρ3=Q2*β+ρ2*(1-β), and so on to obtain ρ f , and obtain the current remaining storage space of the platform as K. f+1 Input into the space occupancy analysis model: Let x = K,

[0031] y=Q f+1 , it is predicted that the time required for the platform to be fully loaded under the current situation is α1*K+α2*Q f+1 +α0, the time selected to filter out the platform data is: the interval from the current time is α1*K+α2*Q f+1 +α0.

[0032] Further, in S400: a remaining storage space threshold is set. If the remaining storage space of the platform is less than the remaining space threshold at the selected time, data is filtered out on the platform: the interval between the time when the monitoring data of different devices stored in the collection platform is uploaded and the time when the filtering is performed is t = {t1, t2, ..., t m}, the corresponding data is retrieved and viewed for a set of times A = {A1, A2, ..., A m}, where m represents the number of equipment monitoring data items. The equipment monitoring data uploaded at the same time is one item. According to the formula Calculate the importance of a random data W j , we get the importance set of m items of data as W = {W1, W2, ..., W j , …, W m}, sort the m items of monitoring data from small to large in order of importance, and filter out the data in the order of arrangement until the remaining storage space of the platform is greater than the remaining space threshold.

[0033] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0034] The present invention collects the time and data volume information of data upload after the remaining different storage spaces and the remaining corresponding storage space of the data storage platform through big data technology, analyzes the reception index of the data after the remaining corresponding storage space in the past according to the collected information of the time and data volume of data upload after the remaining corresponding storage space, takes the remaining storage space and the reception index as the two main factors affecting the full load time of the platform, takes the historical remaining storage space, the reception index and the time when the platform reaches full load of the data storage platform as training data to establish a space occupancy analysis model, analyzes and predicts the time required for the platform to be fully loaded under the current situation according to the space occupancy model, checks the platform full load situation at the predicted time, sets the remaining storage space threshold, and pre-screens the data when it is found that the remaining storage space of the platform is less than the remaining space threshold, rather than waiting until the data is fully loaded and then screening the data, thereby reducing the probability of the data storage platform being fully loaded. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0036] Figure 1 It is a structural diagram of a data storage management system based on a big data model of the present invention;

[0037] Figure 2 It is a flow chart of a data storage management method based on a big data model of the present invention. DETAILED DESCRIPTION

[0038] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0039] Combine the following Figure 1-Figure 2 The present invention is further described with reference to the accompanying drawings and specific embodiments.

[0040] Embodiment 1:

[0041] like Figure 1As shown, this embodiment provides a data storage management system based on a big data model, and the system includes: a historical storage information collection module, a spatial change trend analysis module, a data screening reference module and a data screening management module; the historical storage information of the data storage platform is collected through the historical storage information collection module; the data reception index of the data storage platform after the remaining different storage spaces in the past is analyzed through the spatial change trend analysis module, and training data is composed according to the data historical storage information and the data reception index; a space occupancy analysis model is established through the data screening reference module, and the time for screening the platform data is selected by referring to the space occupancy analysis model; the remaining storage space of the platform is checked at the selected time through the data screening management module, and data screening is performed when the remaining storage space of the platform is less than the remaining space threshold.

[0042] The historical storage information collection module includes a storage space data collection unit, a historical data upload information collection unit and a full load time collection unit. The storage space data collection unit is used to collect the previous storage space data of the data storage platform used to store equipment monitoring data; the historical data upload information collection unit is used to collect the amount of data uploaded to the platform each time after the platform had different remaining storage spaces in the past, as well as the interval duration between two adjacent data uploads; and the full load time collection unit is used to collect the time information taken for the platform to be fully loaded after the platform had different remaining storage spaces in the past.

[0043] The spatial change trend analysis module includes a historical reception index analysis unit and a training data generation unit. The historical reception index analysis unit analyzes the data reception index of the platform after different remaining storage spaces in the past according to the amount of data uploaded to the platform each time after the platform had different remaining storage spaces in the past and the interval time information between two adjacent data uploads; the training data generation unit combines the historical remaining storage space, the data reception index and the time it takes for the platform to be fully loaded after different remaining storage spaces to form training data.

[0044] The data screening reference module includes a space occupancy analysis model establishment unit, a current reception index prediction unit and a system full load time prediction unit. The space occupancy analysis model establishment unit establishes a space occupancy analysis model based on the training data; the current reception index prediction unit predicts the current reception index of the platform based on the changes in the reception index of historical data; the system full load time prediction unit inputs the current remaining storage space of the platform and the current reception index into the space occupancy analysis model, and outputs and predicts the time required for the platform to be fully loaded under the current circumstances.

[0045] The data screening management module includes an importance analysis unit and a storage data screening unit. The importance analysis unit selects the time for screening the data stored in the platform based on the prediction results. When screening, the upload time of different equipment monitoring data and the number of times they are retrieved and viewed are collected, and the importance of the equipment monitoring data is analyzed based on the upload time and the number of times they are retrieved and viewed. The storage data screening unit sorts the equipment monitoring data stored in the platform from small to large according to their importance, and screens the data in the order of arrangement until the remaining storage space of the platform is greater than the remaining space threshold.

[0046] Embodiment 2:

[0047] like Figure 2 As shown, this embodiment provides a data storage management method based on a big data model, which is implemented based on the data storage management system in the embodiment and specifically includes the following steps:

[0048] S100: Collect historical data storage information of the data storage platform, collect the previous storage space data of the data storage platform for storing device monitoring data, and obtain the historical remaining storage space set V = {V1, V2, ..., V i , …,

[0049] V f}, where the elements in set V are arranged from first to last according to the time when the corresponding remaining storage space appears, V f It represents the remaining storage space obtained the closest to the current time before the current time. f represents the number of elements in the set V. The remaining storage space collected on the platform in the past is V i The amount of data uploaded to the platform each time is B = {B1, B2, ..., B n}, where n represents the remaining storage space V i The number of data uploads from the platform to the full load time is collected, and the remaining storage space of the platform is V i The interval length set between two consecutive data uploads is T = {T2, T3, ..., T n}, where T2 indicates that the remaining storage space on the platform is V i The interval between the second data upload time and the first data upload time, the amount of data uploaded to the platform each time after the platform has different remaining storage spaces in the past, and the interval between two adjacent data uploads, the time set spent on the platform being fully loaded after the platform has different remaining storage spaces in the past is collected as R = {R1, R2, ..., R f};

[0050] S200: Analyze the data receiving index of the data storage platform after the remaining different storage spaces in the past, and form training data based on the historical data storage information and the data receiving index. The remaining storage space of the computing platform is V i After the data is received index Q i , the data receiving index set after the remaining different storage spaces on the platform is obtained as Q = {Q1, Q2, ..., Q i , …, Q f}, where B e Indicates that the remaining storage space is V i The amount of data uploaded to the platform for the eth time, T e Indicates that the remaining storage space is V i The interval between the time of the e-th data upload and the time of the e-1-th data upload is used to form a training data set {(V1, Q1, R1), (V2, Q2, R2), …, (V f , Q f , R f )};

[0051] S300: Establish a space occupancy analysis model, select the time to screen out the platform data by referring to the space occupancy analysis model, fit the training data, and establish the space occupancy analysis model: z = α1*x+α2*y+α0, where α1, α2 and α0 represent fitting coefficients, x represents the independent variable referring to the remaining storage space, y represents the independent variable referring to the data reception index, and z represents the dependent variable referring to the time spent on the platform being fully loaded. The data reception index Q of the current platform is predicted according to the following formula f+1 :

[0052] Q f+1 =Q f *β+ρ f *(1-β);

[0053] Among them, β represents the exponential smoothing coefficient, 0<β<1, ρ f represents the exponential smoothing value of the f-th data receiving index, ρ f The solution is: according to Calculate the exponential smoothing value ρ1 of the first data reception index, calculate the exponential smoothing value ρ2 of the second data reception index according to ρ2=Q1*β+ρ1*(1-β), calculate the exponential smoothing value ρ3 of the third data reception index according to ρ3=Q2*β+ρ2*(1-β), and so on to obtain ρ f , and obtain the current remaining storage space of the platform as K. f+1 Input into the space occupancy analysis model: Let x = K,

[0054] y = Q f+1 It is predicted that the time required for the platform to be fully loaded in the current situation is α1*K + α2*Q f+1 + α0, and the time for selecting to screen the platform data is: at an interval of α1*K + α2*Q f+1 + α0 after the current time;

[0055] S400: Check the remaining storage space of the platform at the selected time, perform data screening processing, set the remaining storage space threshold. If the remaining storage space of the platform is less than the remaining space threshold when checked at the selected time, perform data screening processing on the platform: The set of time intervals from the time when different device monitoring data stored in the platform are uploaded to the time of performing the screening processing is t = {t1, t2,..., t m}, and the set of the number of times the corresponding data is retrieved and viewed is A = {A1, A2,..., A m}, where m represents the number of items of device monitoring data, and the device monitoring data uploaded at the same time is one item. According to the formula Calculate the importance degree W of a random item of data j , and obtain the set of importance degrees of m items of data as W = {W1, W2,..., W j ,..., W m}, sort the m items of monitoring data from smallest to largest according to the importance degree, and screen the data in the arranged order until the remaining storage space of the platform is greater than the remaining space threshold;

[0056] For example: The set of time intervals from the time when different device monitoring data stored in the platform are uploaded to the time of performing the screening processing is t = {t1, t2, t3} = {10, 5, 0.2}, with the unit being: days, and the set of the number of times the corresponding data is retrieved and viewed is A = {A1, A2, A3} = {2, 4, 8}. According to the formula Calculate the set of importance degrees of 3 items of data as W = {W1, W2, W3} = {1.69, 3.33, 76.57}, sort the 3 items of monitoring data from smallest to largest according to the importance degree: W1 < W2 < W3. When the remaining storage space of the platform is less than the remaining space threshold, preferentially screen the device monitoring data with the importance degree of W1, and screen the data in the arranged order until the remaining storage space of the platform is greater than the remaining space threshold.

[0057] Finally, it should be noted that the above description is only a preferred example of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data storage management method based on a big data model, characterized in that: The following steps are involved: S100: Collecting historical data storage information from a data storage platform; S200: Analyze the data receiving index of the data storage platform after the remaining different storage spaces in the past, and form training data according to the historical data storage information and the data receiving index; S3 00: Establish a space occupancy analysis model and select the time to filter the platform data by referring to the space occupancy analysis model; S400: Check the remaining storage space of the platform at the selected time and perform data screening; In S100: the previous storage space data of the data storage platform for storing device monitoring data is collected, and the historical remaining storage space set is obtained as V = {V1, V2, ..., V i , …, V f }, where the elements in set V are arranged from first to last according to the time when the corresponding remaining storage space appears, V f It represents the remaining storage space obtained the closest to the current time before the current time. f represents the number of elements in the set V. The remaining storage space collected on the platform in the past is V i The amount of data uploaded to the platform each time is B = {B1, B2, ..., B n }, where n represents the remaining storage space V i The number of data uploads from the platform to the full load time is collected, and the remaining storage space of the platform is V i The interval length set between two consecutive data uploads is T = {T2, T3, ..., T n }, where T2 indicates that the remaining storage space on the platform is V i The interval between the second data upload time and the first data upload time, the amount of data uploaded to the platform each time after the platform has different remaining storage spaces in the past, and the interval between two adjacent data uploads, the time set spent on the platform being fully loaded after the platform has different remaining storage spaces in the past is collected as R = {R1, R2, ..., R f }; In S200: According to the formula The remaining storage space of the computing platform is V i After the data is received index Q i , the data receiving index set after the remaining different storage spaces on the platform is obtained as Q = {Q1, Q2, ..., Q i , …, Q f }, where B e Indicates that the remaining storage space is V i The amount of data uploaded to the platform for the eth time, T e Indicates that the remaining storage space is V i The interval between the time of the e-th data upload and the time of the e-1-th data upload is used to form a training data set {(V1, Q1, R1), (V2, Q2, R2), …, (V f , Q f , R f )}.

2. The data storage management method based on the big data model according to claim 1 is characterized in that: In S300: fit the training data to establish a space occupancy analysis model: z = α1*x+α2*y+α0, where α1, α2 and α0 represent fitting coefficients, x represents an independent variable referring to the remaining storage space, y represents an independent variable referring to the data reception index, and z represents a dependent variable referring to the time it takes for the platform to be fully loaded. The data reception index Q of the current platform is predicted according to the following formula f+1 : Q f+1 =Q f *b+r f *(1-b); Among them, β represents the exponential smoothing coefficient, 0<β<1, ρ f represents the exponential smoothing value of the f-th data receiving index, ρ f The solution is: according to Calculate the exponential smoothing value ρ1 of the first data reception index, calculate the exponential smoothing value ρ2 of the second data reception index according to ρ2=Q1*β+ρ1*(1-β), calculate the exponential smoothing value ρ3 of the third data reception index according to ρ3=Q2*β+ρ2*(1-β), and so on to obtain ρ f , and obtain the current remaining storage space of the platform as K. f+1 Input into the space occupancy analysis model: let x = K, y = Q f+1 , it is predicted that the time required for the platform to be fully loaded under the current situation is α1*K+α2*Qf +1 +α0, the time selected to filter out the platform data is: the interval from the current time is α1*K+α2*Q f+1 +α0.

3. The data storage management method based on the big data model according to claim 2 is characterized in that: In S400: a remaining storage space threshold is set. If the remaining storage space of the platform is less than the remaining space threshold at the selected time, data is filtered out on the platform: the interval between the time when the monitoring data of different devices stored in the collection platform is uploaded and the time when the filtering is performed is t = {t1, t2, ..., t m }, the corresponding data is retrieved and viewed for a set of times A = {A1, A2, ..., A m }, where m represents the number of items of equipment monitoring data. According to the formula Calculate the importance of a random data W j , we get the importance set of m items of data as W = {W1, W2, ..., W j , …, W m }, sort the m items of monitoring data from small to large in order of importance, and filter out the data in the order of arrangement until the remaining storage space of the platform is greater than the remaining space threshold.

4. A data storage management system based on a big data model, applied to the data storage management method based on a big data model as claimed in claim 1, characterized in that: The system includes: a historical storage information acquisition module, a spatial change trend analysis module, a data screening reference module and a data screening management module; The output end of the historical storage information acquisition module is connected to the input end of the spatial change trend analysis module, the output end of the spatial change trend analysis module is connected to the input end of the data screening reference module, and the output end of the data screening reference module is connected to the input end of the data screening management module; The historical storage information collection module is used to collect historical storage information of the data storage platform; the spatial change trend analysis module is used to analyze the data reception index of the data storage platform after the remaining different storage spaces in the past, and training data is composed according to the historical storage information and the data reception index; the data screening reference module is used to establish a space occupancy analysis model, and the time for screening the platform data is selected by referring to the space occupancy analysis model; the data screening management module is used to check the remaining storage space of the platform at the selected time, and perform data screening when the remaining storage space of the platform is less than the remaining space threshold.

5. The data storage management system based on the big data model according to claim 4, characterized in that: The historical storage information collection module includes a storage space data collection unit, a historical data upload information collection unit and a full load time collection unit; The storage space data collection unit is used to collect the previous storage space data of the data storage platform used to store equipment monitoring data; the historical data upload information collection unit is used to collect the amount of data uploaded to the platform each time after the platform had different remaining storage spaces in the past, as well as the interval duration between two adjacent data uploads; the full load time collection unit is used to collect the time information taken for the platform to be fully loaded after the platform had different remaining storage spaces in the past.

6. The data storage management system based on the big data model according to claim 5, characterized in that: The spatial change trend analysis module includes a historical reception index analysis unit and a training data generation unit; The input end of the historical reception index analysis unit is connected to the output end of the historical data upload information collection unit, and the input end of the training data generation unit is connected to the output ends of the historical reception index analysis unit, the storage space data collection unit and the full load time collection unit; The historical reception index analysis unit analyzes the data reception index of the platform after different storage spaces were left in the past according to the amount of data uploaded to the platform each time after different storage spaces were left in the past and the interval time information of two adjacent data uploads; The training data generating unit combines the historical remaining storage space, the data receiving index and the time taken for the platform to be fully loaded after the remaining different storage spaces to form training data.

7. The data storage management system based on the big data model according to claim 6, characterized in that: The data screening reference module includes a space occupancy analysis model building unit, a current reception index prediction unit and a system full load time prediction unit; The input end of the space occupancy analysis model establishment unit is connected to the output end of the training data generation unit, the input end of the current reception index prediction unit is connected to the output end of the historical reception index analysis unit, and the input end of the system full load time prediction unit is connected to the output ends of the space occupancy analysis model establishment unit and the current reception index prediction unit; The space occupancy analysis model establishment unit establishes a space occupancy analysis model based on the training data; the current reception index prediction unit predicts the current reception index of the platform based on the changes in the reception index of historical data; the system full load time prediction unit inputs the current remaining storage space of the platform and the current reception index into the space occupancy analysis model, and outputs and predicts the time required for the platform to be fully loaded under the current circumstances.

8. The data storage management system based on the big data model according to claim 7, characterized in that: The data screening management module includes an importance analysis unit and a stored data screening unit; The input end of the importance analysis unit is connected to the output end of the system full load time prediction unit, and the output end of the importance analysis unit is connected to the input end of the storage data screening unit; The importance analysis unit selects the time for screening the data stored on the platform based on the prediction results. When screening, the upload time and the number of times the data is retrieved for viewing are collected, and the importance of the data is analyzed based on the upload time and the number of times the data is retrieved for viewing; the storage data screening unit sorts the data stored on the platform from small to large in order of importance, and screens the data in the order of arrangement until the remaining storage space of the platform is greater than the remaining space threshold.

Images