Data storage management system and method based on cloud storage

Abstract

The invention relates to the technical field of data storage and management, and discloses a data storage management system and method based on cloud storage, and the system comprises a data distribution and redundancy guarantee module; a data classification and migration strategy module; a data access and service interaction module; and a data processing and trend prediction module. Through distributed storage, redundancy guarantee, data migration and classification strategy and other technologies, cross-region and cross-device data access and sharing can be realized, and high reliability and service continuity of data are ensured; in combination with streaming processing and batch processing frameworks, trend analysis and anomaly detection are performed through a depth feature prediction algorithm, and potential risks and fluctuations can be identified in real time; besides, by utilizing a micro-service architecture and a real-time data synchronization mechanism, continuously changing data requirements can be flexibly adapted, and data security and compliance management are ensured.

Description

Technical Field

[0001] This application relates to the technical field of data storage and management, and in particular to a data storage management system and method based on cloud storage. Background Technology

[0002] Tobacco data typically refers to various data related to product sales, inventory, logistics, and promotional activities in the tobacco industry, including but not limited to sales volume, inventory levels, transportation records, consumer behavior, and the effectiveness of marketing activities. Tobacco data is highly seasonal, regionally diverse, and subject to compliance requirements, and is often affected by factors such as policies, holidays, and climate.

[0003] With the rapid growth of data volume in the tobacco industry, traditional local storage methods can no longer meet the needs of rapid expansion and high reliability. Therefore, cloud storage technology has been widely used in the tobacco industry. Cloud storage technology is a data storage solution based on cloud computing. It integrates different types of storage resources through the network and provides them to tobacco companies in the form of storage services. It integrates multiple technologies such as distributed storage, virtualization, and network transmission. Through the network, it stores various types of data in the tobacco industry (such as sales data, inventory data, logistics data, etc.) on a remote server cluster, realizing cross-regional and cross-device access and sharing of data.

[0004] In existing technologies, traditional storage and data processing architectures generally suffer from single points of failure, relying on a single node or data center. Once a failure occurs, it may lead to the inability to continue using the system, affecting business continuity. In addition, traditional processing methods have poor scalability. As the amount of data grows, it is difficult to flexibly cope with the storage and processing needs of massive amounts of data, resulting in a gradual decline in system performance. At the same time, traditional architectures lack real-time data processing capabilities and cannot effectively cope with high-speed data inflows, resulting in delayed business responses and an inability to capture and respond to rapid market changes in a timely manner. Summary of the Invention

[0005] To address the issues of poor scalability of traditional processing methods, which make it difficult to flexibly cope with the storage and processing needs of massive amounts of data as data volume grows, leading to a gradual decline in system performance, and the insufficient real-time data processing capabilities of traditional architectures, which cannot effectively cope with the high-speed inflow of data, resulting in delayed business response and inability to capture and respond to rapid market changes in a timely manner, this application provides a data storage management system and method based on cloud storage.

[0006] In a first aspect, this application provides a cloud-based data storage management system, which includes: The data distribution and redundancy assurance module is used to distribute data across storage nodes in multiple geographical locations and establish a business continuity assurance mechanism based on cross-regional redundancy strategies. The data classification and migration strategy module classifies data into hot data, cold data, and archived data based on the data access frequency and lifecycle, and formulates migration rules for data from hot storage to cold storage and then to archived storage in conjunction with the business continuity assurance mechanism. The data access and service interaction module, based on migration rules, builds a unified data access interface, and sets up an application programming interface and microservice architecture, configures inter-service call relationships, integrates a real-time data synchronization mechanism, and establishes a data security and compliance management mechanism. The data processing and trend prediction module is used to access real-time data streams and historical data warehouses synchronized with data security and compliance management mechanisms, establish a big data processing platform, combine streaming and batch processing frameworks to process real-time data streams and historical data warehouses respectively, perform trend analysis, identify early warning signals, and provide a visual display interface.

[0007] Optionally, the data distribution and redundancy protection module includes: The primary / standby architecture configuration module deploys distributed storage nodes in multiple geographical regions. For each storage region, a primary / standby architecture is configured. The primary / standby architecture includes a primary node and a standby node. The primary node is responsible for real-time data read and write operations, while the standby node is used for cross-regional data redundancy synchronization and can quickly take over read and write tasks when the primary node fails. The priority strategy formulation module sets up a disaster recovery priority strategy based on the cross-regional redundancy strategy, performs fault detection on each storage node, and sets priorities according to the importance and availability of different nodes. The traffic scheduling and dynamic weight adjustment module configures the global load balancer and monitors the load of each storage node in real time; based on real-time data and access history, it dynamically adjusts the node weights and traffic distribution of the global load balancer.

[0008] Optionally, based on the cross-regional redundancy strategy, a disaster recovery priority strategy is set, fault detection is performed on each storage node, and priorities are set according to the importance and availability of different nodes, including: Storage nodes are categorized based on the business needs and data importance of the region where each storage node is located; Configure a fault monitoring mechanism for each classified storage node and monitor the health status of the storage nodes regularly; Based on the monitored health status, set a fault trigger threshold for each storage node; By combining the classification results of storage nodes with the fault trigger threshold, the importance of nodes to the business and the severity of faults are comprehensively evaluated, and a recovery priority is set for each storage node. Based on the recovery priority of each storage node, a recovery process is developed to address the occurrence of a failure.

[0009] Optionally, a global load balancer is configured, and the load status of each storage node is monitored in real time; based on real-time data and access history, the node weights and traffic distribution of the global load balancer are dynamically adjusted, including: Configure a global load balancer and use it as the unified entry point for all client traffic; Collect data on the current number of requests, average response time, memory usage, network bandwidth utilization, and error rate of each storage node, and monitor the current load of each storage node in real time. Collect access history, combine current load and access history, and substitute into the following formula: ; ; Obtain the dynamic weights of each node. ; Based on the dynamic weights of each node Substitute into the following formula: ; Obtain the traffic allocation for storage node i ; In the formula, This represents the initial weight value of storage node i; Represents the initial weights of all storage nodes. A set; This represents the real-time load score of storage node i; n represents the total number of storage nodes participating in load balancing. This indicates the total traffic that needs to be allocated at present; Represents a minimal constant; This represents all storage nodes from node 1 to n; This indicates the access history factor.

[0010] Optionally, the data access and service interaction module includes: The data access interface construction module is used to define the migration rules of data from hot storage to cold storage and then to archive storage, and to build a unified data access interface; The microservice architecture and interface interface module is used to establish the interface between various service modules and data access interfaces in a microservice architecture. The real-time data synchronization module, based on migration rules and configured with a real-time data synchronization mechanism, ensures that changes between hot data and cold data can be synchronized in a timely manner during the migration of data from hot storage to cold storage and archive storage. The data security and compliance module is used to configure identity authentication and access control, and to establish a data security and compliance management mechanism.

[0011] Optionally, a combination of streaming and batch processing frameworks can be used to process real-time data streams and historical data warehouses respectively, and to perform trend analysis and future trend prediction, including: Real-time data is processed using a stream processing framework, and features are extracted from the processed real-time data to obtain real-time features. Historical data is processed using a batch processing platform, and features are extracted from the processed historical data to obtain a historical feature set. By fusing real-time feature sets with historical feature sets, a complete temporal feature set is obtained; Using deep feature prediction algorithms, we can perform trend analysis and future trend prediction on time series feature sets to identify abnormal fluctuation trends and abnormal behaviors. When abnormal fluctuations or behaviors appear in the prediction results, an early warning mechanism is automatically triggered, and a visual display interface is provided.

[0012] Optionally, deep feature prediction algorithms can be used to perform trend analysis and future trend prediction on time series feature sets, identifying abnormal fluctuation trends and abnormal behaviors, including: We use a one-dimensional convolutional kernel to perform sliding computation on the input temporal feature set to extract local temporal features, and then use max pooling to reduce the dimensionality of the local temporal features. The dimensionality-reduced local temporal features are transformed into query, key, and value vectors through three linear transformations. By calculating the similarity between the query and the key, and using the softmax function for normalization, the attention weight corresponding to each position is obtained. The attention weight is then combined with the value vector in a weighted manner to obtain the attention-weighted feature sequence. By introducing a kernel function, the attention-weighted feature sequence is nonlinearly mapped from a low-dimensional space to a high-dimensional space, resulting in a high-dimensional mapped feature sequence. The high-dimensional mapped feature sequence is used as input to train the ridge regression model in the high-dimensional space, and L2 regularization is used to prevent overfitting. The trained ridge regression model is used to predict data at future time points, outputting predicted values. The predicted values ​​are then compared and analyzed with historical true values ​​to identify potential trend changes, abnormal fluctuations, or risk signals.

[0013] Optionally, the expression for predicting data at future time points using the trained ridge regression model is: ; In the formula, Indicates the current input features The predicted function value; This represents the regression coefficient corresponding to the nth training sample; Represents the kernel matrix; This represents the feature of the nth training sample; This represents the feature vector corresponding to the current time point t that needs to be predicted; n represents the nth training sample; m represents the training sample data.

[0014] Secondly, this application also provides a cloud storage-based data storage management method, which is implemented based on the aforementioned cloud storage-based data storage management method.

[0015] In summary, this application includes at least one of the following beneficial technical effects: 1. This application utilizes technologies such as distributed storage, redundancy assurance, data migration, and classification strategies to achieve cross-regional and cross-device data access and sharing, ensuring high data reliability and business continuity. By combining streaming and batch processing frameworks and using deep feature prediction algorithms for trend analysis and anomaly detection, it can identify potential risks and fluctuations in real time. Furthermore, by leveraging microservice architecture and real-time data synchronization mechanisms, it can flexibly adapt to constantly changing data needs and ensure data security and compliance management.

[0016] 2. This application provides high availability and disaster recovery capabilities through a primary-backup architecture, cross-regional redundancy strategy, and dynamic load balancing mechanism, ensuring continuous data availability in the event of node failure; the priority strategy designs the fault detection and recovery process based on the importance and health status of nodes, further improving the system's fault tolerance and recovery efficiency; at the same time, the global load balancer monitors node load in real time and dynamically adjusts traffic distribution, optimizing resource utilization.

[0017] 3. This application combines streaming and batch processing frameworks to effectively process real-time data streams and historical data warehouses, thereby comprehensively capturing time-series features and performing fusion analysis. The deep feature prediction algorithm utilizes a combination of convolution, attention mechanisms, and kernel functions to extract key time-series features and perform high-dimensional mapping. Finally, it uses a ridge regression model to accurately predict future trends. Attached Figure Description

[0018] Figure 1 This is a flowchart of the system in this application. Detailed Implementation

[0019] The embodiments of this application are described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0020] In the description of this specification, the references to "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples" refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0021] The first embodiment of this application discloses a cloud-based data storage management system, referring to... Figure 1 The data storage and management system includes: The data distribution and redundancy assurance module is used to distribute data across storage nodes in multiple geographical locations and establish a business continuity assurance mechanism based on cross-regional redundancy strategies.

[0022] Preferably, the data distribution and redundancy protection module includes: The primary / standby architecture configuration module deploys distributed storage nodes across multiple geographical regions. For each storage region, a primary / standby architecture is configured. The primary / standby architecture includes a primary node and a standby node. The primary node is responsible for real-time data read and write operations, while the standby node is used for cross-regional data redundancy synchronization and can quickly take over read and write tasks when the primary node fails.

[0023] It should be explained that tobacco data often contains a large amount of historical sales data, market analysis, inventory information, etc. This data usually needs to be stored and processed in multiple locations. In order to achieve high availability and low latency response, the data is stored in distributed nodes in multiple geographical regions, with multiple storage nodes in each region forming a primary and backup architecture.

[0024] The primary-backup architecture includes a primary node and a backup node; 1. Master node: Responsible for receiving data writes from sales points, warehouses, and other locations, and recording real-time sales and inventory changes; 2. Backup node: Synchronizes data with the master node to ensure that service can be restored in the shortest possible time should the master node fail.

[0025] Tobacco data may contain real-time updated information such as inventory, sales, and demand forecasts. A primary / standby architecture ensures that data is not lost. The specific operation is as follows: 1. Data Synchronization: Data on the primary node will be synchronized to the backup node. If synchronous replication is selected, the primary node will wait for the backup node to confirm the data writing before returning a response, thus ensuring data consistency. Asynchronous replication allows for more efficient processing, but inconsistencies may occur for a very short time.

[0026] 2. Failover: When the primary node fails (such as hardware damage, network disconnection, etc.), the backup node will quickly take over the role of the primary node, continue to receive data and provide services; at this time, the backup node becomes the primary node, and all operations are seamlessly switched to ensure that sales data and inventory data are not interrupted.

[0027] Tobacco data may involve market analysis, sales trends, and other information across multiple regions; therefore, cross-regional redundancy is crucial. 1. Data redundancy: The system will back up key data (such as sales, inventory, etc.) across multiple nodes in different geographical regions to ensure that even if a failure occurs in one region, other regions can still continue to provide services. 2. Cross-regional access optimization: By properly configuring the load balancer, we ensure that requests from different regions can be responded to quickly in the nearest region, reducing cross-regional network latency and improving user experience.

[0028] The priority policy formulation module sets disaster recovery priority policies based on cross-regional redundancy policies, performs fault detection on each storage node, and sets priorities according to the importance and availability of different nodes.

[0029] Preferably, based on the cross-regional redundancy strategy, a disaster recovery priority strategy is set, and fault detection is performed on each storage node. The specific implementation steps for setting priorities according to the importance and availability of different nodes are as follows: Storage nodes are categorized based on the business needs and data importance of the region where each storage node is located; Configure a fault monitoring mechanism for each classified storage node and monitor the health status of the storage nodes regularly; Based on the monitored health status, set a fault trigger threshold for each storage node; By combining the classification results of storage nodes with the fault trigger threshold, the importance of nodes to the business and the severity of faults are comprehensively evaluated, and a recovery priority is set for each storage node. Based on the recovery priority of each storage node, a recovery process is developed to address the occurrence of a failure.

[0030] Preferably, a global load balancer is configured, and the load status of each storage node is monitored in real time. The specific implementation steps for dynamically adjusting the node weights and traffic allocation of the global load balancer based on real-time data and access history are as follows: Configure a global load balancer and use it as the unified entry point for all client traffic; Collect data on the current number of requests, average response time, memory usage, network bandwidth utilization, and error rate of each storage node, and monitor the current load of each storage node in real time. Collect access history, combine current load and access history, and substitute into the following formula: ; ; Obtain the dynamic weights of each node. ; Based on the dynamic weights of each node Substitute into the following formula: ; Obtain the traffic allocation for storage node i ; In the formula, This represents the initial weight value of storage node i; Represents the initial weights of all storage nodes. A set; This represents the real-time load score of storage node i; n represents the total number of storage nodes participating in load balancing. This indicates the total traffic that needs to be allocated. Represents a minimal constant; This represents all storage nodes from node 1 to n; This indicates the access history factor.

[0031] Furthermore, it should be noted that, assuming there are three storage nodes A, B, and C, their initial weights are as follows: =10、 =20 and =30; their total weight is calculated to be 60; Calculate the dynamic weights of storage node A, storage node B, and storage node C respectively, taking into account the access history factor during the calculation of the dynamic weights. The calculation yields: =0.167、 =0.333 and =0.5; Use the dynamic weight of each storage node To calculate their flow allocation , Assuming total flow If the number of units is 1000, then for storage node A: Unit flow =0.167 × 1000 = 167 units of flow rate; For storage node B: Unit flow =0.333 × 1000 = 333 units of flow rate; For storage node C: Unit flow =0.500×1000=500 units of flow.

[0032] The traffic scheduling and dynamic weight adjustment module configures the global load balancer and monitors the load of each storage node in real time; based on real-time data and access history, it dynamically adjusts the node weights and traffic distribution of the global load balancer.

[0033] It needs to be explained that the role of a load balancer is to distribute requests reasonably among various nodes based on the load of each storage node; dynamically calculate the load of each node based on monitoring data; for example, when the CPU utilization of a node is too high, its weight will be reduced, thus reducing the traffic load on that node; the load balancer dynamically adjusts the weight of each node based on real-time monitoring data; for example, nodes with lighter loads can have their weight increased, thus receiving more traffic; while nodes with heavy loads will have their weight reduced, receiving less traffic.

[0034] Analyze the access history of each node to identify hotspots or nodes with frequent data requests. Combine this with access patterns to optimize traffic scheduling strategies. For example, for frequently accessed nodes, a higher weight can be set in the load balancer to ensure fast response.

[0035] The data classification and migration strategy module classifies data into hot data, cold data, and archived data based on the data access frequency and lifecycle, and formulates migration rules for data from hot storage to cold storage and then to archived storage in conjunction with the business continuity assurance mechanism.

[0036] It should be explained that the data classification and migration strategy module categorizes data into hot data, cold data, and archived data based on data access frequency and lifecycle, and defines corresponding storage and migration rules. Hot data, which is accessed frequently, is stored in a high-speed storage system; cold data, which is accessed less frequently, is stored in low-cost storage; and archived data is stored on low-cost media for a long time. By monitoring access frequency and setting lifecycle rules, hot data is automatically migrated to cold storage, and cold data is further migrated to archived storage, while ensuring a rapid recovery mechanism to guarantee business continuity.

[0037] Migration rules refer to strategies that automatically move data from one storage medium to another based on data access frequency and storage lifecycle. Specifically, hot data that has been frequently accessed will be automatically migrated to cold storage if it has not been accessed for a long time (e.g., 30 days). Cold data that has not been accessed for a long time (e.g., 180 days) will be migrated to archive storage to reduce storage costs while maintaining long-term data availability. Migration rules ensure that data can be managed and stored efficiently throughout its entire lifecycle.

[0038] Specific examples are illustrated below: Assume a tobacco company operates in multiple markets globally, including the United States, Europe, and the Asia-Pacific region. To ensure high data availability and business continuity, the tobacco company's data is distributed across storage nodes in different geographical regions. Master node: The master node, located in the United States, receives real-time sales and inventory data, and stores real-time sales information, inventory levels, and market demand information for tobacco products. Backup nodes: Backup nodes located in Europe and Asia Pacific will synchronize data with the US primary node in real time to ensure data redundancy. When the US primary node fails, the backup nodes can quickly take over and continue to process business. Sales data, inventory data, and historical market analysis data are all backed up using a cross-regional redundancy strategy to ensure that nodes in other regions can continue to support the business when a failure occurs in one region. Use a load balancer to ensure that users from different regions (such as the European and Asia-Pacific markets) can obtain data through the nearest node, reducing latency for cross-regional access; If the primary node in the United States is unable to respond to requests due to hardware failure, the backup node (such as a node located in Europe) will automatically take over the role of the primary node. Fault detection: By monitoring the health status of the primary node, faults can be detected in real time; when a fault is detected, the backup node is immediately promoted to the primary node and continues to receive sales data and inventory updates. Recovery Priority: Since US market data is crucial to the company's core business, the US primary node has the highest recovery priority; after the backup node takes over, the recovery process will be set to prioritize the recovery of US data. If sales in the US market increase, causing a surge in the load (CPU, memory, etc.) on US nodes, the load balancer will automatically reduce the weight of US nodes and allocate more traffic to lighter-loaded European or Asia-Pacific nodes. By analyzing past sales data access history, the system identifies certain sales peaks in the Asia-Pacific region. During these peak periods, the load balancer will increase the weight of Asia-Pacific nodes to ensure rapid response.

[0039] Tobacco companies process large amounts of historical data, such as market analyses and sales records from many years ago: 1. Hot data: Recent sales data and inventory information, which are frequently accessed and require rapid response, so this data will be stored in high-speed storage systems (such as SSDs) in the United States; 2. Cold data: such as market analysis data from the past three months, which is accessed infrequently but is occasionally used for data analysis. This data is stored in a low-cost storage system (such as an HDD). 3. Archived data: such as sales records from five years ago and historical market analysis data. This data is no longer accessed frequently but must be retained for a long time; it will be migrated to an archive storage system (such as tape or cloud storage).

[0040] The automatic migration rules are as follows: 1. Hot data migration: When sales data has not been accessed for a long time (e.g., more than 30 days), the system will automatically migrate this data to cold storage to reduce storage costs; 2. Cold data migration: If cold data has not been accessed for more than 180 days, the system will automatically migrate it to archive storage to ensure long-term data storage without affecting normal business operations; When a storage node in a certain region fails or becomes overloaded, the load balancer will adjust the traffic distribution based on real-time data to ensure efficient processing of data requests and prevent excessive load on a single region from affecting business operations.

[0041] The data access and service interaction module, based on migration rules, builds a unified data access interface, establishes application programming interfaces and microservice architecture, configures inter-service call relationships, integrates a real-time data synchronization mechanism, and establishes a data security and compliance management mechanism.

[0042] Preferably, the data access and service interaction module includes: The data access interface construction module is used to define the migration rules of data from hot storage to cold storage and then to archive storage, and to build a unified data access interface; The microservice architecture and interface interface module is used to establish the interface between various service modules and data access interfaces in a microservice architecture. The real-time data synchronization module, based on migration rules and configured with a real-time data synchronization mechanism, ensures that changes between hot data and cold data can be synchronized in a timely manner during the migration of data from hot storage to cold storage and archive storage. The data security and compliance module is used to configure identity authentication and access control, and to establish a data security and compliance management mechanism.

[0043] The data processing and trend prediction module is used to access real-time data streams and historical data warehouses synchronized with data security and compliance management mechanisms, establish a big data processing platform, combine streaming and batch processing frameworks to process real-time data streams and historical data warehouses respectively, perform trend analysis, identify early warning signals, and provide a visual display interface.

[0044] Preferably, the specific implementation steps for combining streaming and batch processing frameworks to process real-time data streams and historical data warehouses, and to perform trend analysis and future trend prediction are as follows: Real-time data is processed using a stream processing framework, and features are extracted from the processed real-time data to obtain real-time features. Historical data is processed using a batch processing platform, and features are extracted from the processed historical data to obtain a historical feature set. By fusing real-time feature sets with historical feature sets, a complete temporal feature set is obtained; Using deep feature prediction algorithms, we can perform trend analysis and future trend prediction on time series feature sets to identify abnormal fluctuation trends and abnormal behaviors. When abnormal fluctuations or behaviors appear in the prediction results, an early warning mechanism is automatically triggered, and a visual display interface is provided.

[0045] Preferably, the specific implementation steps for using a deep feature prediction algorithm to perform trend analysis and future trend prediction on a time-series feature set, and to identify abnormal fluctuation trends and abnormal behaviors, are as follows: We use a one-dimensional convolutional kernel to perform sliding computation on the input temporal feature set to extract local temporal features, and then use max pooling to reduce the dimensionality of the local temporal features. The dimensionality-reduced local temporal features are transformed into query, key, and value vectors through three linear transformations. By calculating the similarity between the query and the key, and using the softmax function for normalization, the attention weight corresponding to each position is obtained. The attention weight is then combined with the value vector in a weighted manner to obtain the attention-weighted feature sequence. By introducing a kernel function, the attention-weighted feature sequence is nonlinearly mapped from a low-dimensional space to a high-dimensional space, resulting in a high-dimensional mapped feature sequence. The high-dimensional mapped feature sequence is used as input to train the ridge regression model in the high-dimensional space, and L2 regularization is used to prevent overfitting. The trained ridge regression model is used to predict data at future time points, outputting predicted values. The predicted values ​​are then compared and analyzed with historical true values ​​to identify potential trend changes, abnormal fluctuations, or risk signals.

[0046] Preferably, the expression for predicting data at future time points using the trained ridge regression model is: ; In the formula, Indicates the current input features The predicted function value; This represents the regression coefficient corresponding to the nth training sample; Represents the kernel matrix; This represents the feature of the nth training sample; This represents the feature vector corresponding to the current time point t that needs to be predicted; n represents the nth training sample; m represents the training sample data.

[0047] It needs to be explained that, taking tobacco sales forecasting as an example: Objective: To predict tobacco sales for the coming week.

[0048] Input features include historical sales data, weather data, and the impact of holidays. Features that need to be predicted at the current moment (e.g., dates in the next week); Training data: Assume you have sales data from the past three days as the training set. The sales data for each day and related features constitute a training sample. The details are as follows:

[0049] Kernel function selection: Suppose we choose the RBF kernel function to measure the similarity between historical sales data and the current forecast point; Calculate similarity: for each training sample Calculate its relationship with the current time. The similarity, i.e., the calculation =606; =809; =142; Regression coefficients: The regression coefficients for each training sample are obtained from the training set data using a training algorithm (such as least squares method). , =[0.42, 0.35, 0.40]; Prediction: Substitute the regression coefficient and similarity of each training sample into the formula to obtain the final predicted sales volume. This is approximately equivalent to 857 packages, and since 857 is not significantly different from the actual sales in the last three days, it indicates that there is no significant fluctuation.

[0050] The second embodiment of this application also discloses a data storage management method based on cloud storage, characterized in that the method is based on a data storage management method based on cloud storage.

[0051] It should be noted that the calculation formulas and all parameters involved in the calculations in this application have been dimensionless beforehand. The process of dimensionless processing is well known in the industry and will not be described here.

[0052] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A cloud-based data storage management system, characterized in that, The data storage and management system includes: The data distribution and redundancy assurance module is used to distribute data across storage nodes in multiple geographical locations and establish a business continuity assurance mechanism based on cross-regional redundancy strategies. The data classification and migration strategy module classifies data into hot data, cold data, and archived data based on the data access frequency and lifecycle, and formulates migration rules for data from hot storage to cold storage and then to archived storage in conjunction with the business continuity assurance mechanism. The data access and service interaction module, based on migration rules, builds a unified data access interface, and sets up an application programming interface and microservice architecture, configures inter-service call relationships, integrates a real-time data synchronization mechanism, and establishes a data security and compliance management mechanism. The data processing and trend prediction module is used to access real-time data streams and historical data warehouses synchronized with data security and compliance management mechanisms, establish a big data processing platform, combine streaming and batch processing frameworks to process real-time data streams and historical data warehouses respectively, perform trend analysis, identify early warning signals, and provide a visual display interface.

2. The cloud-based data storage management system according to claim 1, characterized in that, The data distribution and redundancy protection module includes: The primary / backup architecture configuration module deploys distributed storage nodes in multiple geographical regions. For each storage region, a primary / backup architecture is configured. The primary / backup architecture includes a primary node and a backup node. The primary node is responsible for real-time data read and write operations, while the backup node is used for cross-regional data redundancy synchronization and can quickly take over read and write tasks when the primary node fails. The priority strategy formulation module sets up a disaster recovery priority strategy based on the cross-regional redundancy strategy, performs fault detection on each storage node, and sets priorities according to the importance and availability of different nodes. The traffic scheduling and dynamic weight adjustment module configures the global load balancer and monitors the load of each storage node in real time; based on real-time data and access history, it dynamically adjusts the node weights and traffic distribution of the global load balancer.

3. The cloud-based data storage management system according to claim 2, characterized in that, The cross-regional redundancy strategy, which sets a disaster recovery priority strategy, performs fault detection on each storage node and sets priorities based on the importance and availability of different nodes, includes: Storage nodes are categorized based on the business needs and data importance of the region where each storage node is located; Configure a fault monitoring mechanism for each classified storage node and monitor the health status of the storage nodes regularly; Based on the monitored health status, set a fault trigger threshold for each storage node; By combining the classification results of storage nodes with the fault trigger threshold, the importance of nodes to the business and the severity of faults are comprehensively evaluated, and a recovery priority is set for each storage node. Based on the recovery priority of each storage node, a recovery process is developed to address the occurrence of a failure.

4. The cloud-based data storage management system according to claim 3, characterized in that, The configuration of a global load balancer and real-time monitoring of the load on each storage node; dynamically adjusting the node weights and traffic allocation of the global load balancer based on real-time data and access history includes: Configure a global load balancer and use it as the unified entry point for all client traffic; Collect data on the current number of requests, average response time, memory usage, network bandwidth utilization, and error rate of each storage node, and monitor the current load of each storage node in real time. Collect access history, combine current load and access history, and substitute into the following formula: ； ； Obtain the dynamic weights of each node. ; Based on the dynamic weights of each node Substitute into the following formula: ； Obtain the traffic allocation for storage node i ; In the formula, This represents the initial weight value of storage node i; Represents the initial weights of all storage nodes. A set; This represents the real-time load score of storage node i; n represents the total number of storage nodes participating in load balancing. This indicates the total traffic that needs to be allocated at present; Represents a minimal constant; This represents all storage nodes from node 1 to n; This indicates the access history factor.

5. The cloud-based data storage management system according to claim 1, characterized in that, The data access and service interaction module includes: The data access interface construction module is used to define the migration rules of data from hot storage to cold storage and then to archive storage, and to build a unified data access interface; The microservice architecture and interface interface module is used to establish the interface between various service modules and data access interfaces in a microservice architecture. The real-time data synchronization module, based on migration rules and configured with a real-time data synchronization mechanism, ensures that changes between hot data and cold data can be synchronized in a timely manner during the migration of data from hot storage to cold storage and archive storage. The data security and compliance module is used to configure identity authentication and access control, and to establish a data security and compliance management mechanism.

6. The cloud-based data storage management system according to claim 1, characterized in that, The combined streaming and batch processing frameworks are used to process real-time data streams and historical data warehouses respectively, and to perform trend analysis and future trend prediction, including: Real-time data is processed using a stream processing framework, and features are extracted from the processed real-time data to obtain real-time features. Historical data is processed using a batch processing platform, and features are extracted from the processed historical data to obtain a historical feature set. By fusing real-time feature sets with historical feature sets, a complete temporal feature set is obtained; Using deep feature prediction algorithms, we can perform trend analysis and future trend prediction on time series feature sets to identify abnormal fluctuation trends and abnormal behaviors. When abnormal fluctuations or behaviors appear in the prediction results, an early warning mechanism is automatically triggered, and a visual display interface is provided.

7. The cloud-based data storage management system according to claim 6, characterized in that, The method of using a deep feature prediction algorithm to perform trend analysis and future trend prediction on a time series feature set, and to identify abnormal fluctuation trends and abnormal behaviors, includes: We use a one-dimensional convolutional kernel to perform sliding computation on the input temporal feature set to extract local temporal features, and then use max pooling to reduce the dimensionality of the local temporal features. The dimensionality-reduced local temporal features are transformed into query, key, and value vectors through three linear transformations. By calculating the similarity between the query and the key, and using the softmax function for normalization, the attention weight corresponding to each position is obtained. The attention weight is then combined with the value vector in a weighted manner to obtain the attention-weighted feature sequence. By introducing a kernel function, the attention-weighted feature sequence is nonlinearly mapped from a low-dimensional space to a high-dimensional space, resulting in a high-dimensional mapped feature sequence. The high-dimensional mapped feature sequence is used as input to train the ridge regression model in the high-dimensional space, and L2 regularization is used to prevent overfitting. The trained ridge regression model is used to predict data at future time points, outputting predicted values. The predicted values ​​are then compared and analyzed with historical true values ​​to identify potential trend changes, abnormal fluctuations, or risk signals.

8. The cloud-based data storage management system according to claim 7, characterized in that, The expression for predicting data at future time points using the trained ridge regression model is as follows: ； In the formula, Indicates the current input features The predicted function value; This represents the regression coefficient corresponding to the nth training sample; Represents the kernel matrix; This represents the feature of the nth training sample; This represents the feature vector corresponding to the time point t that needs to be predicted; n represents the nth training sample; and m represents the training sample data.

9. A data storage management method based on cloud storage, characterized in that, This method is implemented based on the cloud storage-based data storage management method described in any one of claims 1-8.

Images