Hybrid storage method based on deep learning public cloud storage and local storage

By extracting file features and relationships using deep learning technology, data temperature can be accurately predicted, solving the problem of data temperature differentiation in hybrid storage, reducing migration costs, and improving storage efficiency and user experience.

CN121301294BActive Publication Date: 2026-03-31BEIJING SHENDU SPACE TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-10-15
Publication Date
2026-03-31

AI Technical Summary

Technical Problem

In existing dynamic backup processes, it is difficult to effectively distinguish data temperature, resulting in high migration costs and suboptimal hybrid storage solutions.

Method used

By employing a deep learning-based approach, non-confidential files in public cloud storage and local storage are scanned to extract feature vectors and access timestamp sequences. The relationships between files are analyzed, and a data temperature prediction model is used to determine the data temperature value of the files. Based on the temperature value and migration cost, the files are rationally partitioned to achieve hybrid storage of public cloud storage and local storage.

Benefits of technology

It enables accurate data temperature prediction for different file types, reduces migration costs, improves the efficiency and user experience of hybrid storage, and ensures the rational use of local storage space.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121301294B_ABST
    Figure CN121301294B_ABST
Patent Text Reader

Abstract

The application provides a hybrid storage method of public cloud storage and local storage based on deep learning, comprising: scanning all non-secret files of the public cloud storage and the local storage, including images, videos, audios and texts; pre-processing and feature extraction are performed on each non-secret file to obtain a feature vector (including a file static attribute component, a file dynamic attribute component and a file content semantic component) and an access timestamp sequence of each non-secret file; based on the access timestamp sequence of each non-secret file, an inter-file association relationship corresponding to each non-secret file is analyzed; the feature vector and the inter-file association relationship of the non-secret file are input into a trained data temperature prediction model to obtain a data temperature value of the non-secret file; and according to the data temperature value of each non-secret file, division of the public cloud storage and the local storage is performed on each non-secret file to complete hybrid storage of all non-secret files.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data storage technology, and more specifically, to a hybrid storage method based on deep learning that combines public cloud storage and local storage. Background Technology

[0002] like Figure 1 As shown, the data temperature classification is mainly divided into frozen data, low temperature data, warm data (between low temperature data and high temperature data), high temperature data, and hot data.

[0003] Frozen data, also known as dead data or forgotten data, is data that has not been accessed for more than N days, data that is basically not used by users and businesses, and data that is not expected to be accessed in the future or has a very low probability of being accessed. However, this data usually needs to be stored for a long time and still occupies a certain amount of storage space, such as photos of deceased relatives and friends for many years, and server access logs that are more than 12 months old.

[0004] Low-temperature data, often referred to as cold data in the IT industry, refers to data with low usage rates. Depending on the scenario, it may not have been used for weeks or months. In certain cases, it may still have some value to users and businesses and may be used at some point in the future. Examples include birthday photos, holiday photos, and surveillance data, which may be reviewed once a year or once every few years. This data needs to be reliably stored and accessed within a certain period of time, so it can be stored on low-cost, high-capacity, and slow storage devices.

[0005] Warm data: This type of data falls between low-temperature and high-temperature data. In hybrid storage solutions, the storage allocation of this type of data is a challenge.

[0006] High-temperature data: This type of data is important and may be used intermittently, but the usage volume and frequency are not very frequent. Examples include important data such as medical records and identification documents of healthy individuals. This data needs to be reliably stored and accessed within a certain timeframe, and the decision to store it in low-cost cloud storage or on local high-speed storage devices should be made based on the actual situation.

[0007] Hot data refers to frequently used data that is important to users and business operations. This data typically requires fast and efficient access and is usually stored on high-speed, fast devices such as local NAS or servers. Examples include facial recognition data from attendance machines, recent photos, and data from popular movies and TV shows on the internet.

[0008] Storing low-temperature data on low-cost public cloud storage devices and high-temperature data on local storage devices in users' homes / offices can save costs and improve user experience.

[0009] The advantages of public cloud storage lie in its large storage capacity and low cost; it can be accessed anytime, anywhere with an internet connection; and for files shared by multiple users, only one physical file needs to be stored, with multiple users using the same mapping, which can greatly save storage space and costs. However, public cloud storage also has the following limitations: it requires an internet connection and a high network quality; data access is inconvenient when the network is poor, and playing ultra-high-definition videos will cause buffering; public clouds have content review requirements, and some documents, videos, and photos cannot be saved on cloud storage as they will be filtered out by the review mechanism; for enterprises, financial, budget, sales, and process data are confidential, and for security reasons, enterprises do not want to put them on the internet, and generally store them internally.

[0010] The advantages of private local storage are: it supports direct local access and localized network access with extremely fast speeds; data resides on your own local hard drive, eliminating privacy concerns and risks of confidentiality leaks and security breaches. However, private local storage also has its limitations: storage space is capped, typically ranging from a few terabytes to tens of terabytes; infrequently used data is stored on the hard drive, continuously occupying space even if it's not used for months or even years, and cannot be deleted when occasionally needed, incurring costs; and access speed is limited in an internet environment.

[0011] Therefore, a hybrid storage solution can combine the advantages of both, storing frequently used, private, and confidential data on private devices, while backing up infrequently used data and data without privacy or confidentiality requirements to the cloud; the data in the cloud and on-premises data are dynamically backed up to each other using certain methods to achieve the desired goal.

[0012] However, the current dynamic backup process has the following problems: how to distinguish data temperature, which data should be stored in the cloud and which should be stored locally, and how to reduce migration costs during migration. Summary of the Invention

[0013] The purpose of this application is to provide a hybrid storage method based on deep learning for public cloud storage and local storage, so as to effectively distinguish data temperature, reduce migration costs, realize the division of public cloud storage and local storage, and complete the hybrid storage of all non-confidential files.

[0014] To achieve the above objectives, the embodiments of this application are implemented in the following manner:

[0015] In a first aspect, embodiments of this application provide a hybrid storage method based on deep learning for public cloud storage and local storage, comprising: scanning all non-confidential files in public cloud storage and local storage, wherein the types of non-confidential files include images, videos, audio, and text; preprocessing and extracting features from each non-confidential file to obtain a feature vector and access timestamp sequence for each non-confidential file, wherein the feature vector of each non-confidential file includes static attribute components, dynamic attribute components, and semantic components of file content; analyzing the inter-file relationships corresponding to each non-confidential file based on the access timestamp sequence of each non-confidential file; inputting the feature vectors and inter-file relationships of the non-confidential files into a trained data temperature prediction model to obtain the data temperature value of the non-confidential files; and dividing each non-confidential file into public cloud storage and local storage according to the data temperature value of each non-confidential file, thereby completing the hybrid storage of all non-confidential files.

[0016] In conjunction with the first aspect, in the first possible implementation of the first aspect, preprocessing and feature extraction are performed on each non-confidential file to obtain the feature vector of each non-confidential file. This includes: performing abnormal data processing and data cleaning on each non-confidential file to complete preprocessing; for each preprocessed non-confidential file: extracting the file name, file path, file size, file type, file creation time, and file last modification time of the non-confidential file to generate the static attribute components of the non-confidential file; obtaining the scan timestamp and migration operation direction of each non-confidential file, and obtaining the access timestamp of each non-confidential file, and generating the dynamic attribute components of the non-confidential file based on the migration timestamp, migration operation direction, and access timestamp, wherein the dynamic attribute components of each non-confidential file are processed to maintain a consistent length; analyzing the content of the non-confidential file to determine the semantic components of the file content; and concatenating the static attribute components, dynamic attribute components, and semantic components of the file content of the non-confidential file to obtain the feature vector of the non-confidential file.

[0017] In conjunction with the first possible implementation of the first aspect, the second possible implementation of the first aspect involves analyzing the content of the non-confidential file to determine its semantic components. This includes: for non-confidential files of image type: using a pre-trained multimodal model to analyze the non-confidential file and extracting feature vectors representing the overall content of the image as semantic components of the file content; for non-confidential files of text type: using a pre-trained language model to convert the core content of the non-confidential file into sentence vectors representing semantics as semantic components of the file content; for non-confidential files of audio type: using the librosa library to extract Mel-frequency cepstral coefficient acoustic features as semantic components of the file content; for non-confidential files of video type: using ffmpeg to extract keyframes at fixed time intervals and analyzing the keyframes using a pre-trained multimodal model to form a feature sequence representing the core content of the video as semantic components of the file content.

[0018] In conjunction with the first aspect, in the third possible implementation of the first aspect, the file association relationship corresponding to each non-confidential file is analyzed based on the access timestamp sequence of each non-confidential file. This includes: obtaining the access timestamp sequence of each preprocessed non-confidential file; performing process mining based on the access timestamps to generate an association graph, wherein the topological structure of the association graph reveals the association strength between non-confidential files; and outputting the file association relationship corresponding to each non-confidential file based on the association graph, wherein the file association relationship corresponding to each non-confidential file is processed into a unified dimension.

[0019] In conjunction with the third possible implementation of the first aspect, in the fourth possible implementation of the first aspect, the feature vectors of unclassified files and the inter-file relationships are input into a trained data temperature prediction model to obtain the data temperature values ​​of unclassified files. This includes: inputting the feature vectors of unclassified files and the inter-file relationships into a trained data temperature prediction model; the data temperature prediction model determines the initial temperature value of unclassified files based on the feature vectors of unclassified files, and then determines the data temperature value of each unclassified file based on the initial temperature value of each unclassified file and the inter-file relationships.

[0020] In conjunction with the fourth possible implementation of the first aspect, in the fifth possible implementation of the first aspect, the data temperature prediction model includes an initial temperature prediction module and a data temperature calibration module. The initial temperature prediction module adopts a deep learning model, and the data temperature calibration module adopts a deep learning model or a traditional machine learning model.

[0021] In conjunction with the first aspect, in the sixth possible implementation of the first aspect, each unclassified file is divided into public cloud storage and local storage based on its data temperature value, thus completing the mixed storage of all unclassified files. This includes: based on a set threshold and the data temperature value of each unclassified file, performing a preliminary division of public cloud storage and local storage for each unclassified file; determining whether the remaining space in local storage meets the remaining space condition after migration according to the preliminary division migration plan, where the remaining space condition means that local storage retains at least a preset amount of remaining storage space; if the remaining space in local storage meets the remaining space condition, migrating each unclassified file to public cloud storage and local storage according to the preliminary division migration plan; if the remaining space in local storage does not meet the remaining space condition, performing a migration cost assessment based on the data temperature value and file size of each unclassified file; re-dividing each unclassified file into public cloud storage and local storage based on the migration cost and the remaining space condition in local storage; and migrating each unclassified file to public cloud storage and local storage according to the re-divided migration plan.

[0022] In conjunction with the sixth possible implementation of the first aspect, the seventh possible implementation of the first aspect involves a migration cost assessment based on the data temperature value and file size of each non-confidential file, including:

[0023] The overall migration cost of each non-confidential file stored locally is evaluated using the following methods:

[0024] ,

[0025] ,

[0026] ,

[0027] ,

[0028] in, This represents the overall migration cost when non-confidential files are stored locally. , , As weight, and , This represents the local storage cost of non-confidential files. The constant coefficients, For the probability of access, For file size, This represents the cost of migrating non-confidential files to local storage. The constant coefficients, This indicates that if the non-confidential file is currently located locally, This indicates that if the non-confidential file is currently located in the cloud, This represents the local access cost of a non-confidential file. The constant coefficients, Predicted access duration for non-classified files;

[0029] The overall migration cost of storing each non-confidential file in the cloud is evaluated using the following methods:

[0030] ,

[0031] ,

[0032] ,

[0033] ,

[0034] in, This represents the overall migration cost when non-confidential files are stored in the cloud. , , As weight, and , This indicates the cost of migrating non-confidential files to cloud storage. This indicates the cost of migrating to the cloud for accessing non-confidential files. The cost of accessing non-confidential files in the cloud.

[0035] In conjunction with the seventh possible implementation of the first aspect, the eighth possible implementation of the first aspect involves re-allocating each non-confidential file between public cloud storage and local storage based on the migration cost of each non-confidential file and the remaining space available in local storage. This includes:

[0036] Based on the overall migration cost of storing each non-confidential file locally or in the cloud, construct a cost minimization function:

[0037] ,

[0038] in, To minimize costs, , To plan the number of non-confidential files stored locally, This represents the total number of non-classified documents. To plan the number of non-confidential files stored in the cloud, To plan the storage of the first locally The overall migration cost of a non-confidential file To plan the storage of the first in the cloud The overall migration cost of a non-classified file;

[0039] Obtain the preset constraints:

[0040] ,

[0041] in, This indicates that non-confidential files are stored locally. Indicates the first Temperature values ​​of data in a non-confidential file. , This represents the total number of non-classified documents. To set a threshold;

[0042] ,

[0043] in, This refers to the total size of the local storage space. The size of the space occupied by the secret file. To plan the storage of the first locally The file size of a non-confidential file. The remaining storage space preset for local storage;

[0044] Based on minimizing the cost function and pre-defined constraints, an optimization solution is performed to determine a partitioning scheme for non-confidential files, namely public cloud storage and local storage.

[0045] In conjunction with the first aspect, in the ninth possible implementation of the first aspect, after completing the mixed storage of all non-confidential files, the method further includes: determining whether an index currently exists; if not, creating an index based on all non-confidential files in the mixed storage; if so, updating the index based on all non-confidential files in the mixed storage, wherein the index update includes updating the storage path of the non-confidential files, and the storage path associated with the same non-confidential file is only one of the cloud storage path and the local storage path at any given time.

[0046] Beneficial effects:

[0047] This solution scans all non-confidential files (including images, videos, audio, text, etc.) in the user's public cloud storage and local storage, performs preprocessing and feature extraction to obtain the feature vector (containing static attribute components, dynamic attribute components, and semantic components of file content) and access timestamp sequence for each non-confidential file. It then analyzes the inter-file relationships corresponding to each non-confidential file using the access timestamp sequence, and inputs the feature vectors and inter-file relationships of the non-confidential files into a data temperature prediction model to obtain the data temperature value of each non-confidential file. Based on the data temperature value of each non-confidential file, it divides each non-confidential file into public cloud storage and local storage, completing the hybrid storage of all non-confidential files. This solution can analyze the corresponding static attribute components (file name, file path, file size, file type, file creation time, file last modification time, etc.), dynamic attribute components (timestamp of each migration, migration operation direction, access timestamp, etc.) and semantic components of file content for different file types. Based on this, the extracted feature vectors can comprehensively reflect the characteristics of non-confidential files, which is conducive to accurately predicting the access probability (i.e., data temperature value) in the next cycle (the next cycle is the time interval between two normal scans, such as one week or two weeks), thereby better determining whether the file is stored locally or in the cloud.

[0048] Because some files are interconnected, when a user accesses a particular file, they may also access related non-confidential files (such as multiple photos or videos taken in a specific context). Therefore, when predicting the data temperature value of non-confidential files, the feature vectors of the non-confidential files and the relationships between them are input into a trained data temperature prediction model. Based on the feature vectors of the non-confidential files, the initial temperature value of each non-confidential file is determined. Then, based on the initial temperature value of each non-confidential file and the relationships between them, the data temperature value of each non-confidential file is determined. This effectively measures the influence between related files and more accurately assesses the access probability of each non-confidential file.

[0049] Based on a set threshold and the data temperature value of each non-classified file, each non-classified file is initially divided into public cloud storage and local storage. Depending on the remaining space, it is determined whether to migrate directly or after re-division. During the re-division process, this solution designs an overall migration cost as a standard to measure the migration cost. Different overall migration cost measurement methods are designed for non-classified files stored locally and stored in the cloud. When stored locally, factors such as access probability, file size, current storage location of the non-classified file, its location on the local machine, and the predicted access duration are considered. Local storage cost, local storage migration cost, and local access cost (essentially a reward, therefore negative) are used as measurement indicators to form the overall migration cost of non-classified files stored locally. When stored in the cloud, similarly, factors such as access probability, file size, current storage location of the non-classified file, its location on the local machine, and the predicted access duration are considered. Cloud storage migration cost, cloud access migration cost, and cloud access cost (essentially a penalty, therefore positive) are used as measurement indicators to form the overall migration cost of non-classified files stored in the cloud. Then, a cost minimization function is constructed, and combined with preset constraints (data temperature value constraints and local storage space constraints), an optimization solution is obtained to determine the partitioning scheme for public cloud storage and local storage of non-confidential files. This scheme, under the condition of limited local storage space, can store non-confidential files that are more suitable for local storage locally as much as possible, achieving a more reasonable hybrid storage. Furthermore, this scheme can minimize invalid migrations, improve the hybrid storage effect, and enhance the user experience.

[0050] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0051] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of temperature grading for the data.

[0053] Figure 2 A flowchart illustrating a hybrid storage method combining public cloud storage and local storage based on deep learning, provided in an embodiment of this application. Detailed Implementation

[0054] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0055] This embodiment provides a hybrid storage solution based on deep learning, combining public cloud storage and local storage. It is mainly used in home and office scenarios and is a hybrid storage solution that combines private local storage devices and public cloud storage.

[0056] Local storage devices, such as NAS (Network Attached Storage), are devices or systems that provide storage services over a network. NAS enables data sharing and remote access via a local area network or the internet, supports multiple devices simultaneously reading and writing files online, and is suitable for home, enterprise, and other scenarios.

[0057] In this scenario, this embodiment considers the characteristics of the files themselves and the access patterns of enterprise and home users. It designs a hybrid storage method based on deep learning, combining public cloud storage and local storage, to achieve efficient hybrid data storage. Because of the characteristics of some data (such as photos taken during the Spring Festival or travel photos, where user access to these files exhibits a certain cyclical pattern), even if some data remains unchanged, its "temperature" value may change due to altered objective conditions. Data with a low "temperature" value (H) may revert to high-temperature data, requiring migration back to local storage. For example, as time changes, families reuniting before the Spring Festival are likely to access and browse photos from last year, the year before, or even many years ago. This data might normally be considered low-temperature data, but its access activity increases dramatically before the Spring Festival. Traditional cold and hot backup methods are completely ineffective in handling this.

[0058] Therefore, when allocating files for local and cloud storage, this solution creates a file index after the initial scan (each file is associated with its storage path, which is either the file's local storage path or the cloud storage path), and then performs periodic (e.g., once a week, every Monday at 0:00:00, or every two weeks, or once a month) routine scans. Files that users have a high probability of accessing in the next cycle are stored locally, while files with a low probability of access are stored in the cloud, thus improving the user experience.

[0059] Please see Figure 2 , Figure 2 This is a flowchart illustrating a hybrid storage method based on deep learning for public cloud storage and local storage, provided in an embodiment of this application. The hybrid storage method, running on a server, includes steps S10, S20, S30, S40, and S50.

[0060] First, the server can run step S10.

[0061] Step S10: Scan all non-confidential files in public cloud storage and local storage. Non-confidential file types include images, videos, audio, and text.

[0062] In this embodiment, the server can scan all non-confidential files on public cloud storage and local storage corresponding to the user ID (essentially a unique ID for either a family or an enterprise) when scanning conditions are met (such as a basic scan required for the first authorization of hybrid storage after user login, or a preset periodic normalized scan condition, such as 00:00 every Monday or 00:00 on the 1st of every month; this embodiment takes a weekly normalized scan as an example). Here, non-confidential files refer to files that have not been encrypted by the user; all non-confidential files are within the scope of this solution, while encrypted files are not considered. Non-confidential file types include images, videos, audio, and text.

[0063] After scanning, the server can proceed to step S20.

[0064] Step S20: Preprocess and extract features for each non-confidential file to obtain the feature vector and access timestamp sequence of each non-confidential file. The feature vector of each non-confidential file includes static attribute components, dynamic attribute components, and semantic components of file content.

[0065] In this embodiment, the server can preprocess each non-confidential file. Preprocessing includes removing abnormal data, such as garbled filenames, abnormal dates (e.g., access time is in 1970, or a future time, such as access time being a future time, or file creation time being a future time), and file size being 0. Preprocessing also includes data cleaning. There are many cleaning tools available. This solution uses the Python Panda library as an example. The process is as follows: import data (df.info()), delete useless columns (df.drop(columns=['column name'])), handle missing and outlier values, standardize text and date formats, and export the cleaned data.

[0066] After data preprocessing, the server can perform feature extraction on each preprocessed non-confidential file. For each preprocessed non-confidential file:

[0067] The server can extract the filename, file path, file size, file type, file creation time, and file last modification time of non-classified files, generating static attribute components for the non-classified files. The length of the static attribute components for all non-classified files is uniform. For image files, if the image is a photograph, it also includes the photograph's EXIF ​​information (time of capture, location, altitude, etc.), extracting the corresponding EXIF ​​vector; otherwise, it defaults to a 0 vector of the same length as the EXIF ​​vector, thus maintaining consistent lengths for the static attribute components.

[0068] For file names, since text, video, and some audio types (such as music audio, whose file names are usually in Chinese, English, Japanese, or other mainstream text) generally have a certain ability to reflect the main content of the file in their file names, the file names are converted into fixed-length vectors. However, image files are usually named in various forms (such as names generated by the date and timestamp of the photo being taken, or strings, etc.), which have a weak ability to reflect the main content of the image and are not considered. A fixed-length default 0 vector is used as the feature quantity of the name part. In addition, some audio (such as telephone recordings, conference recordings, etc., whose file names are usually strings of numbers) are almost impossible to reflect the main content in their names, so a fixed-length default 0 vector is also used as the feature quantity of the name part.

[0069] The file path here only considers local or cloud (not specific path), and the feature value is generated in one-hot encoding form. The file type is also generated in one-hot encoding form.

[0070] File size is used as a numerical feature and is standardized using Z-score.

[0071] File creation time and last modification time are periodic features. If the months are simply treated as numbers from 1 to 12, the model will think that December and January are far apart, but in fact they are close together. Therefore, sine and cosine functions are used to map the periodic features onto a circle, thus preserving their periodicity.

[0072] The month feature can be converted into two new features:

[0073] month_sin = sin(2*π*month / 12), (1)

[0074] month_cos = cos(2*π*month / 12), (2)

[0075] In this way (because the year is also extracted as a feature), the model can understand that December of the current year and January of the following year are adjacent, which is crucial for accurately predicting seasonal or periodic file access.

[0076] Of course, there are also some derived features that can be extracted, such as the number of days between the file creation time and the current time, and the number of days between the file last modification time and the current time, which can be processed as numerical features.

[0077] By concatenating the extracted features in a fixed manner, static attribute components of a file can be formed, and the static attribute components of non-confidential files of various file types can maintain a uniform dimension.

[0078] To extract dynamic attribute components from files, the server can obtain the scan timestamps and migration operation directions (uploaded to the cloud or downloaded locally) for each non-confidential file, as well as the access timestamps for each non-confidential file. Then, the server can generate dynamic attribute components for non-confidential files based on the migration timestamps, migration operation directions, and access timestamps. To standardize the dimensions (i.e., the length of the feature components) of dynamic attribute components for different non-confidential files, the representation of migration and access counts needs to be fixed. For example, the migration count can represent the most recent first migration (including migration timestamp and migration direction) and the most recent 9 migrations (or other numbers, such as the most recent 19 or 99 migrations); the access count is also limited, for example, retaining the first access after each migration and the data from the most recent 9 accesses (access timestamps); of course, the access count can also directly retain a fixed number of recent accesses (e.g., the most recent 100 accesses). Based on this, fixed-length dynamic attribute components for files can be formed.

[0079] The features involved in extracting dynamic attribute components of a file mainly include numerical features (such as access counts and migration counts; the actual access counts and migration counts can be retained, but access timestamps and migration timestamps only retain the fixed number of timestamps agreed upon earlier, for a unified dimension), categorical features (such as migration direction), and periodic features (such as migration timestamps and access timestamps). Corresponding processing methods are used to extract these features, ultimately forming dynamic attribute components of the file with a unified dimension. Of course, there are also derived features, such as the number of days between each retained access timestamp and the current timestamp, the number of days between each retained access timestamp, the number of days between each retained migration timestamp and the current timestamp, and the number of days between each retained migration timestamp, etc., calculated based on the current timestamp.

[0080] Furthermore, the server can analyze the content of non-confidential files to determine the semantic components of the file content.

[0081] For non-confidential files containing images: a pre-trained multimodal model (such as the CLIP model) can be used to analyze the non-confidential file and extract feature vectors representing the overall content of the image as semantic components of the file content. This method can understand the abstract concepts of the image (such as "beach" or "meeting"), rather than just identifying isolated objects within it.

[0082] For non-confidential text files: a pre-trained language model (such as Sentence-BERT) can be used to convert the core content of the non-confidential file into sentence vectors representing semantics, which serve as semantic components of the file content. This method differs from simple keyword splitting and can better understand the deeper meaning of the text.

[0083] For non-encrypted audio files: Mel-frequency cepstral coefficients (MFCCs) acoustic features are extracted using the librosa library and used as semantic components of the file content. During feature extraction, global statistical aggregation is used to maintain a consistent dimension for the semantic components of the file content across audio outputs of different durations.

[0084] For non-confidential video files: ffmpeg is used to extract keyframes at fixed time intervals (e.g., one frame every 60 seconds, or other frequencies, such as one frame every 30 seconds). The keyframes are then analyzed using a pre-trained multimodal model (e.g., the CLIP model for image processing) to form a feature sequence representing the core content of the video, which serves as the semantic component of the file content.

[0085] It should be noted that, in order to ensure that the semantic components of the file content of unclassified files of different file types have a uniform length, a feature length needs to be agreed upon. The semantic components of the file content of unclassified files of each file type shall not exceed the maximum agreed length (those that do not reach the agreed length are padded with a 0 vector), thus maintaining a uniform dimension of the semantic components of the file content. The portions padded with 0 vectors need to be masked in subsequent analysis to ensure the accuracy of the analysis.

[0086] Then, the static attribute components, dynamic attribute components, and semantic components of the non-confidential file can be concatenated to obtain the feature vector of the non-confidential file (the feature vectors of non-confidential files of different file types also maintain a unified dimension, providing a basis for model analysis).

[0087] Furthermore, the server can retain a fixed number of access timestamps (such as the first access and the last 99 accesses) for each preprocessed non-confidential file, forming an access timestamp sequence. Additionally, it can retain the actual number of accesses (a numerical value) to reflect the access frequency of non-confidential files, forming an access timestamp sequence with access counts.

[0088] After obtaining the feature vector and access timestamp sequence for each non-secret file, the server can run step S30.

[0089] Step S30: Analyze the inter-file relationships corresponding to each non-confidential file based on the access timestamp sequence of each non-confidential file.

[0090] In this embodiment, the server can obtain the access timestamp sequence of each preprocessed non-confidential file, and then perform process mining based on the access timestamps to generate a correlation graph. The topological structure of the correlation graph reveals the correlation strength between non-confidential files. Based on the correlation graph, the file correlation relationship corresponding to each non-confidential file is output, wherein the file correlation relationship corresponding to each non-confidential file is processed into a unified dimension.

[0091] The association graph mining technique used here can utilize existing solutions (such as the Alpha algorithm in the ProM framework). This involves statistically analyzing the frequency of direct access to file B after file A in a file access sequence to uncover the associations between files. Then, using graph database technology (such as Neo4j), each non-confidential file is treated as a graph node. The association strength mined during the process is used as edge weights, and the statistical features of the timestamp sequence are used as node attributes to construct an association graph (if an existing association graph already exists, it can be updated). Finally, based on the association graph, the file associations corresponding to each non-confidential file are output (e.g., outputting the neighboring nodes directly connected to the node and their edge weights).

[0092] Accordingly, the server can proceed to step S40.

[0093] Step S40: Input the feature vectors of the non-confidential files and the relationships between files into the trained data temperature prediction model to obtain the data temperature values ​​of the non-confidential files.

[0094] In this embodiment, the server can input the feature vectors of non-confidential files and the relationships between files into a trained data temperature prediction model. The data temperature prediction model determines the initial temperature value of non-confidential files based on their feature vectors, and then determines the data temperature value of each non-confidential file based on its initial temperature value and the relationships between files.

[0095] To facilitate understanding of this scheme, the construction and training of the data temperature prediction model will be explained first.

[0096] First, the data temperature prediction model is designed to include two modules: an initial temperature prediction module and a data temperature calibration module. The initial temperature prediction module uses a deep learning model (LSTM is used as an example in this embodiment, but it is not limited to this), while the data temperature calibration module uses a deep learning model (such as MLP) or a traditional machine learning model (such as linear regression). The initial temperature prediction module is mainly used to determine the initial temperature value of non-confidential files based on the feature vectors of the non-confidential files, while the data temperature calibration module is used to determine the data temperature value of each non-confidential file based on the initial temperature value of each non-confidential file and the correlation between files.

[0097] For example, when using an MLP (Multilayer Perceptron) as a data temperature calibration module, it is necessary to limit the fixed input dimension. This can be achieved by aggregating the features of associated nodes (i.e., neighboring nodes) (such as the number of associated nodes, mean of association strength, peak of association strength, valley of association strength, mean of initial temperature of associated nodes, range of initial temperature of associated nodes, variance of initial temperature of associated nodes, etc.) to maintain the uniformity of the input.

[0098] When using linear regression as the data temperature calibration module, the input dimension can be fixed (e.g., a maximum of 100 neighbor nodes, with zeros used for nodes less than 100), and regression learning can be performed using this approach.

[0099] , (3)

[0100] , (4)

[0101] in, For the data temperature value, and As weight, This is the initial temperature value. Therefore, the number of neighboring nodes directly associated with non-confidential data. Therefore, the first non-classified data associated with The weight function of each neighbor node For this non-classified data and its associated third The strength of association between neighboring nodes Based on linear functions, Therefore, the first non-classified data associated with The initial temperature values ​​of the neighboring nodes, This is a bias term.

[0102] This embodiment preferably uses an MLP scheme, which can well balance performance and complexity, but it is not intended to limit this application.

[0103] After establishing the model architecture accordingly, data from the most recent M (M ≥ 12) months is collected, including the following: all non-confidential data stored on local storage devices, all non-confidential data stored in the public cloud, all non-confidential data migrated from local storage devices to the public cloud, all non-confidential data migrated from the public cloud to local storage devices, all non-confidential data in local storage devices that has never been used, and all non-confidential data in the public cloud that has never been used. Additionally, user access data from the most recent M (M ≥ 12) months is collected, including the following: all non-confidential data used on local storage devices and all non-confidential data used in the public cloud.

[0104] After data preprocessing (see the data preprocessing process above), the data is divided according to the scanning period (since the time period of each non-confidential data is relatively long, and the scanning period of this scheme is agreed upon, such as one week). Accordingly, each data can generate multiple training data that can be used for model training. For example, if the scanning period is one week, and assuming that a certain non-confidential data has existed for 24 months, then the data of the first week can be used as the historical segment, and whether the non-confidential data was accessed in the second week can be used as the label. Based on this, feature extraction is performed to generate one training data. Similarly, the data of the first two weeks can be used as the historical segment, and whether the non-confidential data was accessed in the third week can be used as the label to generate another training data, and so on. One data can generate multiple training data. After constructing the training dataset (dividing the training, validation, and test sets in a 7:1.5:1.5 ratio) using this method, the model is trained. The loss function is measured using binary cross-entropy, with AUC (Area Under Curve) as the core metric. AUC measures the model's overall ability to prioritize "files that will be accessed in the future" over "files that will not be accessed in the future." A UC value closer to 1 indicates better model performance. To prevent overfitting, the model's performance on the validation set is continuously monitored during training. If performance no longer improves after several consecutive iterations (e.g., 3, 5, etc.), training is automatically terminated, and the model state at its best performance point is saved.

[0105] Once the model training is complete, it can be deployed on a server to provide support for the operation of this method.

[0106] Accordingly, the server can input the feature vectors of non-confidential files and the relationships between files into the trained data temperature prediction model, and then determine and output the data temperature value of each non-confidential file through the data temperature prediction model.

[0107] After obtaining the data temperature value for each non-confidential file, the server can run step S50.

[0108] Step S50: Based on the data temperature value of each non-confidential file, divide each non-confidential file into public cloud storage and local storage to complete the mixed storage of all non-confidential files.

[0109] In this embodiment, the server can divide each non-confidential file into public cloud storage and local storage based on the data temperature value of each non-confidential file.

[0110] For example, the server can perform an initial division of each non-confidential file into public cloud storage and local storage based on a set threshold and the data temperature value of each non-confidential file.

[0111] The initial overall strategy is as follows: data with no change in temperature value or whose change range does not exceed the stratification (the stratification is designed as follows: less than 0.3 as one stratum, 0.3-0.7 as one stratum, and greater than 0.7 as one stratum) will not be stored in a different location.

[0112] When the data temperature value H > 0.7, if the storage location was originally in the public cloud, it will be migrated to local storage; otherwise, no adjustment is needed.

[0113] When the temperature drops from high to low and the data temperature value H < 0.3, if the file is stored locally, it will be migrated to public cloud storage.

[0114] For medium-temperature data with a temperature value of 0.7 ≥ H ≥ 0.3, if the temperature changes but does not exceed this range, the storage location does not need to be changed, minimizing the number of file migrations.

[0115] Based on the initial overall strategy, the server can estimate whether the remaining space of the local storage meets the remaining space condition after the migration is carried out according to the initial migration plan. The remaining space condition means that the local storage retains at least a preset remaining storage space (e.g., 10%, or a fixed space size).

[0116] If the remaining space in local storage meets the remaining space requirements, each non-classified file will be migrated to public cloud storage and local storage according to the preliminary migration plan, thus completing the mixed storage of non-classified files.

[0117] If the remaining space in local storage does not meet the remaining space requirement, a migration cost assessment needs to be conducted based on the data temperature value and file size of each non-confidential file.

[0118] For example, the server can evaluate the overall migration cost of each non-confidential file when it is stored locally in the following ways:

[0119] , (5)

[0120] , (6)

[0121] , (7)

[0122] , (8)

[0123] in, This represents the overall migration cost when non-confidential files are stored locally. , , As weight, and , This represents the local storage cost of non-confidential files. The constant coefficients, For the probability of access, For file size, This represents the cost of migrating non-confidential files to local storage. The constant coefficients, This indicates that if the non-confidential file is currently located locally, This indicates that if the non-confidential file is currently located in the cloud, This represents the local access cost of a non-confidential file. The constant coefficients, This represents the predicted access duration for unclassified files. Here, the local access cost of an unclassified file is essentially a reward, hence the negative sign. The predicted access duration for unclassified files is set, for example, 10 seconds for photos, 60 seconds for text (this allocation is rather rough because the reward for text access is not very high; downloading is a very quick process, so it won't be set too high). Audio is calculated based on audio duration, and video is calculated based on video duration. The predicted access duration for unclassified files must be in seconds to avoid issues related to unit differences.

[0124] Furthermore, the server can assess the overall migration cost of each non-confidential file when it is stored in the cloud using the following methods:

[0125] , (9)

[0126] , (10)

[0127] , (11)

[0128] , (12)

[0129] in, This represents the overall migration cost when non-confidential files are stored in the cloud. , , As weight, and , This indicates the cost of migrating non-confidential files to cloud storage. This indicates the cost of migrating to the cloud for accessing non-confidential files. The cloud access cost for non-confidential files. Here, the cloud access cost for non-confidential files is essentially a penalty, hence the positive sign. Additionally, the coefficient 1.2 in the cloud storage migration cost considers the difference between uploading and downloading, thus resulting in different values. This issue can also be ignored here; the coefficient 1.2 can be directly removed.

[0130] Based on this, after completing the overall migration cost assessment for each non-classified file (the overall migration cost of storing each non-classified file locally and in the cloud will be assessed), the server can re-divide each non-classified file into public cloud storage and local storage based on the migration cost of each non-classified file and the remaining space available in local storage.

[0131] Specifically, the server can construct a cost-minimizing function based on the overall migration cost of storing each non-confidential file locally or in the cloud:

[0132] , (13)

[0133] in, To minimize costs, , To plan the number of non-confidential files stored locally, This represents the total number of non-classified documents. To plan the number of non-confidential files stored in the cloud, To plan the storage of the first locally The overall migration cost of a non-confidential file To plan the storage of the first in the cloud The overall migration cost of a non-confidential file.

[0134] Obtain the preset constraints:

[0135] , (14)

[0136] in, The condition indicating that non-confidential files are stored locally (i.e., It must not be lower than the set threshold. (Only then can it be stored locally) Indicates the first Temperature values ​​of data in a non-confidential file. , This represents the total number of non-classified documents. Set a threshold (e.g., 0.5).

[0137] ,

[0138] in, This refers to the total size of the local storage space. The size of the space occupied by the secret file. To plan the storage of the first locally The file size of a non-confidential file. The preset remaining storage space for local storage. This condition is a constraint on the remaining space of local storage.

[0139] Then, the server can perform optimization (e.g., greedy algorithm) based on minimizing the cost function and preset constraints (all of which are hard constraints and must be met) to determine the partitioning scheme for public cloud storage and local storage of non-confidential files, and migrate each non-confidential file to public cloud storage and local storage according to the repartitioned migration scheme.

[0140] Of course, if the remaining space in local storage does not meet the remaining space condition, you can directly follow the traditional solution (sort the data according to the size of the temperature value, take the data with the highest temperature value that meets the remaining space condition and store it locally, and store the rest in the cloud), but this solution has a large migration cost each time.

[0141] After the migration of non-confidential data is completed, the server can determine whether an index exists (an index is usually created after the initial low-level scan).

[0142] If the file does not exist (indicating this is likely the first scan), the server needs to create an index based on all non-classified files in the hybrid storage. This creates a system-level index for easy user access and querying, with each non-classified file index associated with its specific storage path (whether it's stored locally or in the cloud).

[0143] If present, the server can update the index based on all non-confidential files in the hybrid storage. The index update includes updating the storage path of the non-confidential files. At any given time, the storage path associated with the same non-confidential file is only one of the cloud storage path and the local storage path. The index update also includes situations such as the addition and deletion of non-confidential files.

[0144] Of course, the system also has functions such as storage space monitoring and reminders, which can prompt users to expand when the remaining storage space is insufficient.

[0145] In summary, this application provides a hybrid storage method based on deep learning for public cloud storage and local storage. It scans all non-confidential files (including images, videos, audio, text, etc.) in the user's public cloud and local storage, performs preprocessing and feature extraction to obtain the feature vector (containing static attribute components, dynamic attribute components, and semantic components of file content) and access timestamp sequence for each non-confidential file. The access timestamp sequence of each non-confidential file is used to analyze the inter-file relationships corresponding to each non-confidential file. The feature vectors and inter-file relationships of the non-confidential files are then input into a data temperature prediction model to obtain the data temperature value of each non-confidential file. Based on the data temperature value of each non-confidential file, each non-confidential file is divided into public cloud storage and local storage, thus completing the hybrid storage of all non-confidential files. This solution can analyze the corresponding static attribute components (file name, file path, file size, file type, file creation time, file last modification time, etc.), dynamic attribute components (timestamp of each migration, migration operation direction, access timestamp, etc.) and semantic components of file content for different file types. Based on this, the extracted feature vectors can comprehensively reflect the characteristics of non-confidential files, which is conducive to accurately predicting the access probability (i.e., data temperature value) in the next cycle (the next cycle is the time interval between two normal scans, such as one week or two weeks), thereby better determining whether the file is stored locally or in the cloud.

[0146] Because some files are interconnected, when a user accesses a particular file, they may also access related non-confidential files (such as multiple photos or videos taken in a specific context). Therefore, when predicting the data temperature value of non-confidential files, the feature vectors of the non-confidential files and the relationships between them are input into a trained data temperature prediction model. Based on the feature vectors of the non-confidential files, the initial temperature value of each non-confidential file is determined. Then, based on the initial temperature value of each non-confidential file and the relationships between them, the data temperature value of each non-confidential file is determined. This effectively measures the influence between related files and more accurately assesses the access probability of each non-confidential file.

[0147] Based on a set threshold and the data temperature value of each non-classified file, each non-classified file is initially divided into public cloud storage and local storage. Depending on the remaining space, it is determined whether to migrate directly or after re-division. During the re-division process, this solution designs an overall migration cost as a standard to measure the migration cost. Different overall migration cost measurement methods are designed for non-classified files stored locally and stored in the cloud. When stored locally, factors such as access probability, file size, current storage location of the non-classified file, its location on the local machine, and the predicted access duration are considered. Local storage cost, local storage migration cost, and local access cost (essentially a reward, therefore negative) are used as measurement indicators to form the overall migration cost of non-classified files stored locally. When stored in the cloud, similarly, factors such as access probability, file size, current storage location of the non-classified file, its location on the local machine, and the predicted access duration are considered. Cloud storage migration cost, cloud access migration cost, and cloud access cost (essentially a penalty, therefore positive) are used as measurement indicators to form the overall migration cost of non-classified files stored in the cloud. Then, a cost minimization function is constructed, and combined with preset constraints (data temperature value constraints and local storage space constraints), an optimization solution is obtained to determine the partitioning scheme for public cloud storage and local storage of non-confidential files. This scheme, under the condition of limited local storage space, can store non-confidential files that are more suitable for local storage locally as much as possible, achieving a more reasonable hybrid storage. Furthermore, this scheme can minimize invalid migrations, improve the hybrid storage effect, and enhance the user experience.

[0148] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0149] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A hybrid storage method based on deep learning of public cloud storage and local storage, characterized in that, The method comprises the following steps: Scanning all non-secret files of public cloud storage and local storage, wherein the non-secret file types include images, videos, audios, and texts; Preprocessing and feature extraction are performed on each non-secret file to obtain a feature vector and an access timestamp sequence of each non-secret file, wherein the feature vector of each non-secret file comprises a file static attribute component, a file dynamic attribute component, and a file content semantic component; Based on the access timestamp sequence of each non-secret file, the inter-file association relationship corresponding to each non-secret file is analyzed; The feature vector and the inter-file association relationship of the non-secret file are input into a trained data temperature prediction model to obtain a data temperature value of the non-secret file; According to the data temperature value of each non-secret file, the public cloud storage and the local storage of each non-secret file are divided, and the hybrid storage of all non-secret files is completed; Preprocessing and feature extraction are performed on each non-secret file to obtain a feature vector of each non-secret file, comprising: Abnormal data processing and data cleaning are performed on each non-secret file to complete preprocessing; For each non-secret file after preprocessing: The file name, file path, file size, file type, file creation time, and file last modification time of the non-secret file are extracted to generate the file static attribute component of the non-secret file, wherein the file creation time and the file last modification time are periodic features, which are mapped to a circle using a sine and cosine function to retain their periodicity, and the month feature is converted into two new features: month_sin=sin(2*π*month / 12) and month_cos=cos(2*π*month / 12); The scanning timestamp, migration operation direction, and access timestamp of each non-secret file are obtained, and based on the migration timestamp, migration operation direction, and access timestamp, the file dynamic attribute component of the non-secret file is generated, wherein the file dynamic attribute component of each non-secret file is processed to have a consistent length; The content of the non-secret file is analyzed to determine the file content semantic component of the non-secret file; The file static attribute component, file dynamic attribute component, and file content semantic component of the non-secret file are spliced to obtain the feature vector of the non-secret file; The feature vector and the inter-file association relationship of the non-secret file are input into a trained data temperature prediction model to obtain a data temperature value of the non-secret file, comprising: The feature vector and the inter-file association relationship of the non-secret file are input into a trained data temperature prediction model; The data temperature prediction model determines an initial temperature value of the non-secret file based on the feature vector of the non-secret file, and determines a data temperature value of each non-secret file based on the initial temperature value of each non-secret file and the inter-file association relationship; The data temperature prediction model comprises an initial temperature prediction module and a data temperature calibration module, the initial temperature prediction module adopts a deep learning model, and the data temperature calibration module adopts a deep learning model or a traditional machine learning model, wherein, when the MLP is used as the data temperature calibration module, the fixed input dimension is limited, and the features of the associated nodes are aggregated to maintain the uniformity of the input, and the features of the associated nodes include the number of associated nodes, the average associated strength, the peak associated strength, the valley associated strength, the average initial temperature of the associated nodes, the initial temperature range of the associated nodes, and the initial temperature variance of the associated nodes; when the linear regression is used as the data temperature calibration module, the fixed input dimension is used for regression learning: , , wherein, is a data temperature value, and is a weight, is an initial temperature value, is a number of neighbor nodes directly associated with this non-secret data, is a weight function for the th neighbor node associated with this non-secret data, is a strength of association between this non-secret data and the th neighbor node associated with this non-secret data, is a linear function based on, is an initial temperature value for the th neighbor node associated with this non-secret data, is a bias term. 2.The hybrid storage method of deep learning based public cloud storage and local storage according to claim 1, characterized in that, The content of the non-secret file is analyzed to determine the file content semantic component of the non-secret file, including: For a non-secret file of the image type: a pre-trained multi-modal model is used to analyze the non-secret file to extract a feature vector representing the overall content of the image as the file content semantic component; For a non-secret file of the text type: a pre-trained language model is used to convert the core content of the non-secret file into a sentence vector representing the semantics as the file content semantic component; For a non-secret file of the audio type: the librosa library is used to extract the mel-frequency cepstrum coefficient acoustic feature as the file content semantic component; For a non-secret file of the video type: the ffmpeg is used to extract key frames at a fixed time interval, and a pre-trained multi-modal model is used to analyze the key frames to form a feature sequence representing the core content of the video as the file content semantic component. 3.The hybrid storage method of deep learning based public cloud storage and local storage according to claim 1, wherein, Based on the access timestamp sequence of each non-secret file, the inter-file association relationship corresponding to each non-secret file is analyzed, including: The access timestamp sequence of each pre-processed non-secret file is obtained; Process mining is performed based on the access timestamp to generate an association graph, wherein the topological structure of the association graph reveals the association strength between the non-secret files; Based on the association graph, the inter-file association relationship corresponding to each non-secret file is output, wherein the inter-file association relationship corresponding to each non-secret file is processed into a uniform dimension. 4.The hybrid storage method of deep learning based public cloud storage and local storage according to claim 1, wherein, According to the data temperature value of each non-secret file, the division of public cloud storage and local storage for each non-secret file is performed to complete the hybrid storage of all non-secret files, including: Based on the set threshold and the data temperature value of each non-secret file, the preliminary division of public cloud storage and local storage for each non-secret file is performed, and it is judged whether the remaining space of the local storage after the migration according to the preliminary division meets the remaining space condition, wherein the remaining space condition means that the local storage retains at least a preset remaining storage space; If the remaining space of the local storage meets the remaining space condition, the migration of public cloud storage and local storage for each non-secret file is performed according to the preliminary division. If the remaining space of the local storage does not satisfy the remaining space condition, migration cost evaluation is performed based on the data temperature value of each non-encrypted file and the file size of the non-encrypted file, and each non-encrypted file is re-divided into the public cloud storage and the local storage based on the migration cost of each non-encrypted file and the remaining space condition of the local storage, and each non-encrypted file is migrated to the public cloud storage and the local storage according to the re-divided migration scheme. 5.The hybrid storage method of deep learning based public cloud storage and local storage according to claim 4, characterized in that, The migration cost evaluation based on the data temperature value of each non-encrypted file and the file size of the non-encrypted file includes: The overall migration cost of each non-encrypted file when stored locally is evaluated in the following manner: , , , , wherein, represents the overall migration cost when the non-secret file is stored locally, , , is a weight, and , represents the local storage cost of the non-secret file, is a constant coefficient, is a probability of access, is a file size, represents the local storage migration cost of the non-secret file, is a constant coefficient, represents that the non-secret file is currently located locally, represents that the non-secret file is currently located in the cloud, represents the local access cost of the non-secret file, is a constant coefficient, is a predicted access duration of the non-secret file; The overall migration cost of each non-encrypted file when stored in the cloud is evaluated in the following manner: , , , , wherein, represents the overall migration cost of non-confidential files when stored in the cloud, , , is a weight, and , represents the cloud storage migration cost of non-confidential files, represents the cloud access migration cost of non-confidential files, cloud access cost of non-confidential files. 6.The hybrid storage method of public cloud storage and local storage based on deep learning according to claim 5, characterized in that, The re-division of each non-encrypted file into the public cloud storage and the local storage based on the migration cost of each non-encrypted file and the remaining space condition of the local storage includes: Based on the overall migration cost of each non-encrypted file when stored locally or in the cloud, a minimum cost function is constructed: , wherein, to minimize the cost, , is the number of non-encrypted files stored locally, is the total number of non-encrypted files, is the number of non-encrypted files stored in the cloud, is the overall migration cost of the th non-encrypted file stored locally, is the overall migration cost of the th non-encrypted file stored in the cloud; The preset constraint condition is obtained: , wherein, represents the condition that the non-secret file is stored locally, represents the data temperature value of the th non-secret file, , is the total amount of non-secret files, is the set threshold value; , wherein, is a total size of the local storage, is a size of the space occupied by the secret file, is a file size of the first non-secret file scheduled to be stored locally, is a file size of the first non-secret file scheduled to be stored locally, is a remaining storage space of the local storage preset. Based on the minimum cost function and the preset constraint condition, optimization is performed to determine the division scheme of the non-encrypted file into the public cloud storage and the local storage. 7.The hybrid storage method of deep learning based public cloud storage and local storage according to claim 1, characterized in that, After the mixed storage of all non-encrypted files is completed, the method further includes: Determine whether there is an index currently; If not, create an index based on all non-encrypted files stored in a mixed manner; If there is, update the index based on all non-encrypted files stored in a mixed manner, wherein the update of the index includes the update of the storage path of the non-encrypted file, and the storage path associated with the same non-encrypted file is only one of the cloud storage path and the local storage path at the same time.

Citation Information

Patent Citations

  • File storage processing method and system

    CN106202070A

  • Multi-level cloud storage cluster construction and data storage method

    CN120162003A

  • Terminal data secure transmission method and system for cloud computer

    CN120475023A