A data retrieval method and system based on cloud storage
By employing hierarchical indexing and encrypted data retrieval methods, the problems of low retrieval efficiency and insufficient security in cloud storage are solved, achieving efficient and secure data retrieval.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIAN HUAQI INTERACTIVE INFORMATION TECHNOLOGY CO LTD
- Filing Date
- 2026-05-06
- Publication Date
- 2026-07-31
AI Technical Summary
Existing cloud storage-based data retrieval methods suffer from long query times, low retrieval efficiency, lack of access control and data encryption mechanisms, leading to security risks, and low retrieval accuracy.
A hierarchical index structure is adopted, including a first-level global index and a second-level local index. Combined with encryption processing and permission verification in data preprocessing, the matching degree is calculated using the cosine similarity algorithm, and the retrieval logs are recorded and anonymized.
It significantly improves search efficiency and accuracy, prevents data leaks and unauthorized searches, and protects data and user privacy.
Smart Images
Figure CN122489623A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a data retrieval method and system based on cloud storage, belonging to the field of cloud storage technology. Background Technology
[0002] With the rapid development of cloud computing technology, cloud storage has been widely used in various fields such as enterprise office, medical and health care, financial services, scientific research and education due to its advantages such as large capacity, low cost, strong scalability and convenient access. Massive amounts of structured data (such as tables and database records) and unstructured data (such as text, images, videos and audio) are continuously stored on cloud storage platforms, and the data scale is showing an exponential growth trend.
[0003] Existing cloud storage-based data retrieval methods and systems can generally meet normal usage needs. However, with the continuous growth of stored data volume and the increasing diversification of data types, the following technical shortcomings have gradually emerged: Traditional retrieval often uses a single index structure, which takes a long time to query when the data volume is large, and retrieval efficiency needs to be improved. At the same time, the lack of effective access control and data encryption mechanisms during the retrieval process makes it easy for data leakage, unauthorized retrieval, and other security risks to occur, failing to guarantee data and user privacy security. In addition, traditional retrieval is not accurate enough in calculating the matching degree of search keywords, which easily leads to false or missed searches, and the accuracy of retrieval needs to be improved. Based on this, the present invention provides a cloud storage-based data retrieval method and system. Summary of the Invention
[0004] The technical problems to be solved by this invention are: queries using a single index are time-consuming, retrieval efficiency needs to be improved, there is a lack of effective access control and data encryption mechanisms, data and user privacy security cannot be guaranteed, the matching degree calculation of search keywords is not accurate enough, and the retrieval accuracy needs to be improved.
[0005] To address the aforementioned problems, this invention provides a data retrieval method based on cloud storage. The proposed technical solution includes the following steps:
[0006] S1: Data preprocessing: Obtain the data to be stored in the cloud storage node, classify and encode the data, extract the core features of the data, generate feature labels, and encrypt the data to obtain encrypted data;
[0007] S2: Index Construction: Construct a hierarchical index based on feature tags. The hierarchical index includes a first-level global index and a second-level local index. The first-level global index is associated with the feature tag set of all cloud storage nodes, and the second-level local index corresponds to the data feature tags within a single cloud storage node. The hierarchical index is stored in the index library of the cloud server.
[0008] S3: Retrieval Request Reception and Parsing: Receive retrieval requests sent by users, parse the keywords in the retrieval request, extract the target features corresponding to the retrieval keywords, verify the permissions of the retrieval request, and proceed to the next step if the verification is successful; otherwise, the retrieval is rejected.
[0009] S4: Index Matching and Data Location: Based on the target features, query the first-level global index to determine the matching cloud storage node, and then locate the target data within the node through the second-level local index. Calculate the matching degree between the target features and the data feature labels, and filter out candidate data with a matching degree higher than the preset threshold.
[0010] S5: Data Decryption and Feedback: Call the decryption algorithm corresponding to the encryption to decrypt the candidate data, sort them from high to low matching degree, and feed the sorted search results back to the user. At the same time, record the search log, which includes search keywords, search time, search results and user information.
[0011] Furthermore, in step S1, the core features include data type, creation time, keywords, and data size. The feature tags are stored in key-value pair format. The encryption process uses a symmetric encryption algorithm. The encryption key is managed uniformly by the cloud server.
[0012] Furthermore, in step S2, the hierarchical index adopts a tree structure, the nodes of the first-level global index correspond to cloud storage nodes, the nodes of the second-level local index correspond to specific data within a single cloud storage node, and the index database adopts distributed storage that is updated synchronously with the cloud storage nodes.
[0013] Furthermore, in step S3, the permission verification includes user authentication and retrieval permission verification. The user authentication is achieved through account password, dynamic verification code or biometric identification. The retrieval permission is divided according to user level, with different levels of users corresponding to different retrieval ranges.
[0014] Furthermore, in step S4, the calculation uses a cosine similarity algorithm, and the preset threshold can be dynamically adjusted according to user needs or system performance. The range of the preset threshold is 0.6-0.9. If no matching candidate data is found, a no-match result prompt is returned, and relevant search keywords are recommended.
[0015] Furthermore, in step S5, the search logs are periodically backed up to cloud storage nodes, with the backup cycle set to 1-7 days. Users can query their own search history, and the cloud server performs anonymization processing on the search logs to protect user privacy.
[0016] A cloud storage-based data retrieval system, used to implement the aforementioned cloud storage data retrieval method, includes:
[0017] Data preprocessing module: used for classifying, encoding, extracting features, generating tags, and encrypting cloud storage data;
[0018] Index building module: used to build hierarchical indexes, which include a first-level global index and a second-level local index, and realize the storage, updating and maintenance of the indexes;
[0019] Search request processing module: used to receive user search requests, parse keywords, extract target features, and perform search permission verification;
[0020] Index matching module: used to match hierarchical indexes based on target features, locate target data, calculate matching degree and filter candidate data;
[0021] Data decryption and feedback module: used to decrypt candidate data, sort it, and then send the search results back to the user terminal, and record and back up the search log;
[0022] Cloud storage nodes: used to store encrypted data and retrieve log backups;
[0023] The cloud server is electrically connected to the data preprocessing module, index building module, retrieval request processing module, index matching module, data decryption and feedback module, and cloud storage node, respectively. It is used to manage the index database, keys, and user permissions, coordinate the operation of each module, and realize the synchronous update of data and index.
[0024] Furthermore, the cloud storage nodes are deployed in a distributed manner, including at least two nodes. Data synchronization and backup are implemented between the nodes to automatically switch to the backup node when any node fails, ensuring the normal operation of the retrieval service.
[0025] Furthermore, it also includes an anomaly monitoring module electrically connected to the cloud server, used to monitor abnormal behavior during the retrieval process. The abnormal behavior includes malicious retrieval, unauthorized access, data leakage, etc. When abnormal behavior is detected, the retrieval operation is immediately terminated and an early warning message is sent to the administrator.
[0026] The beneficial effects of this invention are:
[0027] Employing a hierarchical index structure, the first-level global index quickly locates cloud storage nodes, while the second-level local index accurately locates data within those nodes. This avoids the time-consuming nature of single-index queries, significantly improving retrieval efficiency and making it suitable for large-scale cloud storage data retrieval. By extracting core data features to generate feature labels and using a cosine similarity algorithm to calculate matching degrees, high-matching candidate data is selected, reducing false and missed searches and improving retrieval accuracy. Stored data is encrypted, strict permission verification is implemented during the retrieval process, and retrieval logs are anonymized and backed up, effectively preventing data leakage, unauthorized retrieval, and other security risks, ensuring data and user privacy security. Attached Figure Description
[0028] Figure 1 This is a flowchart of the overall data retrieval process of the present invention.
[0029] Figure 2 This is a flowchart of the data preprocessing and index construction sub-process of the present invention.
[0030] Figure 3 This is a flowchart of the retrieval request processing and result feedback sub-process of the present invention.
[0031] Figure 4 This is a block diagram showing the hierarchical connection of the overall system modules of the present invention.
[0032] Figure 5 This is a diagram showing the distributed deployment connection of cloud storage nodes in this invention. Detailed Implementation
[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example 1
[0034] like Figure 1 , 2 As shown in Figure 3: This embodiment provides a data retrieval method based on cloud storage, and the specific steps are as follows:
[0035] S1: Data Preprocessing: Acquire various types of data to be stored, such as text, images, and videos, and classify and encode them according to data type (text, image, video) using UTF-8 encoding format; extract the core features of each data type, including data type, creation time, keywords, and data size. For example, extract core keywords from text data and pixel features and tag keywords from image data to generate key-value pair feature labels; encrypt the data using the AES symmetric encryption algorithm, with the key uniformly generated and managed by the cloud server to ensure data storage security.
[0036] S2: Index Construction: A hierarchical index with a tree structure is constructed based on feature labels. The hierarchical index includes a first-level global index and a second-level local index. The first-level global index associates the feature label sets of all cloud storage nodes, with each node corresponding to a cloud storage node, recording the storage range and core feature labels of that node. The second-level local index corresponds to the data feature labels within a single cloud storage node. The hierarchical index is stored in the index library of the cloud server. The nodes of the second-level local index correspond to the specific data within a single cloud storage node, achieving precise data location. The index library adopts distributed storage and is updated synchronously with the cloud storage nodes. When data is added, deleted, or modified, the index is updated synchronously to ensure the accuracy of the index.
[0037] S3: Retrieval Request Reception and Parsing: Users send retrieval requests via their terminal devices. After receiving the request, the cloud server parses the keywords in the request, extracts the target features corresponding to the search keywords, and verifies the retrieval request's permissions. If verification is successful, the next step is executed; otherwise, the retrieval is rejected. The permission verification includes user authentication and retrieval permission verification. User authentication is achieved through account passwords, dynamic verification codes, or biometric identification. Retrieval permissions are divided according to user levels, with different levels corresponding to different retrieval scopes. Specifically, for example, if a user searches for "text data in October 2024," the target features extracted are "data type = text, creation time = October 2024." Simultaneously, permission verification is performed on the user, requiring them to enter their account password and dynamic verification code. If verification is successful, the retrieval scope is determined based on the user level (ordinary users can only retrieve data within their own permissions, while administrators can retrieve all data). If verification fails, an insufficient permission prompt is returned, and the retrieval is rejected.
[0038] S4. Index Matching and Data Location: Based on the target features, the first-level global index is queried to determine the matching cloud storage node. Then, the target data within the node is located through the second-level local index. The matching degree between the target features and the data feature labels is calculated, and candidate data with a matching degree higher than a preset threshold is filtered out. The calculation uses the cosine similarity algorithm. The preset threshold can be dynamically adjusted according to user needs or system performance, and the range of the preset threshold is 0.6-0.9. If no matching candidate data is found, a no-match result prompt is returned, and relevant search keywords are recommended. Specifically, based on the extracted target features, the first-level global index is queried to determine the matching cloud storage node storing the October 2024 text data. Then, the data within the node that meets the conditions is located through the second-level local index of the node. The matching degree between the target features and the data feature labels is calculated using the cosine similarity algorithm. The preset threshold is set to 0.7, and candidate data with a matching degree ≥ 0.7 is filtered out. If no candidate data is found, a no-match result prompt is returned, and relevant keywords are recommended based on the user's search keywords, such as "September 2024 text data".
[0039] S5: Data Decryption and Feedback: The cloud server calls the AES decryption algorithm and uses the corresponding key to decrypt the candidate data. The data is then sorted from highest to lowest matching degree, and the sorted search results (including data name, creation time, and preview link) are fed back to the user's terminal. Simultaneously, a search log is recorded, including search keywords, search time, search results, and user account. The search log is periodically backed up to the cloud storage node, with a backup cycle of 1-7 days. Users can query their own search history. The cloud server performs anonymization processing on the search logs to protect user privacy. For example, if the search logs are backed up to the cloud storage node daily, the backup data is anonymized to hide user privacy information. Users can query their own search history through their terminal. Example 2
[0040] like Figure 4 , 5 As shown: This embodiment provides a cloud storage-based data retrieval system for implementing the retrieval method described in Embodiment 1, including the following modules:
[0041] Data preprocessing module: Deployed on a cloud server, it is used to receive data to be stored, perform classification coding, feature extraction, label generation and encryption processing. The encoding adopts UTF-8 format and the encryption adopts AES symmetric encryption algorithm. The generated feature labels and encrypted data are sent to the index building module and cloud storage node respectively.
[0042] Index building module: Communicates with data preprocessing module and cloud storage nodes to build a tree-structured hierarchical index. The hierarchical index includes a first-level global index and a second-level local index, enabling the storage, updating and maintenance of the index. The index database adopts distributed storage to ensure that the index is synchronized with the data.
[0043] Search request processing module: It is used to receive search requests sent by user terminals, parse search keywords, extract target features, perform user authentication and search permission verification. If the verification is successful, the target features are sent to the index matching module; if the verification fails, a search rejection prompt is returned.
[0044] Index matching module: used to match hierarchical indexes based on target features, locate target data, calculate matching degree using cosine similarity algorithm, filter out candidate data that meets preset thresholds, and send candidate data information to data decryption and feedback module;
[0045] Data decryption and feedback module: This module is used to call the decryption algorithm to decrypt the candidate data, sort the decrypted candidate data according to the matching degree, feed back the search results to the user terminal, record the search log, and back up the log to the cloud storage node.
[0046] Cloud storage nodes: They are deployed in a distributed manner, including at least two nodes. Data is synchronized and backed up between nodes to automatically switch to the backup node when any node fails, ensuring the normal operation of the retrieval business. Data is synchronized and backed up between nodes to store encrypted data and retrieval log backup. When any node fails, it automatically switches to the backup node to ensure data security and the normal operation of the retrieval business.
[0047] The cloud server, as the core of the system, is electrically connected to the data preprocessing module, index building module, retrieval request processing module, index matching module, data decryption and feedback module, and cloud storage nodes. It is used to manage the index library, encryption keys, and user permissions, coordinate the collaborative work of each module, realize the synchronous update of data and index, and handle system abnormal situations.
[0048] Anomaly monitoring module: Electrically connected to the cloud server, it is used to monitor abnormal behavior during the retrieval process in real time, including malicious frequent retrieval, unauthorized permission attempts, data leakage, etc. When an anomaly is detected, the retrieval operation is immediately terminated, an alert is sent to the administrator, and an anomaly log is recorded for subsequent investigation.
[0049] Specifically, the data preprocessing module connects to the cloud server, index building module, and cloud storage nodes, and is deployed on the cloud server. It sends the generated feature tags to the index building module, sends encrypted data to the cloud storage nodes, and receives key management instructions from the cloud server. The index building module connects to the data preprocessing module, cloud storage nodes, and cloud server. It communicates with the data preprocessing module to obtain feature tags for index building, and synchronously updates the index with the cloud storage nodes (adjusting the index synchronously when data changes). The index library is managed by the cloud server. The retrieval request processing module connects to the user terminal, cloud server, and index matching module. It receives retrieval requests from the user terminal, interacts with the cloud server to complete permission verification, and sends the parsed target features to the index matching module. The index matching module connects to the index building module, cloud storage nodes, and data decryption and feedback module: it obtains hierarchical indexes from the index building module for matching, interacts with the cloud storage nodes to locate target data, and selects candidate data information. The data is sent to the data decryption and feedback module; the data decryption and feedback module connects to the cloud server, user terminal, and cloud storage node, obtains the decryption key from the cloud server, feeds back the search results to the user terminal, and backs up the search log to the cloud storage node; the cloud storage node connects to various related modules and the cloud server: it adopts a distributed deployment (interconnected nodes achieve synchronous data backup), receives encrypted data from the data preprocessing module and log backups from the data decryption and feedback module, is managed by the cloud server, and in case of failure, the cloud server triggers a switchover to a backup node; the anomaly monitoring module connects to the cloud server, is deployed on the cloud server, and monitors the running status of modules such as search request processing and index matching in real time. In case of anomalies, it sends an alert to the cloud server, which then terminates the search operation; the cloud server connects to all modules and cloud storage nodes, acts as the core coordination unit, manages the index library, keys, and user permissions, interacts with various modules and cloud storage nodes, coordinates the collaborative operation of each part, and handles abnormal situations.
[0050] In summary, compared with the prior art, the present invention has the following advantages:
[0051] It adopts a hierarchical index structure, with a first-level global index to quickly locate cloud storage nodes and a second-level local index to accurately locate data within the nodes. This avoids the problem of time-consuming queries using a single index, greatly improves retrieval efficiency, and is suitable for large-scale cloud storage data retrieval.
[0052] By extracting core features from the data to generate feature labels, and using the cosine similarity algorithm to calculate the matching degree, candidate data with high matching degree are selected, reducing false searches and missed searches, and improving search accuracy.
[0053] The stored data is encrypted, strict permission verification is performed during the retrieval process, and the retrieval logs are backed up in anonymized form to effectively prevent security risks such as data leakage and unauthorized retrieval, and to protect data and user privacy.
[0054] The cloud storage nodes are deployed in a distributed manner to achieve synchronous data backup and automatic switching to backup nodes in case of failure, ensuring uninterrupted retrieval services. At the same time, an anomaly monitoring module is set up to promptly detect and handle retrieval anomalies, thereby improving retrieval stability.
[0055] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.
Claims
1. A cloud storage based data retrieval approach, characterized in that: Includes the following steps: S1: Data preprocessing: Obtain the data to be stored in the cloud storage node, classify and encode the data, extract the core features of the data, generate feature labels, and encrypt the data to obtain encrypted data; S2: Index Construction: Construct a hierarchical index based on feature tags. The hierarchical index includes a first-level global index and a second-level local index. The first-level global index is associated with the feature tag set of all cloud storage nodes, and the second-level local index corresponds to the data feature tags within a single cloud storage node. The hierarchical index is stored in the index library of the cloud server. S3: Retrieval Request Reception and Parsing: Receive retrieval requests sent by users, parse the keywords in the retrieval request, extract the target features corresponding to the retrieval keywords, verify the permissions of the retrieval request, and proceed to the next step if the verification is successful; otherwise, the retrieval is rejected. S4: Index Matching and Data Location: Based on the target features, query the first-level global index to determine the matching cloud storage node, and then locate the target data within the node through the second-level local index. Calculate the matching degree between the target features and the data feature labels, and filter out candidate data with a matching degree higher than the preset threshold. S5: Data Decryption and Feedback: Call the decryption algorithm corresponding to the encryption to decrypt the candidate data, sort them from high to low matching degree, and feed the sorted search results back to the user. At the same time, record the search log, which includes search keywords, search time, search results and user information. 2.The cloud storage-based data retrieval method of claim 1, wherein: In step S1, the core features include data type, creation time, keywords, and data size. The feature tags are stored in key-value pair format. The encryption process uses a symmetric encryption algorithm. The encryption key is managed uniformly by the cloud server. 3.The cloud storage-based data retrieval method of claim 1, wherein: In step S2, the hierarchical index adopts a tree structure, the nodes of the first-level global index correspond to cloud storage nodes, the nodes of the second-level local index correspond to specific data within a single cloud storage node, and the index database adopts distributed storage that is updated synchronously with the cloud storage nodes.
4. The data retrieval method based on cloud storage according to claim 1, characterized in that: In step S3, the permission verification includes user authentication and retrieval permission verification. The user authentication is achieved through account password, dynamic verification code or biometric identification. The retrieval permission is divided according to user level, and different user levels correspond to different retrieval ranges.
5. The cloud storage based data retrieval method of claim 1, wherein: In step S4, the calculation uses a cosine similarity algorithm. The preset threshold can be dynamically adjusted according to user needs or system performance, and the range of the preset threshold is 0.6-0.
9. If no matching candidate data is found, a no-match result prompt is returned, and relevant search keywords are recommended.
6. The cloud storage based data retrieval method of claim 1, wherein: In step S5, the search logs are periodically backed up to cloud storage nodes. The backup cycle can be set to 1-7 days. Users can query their own search history. The cloud server performs anonymization processing on the search logs to protect user privacy.
7. A cloud storage based data retrieval system, characterized by, include: Data preprocessing module: used for classifying, encoding, extracting features, generating tags, and encrypting cloud storage data; Index building module: used to build hierarchical indexes, which include a first-level global index and a second-level local index, and realize the storage, updating and maintenance of the indexes; Search request processing module: used to receive user search requests, parse keywords, extract target features, and perform search permission verification; Index matching module: used to match hierarchical indexes based on target features, locate target data, calculate matching degree and filter candidate data; Data decryption and feedback module: used to decrypt candidate data, sort it, and then send the search results back to the user terminal, and record and back up the search log; Cloud storage nodes: used to store encrypted data and retrieve log backups; The cloud server is electrically connected to the data preprocessing module, index building module, retrieval request processing module, index matching module, data decryption and feedback module, and cloud storage node, respectively. It is used to manage the index database, keys, user permissions, coordinate the operation of each module, and realize the synchronous update of data and index.
8. The cloud storage based data retrieval system of claim 7, wherein: The cloud storage nodes are deployed in a distributed manner, including at least two nodes. Data is synchronized and backed up between the nodes so that when any node fails, the system can automatically switch to the backup node to ensure the normal operation of the retrieval service.
9. The cloud storage based data retrieval system of claim 7, wherein: It also includes an anomaly monitoring module electrically connected to the cloud server, used to monitor abnormal behavior during the retrieval process. The abnormal behavior includes malicious retrieval, unauthorized access, data leakage, etc. When an abnormal behavior is detected, the retrieval operation is immediately terminated and an early warning message is sent to the administrator.