Quantum storage system based on cloud desktop data

Abstract

The invention provides a quantum storage system based on cloud desktop data. The quantum storage system comprises a cloud desktop client, a big data processing center and a quantum storage database, the cloud desktop client receives data issued by the cloud desktop through a plurality of peripherals, and arranges the data to obtain a preprocessed data set; the big data processing center comprises a data cleaning module and a digital fingerprint module, the data cleaning module performs data cleaning on the preprocessed data set, and the digital fingerprint module extracts digital fingerprints from the cleaned data and generates a fingerprint data set by taking the digital fingerprints as classification labels of the data; and the quantum storage database performs data block dynamic storage on the data according to the classification labels, and performs data compression by adopting a quantum technology. Through the extraction and embedding technology of the digital fingerprints, batch classification of the cloud desktop big data is realized, the data is subjected to data block dynamic storage based on a load balancing idea, and data compression is performed in combination with a quantum technology, so that the storage efficiency of the cloud desktop big data is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of big data storage, and in particular relates to a quantum storage system based on cloud desktop data. Background Art

[0002] In today's wave of digital office, cloud desktop systems have been widely used due to their convenience and flexibility. In recent years, with the rapid development of information technology, especially the rise of cloud computing and big data industries, the total amount of digital information has shown an explosive growth trend. In a small office environment, with a limited number of users and a relatively single type of peripherals, traditional data storage methods may be able to barely cope with the amount of cloud desktop data. However, once faced with large-scale enterprise-level applications, with hundreds or even thousands of users, and a wide variety of diversified peripherals, such as various professional plotters, high-resolution scanners, special models of printers, etc., traditional data storage methods are stretched.

[0003] At present, the growth rate of data volume has exceeded the growth rate of traditional physical storage space (disks, tapes, optical disks, RAM, etc.), and the efficient storage of big data has become a problem. And with the surge in data volume, the classification of large quantities of data is also an urgent problem to be solved, which is of great significance for data query and storage. There are usually two technologies to reduce the amount of data storage, one is traditional data compression, and the other is the data deduplication technology (Data Deduplication, Dedup) that has emerged in recent years, referred to as deduplication technology. However, these methods are not ideal for cloud desktop data storage in terms of security, configuration efficiency and accuracy, and cannot meet the growing business needs. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a quantum storage system based on cloud desktop data.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a quantum storage system based on cloud desktop data, comprising: a cloud desktop client, a big data processing center and a quantum storage database; the cloud desktop client receives data sent by the cloud desktop through a number of peripherals, and sorts out a preprocessed data set; the big data processing center comprises a data cleaning module and a digital fingerprint module, the data cleaning module cleans the preprocessed data set, the digital fingerprint module extracts a digital fingerprint from the cleaned data, and uses the digital fingerprint as a classification label for the data to generate a fingerprint data set; the quantum storage database dynamically stores the data in data blocks according to the classification label, and uses quantum technology to compress the data.

[0007] Optionally, the cloud desktop client divides the received data according to application scenarios, associates the received data with the corresponding peripherals and application scenarios, and stores them in the form of vector space data to generate a preprocessed data set.

[0008] Optionally, the data cleaning module first deletes null values and duplicate values in the preprocessed data set, and then uses the Z-score method to remove outliers for the data received by the same peripheral under the same application scenario.

[0009] Optionally, the digital fingerprint module reads the cleaned vector space data, extracts digital fingerprints, and embeds the digital fingerprints into the corresponding vector space data to generate a fingerprint data set.

[0010] Optionally, in the fingerprint data set, a distributed storage hash table is used to store digital fingerprints; when a digital fingerprint is stored in the distributed storage hash table, the digital fingerprint is used as the Key, and the locality list composed of all digital fingerprints with file locality to the Key is used as the Value for storage; an index of Key and Value is constructed, where the Key is used as the classification label, and the fingerprint data corresponding to the Key and Value is the data under this classification.

[0011] Optionally, when the quantum storage database dynamically stores data in data blocks according to classification labels, first calculate the data usage density of each classification through the following formula:

[0012] Composite Density=w u ·User Density+w t ·Traffic Density+w c ·ConnectionDensity

[0013] In the formula, Composite Density represents the data usage density of the current classification, User Density represents the control command density of the current classification; Traffic Density represents the data traffic density of the current classification, ConnectionDensity represents the data connection density of the current classification, w u 、w t and w c are the weights for control commands, data traffic, and data connections respectively;

[0014] According to the data usage density of each classification, allocate data blocks to the data under each classification for data storage, where the data usage density is proportional to the free space size of the data block.

[0015] Optionally, the control command density, data traffic density, and data connection density are calculated respectively through the following formulas:

[0016]

[0017] In the formula, UserData represents the amount of control command data, TotalData represents the total amount of data, TrafficArea represents the area of the regional network through which the data flows, TotalConnection represents the amount of data connections, including the data connections that are being used and have been established, and ConnectionArea represents the area of the data connection region.

[0018] Optionally, the quantum storage database uses quantum technology for data compression, specifically:

[0019] First, map the data stored in each database to a quantum state, globally compress the data in the quantum state using the quantum Fourier transform, and then measure the correlation relationship between the data by calculating the quantum entanglement degree to remove the information redundancy of the data.

[0020] The beneficial effects of the present invention are as follows:

[0021] 1) Based on digital fingerprint extraction and embedding technology, the present invention constructs an innovative data processing mechanism specifically for complex data in the cloud desktop big data environment. By extracting digital fingerprints from massive data as data identifiers, it can clearly reflect the characteristics and attributes of each piece of data; using fingerprint embedding technology to closely associate the digital fingerprints with the corresponding data, and taking the digital fingerprints as indexes and file locality as the aggregation condition, batch classification of big data is realized.

[0022] 2) Based on the concept of load balancing, the present invention further optimizes the data storage strategy. The data is not stored in a fixed manner, but the data blocks are dynamically stored according to the data usage density. The present invention dynamically determines the storage location of each category of data according to the real-time load situation, realizes the load balancing of the entire storage system, and avoids the performance bottleneck caused by local overload.

[0023] 3) The present invention uses quantum technology for data compression, which further reduces the space required for data storage while ensuring the integrity and accuracy of the data, and improves the data storage efficiency. Description of the Drawings

[0024] Figure 1 is an architecture diagram of a quantum storage system based on cloud desktop data.

[0025] Figure 2 is a flowchart of the digital fingerprint module generating a fingerprint data set.

[0026] Figure 3 is a schematic diagram of quantum entanglement in the quantum storage process. DETAILED DESCRIPTION

[0027] The present invention will now be described in further detail with reference to the accompanying drawings.

[0028] In view of the current security, configuration efficiency and accuracy requirements of cloud desktop data storage, as well as the growing business needs, the present invention proposes a quantum storage system based on cloud desktop data, whose architecture is as follows: Figure 1 As shown in the figure, we first built a cloud-edge-end intelligent framework to perform correlation analysis on the information sent down in the cloud-edge-end intelligent framework integrated communication network, maximize the scope of computing and transmission tasks, thereby maximizing data storage capacity, improving storage security, configuration efficiency and accuracy, and effectively solving the dual limitations of massive cloud data in hardware resources and data resources. Figure 1 As shown, the system mainly includes cloud desktop client, big data processing center and quantum storage database.

[0029] The cloud desktop client receives the data sent by the cloud desktop through several peripherals. The cloud desktop client divides the received data into application scenarios. The cloud desktop peripherals are configured with data sets of different application scenarios P ​​= {P a1 , P a2 , ..., P st , ..., P nN}, where P st represents the tth peripheral in the sth application scenario, n is the total number of scenarios, and N is the total number of peripherals; the peripherals themselves are divided into clients, PCs, mobile phones, tablets, etc., and the peripherals are also set with a quantity set Q = {Q1, Q2, ..., Q t , ..., Q n}. The cloud desktop client associates the received cloud desktop data with the peripherals and application scenarios corresponding to the data and stores them in the form of vector space data to generate a preprocessed data set. Vector space data can accurately reflect the temporal and spatial characteristics of the data and has a compact data structure, which facilitates the subsequent extraction of digital fingerprints with file locality. File locality is an important concept in the computer field to describe data patterns. It is mainly divided into two aspects: time locality and space locality. The operating system and hardware will use file locality to improve data access efficiency, such as caching mechanism.

[0030] The big data processing center includes a data cleaning module and a digital fingerprint module. When the data cleaning module performs data cleaning, it first deletes the null values ​​and duplicate values ​​in the preprocessed data set, and then uses the Z-score method to remove abnormal values ​​for the data received by the same peripheral in the same application scenario. The digital fingerprint module reads the cleaned vector space data and extracts the digital fingerprint, embeds the digital fingerprint into the corresponding vector space data, and generates a fingerprint data set. The process is as follows: Figure 2 shown.

[0031] Digital fingerprint technology consists of three parts: fingerprint encoding (extraction), fingerprint embedding, and fingerprint detection and tracking. The fingerprint encoding process ensures the uniqueness of the fingerprint information distributed to users; the fingerprint embedding process mainly focuses on the usability of the carrier data after embedding and the robustness of the fingerprint; the detection and tracking process generally depends on the detection rate and the error rate. Finally, the interaction and identification between entities are achieved through protocols.

[0032] In the index service, the scale of digital fingerprint data can reach several terabytes or even dozens of terabytes. Traditional databases cannot meet the storage and fast search requirements of such large-scale data, and security is also an issue that cannot be ignored. Therefore, in the fingerprint data set of this embodiment, the digital fingerprints in the index service are stored by a distributed hash table, which improves the scalability of index storage and the distributed access characteristics. This embodiment uses a Distributed HashTable (DHT) to store digital fingerprints. In order to save the data set locality information in the index service, when a digital fingerprint is stored in the distributed storage hash table, the digital fingerprint is used as the Key, and the locality list composed of all digital fingerprints with file locality to this Key is stored as the Value; an index of Key and Value is constructed, where Key is used as the classification label, and the fingerprint data corresponding to Key and Value is the data under this classification. This effectively avoids the situation where when there are too many digital fingerprints due to large files, the same locality list as the Value will be stored multiple times, causing serious waste of DHT storage. With the fingerprint data set constructed in this way, multiple data with file locality of digital fingerprints can be searched according to the Key. In terms of hardware support, the RCID method can be used for scheduling, and with a high probability, the files with file locality are handed over to the same service node, which is beneficial to the more efficient operation of caching and prefetching technologies. The caching and prefetching nodes provide the same Key / Value-based access interface as the DHT, also using the digital fingerprint as the Key and its locality list as the Value.

[0033] The quantum storage database dynamically stores data in data blocks according to classification tags and uses quantum technology for data compression. This embodiment is based on the concept of load balancing and considers the allocation of storage space based on data usage density. In the process of building an efficient data storage system, it is a key link to allocate data blocks according to the usage density of each classification data, so as to achieve dynamic storage of data. Data usage density, that is, the frequency of access and invocation of a certain classification data per unit time, plays a guiding role in the data storage strategy. Specifically, the system will deeply analyze the composition and usage of each classification data, count its access frequency in different time periods, and calculate the data usage density based on this. For classifications with a high data usage density, this means that the classification data is frequently invoked. To ensure the fast reading and writing and efficient use of data, it is necessary to allocate data blocks with sufficient free space, reduce the waiting time caused by insufficient space during data storage and reading, and improve data processing efficiency. On the contrary, for classifications with a low data usage density, the data is accessed less frequently, and the system can correspondingly allocate data blocks with relatively small free space to effectively avoid waste of storage space. Through this allocation method, the performance advantages of dynamic data block storage can be fully utilized to ensure that all types of data can achieve the best results during storage and invocation.

[0034] Data usage density mainly considers three aspects of factors: control commands, data traffic, and data connections. Control commands cover a series of operation instructions for data, from data creation, modification to deletion. The execution of each control command means the need for data storage and invocation. For example, frequently issued data query instructions mean a high-frequency demand for specific data, which will undoubtedly increase the data usage density of the classification to which the data belongs. Data traffic intuitively reflects the scale of data flow within the region. The duration and initiation times of data connections are all closely related to data usage density. A stable and long-term data connection often means continuous data interaction, thereby increasing data usage density. Considering these three aspects of factors comprehensively, when dynamically storing data blocks, first calculate the data usage density of each classification through the following formula:

[0035] Composite Density=w u ·User Density+w t ·Traffic Density+w c ·ConnectionDensity

[0036] In the formula, Composite Density represents the data usage density of the current classification, User Density represents the control command density of the current classification; Traffic Density represents the data flow density of the current classification, Connection Density represents the data connection density of the current classification, and w u 、w t and w c are the weights used to control commands, data traffic, and data connections respectively;

[0037] Among them, the control command density, data flow density and data connection density are calculated using the following formulas:

[0038]

[0039] In the formula, UserData represents the amount of control command data, TotalData represents the total amount of data, TrafficArea represents the area of ​​the regional network through which the data flows, TotalConnection represents the amount of data connections, including data connections in use and established, and ConnectionArea represents the area of ​​the data connection area.

[0040] Finally, according to the data usage density of each category, data blocks are allocated to the data under each category for data storage, wherein the data usage density is proportional to the size of the free space of the data block.

[0041] Quantum storage can utilize the polymorphism and superposition characteristics of quantum states, break through the physical limitations of traditional storage technology, and achieve unprecedented storage density. In the quantum storage database of this embodiment, quantum technology is used for data compression. First, the data stored in each database is mapped to a quantum state, and the quantum state data is globally compressed using the quantum Fourier transform. Through this process, the data that was originally scattered and occupied a large storage space can be optimized and integrated at the quantum level. While retaining key information as much as possible, the resources occupied by the data are greatly reduced, and efficient global compression of the data is achieved. Then, the correlation between the data is measured by calculating the quantum entanglement degree, and the information redundancy of the data is removed. Quantum entanglement describes a non-classical strong correlation characteristic between multiple quantum states. Its principle is as follows: Figure 3 As shown. In the data compression scenario, the degree of entanglement between different quantum state data reflects the correlation between the data they represent. Data with high entanglement needs to be retained; while data with low entanglement can be further judged whether it is information redundancy and removed.

[0042] In summary, in the cloud-edge-end intelligent framework of the present invention, the data and application scenarios sent by the cloud desktop are associated and analyzed with peripherals, and the diverse cloud desktop data is effectively classified to improve the storage security, configuration efficiency, and accuracy of associated data; digital fingerprints are generated based on the cloud desktop data to meet the storage and fast search requirements of large-scale data; the concept of load balancing is combined with the calculation of data usage density to achieve dynamic storage of data blocks for various types of data, optimizing the storage mode; finally, the quantum storage technology is adopted for the stored cloud desktop data set to further compress the storage space.

[0043] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, several improvements and refinements made without departing from the principle of the present invention should be regarded as within the protection scope of the present invention.

Claims

1. A quantum storage system based on cloud desktop data, characterized in that: include: Cloud desktop client, big data processing center and quantum storage database; The cloud desktop client receives the data sent by the cloud desktop through several peripherals, and sorts out the data to obtain a pre-processed data set; The big data processing center includes a data cleaning module and a digital fingerprint module. The data cleaning module cleans the preprocessed data set, and the digital fingerprint module extracts the digital fingerprint from the cleaned data, and uses the digital fingerprint as a classification label for the data to generate a fingerprint data set; the quantum storage database dynamically stores the data in data blocks according to the classification labels, and uses quantum technology to compress the data.

2. A quantum storage system based on cloud desktop data as claimed in claim 1, characterized in that: The cloud desktop client divides the received data into application scenarios, associates the received data and the peripherals and application scenarios corresponding to the data, stores them in the form of vector space data, and generates a preprocessed data set.

3. A quantum storage system based on cloud desktop data as claimed in claim 1, characterized in that: The data cleaning module first deletes null values ​​and duplicate values ​​in the preprocessed data set, and then uses the Z-score method to remove outliers for the data received by the same peripheral in the same application scenario.

4. A quantum storage system based on cloud desktop data as claimed in claim 2, characterized in that: The digital fingerprint module reads the cleaned vector space data and extracts the digital fingerprint, embeds the digital fingerprint into the corresponding vector space data, and generates a fingerprint data set.

5. A quantum storage system based on cloud desktop data as claimed in claim 4, characterized in that: In the fingerprint data set, a distributed storage hash table is used to store digital fingerprints; when a digital fingerprint is stored in a distributed storage hash table, the digital fingerprint is used as the Key, and a locality list consisting of all digital fingerprints that have file locality with the Key is stored as the Value; an index of the Key and the Value is constructed, wherein the Key is used as a classification label, and the fingerprint data corresponding to the Key and the Value is the data under the classification.

6. A quantum storage system based on cloud desktop data as claimed in claim 1, characterized in that: When the quantum storage database dynamically stores data blocks according to classification labels, the data usage density of each classification is first calculated by the following formula: Composite Density=w u ·User Density+w t ·Traffic Density+w c ·ConnectionDensity In the formula, Composite Density represents the data usage density of the current classification, User Density represents the control command density of the current classification; Traffic Density represents the data flow density of the current classification, Connection Density represents the data connection density of the current classification, and w u 、w t and w c are the weights used to control commands, data traffic, and data connections respectively; According to the data usage density of each category, data blocks are allocated to the data under each category for data storage, wherein the data usage density is proportional to the size of the free space of the data block.

7. A quantum storage system based on cloud desktop data as claimed in claim 6, characterized in that: The control command density, data flow density and data connection density are calculated using the following formulas: In the formula, UserData represents the amount of control command data, TotalData represents the total amount of data, TrafficArea represents the area of ​​the regional network through which the data flows, TotalConnection represents the amount of data connections, including data connections in use and established, and ConnectionArea represents the area of ​​the data connection area.

8. A quantum storage system based on cloud desktop data as claimed in claim 1, characterized in that: The quantum storage database uses quantum technology to compress data, specifically: First, the data stored in each database is mapped into quantum states, and the quantum state data is globally compressed using quantum Fourier transform. Then, the correlation between the data is measured by calculating the quantum entanglement degree to remove information redundancy of the data.

Images