1841 results about "Block (data storage)" patented technology

The invention discloses an optical fiber data storage management system and method based on big data, and relates to the technical field of storage management. According to the optical fiber data storage management system and method based on big data, the spatial-temporal characteristics of the historical access records are extracted through the LSTM network, the data heat value is calculated in combination with the time decay function, and the thermal distribution diagram of the data blocks is generated. And based on the thermal distribution map, constructing a storage resource allocation model by using a deep reinforcement learning algorithm, and dynamically adjusting the optical path priority and copy distribution. A data prefetching strategy is optimized through the spatio-temporal joint index and the vector similarity index, and a high-frequency data cache index distribution diagram is generated. And comprehensively considering the data access delay, the index hit rate and the copy migration frequency, obtaining an optical fiber storage efficiency index, and adjusting a storage strategy according to the index. According to the method, the dynamic scheduling and utilization efficiency of data storage resources is effectively improved, the data access delay is reduced, the copy management is optimized, and the overall performance of a storage system is remarkably improved.

The invention relates to the technical field of data processing, in particular to a cloud backup platform data deduplication method based on distributed storage. Comprising the following steps of: performing self-adaptive blocking on data by utilizing a dynamic blocking algorithm, generating a unique identification code and extracting multi-dimensional feature information; establishing a global index through a distributed hash table, mapping a data position and maintaining index consistency; performing redundancy analysis and similarity matching based on a deep learning algorithm, and executing a duplicate removal strategy; according to the access frequency and the load condition, distributed storage of data blocks is realized by adopting a hierarchical graph division algorithm; and a partitioning strategy, index frequency and storage layout are dynamically optimized by using a reinforcement learning algorithm, and global optimization of performance and resources is realized. According to the invention, high efficiency, accuracy and dynamic optimization of data deduplication are realized, and the data storage and management efficiency of the cloud backup platform is improved.

The invention provides a data all-flash storage optimization method and system based on cloud computing, and the method comprises the steps: receiving a storage operation request stream of a user terminal through collecting a multi-dimensional performance data set of each node of an all-flash storage cluster in real time, wherein the multi-dimensional performance data set comprises a storage space fragmentation rate, an input / output request queue depth and a network link bandwidth utilization rate; analyzing a to-be-stored data block identifier set and an operation mode label, traversing a historical access record library based on to-be-stored data block identifiers, extracting historical access time sequence characteristics and physical storage position proximity of associated data blocks, and combining the multi-dimensional performance data and the historical access time sequence characteristics to obtain a data block identifier set; according to the method, a dynamic access popularity prediction model in a preset time period is constructed, and finally, a cross-node fragmentation storage topological graph and a cache preloading strategy matrix are generated according to the proximity of the dynamic access popularity prediction model and a physical storage position, so that the optimization of all-flash storage is realized, and the data storage performance in a cloud computing environment is improved.

The invention relates to the technical field of data storage and processing, in particular to an RAID card static cache management method and device based on data popularity, and the method comprises the steps that the physical position of a data block needing to be accessed in an SSD array is acquired according to a read-write request; historical data access information of the RAID card is collected, a historical data set is generated, and a neural network model is trained by using the historical data set to obtain a cache management model; predicting and outputting data blocks which are possibly accessed in the future and the access probability of the data blocks through the cache management model; the RAID controller dynamically adjusts a cache strategy in combination with an LRU strategy according to a prediction result output by the cache management model and a read-write request of a file system, and optimizes a storage position and an updating mechanism of a data block in a cache space; and when the hit rate or the data access efficiency does not reach the preset value, adjusting the parameters of the cache strategy and updating the cache strategy. According to the method, cache resources can be more effectively distributed, the cache hit rate is improved, the cache replacement overhead is reduced, and therefore the performance of a storage system is optimized.

The invention relates to the technical field of data storage, in particular to a dynamic data distribution storage method and system for an RAID (redundant array of independent disks), and the method comprises the following steps: based on the access frequency of data blocks in a disk, recording the access frequency and timestamp of each data block, carrying out heat rating on the data blocks, and determining thresholds of hot data and cold data. According to the method, the dynamic distribution of data storage is effectively optimized by analyzing the access frequency of the data blocks, and the performance and the economic efficiency of a storage system are remarkably improved by migrating the frequently accessed data to the cache disk and migrating the less accessed data to the low-cost storage; according to the method, unnecessary occupation of high-performance storage resources is reduced, the operation cost is also reduced, heat rating is performed on the data blocks, and the storage positions of the data blocks are correspondingly adjusted, so that the data processing speed is higher, meanwhile, the data security is improved, when a hard disk fails, data recovery is faster, and the service continuity is ensured.

The invention relates to the technical field of visual rendering, and provides a three-dimensional data visual rendering method and a visual rendering engine system.The method comprises the steps that space segmentation processing is conducted on three-dimensional model data to be rendered through the LOD technology to generate block slice data with different geometric detail levels, organizing the block slice data through a tree structure with a spatial relationship, and outputting the tree structure as an index file; the method comprises the following steps: adaptively screening block and slice data of different levels of detail based on a user vision field range, storing the block and slice data into a rendering request queue, querying block and slice data to be rendered from the rendering request queue according to index codes of the block and slice data, and updating a layer browser database; according to the method, to-be-rendered block slice data are downloaded and analyzed in parallel in batches in a separated collaborative operation mode, and a rendering task is executed according to a data analysis result, so that the rendering scheduling frame rate is remarkably improved, and rapid visualization of urban mass three-dimensional slice data at a webpage end is realized.

The invention provides a block chain-based clinical test data integrity system, and relates to the technical field of data processing, and the system comprises the following components: a multi-source heterogeneous data acquisition and preprocessing module, a multi-level encryption and under-chain data storage module, an alliance block chain network, and an alliance block chain network maintenance and verification shared distributed account book. The distributed account book only stores all digitally signed clinical test data fingerprints, metadata and operation logs, does not store original data, and is linked with the intelligent contract automatic management module, so that the clinical test data cannot be tampered in the whole life cycle from the source to archiving. Hash anchoring, digital signature and decentralized verification mechanisms of the block chain technically eradicate the possibility of unauthorized tampering of data, and ensure scientificity and reliability of clinical test results.

The invention discloses a data distributed storage and reading method and system based on multi-modal protection, and relates to the technical field of multi-modal data encryption storage, and the method comprises the steps: receiving input data, and preprocessing the input data into multi-modal segmentation data; executing multi-key cooperative encryption on the data; carrying out distributed storage on the encrypted data; signing and broadcasting the distributed stored node information based on the block chain; and receiving a data query request of a user, obtaining the corresponding encrypted segmented data through the permission vector of the user, and decrypting and recombining the encrypted segmented data to obtain query data. Through the multi-modal feature fusion technology, the security of data storage is improved, meanwhile, a multi-level encryption system is constructed through multi-key cooperation encryption, the security of key distribution is effectively guaranteed, a distributed storage strategy based on dynamic weight is adopted, intelligent selection and load balancing of storage nodes are achieved, and the security of data storage is improved. And the data storage reliability and the system expansibility are improved.

The invention provides an unstructured data storage optimization method and system based on information entropy and a block chain, and relates to the technical field of data storage, and the method comprises the steps: building a block chain storage network to obtain data, recognizing redundant data through dimension reduction, mapping and hyperplane construction, and removing the redundant data based on fuzzy similarity. The to-be-stored data is partitioned, the access frequency is calculated, the consistent Hash algorithm is executed to determine the storage position, the access index and the verification sequence are established, and the data integrity verification is executed, so that the data access efficiency can be improved, and the storage security can be enhanced.

The invention provides a real-time deep sea subsurface buoy monitoring system, which belongs to the technical field of deep sea measurement, and comprises a control chip, a multi-parameter sensor array, a data storage device and a power supply, high-precision monitoring is realized through the following steps: recording ocean data acquired by a multi-parameter sensor; correcting the sound wave propagation model and calculating the relative position of the subsurface buoy; constructing a three-dimensional data matrix and realizing data compression; identifying abnormal data by using a matrix disorder index; processing the time sequence data by adopting a sliding window weighted average method and eliminating jump; carrying out distributed preprocessing on the multi-source heterogeneous data and carrying out parallel processing by applying a matrix partitioning technology; accumulated errors are corrected by combining historical data and applying a Kalman filtering algorithm, and the problems of environment interference and sensor drift are solved through a deep sea environment disturbance compensation network model and a self-adaptive weighted error optimization function.

The invention discloses a block chain-based performance-lossless watermark credible traceability method and system. The method comprises the following steps of: embedding extended watermark information based on distribution consistent sampling in a potential space of a generative model to obtain a generation content of an embedded watermark; converting the watermark metadata into a hash value, storing the hash value in a block chain, and executing node trust management and a shared data storage and consensus mechanism; inverting the generated content embedded with the watermark to extract watermark information and calculating a hash value, comparing the hash value with the hash value of the watermark information stored in the block chain, if the comparison is consistent, passing the watermark verification, and if the comparison is inconsistent, failing the watermark verification; and when the watermark verification fails, performing node signature verification, if the node signature verification fails or the data is abnormal, starting an on-chain traceability process, tracking a data generation path according to the watermark metadata, and searching an abnormal node or data transmission point. According to the method, lossless embedding, credible circulation and reliable traceability of watermark information can be realized while high quality of generated content is ensured.

The invention relates to a health data encryption storage method and device, equipment and a storage medium, and the method comprises the steps: obtaining to-be-processed health data, carrying out the sensitivity analysis and grade division of the health data, and obtaining health data sub-blocks; performing encryption preprocessing on the health data sub-blocks, and performing group data protection according to a preset differential privacy algorithm to obtain encrypted data blocks; generating an initial master key through a hardware security module, and performing derived hierarchical encryption on the encrypted data block to obtain a data encryption key; performing fragmentation processing and regular distributed storage on the data encryption key to obtain a dynamic security key; and performing metadata separation storage on the encrypted data according to the dynamic security key, and performing data permission determination according to a preset role-based access control mechanism to obtain an encrypted isolation database. According to the invention, efficient security protection can be maintained under different application scenes and access permissions, and the risk of data leakage is reduced.

The invention relates to the technical field of block chains, and discloses a data encryption system for evidence collection and storage based on a block chain technology. According to the data encryption system for evidence collection and storage based on the block chain technology, a hash value can be generated based on a timestamp of data chaining, so that integrity of storage data is regulated and controlled, data tampering detection is realized, a user permission access range can be regulated and controlled in combination with a key distribution mechanism, sensitive data leakage is effectively prevented, and user experience is improved. The consistency of the encryption step is verified according to the intelligent contract execution logic, the multi-node synchronization error is eliminated, the error of the encrypted data storage link is verified through the Hash verification algorithm, and the consistency of the block data is ensured.

The invention discloses a data dynamic migration and storage method, device and equipment and a medium, and relates to the technical field of computers. The method comprises the steps that environment parameters of a current mechanical hard disk in a distributed cluster are collected; wherein the environment parameters comprise a vibration parameter, a temperature parameter and a noise parameter; quantifying the input and output delay performance of the current mechanical hard disk by using the environmental parameters of the current mechanical hard disk; if the input and output delay performance of the current mechanical hard disk is greater than a preset threshold value, searching a target mechanical hard disk combination in the distributed cluster; and migrating and storing the target data block of the current mechanical hard disk into the target mechanical hard disk combination based on an erasure code and a multi-copy mechanism. Through the scheme, the reliability of storing the data in the mechanical hard disk in the distributed cluster is remarkably improved.

The invention discloses a logistics data credible evidence storage and tracing method based on a block chain, and the method comprises the following steps: collecting original logistics data, and executing data standardization processing; constructing a layered Merkle mountain structure, and calculating to obtain a main root commitment of the layered Merkle mountain; dividing a layered Merkle mountain range structure, and calculating to obtain a windowed subtree root set; configuring a revision branch record structure, generating a collision pair by using chameleon hash, and generating a revision proof record; a multi-index root structure is constructed, a cross-root sharing mechanism is established, and a multi-dimensional tracing index set is generated; and based on the hierarchical Merkle mountain range main root commitment, the windowed subtree root set, the multi-dimensional traceability index set and the revision proof record, executing anchoring consistency verification, and outputting a credible evidence storage result and a traceability verification result. According to the invention, the logistics data evidence storage integrity and the multi-dimensional tracing efficiency are improved.

The invention discloses a building industry human resource management system based on a block chain technology, and aims to solve the problems that a current building industry human resource management system is slow in processing speed and difficult to meet flexible employment and rapid recruitment requirements of the building industry, so that transaction processing is delayed, sensitive information of employees may be leaked, a decentralized architecture needs multi-party coordination, and the efficiency is high. The system comprises a block chain module which is used for data storage, security and automatic execution of a consensus mechanism; and the business management module is used for carrying out precise management on complex scenes in the building industry through characteristics of decentralization, non-tampering and the like. According to the method, a decentralized distributed architecture is adopted, the data is stored on a plurality of nodes, the security of the data is ensured, each piece of data is recorded on the block chain and cannot be tampered by any person, the authenticity and integrity of the data are ensured, intermediate links can be reduced, and the management cost and risk are reduced.

The invention provides a dynamic video memory allocation method and system based on data characteristics, and relates to the technical field of data storage, and the method comprises the following steps: pre-allocating a plurality of video memory blocks with different sizes in a GP U video memory to form a three-stage video memory pool; acquiring data features input by the model in real time, and establishing a multi-dimensional feature vector; constructing a time sequence prediction model based on an LSTM network, and predicting a video memory demand value by inputting a feature vector sequence subjected to continuous multiple reasoning; selecting a hierarchical distribution strategy according to the predicted video memory demand value and the scene feature matrix; constructing a video memory dependency graph, dynamically tracking the earliest release time of a video memory block, and optimizing the utilization rate of the video memory through a multiplexing strategy; monitoring a GPU video memory state index; triggering a video memory optimization strategy according to the video memory state index; through accurate prediction of video memory demands, reasonable hierarchical distribution, optimization of video memory utilization rate and coping with different scenes, the video memory management problem in deep learning model reasoning is solved, and the video memory use efficiency and the system stability are improved.

The invention provides a multi-modal process data trusted storage method and system combined with a block chain, and belongs to the technical field of data storage, and the method comprises the following steps: obtaining multi-modal process data; constructing a metadata minimum set covering the full life cycle of the multi-modal process data; performing under-chain storage on the multi-modal process data and the metadata minimum set, and verifying the multi-modal process data to obtain verification process data; recording the digital signature evidence of the verification process data on a block chain to obtain an on-chain shared signature evidence of the verification process data; based on the on-chain shared signature evidence, establishing a mapping relationship between the hash value of the on-chain shared signature evidence and the under-chain verification process data; and storing the verification process data and the mapping relationship in a distributed process database. According to the method, the block chain and the distributed storage are combined to construct a multi-modal process data trusted storage system with coordination of on-chain and off-chain, so that the trusted storage of the multi-modal process data in the whole life cycle is realized.

The invention provides a data access control method and device based on a block chain and improved CP-ABE, and belongs to the technical field of block chains. The method comprises the following steps: an authority generates key parameters including a system public key and a master key, and uploads the master key to a block chain; the user sends an attribute set and an identity label of the user to an authority, the identity label of the user is hidden, an attribute private key, a search private key, a conversion key and a recovery key are generated for the user, the conversion key is sent to the cloud server, and the attribute private key, the search private key and the recovery key are sent to the user, and the user comprises a data owner and a data visitor; the data owner carries out offline encryption, online encryption and index encryption on the file in sequence, and index information is linked; the data accessor performs data access; and the data accessor decrypts the data. According to the method, the problems in data storage and access control are effectively solved, the integrity of user data is protected, and the calculation burden of a data access user is reduced.

The embodiment of the invention provides a memory recovery method and device, electronic equipment and a storage medium, and relates to the technical field of data storage. Under the condition that both the file system and the storage device support the perceptual garbage collection function, the device state of the storage device is obtained in response to the starting of the perceptual garbage collection function; according to the equipment state, determining whether an execution condition for sensing a garbage collection function is met or not; and if the execution condition of the garbage collection sensing function is met, the file system is linked to perform memory collection on the storage device, so that the file system can know the data blocks moved by the storage device, the file system is prevented from moving the data blocks moved by the storage device again, and the efficiency of the file system is improved. In addition, invalid blocks in the file system and the storage device can be synchronized, the situation that invalid data blocks in the file system are moved when the storage device executes memory recovery is avoided, and therefore the service life expenditure of the storage device and the write-in amplification value of the storage device are reduced.

The invention relates to the field of legal service management, and discloses a legal service management system based on a block chain, and the system comprises the following modules: a block chain network, an intelligent contract module, a data storage module, a user interface module, a feedback and monitoring module and a user management module, the non-tampering property and the transparency are ensured; the intelligent contract module automatically executes contract terms and supports intelligent management; the data storage module adopts a hierarchical structure, and on-chain data and off-chain data are stored in a distinguished manner, so that the storage and access efficiency is optimized; and the user interface module provides user-friendly operation experience through a visual tool. By using the block chain technology, all contract execution and compliance audit data are encrypted and stored in a distributed mode, it is ensured that tampering is avoided and whole-course traceability is achieved, and a user and related mechanisms can verify the authenticity and integrity of the contract data at any time.

The invention discloses a mine digital asset storage management method, which comprises the following steps of: extracting characteristics of mine data, dividing the mine data into hot data, warm data and cold data according to the characteristics, selecting different storage devices for the data, and dynamically adjusting migration of the data among the different storage devices based on access frequency and business requirements; future data storage requirements are predicted through a machine learning algorithm, and storage resources are scheduled; performing full life cycle management on the data, performing business value evaluation on the data, and adjusting storage priority and resource allocation based on the data value; carrying out full-life-cycle encryption on the data, and recording data operation through a block chain technology; energy consumption of the storage equipment is monitored in real time, and storage energy efficiency and carbon footprint management are adjusted. The problems of low storage efficiency, high storage cost and insufficient energy consumption optimization caused by data type diversity, access frequency difference and insufficient storage resource utilization in mine data management can be solved.

The invention relates to the technical field of data storage, and discloses a lossless compression algorithm-based data storage method, which comprises the following steps of: receiving to-be-stored data through a data interface, automatically identifying a data format by using a data analysis tool, and performing feature analysis to obtain the size, the type and the complexity of the data; through deep analysis of data features, the method can flexibly select a proper Huffman coding processing mode, and optimizes a compression process according to different data types and complexity. In addition, a context modeling optimization mechanism is introduced, the coding mode is dynamically adjusted, and the compression ratio and the storage efficiency are further improved; efficient storage and reliability verification of data are ensured through a block storage strategy in combination with metadata management; the problem that an existing compression algorithm is low in efficiency is solved, the integrity and reliability of data storage are improved, and the method is suitable for storage requirements of large-scale data and diversified data.

The invention discloses an intelligent traffic data storage method, system and device based on the Internet of Things and a medium, and relates to the technical field of electric digital data processing, and the method comprises the steps that an edge node packages received data and inserts the data into a local directed acyclic graph account book; dividing the verified data into Level-0 fragments according to a fixed duration window, and performing variable-length fragmentation based on a Rabin fingerprint algorithm; calculating Hash values of the variable-length fragments, and performing duplicate removal through a Bloom filter and a Hash index table in sequence; generating a local erasure block and a global erasure block for the deduplicated fragments, and storing complete copies of the key fragments on a plurality of storage nodes; and periodically checking a local erasure block and a complete copy, and triggering self-healing reconstruction when detecting that the fragment is lost. According to the method, the technical problems of high delay of distributed write-in consistency, high storage redundancy, complex cross-node time sequence correction, slow data recovery, poor system expansibility and the like are solved.

The invention relates to the technical field of data security, in particular to a block chain-based big data security data storage method, which comprises the following steps of acquiring a write-in request, analyzing a field path and a label, matching a structure number and state information, sorting a conflict path, compressing a number sequence, distributing a compression path, positioning a writable area and updating a path pointer. And writing data and recording index and verification information to obtain a safe chain-entering data set. According to the method, a clear mapping relation is formed through binding of field paths and label numbers, path conflicts are subjected to number rearrangement and hierarchical compression processing, the structural coherence of field distribution is enhanced, field write-in positioning is completed in combination with node capacity, position indexes and verification identifiers are generated, and unified binding of field contents, the path numbers and access positions is achieved; the writing accuracy and the structure traceability are guaranteed, and the regularity of chain segment storage and the orderliness of operation are improved.

The invention relates to the technical field of storage, in particular to an SSD wear leveling method and device based on dynamic data migration and a medium. The method comprises the following steps: monitoring access frequency, recording read-write times of a logic block in real time, and identifying hotspot data; tracking a wear state, counting the erasure times of the physical flash memory block, and evaluating the wear degree of the physical flash memory block; data migration decision: determining whether to migrate data based on the access frequency and the wear frequency; preferentially migrating data with high access frequency and high wear degree according to access frequency and wear program sorting; and data migration execution: migrating the hotspot data from the high-wear area to the low-wear area, and updating the mapping relation. According to the method, the data storage position can be intelligently adjusted according to the real-time data access characteristics and the abrasion state, and therefore the service life of the SSD is more efficiently prolonged.

The invention relates to the technical field of block chains, and discloses a distributed data security storage system and method based on a block chain, and the system comprises a dynamic fragmentation encryption module which can automatically select a fragmentation strategy based on a data sensitivity level; the data sensitivity is set to be S, the data sensitivity level is determined through the following formula: # imgabs0 #, wi is the weight of the ith factor influencing the data sensitivity, fi (D) is the quantized value of the data D under the ith factor, and for low-sensitivity data (S is smaller than or equal to Slow, and Slow is a preset low-sensitivity threshold), ECIES asymmetric encryption and random fragmentation storage are adopted; and high and low sensitive data. Through the dynamic fragmentation encryption module, hierarchical storage and flexible encryption of data are realized, and storage redundancy is reduced; the cross-chain verification engine adopts zero-knowledge proof, and cross-chain verification is safely and efficiently completed; the storage node reputation system is combined with LSTM to predict the node failure probability, and reliable data storage is guaranteed; all work processes are tightly matched, and the whole process ensures data safety and efficient circulation and management.

The invention discloses a data asset secure storage method based on a block chain technology, and the method comprises the following steps: data preprocessing and fragmentation storage: carrying out the preprocessing of to-be-stored original data assets, extracting metadata information, and generating a unique hash identifier; through multi-chain collaborative storage, dynamic hierarchical encryption and cross-chain redundant backup, the security of data assets is remarkably improved, fragment storage is combined with a heterogeneous block chain network, the data storage efficiency is improved, and single-point failure and centralized attack risks are avoided; the anti-cracking capability is enhanced through dynamic encryption and distributed key management; a cross-chain synchronization mechanism ensures high availability of data, so that the data can be efficiently circulated in different block chains; the operation and maintenance cost is reduced through automatic verification and recovery driven by the intelligent contract, the auditing tracking and life cycle management functions meet the compliance requirement, and the method is suitable for data storage scenes in the high-sensitivity fields of finance, medical treatment and the like.

The invention relates to the technical field of distributed data storage, and discloses a data index construction method for distributed data storage, comprising the following steps: constructing a hypergraph model of a distributed storage system through a high-order graph theory, the hypergraph model comprising a plurality of storage nodes, a plurality of data blocks and a plurality of query requests, the method comprises the following steps: performing load balancing optimization on storage nodes and a query request on the basis of a minimum cut theorem in a high-order graph model, mapping data blocks and the storage nodes to a consistent hash ring according to a consistent hash algorithm, reducing data migration through a virtual node mechanism, storing the data blocks on a plurality of storage nodes in a redundant manner by adopting a redundant coding method, and storing the data blocks on the storage nodes in a redundant manner. And a load distribution strategy of the storage nodes is adjusted through a game theory model, and load adjustment is optimized by adopting an optimal control method. The method can ensure high availability, expansibility and reliability of the system while improving the efficiency of the data storage system, and is suitable for a large-scale distributed data storage system.

The invention discloses a smart home early warning method and system based on dual authentication, and relates to the technical field of smart home. Four-element data of an early warning event is collected through an edge computing gateway, the four-element data is coded and packaged into a data packet, and the data packet is transmitted to a cache region of the edge computing gateway for further processing; according to the smart home early warning system based on dual authentication, through an equipment identity dual authentication mechanism, the security of equipment access is doubly guaranteed from a physical layer and a logic layer, intrusion of unknown equipment is effectively prevented, the risk of data leakage is reduced, in the aspect of data storage, an on-chain and off-chain collaborative storage mode is adopted, and the security of equipment access is improved. The authenticity and integrity of event key information are ensured by using the non-tampering characteristic of the block chain on the chain, and the security of original data is protected through distributed storage and encryption under the chain, so that the privacy and security of user data are protected comprehensively, and the security of the data of the smart home system is ensured.