Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

7657 results about "Data chunk" patented technology

Multi-modal document retrieval enhancement generation method based on large model

The invention belongs to the technical field of multi-modal data processing, and particularly relates to a multi-modal document retrieval enhancement generation method based on a large model, which comprises the following steps: receiving query content input by a user for a multi-modal document; processing the query content by adopting an embedded model, generating a query vector representing user query semantic information, and storing the query vector in a vector database; analyzing the multi-modal document to obtain long text information, segmenting the long text information into data blocks by adopting a recursive partitioning strategy, and numbering and marking the data blocks; carrying out vectorization processing on the data blocks by adopting an embedded model to generate document vectors, storing the document vectors into a vector database, and constructing a hierarchical index structure; retrieving in a vector database based on the query vector, and returning a retrieval result; and processing a retrieval result by utilizing a large language model to generate response content conforming to the query intention of the user. According to the method, the multi-modal document can be effectively analyzed and processed, and the accuracy and comprehensiveness of analysis are improved.
Owner:杭州长望智创科技有限公司

Large language model reasoning acceleration method and system based on dynamic video memory compression and memory isomerism

The invention discloses a big language model reasoning optimization method and system based on dynamic video memory compression and memory isomerism, and intelligent management of video memory resources is realized by integrating a dynamic compression strategy of KV Cache and a memory parallel architecture. The method comprises the following steps: 1) analyzing the spatial-temporal characteristics of the KV Cache in real time, adaptively selecting a quantization compression algorithm, a rarefaction algorithm or a low-rank decomposition algorithm, performing hierarchical storage based on attention head importance scores, keeping high precision of a core head, and implementing low-bit quantization on a secondary head; (2) the compressed inactive data are divided into a plurality of data blocks to be stored in a system memory, a parallel data channel group is established according to the number of physical channels, the compressed blocks are concurrently read through multiple channels during loading, and parallel decompression of a sparse matrix is accelerated through a GPU tensor core; and 3) constructing a KV Cache multiplexing mechanism and a parallel channel, and parallelizing a compression / decompression process and model calculation by adopting a hardware acceleration compression and asynchronous pipeline mechanism.
Owner:HANGZHOU AMTD YINGANG DIGITAL TECH CO LTD

Integrated error correction code (ECC) and parity protection in memory control circuits for increased memory utilization

A codeword read from memory includes data blocks including data and supplemental blocks including error correction code (ECC) symbols for detecting and correcting data errors. Metadata can be stored in the supplemental blocks to increase memory utilization but using bits of the supplemental blocks for metadata leaves too few bits remaining for the ECC symbols. To maintain error protection, the supplemental blocks include ECC symbols to protect a first data portion of the codeword and parity bits configured to protect a second data portion of the codeword. Errors in the first data portion can be located and corrected using the ECC symbols. Errors in the second data portion can be detected by the parity. For example, the first data portion is encoded based on the second data portion, so locations of parity errors correspond to locations of symbol errors, and parity errors can be corrected.
Owner:AMPERE COMPUTING LLC

Three-dimensional model adaptive generation method based on video and point cloud data

The embodiment of the invention provides a three-dimensional model adaptive generation method based on video and point cloud data, scene data is collected through a multi-view camera array and a depth scanning array, and key features are extracted and screened by using a hierarchical feature extraction network and an attention mechanism. And organizing local data blocks by adopting a hierarchical index tree based on a union-check set, and realizing self-adaptive parameter optimization by combining integrity and accuracy scores. Meanwhile, efficient reconstruction and material mapping of a local three-dimensional grid are realized through a parallel operation channel and a dynamic calculation resource scheduling mechanism. According to the method, the precision and efficiency problems of a traditional three-dimensional reconstruction scheme in processing a large-scale scene are effectively solved, and the model generation quality is improved.
Owner:UNIVERSAL UBIQUITOUS TECH CO LTD

Intelligent factory data processing method and system based on industrial internet

The invention provides an intelligent factory data processing method and system based on the industrial Internet, and the method comprises the steps: firstly obtaining a real-time industrial data set collected by a multi-mode sensor in a target production region, covering various data, such as equipment vibration signals, carrying out the time sequence synchronization processing of the real-time industrial data set, and generating a synchronization data block; the method comprises the following steps of: extracting dynamic state characteristics of a production line from the data, calling a pre-trained anomaly detection model to carry out multi-dimensional anomaly detection, outputting an anomaly detection result, matching a preset expert knowledge base according to the anomaly detection result, and generating a dynamic control instruction set containing equipment adjustment parameters and the like; finally, production parameter configuration is adjusted based on the dynamic control instruction set and fed back to the industrial control terminal in real time, optimization of current production resource configuration is achieved, and the production efficiency and the resource utilization rate of an intelligent factory are effectively improved.
Owner:HIMIT (SHENZHEN) TECH CO LTD

Parallel synchronization method and system for unstructured files

The invention provides an unstructured file parallel synchronization method and system, and relates to the technical field of file synchronization, and the method comprises the steps: carrying out the intelligent partitioning of a target file through a pre-training file partitioning model, and generating a double-layer cascade index number and a data hash value; and then, according to the data transmission priority model, transmitting the data sub-blocks to a target end database through a self-adaptive parallel transmission channel, and performing asynchronous verification. And finally, recombining the data sub-blocks based on the double-layer cascade index number and the data block association graph by utilizing a pre-training file recombining model, generating a synchronous target file, and submitting confirmation information to the block chain network. According to the method, the unstructured file synchronization efficiency can be improved, the data integrity and security are guaranteed, and the network transmission pressure and the storage cost are reduced.
Owner:北京科杰科技有限公司

Time sequence database data compression method, system and device and storage medium

The invention relates to the technical field of wind power generation equipment data processing. The invention provides a time sequence database data compression method, system and device and a storage medium. The method comprises the steps of dynamically partitioning time sequence data based on a preset time window; hybrid coding compression is performed on each data block, and a composite strategy of difference coding and run length coding is adopted for numerical data; a metadata index of the compression blocks is established, the time range, the data feature statistics and the compression parameters of each data block are recorded, and the metadata index comprises extreme value distribution, variance features and data fluctuation frequency indexes; and dynamically adjusting a compression strategy according to a historical data feature analysis result, predicting data fluctuation modes of different equipment sensors through a machine learning model, and automatically selecting an optimal coding combination and compression granularity for subsequent data blocks. The problems that when wind power plant time sequence data are processed through an existing method, time sequence characteristics are difficult to adapt, the storage cost is high and the query efficiency is low are solved.
Owner:HUANENG DINGBIAN NEW ENERGY POWER GENERATION CO LTD +1

Operator optimization method, electronic device, storage medium and program product

The invention relates to the technical field of artificial intelligence chips, and provides an operator optimization method, electronic equipment, a storage medium and a program product, and the method comprises the steps: determining the block size of operator data in each dimension based on the batch size and mask mode of the operator data and the hardware parameters of computing equipment, each dimension comprises a batch dimension and a sequence length dimension; segmenting the operator data based on the block size of the operator data in each dimension to obtain a plurality of data blocks; and distributing the calculation tasks corresponding to the plurality of data blocks to a plurality of processing units on the calculation equipment, and performing parallel execution on the calculation tasks corresponding to the plurality of data blocks based on the plurality of processing units. According to the method and the device, the data are segmented in the batch dimension and the sequence length dimension at the same time, so that when the data batch is small, a plurality of processing units on the computing equipment can also participate in computing at the same time, hardware resources are prevented from being idle, the utilization rate of the hardware resources is improved, and the overall computing efficiency is improved.
Owner:SHANGHAI BIREN TECH CO LTD

Multi-level cache management method, system and device for solid-state storage device and medium

The invention relates to the technical field of computer storage, in particular to a multi-level cache management method, system and equipment for solid-state storage equipment and a medium. The method comprises the following steps that multiple layers of caches are configured in the solid-state storage device and comprise a high-speed cache layer, a medium-speed cache layer and a low-speed cache layer, and each layer of cache has different access speeds and capacities and is used for storing data blocks with different priorities and access frequencies; dynamically scheduling the data blocks to a proper cache layer through a cache allocation algorithm according to the real-time access frequency, size, life cycle and importance of the data blocks; establishing a comprehensive data block priority evaluation model, and calculating a priority score of each data block; according to the multi-level cache management method, by configuring the multi-layer cache, the priority evaluation model and the multi-path scheduling mechanism, dynamic scheduling and load balancing of the data blocks among different cache layers are achieved, and the overall efficiency of data access is improved.
Owner:SHENZHEN MICRO INNOVATION IND CO LTD

Distributed storage and indexing method

The invention relates to the technical field of information retrieval, and discloses a distributed storage and indexing method, which comprises the following steps of: dynamically fragmenting an original file through an improved consistent Hash algorithm to generate a plurality of data blocks with timestamps; constructing a three-dimensional Bloom filter index matrix containing timestamps, data types and content features for the data blocks with the timestamps, associating a B + tree local index with an inverted global index through a hierarchical index structure, and establishing an index update priority queue by adopting a two-channel synchronization mechanism, performing real-time increment synchronization on the hotspot index through a heartbeat mechanism, performing batch synchronization on the cold data layer index according to a cold data synchronization period, predicting a data distribution probability through a distributed query statistics probability table during query, and initiating multi-path query in parallel based on probability weight, and the query path is dynamically optimized according to the node group storage medium type and the inter-node network transmission delay, so that rapid distribution adjustment and access of distributed storage are realized.
Owner:SHENYANG LIUFANG INFORMATION TECHNOLOGY CO LTD

Data full-flash storage optimization method and system based on cloud computing

The invention provides a data all-flash storage optimization method and system based on cloud computing, and the method comprises the steps: receiving a storage operation request stream of a user terminal through collecting a multi-dimensional performance data set of each node of an all-flash storage cluster in real time, wherein the multi-dimensional performance data set comprises a storage space fragmentation rate, an input / output request queue depth and a network link bandwidth utilization rate; analyzing a to-be-stored data block identifier set and an operation mode label, traversing a historical access record library based on to-be-stored data block identifiers, extracting historical access time sequence characteristics and physical storage position proximity of associated data blocks, and combining the multi-dimensional performance data and the historical access time sequence characteristics to obtain a data block identifier set; according to the method, a dynamic access popularity prediction model in a preset time period is constructed, and finally, a cross-node fragmentation storage topological graph and a cache preloading strategy matrix are generated according to the proximity of the dynamic access popularity prediction model and a physical storage position, so that the optimization of all-flash storage is realized, and the data storage performance in a cloud computing environment is improved.
Owner:YUGE ELECTRONIC TECH CO LTD

AI platform for processing and querying specified objects

An AI based system and method for processing and querying files. A method of processing files for an artificial intelligence (AI) querying service includes: hierarchically parsing the files into a set of hierarchically connected data chunks; generating metadata for each of the hierarchically connected data chunks, wherein the metadata includes hierarchical information; processing the hierarchically connected data chunks and metadata with an embedding model to generate vector embeddings that include the hierarchical information; generating textual summaries from the hierarchically connected data chunks; and storing the vector embeddings, textual summaries, and hierarchically connected data chunks for the AI querying service.
Owner:UTECH PRODUCTS INC

Heterogeneous computing thread block optimal scheduling method and system based on dynamic topology mapping

The invention belongs to the field of parallel computing architecture optimization, and relates to a matrix multiplication acceleration method and system based on dynamic computing resource mapping, and the method comprises the steps: constructing a dynamic topology model driven by tensor dimension features, and generating a thread block distribution mode according to matrix parameters and GPU hardware information; constructing a multi-dimensional resource scheduling strategy library, dynamically selecting an optimal thread block distribution strategy from the multi-dimensional resource scheduling strategy library, and generating a binding relationship between the thread blocks and the data blocks; calculating collaborative access logic of thread blocks and storage hierarchies based on block parameters and dynamic mapping function optimization; distributed calculation is carried out, calculation and data transmission are parallelized through pipelining and a double-buffering mechanism, and result aggregation across calculation units is completed synchronously through atomic operation and a barrier. According to the method, discontinuous memory access conflicts can be effectively reduced, the execution efficiency of the calculation instruction and the utilization rate of the cache space are improved, the parallel calculation process of accelerating and optimizing the general matrix multiplication is realized, and the data processing efficiency is improved.
Owner:SOUTH CHINA UNIV OF TECH

RAID card static cache management method and device based on data popularity

The invention relates to the technical field of data storage and processing, in particular to an RAID card static cache management method and device based on data popularity, and the method comprises the steps that the physical position of a data block needing to be accessed in an SSD array is acquired according to a read-write request; historical data access information of the RAID card is collected, a historical data set is generated, and a neural network model is trained by using the historical data set to obtain a cache management model; predicting and outputting data blocks which are possibly accessed in the future and the access probability of the data blocks through the cache management model; the RAID controller dynamically adjusts a cache strategy in combination with an LRU strategy according to a prediction result output by the cache management model and a read-write request of a file system, and optimizes a storage position and an updating mechanism of a data block in a cache space; and when the hit rate or the data access efficiency does not reach the preset value, adjusting the parameters of the cache strategy and updating the cache strategy. According to the method, cache resources can be more effectively distributed, the cache hit rate is improved, the cache replacement overhead is reduced, and therefore the performance of a storage system is optimized.
Owner:SOUTH CHINA UNIV OF TECH

File uploading method and device, storage medium, electronic equipment and program product

The invention discloses a file uploading method and device, a storage medium, electronic equipment and a program product, and relates to the technical field of computer networks and distributed storage, and the method comprises the steps: collecting metadata of a to-be-uploaded file and a network state of an uploading path between a client and a server in real time; calculating the size of a target fragment of the to-be-uploaded file for data uploading between the client and the server through the metadata and the network state; segmenting the to-be-uploaded file based on the target fragment size to obtain a plurality of data blocks; according to the method, the plurality of data blocks are used for uploading the file from the client to the server, and the uploading parameters of the different data blocks are recorded, so that the uploading result of the file to be uploaded transmitted to the server is determined through the uploading parameters, the technical problem that the uploading efficiency and robustness of the object storage system are poor is solved, and the uploading efficiency of the object storage system is improved. The technical effect of improving the uploading efficiency and robustness of the object storage system is achieved.
Owner:JINAN INSPUR DATA TECH CO LTD

Dynamic data distribution storage method and system for RAID (Redundant Array of Independent Disks)

The invention relates to the technical field of data storage, in particular to a dynamic data distribution storage method and system for an RAID (redundant array of independent disks), and the method comprises the following steps: based on the access frequency of data blocks in a disk, recording the access frequency and timestamp of each data block, carrying out heat rating on the data blocks, and determining thresholds of hot data and cold data. According to the method, the dynamic distribution of data storage is effectively optimized by analyzing the access frequency of the data blocks, and the performance and the economic efficiency of a storage system are remarkably improved by migrating the frequently accessed data to the cache disk and migrating the less accessed data to the low-cost storage; according to the method, unnecessary occupation of high-performance storage resources is reduced, the operation cost is also reduced, heat rating is performed on the data blocks, and the storage positions of the data blocks are correspondingly adjusted, so that the data processing speed is higher, meanwhile, the data security is improved, when a hard disk fails, data recovery is faster, and the service continuity is ensured.
Owner:SHANGHAI FAITH INFORMATION TECH CO LTD

Data association maintenance processing method and system based on file reference

The invention provides a data association maintenance processing method and system based on file reference, and relates to the technical field of data association maintenance, which comprises the following steps: constructing a directed reference graph, identifying direct and indirect reference paths, and constructing a reference dependence intensity matrix in combination with a time sequence, an interaction amount and a reference propagation probability. And carrying out file clustering based on the matrix, and determining a core reference file set. And when the core file is updated, acquiring incremental update data by adopting data block level difference comparison, and merging the data blocks based on the content similarity and the semantic association degree. After the merged data blocks are segmented according to threshold values and semantic boundaries, updating is propagated to a target node through a multi-path updating propagation network, and the data consistency is ensured through version consistency check and a compensation updating mechanism. According to the method, data association between files can be effectively maintained, and the data updating efficiency and consistency are improved.
Owner:NORTH CHINA MUNICIPAL ENG DESIGN & RES INST

Large model data distributed management method and device, equipment and storage medium

The invention discloses a large model data distributed management method and device, equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: segmenting large model training data to obtain a plurality of data blocks, and determining a predicted access frequency based on a long short-term memory network model and the historical access frequency of the data blocks; caching the large model training data corresponding to the data block to a corresponding data cache layer by utilizing the predicted access frequency; constructing a resource portrait by using the static attribute and the dynamic index of the GPU node, and allocating the large model training task to a target GPU node by using a preset hybrid strategy and the resource portrait based on the predicted access frequency and the storage position corresponding to the data block in the data cache layer; when it is monitored that the large model training task on the target GPU node is executed, periodic snapshot is conducted on the large model training task through a distributed snapshot algorithm, and the obtained complete data state is stored in a distributed storage center. And the resource vacancy rate is reduced.
Owner:SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD

Multi-scene adaptive communication method and system of VOIP product

The invention relates to the technical field of VOIP product communication, and discloses a multi-scene adaptive communication method and system of a VOIP product. The method comprises the following steps: carrying out collection and feature extraction on network state parameters to obtain a scene category, and calling reference communication parameters corresponding to the scene category from a scene parameter library; calculating a composite cumulative distribution function based on the reference communication parameters to obtain an optimized communication parameter combination; applying the optimized communication parameter combination to an edge cloud collaborative coding and decoding framework, and carrying out coding and decoding processing on audio and video data to obtain coded communication data; performing block processing on the encoded communication data, and determining a storage position and a communication node of a data block; according to the method, the communication nodes are divided into the master node and the slave node, and the master node and the slave node are subjected to multi-objective optimization to obtain the data transmission path, so that the dynamic optimization of the communication parameters is realized, a VOIP product has scene self-adaptive capability, and the communication service quality of the VOIP product is improved.
Owner:SHENZHEN SKYRISE TECH CO LTD

Agile remote sensing satellite on-satellite full-autonomous task planning system

The invention discloses an on-satellite full-autonomous task planning system for an agile remote sensing satellite. Comprising an imaging target database module, a high-precision orbit prediction and orbit calculation module, a target visible arc section calculation module, an imaging task preprocessing module, a subsystem constraint model module, an imaging task planning module, a data transmission task planning module, a solid memory autonomous erasing module, an instruction data block generation module and a satellite state real-time monitoring module. By carrying out constraint modeling on subsystems or single machines such as load, data transmission, control and the like, an on-satellite full-autonomous operation process from receiving of subsystem or single machine instruction blocks from user requirements is established, satellite control is effectively simplified, and remote control uploading efficiency and on-satellite autonomous operation capacity are improved. And through a satellite state real-time monitoring module, the execution condition of tasks on the satellite, satellite state changes and the like are monitored in real time, task execution stopping, whole satellite sun-facing or re-planning are further triggered, task requirements and environment changes are quickly responded, task plans are formulated and adjusted in real time, and the task execution efficiency is improved.
Owner:BEIJING INST OF CONTROL ENG

Solid state disk data reading method and system, storage medium and program product

The invention discloses a solid state disk data reading method and system, a storage medium and a program product, and the method comprises the steps: calculating a temperature change rate and a temperature accumulation effect coefficient of a to-be-read data block according to a current temperature value and a historical temperature record; dividing the physical page address into a plurality of subintervals, and determining a loss level; calculating an initial reading voltage threshold value; compensating the initial reading voltage threshold value to obtain a target reading voltage threshold value; performing parallel reading on the to-be-read data block by adopting the target reading voltage threshold value to obtain original reading data; performing crosstalk correction on the original read data to obtain corrected data; when the reliability verification result of the corrected data of any subinterval is lower than a preset threshold value, error correction processing is carried out; and updating the loss level of the sub-interval according to the verification result of the reliability verification, and re-dividing the sub-interval of the data block to be read. The data reading delay is reduced, and the performance of the solid state disk in a scene of frequently reading small files is improved.
Owner:SHENZHEN WEIKEWEIYE ELECTRONIC TECH CO LTD

Database load optimization method and device based on dynamic fragmentation, equipment and medium

The invention relates to the technical field of big data and the field of financial science and technology, and discloses a database load optimization method based on dynamic fragmentation, which comprises the following steps: establishing an association relationship between fragments and storage nodes by configuring basic logic fragments and dynamic logic fragments; monitoring a data access record, identifying hotspot data and marking the hotspot data as a to-be-migrated data block; when the fragment load exceeds a threshold value, migrating the high-load data block to the dynamic logic fragment; and when the access frequency is reduced, moving the low-load data block back to the basic logic fragment. According to the method, through configuration and management of the dynamic logic fragments, in combination with real-time monitored access records and load state analysis, the high-load data blocks are recognized and migrated to the dynamic logic fragments for storage, and the overload problem of the basic logic fragments is relieved. Meanwhile, the low-load data blocks are moved back to the basic logic fragments, dynamic adjustment and balance of storage resources are further achieved, the load balancing capacity of the system is improved, and the requirements for high concurrency and real-time load changes in a financial scene are met.
Owner:JINZHUAN INFORMATION TECHNOLOGY CO LTD

Operator parallel processing method, electronic equipment, storage medium and program product

The invention relates to the technical field of artificial intelligence, and provides an operator parallel processing method, electronic equipment, a storage medium and a program product, and the method comprises the steps: determining a step of data dependence based on each processing step of a first operator and a second operator; and performing parallel execution on each processing step of the first operator and the second operator by applying a synchronization strategy and a segmentation strategy based on the plurality of cooperative thread bundles. According to the method, the parallel fusion processing of operators is realized through the steps of determining data dependence, segmenting the data, realizing parallel execution by utilizing the cooperative thread bundle, inserting synchronization between the data dependence steps and the like, the calculation performance is remarkably improved, and meanwhile, the data blocks after segmentation are multiplexed, so that the data processing efficiency is improved. The data dependence in the operators and between the operators can be reduced, and the overhead of data reading, instruction synchronization and the like can be reduced.
Owner:SHANGHAI BIREN TECH CO LTD

Electronic data storage verification method and system based on hierarchical hash and smart contract

The invention provides an electronic data evidence storage verification method and system based on hierarchical hash and an intelligent contract, and relates to the technical field of block chains and data security, and the method comprises the steps: obtaining to-be-stored electronic data; to-be-stored electronic data is divided into data blocks according to a preset rule, a unique identifier and a timestamp are added to each data block, and a standardized data unit is generated; performing hierarchical hash calculation on the standardized data unit, generating a leaf node hash value by each data block, aggregating the leaf node hash values layer by layer according to a Merkle tree structure, and generating root hash; calling a block chain smart contract, submitting the tree structure hash, the timestamp and the public key encrypted signature to a block chain network, after verification of a consensus node, writing the evidence information into the block chain, analyzing a hash path submitted by a user by the smart contract to perform data verification, if the hash path is consistent with the data verification, returning verification success, otherwise, triggering an exception handling contract; through the dynamic partitioning algorithm and the layered hash structure, the calculation overhead of large file processing is reduced.
Owner:SHANDONG UNIV OF POLITICAL SCI & LAW

Distributed account book remote data synchronization method and system

The invention relates to the technical field of data synchronization, and discloses a distributed account book remote data synchronization method and system, and the method comprises the steps: carrying out the Hash mapping calculation of all nodes of a distributed account book system, and obtaining a distributed storage architecture; performing hierarchical network construction on the nodes, and setting an inter-region remote synchronization link and an intra-region local synchronization link to obtain a double-layer communication architecture; performing parallel compression coding, digital signature generation, transmission scheduling arrangement and consistency verification processing on the data blocks to obtain a node synchronous data set; calculating a correlation coefficient of node processing delay and network transmission delay to obtain a synchronous delay analysis model; based on a data block corresponding to the synchronous delay analysis model, adding a time sequence identifier and calculating an integrity check code, and generating a synchronous reliability guarantee mechanism; according to the method, the consistency and the traceability of the distributed account book data are ensured.
Owner:SHENZHEN LINGZHU TECH CO LTD +1

File high-speed transmission system based on multilink aggregation

PendingCN120358230ATransmissionHigh level techniquesFile transmissionDynamic load balancing algorithm
The invention provides a file high-speed transmission method based on multi-link aggregation, and solves the problems of low transmission efficiency and insufficient stability of a traditional single link. The method comprises the following steps: detecting network link performance parameters, and screening available links; according to the number of links, the average bandwidth and # imgabs0 # limitation, dynamically segmenting a large file into adaptive data blocks, numbering the data blocks, and adding information such as a position to a head; an improved dynamic load balancing algorithm is adopted, link quality (including bandwidth utilization rate, delay, packet loss rate, stability and dynamic adjustment of weight along with a network environment) and data block priority (distinguished according to types and sizes and the weight is adjusted according to file characteristics) are synthesized, data blocks are distributed to different links for parallel transmission, and pre-distribution verification ensures link load safety; and recombining the data blocks by the receiving end according to the numbers to realize file recovery. The system significantly improves the transmission efficiency through multi-link aggregation and dynamic strategy adjustment.
Owner:南京通达海软件有限公司

Data distributed storage method and system based on secret sharing technology

The invention discloses a data distributed storage method and system based on a secret sharing technology, belongs to the technical field of data storage, and aims to solve the technical problem of how to realize the security and privacy of data in distributed storage based on the secret sharing technology. Comprising the following steps: dividing original data into a plurality of data blocks, and encrypting the data blocks through a symmetric key; segmenting the symmetric key into a plurality of key fragments based on a Shamir threshold scheme, and storing the plurality of key fragments in a plurality of different nodes in a distributed manner; dividing the encrypted data block into a plurality of data fragments based on a Shamir threshold scheme, and storing the plurality of data fragments to a plurality of different nodes in a distributed manner; the method comprises the steps of collecting key fragments based on a Shamir threshold scheme, carrying out symmetric key reconstruction on the key fragments collected based on the Shamir threshold scheme, collecting a predetermined number of data fragments based on the Shamir threshold scheme, carrying out encrypted data block reconstruction on the data fragments, decrypting the reconstructed encrypted data block based on the reconstructed symmetric key, and recovering the data block.
Owner:INSPUR SOFTWARE TECH CO LTD

Distributed storage method and system for clinical test data

The invention relates to the technical field of data distributed storage, and discloses a distributed storage method and system for clinical test data, and the method comprises the steps: carrying out the ring signature encryption processing of the patient test data of a plurality of data centers, and obtaining a patient anonymization identifier and standardized clinical data; performing two-stage random storage node configuration to obtain an optimal storage node configuration scheme; constructing an LPPMM-DA data aggregation model based on the optimal storage node configuration scheme, and performing fragmentation processing on the patient test data to obtain a plurality of target data fragments; performing dynamic operation envelope analysis and alliance graph game calculation on the multiple target data fragments to obtain a storage resource scheduling scheme; according to the storage resource scheduling scheme and the anonymization identifier, distributed data block storage and access authentication are executed on the multiple target data fragments, data block storage distribution information is generated, a complete data access control system is constructed, and safe and reliable storage of clinical test data is ensured.
Owner:BEIJING HOUPU PHARM TECH CO LTD

Page progressive rendering method and system based on streaming data

The invention relates to the technical field of page rendering, and discloses a progressive page rendering method and system based on streaming data, and the method comprises the steps: carrying out the data segmentation through obtaining a data stream, user interaction data and equipment performance parameters, and obtaining data blocks; then, performing cache management in combination with the data, and constructing a multi-level cache pool; thirdly, performing priority grading and sorting on the data blocks to form a rendering task queue; and according to the equipment performance parameters, optimizing a task sequence and obtaining an optimized task sequence. Next, combining the optimized task sequence and user interaction data, predicting data about to enter a viewport, and generating a viewport pre-rendering task; and finally, according to the viewport pre-rendering task, the multi-level cache pool and the equipment performance parameters, performing rendering strategy optimization to obtain a dynamically adjusted rendering task flow. The method can realize dynamic resource scheduling.
Owner:DEEP BLUE INTERNET (BEIJING) TECHNOLOGY CO LTD

Object-based storage with garbage collection and data consolidation

Embodiments are directed to a file system that include object stores. An object store for write requests may be provided. Write ahead log (WAL) entries that include data blocks may be generated. A WAL object may be generated based on the WAL entries and stored in the object store. An in-memory overlay may be updated to associate the data blocks with the WAL object. A checkpoint operation may be executed to: generate an index object that includes index entries that associate other data blocks with data objects stored in the object store; update the index object to include index entries that associate the data blocks with the WAL object; store the updated index object in the object store; update the in-memory overlay to remove the association of the data blocks and the WAL object and update the in memory WAL to remove records of successfully checkpointed WAL objects.
Owner:QUMULO INC