1325 results about "Data storage system" patented technology

A data storage system can implement a key-value engine configured for tunable read, write, and space amplification. The key-value engine can support multi-versioning, synchronous and asynchronous key updates, and read snapshots. The key-value engine is highly scalable and can support generalized parallel, in-memory computation. Experimental results demonstrate that a key-value engine consistent with disclosed embodiments can outperform a state-of-the-art production LSM-based key-value store in a wide range of metrics.

The invention relates to the technical field of natural language processing and artificial intelligence, and discloses a silk path knowledge base intelligent question answering system based on a large language model, comprising a silk path data acquisition module used for acquiring multi-source silk path knowledge data; the silk path knowledge base construction module is used for processing the multi-source silk path knowledge data to construct a structured knowledge base and an unstructured knowledge base; the data storage system module is used for storing a structured knowledge base and an unstructured knowledge base and providing a structured retrieval interface and an unstructured retrieval interface; and the man-machine interaction module is used for identifying a user intention according to the user question and outputting an answer corresponding to the user question by adopting a corresponding answer generation mode based on the user intention. According to the method, efficient and accurate silk path knowledge questions and answers are realized by constructing the multi-modal knowledge base and optimizing a retrieval algorithm and a prompt engineering technology.

The invention discloses a data storage system of a digital country based on cloud computing, which comprises a client, a data transmission module and a cloud computing center, relates to the technical field of data storage, stores hot data on a high-performance hot data storage layer, ensures that frequently accessed data can be quickly responded, improves the system performance, and at the same time, stores the hot data on a high-performance hot data storage layer. The cold data is stored in a low-performance cold data storage layer, the storage cost is reduced, effective balance of performance and cost is achieved, the data is automatically migrated among different storage layers along with change of data access frequency, so that the performance and the cost are balanced, the frequently accessed data is migrated into a cache, the access speed of the data is increased, and the data access efficiency is improved. And establishing a relationship between a time interval and a data type of historical data, predicting data which are possibly accessed in a given time interval, adding the data into the cache, dynamically adjusting the cache, and ensuring the validity of the cache.

The invention belongs to the technical field of intelligent power grids, and particularly relates to an intelligent power grid monitoring data storage system which comprises a data acquisition module, a data preprocessing module, a hierarchical data storage module, an intelligent data retrieval and management module and a data analysis and visualization module. Power grid operation data are collected in real time through various sensors; the data preprocessing module is used for cleaning, denoising and formatting the collected data and identifying data abnormal values by adopting a dynamic threshold detection method; the hierarchical data storage module adopts a hierarchical storage strategy of hot data, temperature data and cold data, stores the data in different media respectively, and introduces a time sequence database; the intelligent data retrieval and management module retrieves the stored data by using a multi-condition complex query algorithm, and improves the query efficiency through an intelligent data indexing and retrieval mechanism; and the data analysis and visualization module generates a report and a visualization chart through an advanced data analysis algorithm.

A method is provided for caching a query submitted to a graph data storage system, including receiving a query from a query caller directed to the graph data storage system, and examining the query, via a cache suitability determination engine, to determine if the query is suitable for caching under a first graph query caching policy based on the query caller being a trusted caller and / or the query scenario being a trusted query scenario. If suitable, generating a query execution plan for the query via a graph query plan generating engine and examining the query plan to determine if the query plan is suitable for caching under the first policy. If not suitable, caching the query under a second or a third graph caching policy based on the geographic location of a memory system on which the query plan is stored, and providing the query execution plan.

A method and apparatus for cache warming in a distributed storage system is described. The method can include detecting a destructive change to one or more nodes of an existing cluster of cache data nodes. The method can also include allocating a new cluster of cache data nodes in a write-only mode, and streaming data from each cache data node of the existing cluster to cache data nodes of the new cluster. The method can further include servicing a data access request from a selected cache data node of the existing cluster while writing data from the data access request to a selected cache data node of the new cluster. Furthermore, the method can include in response to a determination that data from the cache data nodes of the existing cluster has been successfully streamed to the new cluster, servicing new data access requests with the new cluster.

The invention relates to a data access prediction scheduling method and system based on a dynamic threshold value, and belongs to the technical field of data processing. The method comprises the following steps of: dynamically adjusting an abnormal detection threshold in an access behavior by utilizing reinforcement learning and an error feedback mechanism, realizing preposition identification of potential high-frequency access data by combining abnormal point classification processing and access trend prediction, and generating a scheduling request to drive cache preheating or interface priority scheduling of the data. According to the method, access behavior modeling, anomaly recognition, trend prediction and strategy feedback mechanisms are combined, interface access optimization requirements in a mass data storage system are met, and the key problems that access behaviors are unpredictable, scheduling response lags and the like are solved.

The invention discloses a bar code and RFID dual-mode fused tamper-proof data storage system, and relates to the field of data security storage, which comprises the following steps: uploading complete data to be stored into a scanning monitoring platform, splitting the data into static data and dynamic data, writing the static data into a bar code module, and writing the dynamic data into the bar code module; randomly distributed initial fluorescent particles are generated on the surface of the bar code data label through a scanning monitoring platform, a unique initial optical fingerprint is generated, if a bar code module is tried to be written or modified, the distribution position of the initial fluorescent particles is modified, an abnormal optical fingerprint is obtained, and an alarm is given; and writing the dynamic data into the RFID module, generating a dynamic key in each communication, and performing cross validation on the hash value based on the static data and the dynamic data. The method has the advantages that a three-dimensional protection system is constructed through a dual tamper-proof mechanism of a physical layer of fluorescent particle displacement and a password layer of a dynamic key in combination with cross validation of a hash tree, and the tamper-proof capability is improved.

A computer-based system and corresponding computer-implemented method process input data queries in a manner enabling advanced functionality for data analytics. The method transforms a query plan tree into a query strategy tree. The query plan tree is constructed from an input data query associated with a computation workload. The method compiles the query strategy tree into dataflow graph(s) (DFG(s)). The method transmits the DFG(s) for execution via a virtual platform. The method monitors execution of the DFG(s). The method outputs, based on a result of the execution monitored, a response to the input data query. The result is received from the virtual platform and represents at least one computational result of processing the computation workload by the virtual platform. The system and method enable rapid and efficient retrieval and analysis of data stored in data lakes and other data storage systems, in response to user queries.

A distributed data storage system includes clusters of data storage appliances interconnected by an inter-cluster (IC) network having an IC namespace. Storage processing (SP) nodes exchange management traffic using mesh network (MN) addresses of a separate MN namespace. Gateways provide IC tunnels for routing management traffic among the clusters using IC network addresses. Operation includes, in each gateway for traffic from a local SP node destined for a remote SP node of another cluster, (1) performing a routing check based on a routing rule associating an MN destination address with a dummy MN address further associated with a respective IC tunnel, (2) applying a tunnel configuration rule of the IC tunnel associating the MN destination address with the IC network address of a remote gateway for the remote SP, and (3) forwarding the traffic on the respective IC tunnel using the IC network address of the remote gateway.

The embodiment of the invention discloses a hard disk fault prediction method and device, electronic equipment and a storage medium, and relates to the technical field of reliability of data storage systems.The method comprises the steps that hard disk data are obtained, data cleaning and feature selection are conducted on the hard disk data to obtain feature data, and the feature data are stored in a database; performing standardization and normalization on the feature data to obtain a data set, using a long-short term memory network as a deep learning model, designing a model structure, introducing a regularization technology and a dropout layer to construct an initial model, training and optimizing the initial model by using the data set to obtain a trained prediction model, and performing prediction on the trained prediction model. And performing state monitoring, fault early warning and maintenance on the hard disk by using the prediction model. According to the method, efficient and accurate hard disk state monitoring, fault early warning and maintenance are achieved through data preprocessing and the LSTM network deep learning model, and the problems that in the prior art, changes of the running state of the hard disk cannot be accurately captured, and data loss and system crash cannot be prevented are solved.

The invention discloses an equipment management method based on data acquisition, and relates to the technical field of equipment management, and the method comprises the following steps: S1, selecting and configuring a sensor; s2, establishing a data acquisition protocol; s3, deploying data acquisition hardware; s4, designing and applying a data storage system; s5, performing data cleaning and preprocessing; s6, deploying an analysis model; s7, establishing a real-time monitoring system; s8, optimizing a maintenance strategy; and S9, performing regular evaluation and optimization. According to the equipment management method based on data acquisition, the running state of the equipment is monitored in real time through real-time data acquisition and analysis, and potential faults and performance problems are found in time. In combination with an advanced data processing technology, such as a machine learning analysis model, future behaviors of the equipment can be learned and predicted from historical data, so that preventive maintenance is realized, and risks of equipment failure and production interruption are reduced. In addition, a condition-based maintenance policy replaces traditional periodic maintenance.

The invention relates to the technical field of distributed data storage, and discloses a data index construction method for distributed data storage, comprising the following steps: constructing a hypergraph model of a distributed storage system through a high-order graph theory, the hypergraph model comprising a plurality of storage nodes, a plurality of data blocks and a plurality of query requests, the method comprises the following steps: performing load balancing optimization on storage nodes and a query request on the basis of a minimum cut theorem in a high-order graph model, mapping data blocks and the storage nodes to a consistent hash ring according to a consistent hash algorithm, reducing data migration through a virtual node mechanism, storing the data blocks on a plurality of storage nodes in a redundant manner by adopting a redundant coding method, and storing the data blocks on the storage nodes in a redundant manner. And a load distribution strategy of the storage nodes is adjusted through a game theory model, and load adjustment is optimized by adopting an optimal control method. The method can ensure high availability, expansibility and reliability of the system while improving the efficiency of the data storage system, and is suitable for a large-scale distributed data storage system.

A system and method provide streamlined restricted access to a secure server through a communications network. A client identifier parameter value is established and uniquely associated with a user registering with an authentication server, and is stored in at least first and second predetermined storage forms within a data storage system, the first form readable exclusively by a client device of the user and the second form readable by the authentication server. The client device then authenticates by retrieving the client identifier parameter value from the data storage system and providing it to the authentication server, which independently retrieves the client identifier parameter value from the data storage system for comparison, and initiates an interactive communication session between the client device and the secure server responsive to the comparison. Between comparisons, the client identifier parameter values are stored exclusively on the data storage system and deleted from all other devices.

An embodiment of the disclosed technology reduces direct data transmission and reception between a plurality of host devices and enables data sharing by a data storage device, through management of allocation information and access mode information of memory regions included in the data storage device used by the plurality of host devices, thereby improving data processing performance by the plurality of host devices and the use efficiency of the data storage device.

A dynamic marketplace system leveraging store and warehouse mobility features a neuroevolution (NE) engine, an electronic device, and a request handler facilitating communication between the electronic device and the NE engine. The NE engine interfaces with a data storage system and an event handler receiving real-time event data from a public cloud services processor. An intentions handler interprets user intention data to predict user behavior. The NE engine, integrated with a processor, generates a predictive evolutionary model for the marketplace based on request, event, and intention data. An AI agent processor within the NE engine creates a recommendation model for mobile retail vendors, devises route plans, and deploys vendors to strategic locations.

Systems and methods for source chain management with identity-based tokens and enhanced security are disclosed. A processing system receives input that includes a decentralized identifier (DID) uniquely associated with an entity. The processing system invokes a smart contract deployed on a distributed ledger to mint a non-transferable identity-based token permanently linked to the DID. Metadata generated for the token includes cryptographic references to a hierarchical source chain structure stored in a decentralized data storage system, along with cryptographic verification data. Events associated with the DID are memorialized by appending granular event details, including timestamps, data changes, and origin identifiers, into the hierarchical source chain. Authorized access to event details and tokens is provided via an authenticated interface that securely retrieves and reconstructs encrypted data shards distributed across decentralized nodes, enhancing data security, immutability, and verifiability in decentralized environments.

The invention relates to the field of heterogeneous data storage, in particular to a heterogeneous data storage method and system for a civil aviation data platform, and the method comprises the steps: collecting metadata of each subsystem in the heterogeneous data storage system in real time, inputting the metadata into a pre-constructed metadata unified model, and carrying out standardization processing to generate a global metadata view; establishing a data traceability tracking mechanism according to the global metadata view, and mapping dispersed metadata items and data entity operation records according to the data traceability tracking mechanism to generate a traceable metadata change track; and a metadata quality label is embedded in the metadata unified model, a metadata knowledge graph is constructed according to the metadata change track and the metadata quality label, and a resource allocation strategy of the heterogeneous data storage system is dynamically optimized according to an analysis result of the metadata knowledge graph. The problem of low storage management efficiency caused by difficulty in integrating and tracing metadata in a heterogeneous data storage system in civil aviation data is solved.

The invention provides a multi-dimensional accounting data integration and analysis system, and belongs to the technical field of accounting information processing, the multi-dimensional accounting data integration and analysis system comprises a data acquisition system, a data source interface unit of the data acquisition system can be connected with a plurality of data sources, and an acquisition control unit manages acquisition; the data integration system comprises a preprocessing module (format conversion, data cleaning and fusion modules); the data analysis system is provided with an analysis dimension setting module, an algorithm execution module and a result visualization module; the data storage system comprises a storage architecture management and index module, solves the problems that a traditional method is difficult in data source integration and single in analysis dimension, can effectively integrate multi-dimensional accounting data, improves the comprehensive analysis capability, improves the data security and query performance, and visually presents an analysis result to facilitate decision making.

The invention discloses an electric meter data storage method of a single-phase cost-control electric energy meter, and relates to the technical field of intelligent electric meter data processing, and the storage method comprises the following specific steps: S100, data acquisition and characteristic pro-analysis: acquiring voltage, current and power data of the single-phase cost-control electric energy meter through a voltage sensor and a current sensor, by integrating the voltage sensor and the current sensor, real-time and accurate acquisition of key power utilization data of voltage, current and power of the single-phase charge-control electric energy meter is realized, and by introducing a dynamic time window setting mechanism, the length of a time window is automatically adjusted according to the fluctuation amplitude of the data, so that the accuracy of power utilization is improved. It is ensured that a window is shortened to capture transient changes when data fluctuates violently, the window is prolonged to reduce redundant collection when the data is stable, the flexibility and accuracy of data collection are improved through intelligent adjustment, subsequent data processing can be conducted based on richer and more detailed feature information, and the data collection efficiency is improved. And the intelligent level of the whole data storage system is obviously improved.

System-level garbage collection is performed in an SSD-based data storage system employing log-structured writing of data. Respective reclaim unit sizes of the storage devices are identified, wherein each reclaim unit is a unit of programming and erasure used by device-level garbage collection to generate erased reclaim units for allocation for physical data storage. The data is arranged across the storage devices in first and second data units, each first data unit being of a first size U equal to the product of R and a stripe width W, each second data unit including a respective set of first data units which occupy a maximum-size reclaim unit of a storage device among those storing the first data units. The system-level garbage collection is operated with second-data-unit granularity to regularly generate empty second data units for allocation in the log-structured writing of data, reducing write amplification and device wear.

The invention discloses a self-healing distributed data storage method and system. The system comprises a message bus and a distributed cluster, the message bus is used for realizing cluster, node and data management; the distributed cluster comprises a master node and a plurality of slave nodes, and each node comprises a management module, a monitoring module and a storage module; the master node is used for processing all write operations, maintaining the latest state of data, and synchronizing data change to the slave nodes to ensure the consistency of the data; the slave nodes are used for data replication and fault recovery, and data consistency is maintained; when the master node breaks down, the slave nodes are promoted to be the master node according to a self-healing mechanism; the management module is used for communicating with a message bus to realize cluster management and system self-healing; the storage module is responsible for data storage, and data storage is optimized by adopting a data fragmentation strategy. According to the method, system self-healing can be automatically completed when a physical node or a network fails, and the characteristics of efficient access and high availability of the distributed data storage system are achieved.

The present invention provides a high-frequency industrial time series data storage system based on double caching. A double caching strategy is used to isolate data acquisition and data storage, the structure is simple, and the modules are clear, ensuring that the data processing process is not blocked and other associated modules are not affected, improving the overall stability of the system, ensuring the reliability and timeliness of data storage, and achieving the function of storing time series data in massive, high-concurrency, high-frequency, and other scenarios. The present invention has higher performance and better expandability. By means of hierarchical design and reasonable functional module division, the requirements of high throughput, low delay and high expandability are met, thereby better supporting the application of industrial time series data. According to the present invention, a message layer and a caching layer are introduced, achieving real-time data processing and caching, improving the data processing efficiency, and reducing the data processing delay.

Data payloads from an external data storage system are processed in an observability pipeline system. In some aspects, the observability pipeline system defines a leader role and worker roles. The leader role generates a data discovery task based on configuration information for a data collection task. A worker role executes the data discovery task, which includes communicating with an external data storage system to identify a data payload that is stored on the external data storage system and contains event data that meet event filter criteria. The leader role generates data collection tasks based on the data payload. Worker roles execute the data collection tasks. Executing a data collection task includes: communicating with the external data storage system to obtain a subset of filtered event data from the data payload; and streaming the subset of filtered event data to an observability pipeline process.

Data storage systems are disclosed. In some implementations, a data storage device includes a controller coupled to a plurality of memory devices to control memory operations of the plurality of memory devices, and including M communication lanes, and an interface device configured to include N communication lanes to be connected to a host device that includes N communication lanes and to the M communication lanes of the controller, wherein the interface device includes a connector configured to connect the N communication lanes of the host device to the N communication lanes of the interface device, and a selector configured to, in response to a host port type identification signal received from the host device, selectively activate M communication lanes of the interface device, out of the N communication lanes of the interface device, to transmit or receive the data to or from the M communication lanes of the controller.

Disclosed systems and methods convert user-supplied textual content into animated videos. Textual input is received and processed to identify narrative elements such as characters, settings, and events. These elements are then transformed into visual scene components. Still images generated based on these components are subsequently animated in line with the narrative context. The system can automatically implement storytelling techniques adapted to incorporate neuroscience principles. The system also synthesizes speech for dialogues or narrations using voice synthesis technology that considers emotional markers, tone, and pace. Generated media and metadata are stored in a data storage system that maintains data integrity and enables efficient retrieval. Users can interact with an export interface to choose video resolution, format, and sharing options. A feedback system employing machine learning algorithms collects and analyzes user feedback for real-time adjustments to the generated animated video.

The invention is suitable for the technical field of payment data system vulnerability assessment, and provides a payment data security assessment method and system.The method comprises the steps that after vulnerability scanning and penetration testing operation is conducted on a payment data storage system, a plurality of scanned security defects are determined, and acquiring a preliminary risk priority evaluation index allocated to each security defect, and acquiring a historical operation log of the payment data storage system. According to the method, the mechanism of double correction factors is introduced, so that the repair priority evaluation of the potential risk vulnerabilities in the payment data storage system is optimized. Firstly, in combination with a hysteresis effect correction factor and a secret key updating frequency deviation correction factor, the accuracy and timeliness of vulnerability repair are effectively improved;

The invention discloses a database data storage system based on multi-level cache, and relates to the technical field of database data storage systems, the database data storage system comprises a basic storage layer, a first-level cache layer, a second-level cache layer, a cache cooperative control module and a data read-write scheduling module, the basic storage layer is used for persistently storing total data of a database; the first-level cache layer is divided into a plurality of hot data partitions; and the second-level cache layer is provided with quasi hot data partitions in one-to-one correspondence with the hot data partitions. According to the database data storage system based on the multi-level cache, the system constructs a'high frequency-secondary high frequency-full quantity 'layered architecture through the first-level cache, the second-level cache and the basic storage layer, dynamic flow of data is achieved in cooperation with a data aging counter and a heat recovery detector, heat reduction data is migrated from the first-level cache to the second-level cache, and the heat reduction data is transferred to the second-level cache. And the heat recovery data returns to the first-level cache from the second-level cache to ensure that the high-frequency data is always kept in a cache layer.

The invention discloses an intelligent RAG enhanced large model auxiliary writing system, and relates to the technical field of artificial intelligence auxiliary writing. In order to overcome the defects of an existing writing auxiliary tool, the adopted scheme comprises an interface and user layer for receiving a writing demand and redisplaying a generation result; the service layer is used for deploying three types of intelligent agents including writing, interactive writing enhancement and checking; the platform layer is used for carrying out industry adaptation analysis, mixed retrieval and RAG enhancement on writing demands and then sending the writing demands to the large model layer; three intelligent agents of writing, interactive writing enhancement and checking are called in sequence based on the output of the large model layer, and then a text is redisplayed to the interface and user layer after fact accuracy verification is completed through a tense knowledge graph; the data layer is used for providing a multi-type data storage system; the large model layer outputs a paragraph-level text and a reference index based on a writing large model, the paragraph-level text is returned to the platform layer, and reference index information is fed back to the interpretability and security layer for tracing; and the infrastructure layer is used for providing bottom layer resource support for each layer.

A computer-based system and corresponding computer-implemented method process computation workloads to enable advanced functionality for data analytics. An execution resource is selected from a set of execution resources of a virtual machine (VM). The resource is for executing a VM instruction. The VM instruction is transformed into machine code for the resource selected. The code is executed via the resource selected. The executing furthers execution by the VM of a dataflow graph (DFG) including a compute node. The compute node has a set of VM instructions including the VM instruction. The DFG corresponds to a portion of a computation workload associated with a user data query. An output of the execution of the DFG represents a result of processing the workload and contributes to a response to the query. The system and method enable rapid and efficient retrieval and analysis of data in data storage systems, based on computation workloads.