30 results about "In-memory database" patented technology

The application relates to the technical field of flexible direct current power transmission, and particularly provides a lightweight real-time memory database transaction concurrent control method and device, which comprises the following steps: S101. Classifying to-be-processed transactions through cluster analysis to determine the cluster to which the to-be-processed transactions belong; S102. Obtaining a transaction lock corresponding to the to-be-processed transaction in the cluster to which the to-be-processed transaction belongs and a data lock called by the to-be-processed transaction, and executing S103 under the condition that the acquisition state is successful; S103. Conflict verification is performed on the to-be-processed transaction, if there is a conflict, the to-be-processed transaction is prevented, otherwise, the to-be-processed transaction is executed. The technical scheme provided by the application meets the demand of high-efficiency access of Internet of Things data and improves the transaction execution performance of the real-time memory database.

This invention discloses a method and system for accurate electricity consumption data supplementation, relating to the field of electricity consumption data acquisition technology. It includes: the master station dynamically initializes and records the missing curve data information for each electricity meter based on a memory database; when accurate supplementation is triggered, using a single acquisition terminal as the smallest unit, it extracts only the missing data point information from the memory database to generate the missing data point sequence for each electricity meter; through an extended communication protocol for accurate supplementation, it generates an accurate supplementation notification instruction carrying only the missing data point information and sends it to the acquisition terminal; the acquisition terminal extracts the missing data point sequence, retrieves only the missing data corresponding to the missing data point sequence from its local cache, and reports it in batches to the master station; after receiving the missing data, the master station compares and verifies it based on locally maintained missing data status information; after successful verification, it completes the supplementation and updates the missing data marker. This invention effectively improves the efficiency of electricity consumption data supplementation and reduces the resource consumption across the entire link.

A computer-implemented method for regulating delivery frequency of outbound communications. The method includes receiving message context data comprising one or more of a user identifier, a client identifier, a campaign identifier, a channel identifier, a label identifier, and a trigger identifier. A counter-field identifier is determined that encodes the channel identifier, label identifier, trigger identifier, campaign identifier, and a temporal-interval indicator. A counter field identified by the counter-field identifier is created. A value stored in the counter field is incremented through an atomic operation. The value is compared with a limit stored in a rule set. An allow signal is output when the value does not exceed the limit and a suppress signal is output when the value exceeds the limit. An engagement-counter record is updated to the in-memory database.

The application discloses a kind of distributed log's memory database parallel recovery method and system, belong to computer database technical field, the method is: the command log generated in memory database is stored by log partitioning and is physically isolated, and the metadata of each log partition is recorded;Based on pre-received recovery instruction and the metadata, the corresponding recovery plan is generated for each log partition;Parallelly start multiple recovery threads, and based on the recovery plan, the snapshot information pre-acquired in each recovery thread is loaded corresponding log partition, and according to the recovery plan and the snapshot information, the log fragment that needs to be replayed is determined, and the log fragment that needs to be replayed is replayed to corresponding log partition, to carry out parallel recovery to memory database.Therefore, by implementing the present application, the problem that the data recovery reliability of prior art distributed log memory data recovery process is complex and dependent on manual intervention can be solved.

The application relates to a smart rail transit system multi-device association method, system, device and storage medium, the method comprising the following steps: S1, according to whether a device state is affected, an association quantity and association reference information, type classification is performed on a device association relationship, a metadata template under each type is constructed, and the metadata template is stored in a Key-Value in-memory database; S2, based on the metadata template, a device association relationship static model based on a functional requirement is constructed and data storage is performed; S3, device association relationship data is loaded from the Key-Value in-memory database, a device association relationship dynamic model is constructed, and dynamic attribute association is performed; S4, an observer for listening to a device state change event is created on the device, and the device association relationship is updated in real time according to a listening result. Compared with the prior art, the application has the advantages of high reliability, good real-time performance and high flexibility.

The application provides a concurrent memory database splitting method and relates to the technical field of databases. The ART index is improved to obtain an IART index, and an IARTC algorithm is proposed based on the IART index. The IARTC algorithm uses the IART index as a splitting index to drive database splitting. Meanwhile, a new database splitting concurrent algorithm PIARTC is obtained based on the IARTC algorithm. The PIARTC algorithm controls locking and unlocking of nodes on the IART index through an optimistic lock coupling concurrent strategy to realize concurrent splitting and index updating, so that small locking granularity is maintained in the concurrent execution process. Compared with existing algorithms, the two adaptive indexes proposed by the application have good query performance in single-core and multi-core systems in the case of facing dynamic environments and large-scale data set queries, and still maintain the lightweight characteristics of the original database splitting algorithm.

The application provides a blockchain data query method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a target block interval in the case that a new block is generated in a blockchain, wherein the target block interval comprises blocks that may cause a fork and rollback; determining initial data in each block in the target block interval; analyzing the initial data according to a preset condition to generate intermediate data; associating the initial data and the intermediate data corresponding to the same block in the target block interval to obtain target data; and importing the target data into an in-memory database to enable a decentralized application to query data in the in-memory database. The application improves the data query efficiency.

A data processing method applicable to database systems or distributed storage systems, particularly suitable for systems with high memory usage, such as in-memory databases, is disclosed. The method includes: during a snapshot period, writing at least one first object from memory to a snapshot file; wherein the snapshot file includes a first version number and first data content for each first object; writing a second version number and second data content of a second object corresponding to a write request within the snapshot period to a log file; wherein the second object is contained within at least one first object; and wherein the first version number of the first object in the snapshot file and the second version number of the second object in the log file are used to restore the data corresponding to the memory object. Because generating a snapshot does not require blocking write requests, database access latency is reduced. Furthermore, restoring data by comparing the current version number ensures data consistency after restoration.

The application discloses a database management method and system based on mixed index and data layering, and belongs to the technical field of database storage. The method is as follows: according to the use frequency and access attribute of stored data, setting several types of data tables, setting corresponding index modes for different types of data tables; judging the type of the data table to be accessed through a received transaction request, generating an access index; if the access index is a primary index, performing data processing in the node memory through the primary index; if the access index is a mixed index, performing data processing in the node memory according to the data storage position provided by the mixed index. Therefore, by implementing the application, the problem that the existing memory database cannot process massive data when processing high-concurrency tasks can be solved.

This invention discloses an inventory management method and system for a cloud-synchronized Netman shopping cart management system. The method creates a pre-occupancy pool in a Redis in-memory database, containing total inventory, available inventory, cart-occupied inventory, and order-locked inventory. Atomic inventory pre-occupancy operations are implemented using Lua scripts. When a user adds an item to their cart, the system reads available inventory, queries existing occupancy records, calculates the additional occupancy quantity, atomically updates the inventory pool fields, and writes the occupancy information. The system uses a Workerman timer to implement an automatic release mechanism after 1800 seconds, dynamically adjusting the release time based on the inventory occupancy rate. During checkout, the system uses Lua scripts to convert cart occupancy into order locks. This invention solves the performance bottleneck problem of inventory verification in high-concurrency scenarios, avoids overselling and inventory inconsistencies, reduces response time to milliseconds compared to traditional solutions, increases concurrency processing capacity by five times, and improves inventory utilization by 30%.

An orchestration platform leverages an in-memory database for guaranteed synchronous processing of requests from origin devices. The in-memory database is created on a per-transaction basis and localized to a synchronous orchestration engine, which is a part of an orchestration platform. The in-memory database is serialized and stored in memory. To update the in-memory database, the in-memory database is deserialized and one or more database operations can be performed on the in-memory database. The updated in-memory database is communicated to a return to caller service which deserializes and extracts data from the updated in-memory database. The data is stored in a disk-based database that is accessible by an asynchronous orchestration engine. At this point, the updated in-memory database no longer exists.

The application discloses a memory management method and a replaceable cache strategy and system under a DDR-CXL heterogeneous memory structure, and aims at the problem that an existing memory database lacks a NUMA-aware memory management and an efficient cache mechanism under a heterogeneous memory environment.The application constructs a hierarchical memory structure composed of a DDR memory and a CXL extended memory, wherein the DDR memory serves as a cache layer, and the CXL memory serves as a large-capacity data storage layer.On this basis, a NUMA-aware memory allocator is designed to realize unified abstract management of different memory nodes and support dynamic migration of data between the DDR and the CXL memory.Meanwhile, a pluggable cache strategy management mechanism is proposed to support multiple cache replacement strategies through a unified interface, and a cache admission strategy based on access statistics is introduced to reduce invalid data migration and improve cache utilization.The application realizes decoupling of memory management and cache strategy through a plug-in framework, and can improve system performance while enhancing flexibility and scalability of the system.

In an example embodiment, a solution is provided that selects a portion of a non-in-memory database to migrate prior to a database migration option (DMO) performing a downtime portion of the migration. While it is typically not possible for the entire database to be migrated prior to the downtime portion due to the need for certain portions of the database to be used for the DMO portion itself, a substantial portion of the non-in-memory database can be migrated prior to the DMO performing the downtime portion of the migration by using specialized techniques. By migrating a substantial portion of the database prior to the downtime portion, the size of the portion of the database that needs to be migrated during the downtime portion is reduced, thus substantially reducing the length of time needed for the downtime portion.

Embodiments may include a collected statistics data store that contains statistics about information accessed via an in-memory database management system that is executing in a cloud computing environment. A configuration advisor executing in the cloud computing environment may retrieve the statistics about information accessed via the in-memory database management system. Based on the retrieved statistics, the configuration advisor may automatically generate a data storage recommendation for the in-memory database management system. The data storage recommendation may be, for example, associated with page-loadable or column-loadable units and a persistence memory layer. According to some embodiments, the data storage recommendation may be further based on a storage cost, a performance metric, an access frequency, an object threshold (e.g., a threshold for a data column, a data partition, or a data table), recommendation rule-based heuristics, etc.

The invention relates to a process management system and method, in particular to a complex equipment-oriented intelligent management system and method capable of arranging processes and rules, and aims to solve the problems of high coupling of process control and business rules, coding solidification and the like in the prior art. The system comprises a core service component, the core service component comprises a process model library module and a rule model library module which are used for providing a visual arrangement interface, a process engine module for driving process execution and a rule engine module for executing a rule model, and all the modules are in decoupling connection through message middleware; the database component comprises a relational database and a memory database; and a service middleware component. The method comprises the steps of flow and rule visual modeling and association configuration; performing breakpoint debugging execution of the task process; and the rule engine dynamically generates data or instructions according to the model and interacts. According to the invention, decoupling of processes, rules and execution is realized, and the flexibility, maintainability and test intelligence level of the system are improved.

This invention relates to a data modeling technology, specifically a multi-module integrated real-time database system. It includes multiple parallel magnetic core memories, each comprising a synchronization host that establishes unidirectional communication with multiple application systems via a filtering channel controller. The magnetic core memories also include a memory database that interacts with the synchronization host. The synchronization host is also connected to multiple data interfaces for real-time data acquisition and supports multiple real-time data acquisition sources. By constructing a distributed memory database using the synchronization host, memory database, and filtering channel controller, a large amount of real-time data is first received and stored in memory. The memory data is then instantiated in a process-by-process manner, forming a system architecture with excellent scalability. The overall system performance can be adjusted by adding a corresponding number of magnetic core memories according to actual application requirements.

A method for processing batch and live streaming data is disclosed. The method may include receiving a first data object from a data source, the first data object being of information technology event data; processing the first data object; determining, by utilizing a message queue, a first identifier for the first data object; storing the processed first data object in a data sink layer; transferring the first data object to a data warehouse configured to apply an analytic algorithm to the first data object, wherein the transfer is based on the first identifier; determining, by applying the analytic algorithm to the first data object, a third data object; transferring the first data object and third data object to an in-memory database; retrieving and presenting the first data object and / or third data object from the in-memory database to present in a presentation layer of a user interface.

This application provides a data processing method, apparatus, device, storage medium, and program product, which can be applied to the fields of distributed technology and financial technology. The data processing method includes: receiving a data processing instruction for a target product from a scheduling module; sending a data processing request to a data synchronization component, causing the data synchronization component to extract data to be updated associated with the target product from multiple database shards of a persistent database, and aggregating and storing the data to be updated in a memory database corresponding to the target transaction node; and processing the data to be updated in the memory database to obtain updated target data.

This invention discloses a distributed cache coherence system and method based on CXL tunnel transmission coherent bus, belonging to the fields of distributed computing and high-speed interconnect bus technology. Addressing the technical shortcomings of existing distributed architectures based on the CXL protocol, such as high cross-node cache coherence latency, inability to scale without interruption, and low deployment density, this invention encapsulates the coherent bus messages inside the master computing node into the CXL link layer Flit unit for tunnel transmission through a protocol conversion unit. It maps the remote CXL memory pool to the local coherent bus home node, and simultaneously includes a dynamic, uninterrupted topology management mechanism, a high-speed signal and power supply integrated interconnection interface, and an integrated electromagnetic shielding heat dissipation structure. This invention reduces cross-node memory access latency by more than 70% compared to existing solutions, achieves cluster availability of 99.999%, and increases deployment density by more than 40%. It can be widely applied in scenarios such as artificial intelligence computing clusters, heterogeneous computing resource pooling, and distributed memory databases.

The application belongs to the technical field of unmanned aerial vehicle application data processing, and particularly relates to a multi-modal data storage and information retrieval system and method, which comprises: a data access and preprocessing layer, receiving multi-modal data from different platforms and preprocessing, mapping the preprocessed multi-modal data into a high-dimensional semantic space by using a multi-modal large model to generate a unified semantic feature vector; a data storage and index layer, storing the original multi-modal data and the unified semantic feature vector thereof into a relational database, a distributed file system, a vector database or an in-memory database respectively, and constructing a multi-level index structure according to the stored multi-modal data; an application service layer, used for user interaction, receiving a retrieval instruction, and displaying a retrieval result; a retrieval and correlation analysis layer, performing retrieval based on the retrieval instruction, and supporting correlation analysis on three index results of a vector index, a structured index and a space-time index, generating a correlation result and fusing and sorting the index results.

Embodiments of the present application provide a memory database query method and related equipment. The memory database query method is applied to a memory database, and the method comprises: establishing an operator index according to a secondary index of the memory database, the operator index comprising a prefix index, a suffix index, a substring index, a regular index and a regular substring index; receiving a data query instruction, determining an optimal operator index in the operator index; and performing a query using the optimal operator index to obtain a query result. The technical solution of the embodiments of the present application introduces multiple indexes in the memory database, dynamically selects an optimal operator index according to a query instruction, optimizes the query process, enables the memory database to cope with different types of query instructions, improves the overall throughput of the system, and reduces unnecessary computing overhead.

This application provides a call detail record (CDR) data processing method, apparatus, electronic device, and storage medium. The method includes: acquiring CDR data, which includes uplink data and downlink data, wherein the downlink data includes a submission form and a corresponding receipt; determining a message identifier for the CDR data based on the submission form and the corresponding receipt; setting the message identifier as an identifier for a key-value pair and setting the CDR data as the specific value of the key-value pair; and then caching the key-value pair in an in-memory database, which is used to retrieve the CDR data corresponding to the message identifier. In the above implementation, since the in-memory database is directly stored in memory, and the access speed of the in-memory database is much faster than that of a distributed file system, the data user can retrieve the CDR data corresponding to the message identifier almost in real time, thereby effectively improving the real-time performance of associating CDR data.

The application discloses a data backup method and system of an in-memory database, and belongs to the technical field of in-memory database backup.The application determines consistent data and log data of each data partition in the in-memory database through a write-time copy mechanism; then, the consistent data is split to obtain a plurality of data blocks, and hash fingerprint data of each data block is calculated, and then a current hash signature digest tree of each data partition is constructed based on the hash fingerprint data; subsequently, by comparing nodes of a historical hash signature digest tree and the current hash signature digest tree, incremental data of the data partition is obtained; then, the incremental data and the hash fingerprint data are used to generate incremental backup data, and the incremental backup data is associated with the log data to form backup metadata, and the in-memory database is backed up with the backup metadata, so that the efficiency of data backup of the in-memory database can be improved, and the storage space occupation and consumption can be reduced.

The application provides a kind of multitask multi-star telemetry data efficient processing method, system and electronic equipment, it is related to satellite ground telemetry data processing technical field. Including: based on the resource state evaluation result of each computing node in server cluster, task plug-in is dynamically distributed to optimal computing node;Task registration information is uploaded to configuration center, and the telemetry parameter configuration information contained in information is stored in the form of key-value pair in memory database;According to the assigned task plug-in, task telemetry data is extracted from the specified source, and the data is subdivided level by level based on the data shunt model;Based on the obtained telemetry data stream of each frequency band, telemetry source package is extracted, and telemetry source package analysis task is distributed through thread pool.Based on telemetry parameter configuration information, all parameters in source package are calculated by using parameter calculation virtual thread parallel. Solve the defect that high code rate telemetry data cannot be processed in real time in the scene of multi-task multi-star.

This disclosure describes systems, software, and computer implemented methods for generating a flexibly scalable database with native internationalization. This can include partitioning a dataset into a group of subsets based on one or more attributes; initiating a plurality of in-memory database instances in a containerized computing environment, wherein there is at least one in-memory database instance in the plurality of in-memory database instances for each database subset of the group of subsets; loading, by a particular in-memory database instance, a particular database subset; receiving a query at a database gateway from an application layer, wherein the query identifies a particular attribute of data to be queried; identifying, by the database gateway, the particular database subset; directing, by the database gateway, the query to the particular in-memory database instance, and receiving a result; and returning the result to the application layer.

The application relates to the technical field of databases, and discloses a database field duplicate value detection method, equipment and medium, which are applied to migration of unique identifier data of different types of databases to a PostgreSQL database, and comprise the following steps: in a PostgreSQL plug-in, a target table for persistently storing existing unique identifier data is created; an instance of an in-memory database is started, a communication connection with the in-memory database is established, and a hook function for duplicate detection when a unique identifier is generated in the PostgreSQL database is registered; when a duplicate checking function of the PostgreSQL plug-in is enabled, existing unique identifier data that needs to be prevented from being duplicated is imported into the target table, and data in the target table is synchronized to the in-memory database, wherein a unique identifier of each piece of data is stored as a key and is associated with a predetermined value; when an operation of generating a new unique identifier is performed in the PostgreSQL database, the hook function is called to perform a duplicate checking operation.

Embodiments of the present application provide a task processing method, a processor, a server and a storage medium for multiple operating systems, which are applied to an intermediate service cluster. The task processing method comprises the following steps: calling a task acquisition interface of a host service to acquire a to-be-processed task list in the host service; caching the to-be-processed tasks in the task list to a memory database of the intermediate service cluster; acquiring the to-be-processed tasks from the memory database; parsing the to-be-processed tasks to generate corresponding task messages; in the case that it is determined according to the task messages that the to-be-processed tasks are compliance tasks, sending the task messages to a message queue; calling a task processing interface of a scheduling platform to send a task processing instruction to the scheduling platform, so that the scheduling platform processes the to-be-processed tasks corresponding to the task messages in the message queue according to the task processing instruction. The to-be-processed task list in the host service is acquired by the intermediate service cluster and cached to the memory database, so that the task information can be ensured not to be lost.

The application belongs to the technical field of artificial intelligence and scientific data processing, and relates to an intelligent understanding and analysis method for large-scale scientific data and a trusted intelligent agent system. The method comprises the following steps: pre-processing input heterogeneous scientific data, standardizing file formats, and avoiding memory overflow through a streaming conversion mechanism; constructing a dynamic file association knowledge graph, capturing complex semantic dependency relationships among multiple files through a semantic abstraction and similarity merging strategy; planning an optimal file workflow based on user queries and the dynamic file association knowledge graph, generating constraint executable code and executing it in an in-memory database; and generating a trusted analysis result containing reasoning tracks, evidence chains and confidence evaluation through an evidence-to-answer mechanism. The application can realize efficient processing of large-scale heterogeneous data, accurate capture of multi-file association and result traceability, significantly improve the applicability and reliability of scientific research analysis, and support real-time analysis of large-scale multidisciplinary data sets.

The application discloses a space-time trajectory data stream-batch integrated processing method and device and a medium, relates to the field of data processing, and comprises the following steps: receiving original space-time trajectory data preprocessing, distributing to multiple independent topics based on device attribute identification; subscribing and consuming standard space-time trajectory data through a first stream processing job, performing trajectory segmentation on real-time data stream; writing extremely short trajectory segment data into an in-memory database, and writing real-time trajectory segment data into a unified partition table of a database; in parallel, subscribing and consuming standard space-time trajectory data through a second stream processing job, and writing real-time data stream into a data lake storage; triggering a batch processing job, performing trajectory optimization, and updating the unified partition table. Through the parallel processing data and the fusion mechanism of the unified partition table, a unified query view containing real-time segments and optimized trajectories is provided, and real-time trajectory monitoring and historical trajectory optimization are organically unified.