251 results about "In-memory database" patented technology

The invention relates to a method and a system for storing and backing up electric power data of a data-in-data station based on use popularity, and belongs to the technical field of data-in-data stations, and the method comprises the following steps: classifying electric power business data into hot data, temperature data and cold data based on preset access frequency, time window and data volume index; according to a classification result and a business importance degree, storing hot data in an SSD cloud disk or a memory database, storing temperature data in a distributed column database, and compressing cold data and then archiving the cold data to an object storage or a tape library; dynamically configuring differentiated backup strategies for different types of data; automatic scheduling of storage and backup is realized through a cloud platform component, and hot switching of master and slave databases, OSS archived data recovery by a DataWorks tool and a remote multi-copy recovery mechanism are supported; and migrating data in advance in combination with a historical access prediction popularity trend, and triggering cold data compression and archiving when storage resources exceed 80%. The method is suitable for power industry scenes, the storage cost is optimized, and the data calling efficiency is improved.

The invention discloses a real-time interaction and intelligent management method for a multi-dimensional data space, and relates to the technical field of data interaction management, and the method comprises the following steps: I, collecting multi-source data, including structured data, unstructured data and spatio-temporal data, in real time through a diversified collection terminal, and dynamically adjusting the sampling frequency based on a data entropy value, carrying out cross-device space-time synchronization calibration and edge end lightweight processing on the acquired data; iI, preprocessing the data flow in real time, fusing the multi-modal data, and the like. According to the method, the sampling frequency is dynamically adjusted based on the data entropy value, and in combination with edge end lightweight processing, invalid data transmission is reduced, and the key data acquisition efficiency is improved; hot data is stored in a time sequence database and a memory database, real-time monitoring of an emergency scene and second-level response of emergency command are supported, dynamic capacity expansion and shrinkage of computing resources are matched, edge-cloud end collaboratively distributes tasks, network delay is reduced, and system stability and response speed in a high-concurrency scene are guaranteed.

The invention provides a final consistency processing system and method for transaction messages. The method comprises the following steps: acquiring transaction state metadata and a service message body about a transaction message in a distributed system, storing the service message body into a distributed message queue, and storing the transaction state metadata into a memory database; constructing a transaction state index structure; determining a physical offset based on the business message body, creating a transaction state record in the memory database based on the physical offset, and generating a to-be-rechecked transaction task according to the transaction state record; based on the transaction creation time and the system load factor, determining a rechecking interval, and executing the to-be-rechecked transaction task; when it is detected that the transaction state is changed into a transaction confirmation state from a to-be-confirmed state, pulling a service message body according to the physical offset; and performing double confirmation on the pulled service message body. The problems of performance bottleneck and low recovery efficiency caused by strong coupling of the transaction state and the message body are solved.

The invention relates to the field of financial science and technology, and particularly discloses a cross-platform financial data integration and real-time analysis system and method. The system is characterized in that a distributed heterogeneous data acquisition module accesses multi-source data in parallel through a pluggable protocol adapter; the streaming data cleaning engine realizes field-level normalization based on a dynamic template and semantic mapping; the hybrid storage architecture forms a three-level system by a memory database, a time sequence database and distributed file storage; the stream processing core engine adopts an elastic sliding window to execute CEP complex event processing; and the dynamic analysis decision module integrates online machine learning to generate a real-time transaction signal. The method comprises the steps of multi-source concurrent acquisition and microsecond-level timestamp injection, context sensing data standardization, dual-channel data storage, event time window aggregation, dynamic risk calculation and block chain audit evidence storage. The defects of low data fusion efficiency, high real-time analysis delay and the like in the prior art are overcome, and cross-platform data processing timeliness and decision accuracy are remarkably improved.

The invention relates to the field of data processing, and discloses a cloud computing-based data processing method, which comprises the following steps of: constructing a reference count management micro-service cluster at a cloud end, receiving a reference count request carrying a count type through an API (Application Program Interface) gateway, and analyzing the reference count request into a structured operation instruction; reading a first state identifier of the referenced object and a second state identifier of the reference object from the distributed cache system according to the metadata snapshot; according to the method, through dynamic scheduling of a reference counting management micro-service cluster and a container arrangement engine, efficient allocation and scheduling of computing resources can be realized, maximization of resource utilization is ensured, resource waste is avoided, and high-speed query capability of a distributed cache system and a memory database is utilized; the state identification of the object can be retrieved at a millisecond level, the system response speed is improved, and the performance requirement under a high-concurrency scene is met.

The invention provides a concurrent debugging method and system based on sandbox-LLM (Language Model) session bidirectional binding. The method comprises the following steps: distributing a global unique session identifier for each LLM session; writing the globally unique session identifier and the sandbox instance identifier into a distributed memory database to form a bidirectional binding table, and marking the initial state as a binding state; realizing unvarnished transmission of a full-link global unique session identifier through an environment variable and an HTTP / gRPC header field; during sandbox thermal migration, the old instance atomically updates a binding table entry state into a migration state through a distributed mutual exclusion lock, and the new instance is written into the binding state after being preheated within a preset time; the log agent identifies an aggregation index according to a sandbox instance, collects average generation delay according to a preset frequency, accumulates a data collection number and GPU video memory occupation; and periodically polling the binding table, and cleaning entries which exceed a preset threshold value of the distributed mutual exclusion lock and are in a recovery state.

The embodiment of the invention provides a distributed data synchronization system, method and device, a medium and a product. The method comprises the following steps: sending a to-be-synchronized message to a main service, an observer service and each standby service in a multicast manner through a sending service, and sending a main message serial number corresponding to the to-be-synchronized message to each standby service and the observer service by the main service, sending the message to be synchronized to each standby service, so that each standby service and the observer service determine whether the message is continuous according to the message to be synchronized and the serial number of the main message, if not, acquiring a missing incremental message from the main service, otherwise, updating respective corresponding memory databases through the main service and each standby service on the basis of the message which is confirmed to be continuous, and simultaneously, sending the message to the observer service. And sending the to-be-synchronized message to the remote disaster recovery service through the observer service, so that the remote disaster recovery service updates the corresponding memory database according to the received to-be-synchronized message. The method is used for achieving the effects of improving the data consistency of the databases in the transaction system and improving the reliability and stability of data synchronization of the transaction system.

The invention relates to a memory allocation method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: receiving a memory allocation request of to-be-stored data to a memory database, wherein the memory allocation request comprises a data storage amount of the to-be-stored data; determining a target memory block from a virtual process space of the memory database according to the data storage amount; the virtual process space is a storage space mapped to a disk file based on a memory mapping mechanism, and data stored in the virtual process space is synchronously stored in the disk file; the size of the target memory block is greater than or equal to the data storage amount; and distributing the target memory block for the to-be-stored data, wherein the target memory block is used for storing the to-be-stored data. By adopting the method, the storage capacity and the storage efficiency of the memory database can be improved.

The invention provides a memory database starting method, system and device and a medium, and belongs to the technical field of computers. The method comprises the following steps: detecting a current project scale and a performance index of an original server, and determining whether to trigger a memory processing mechanism; if a memory processing mechanism is triggered, selecting a memory server according to high-performance computing and expansibility requirements, accessing an original server by utilizing a hot plug technology, configuring a memory database and initializing a memory computing cluster; configuring a to-be-synchronized database table by utilizing a metadata management function, loading original data to a memory database through a full-amount synchronization and incremental updating mechanism, and starting real-time data synchronization; starting a memory computing cluster, automatically detecting a memory processing state on an original server when a service task is triggered, and routing a large quantity of data tasks meeting conditions to a memory server to execute parallel computing. According to the method, the memory can be dynamically started for data processing, the operation efficiency is improved, and the user experience is improved.

The invention discloses a hand-eye cooperative robot control system and method based on dynamic operator arrangement, an upper computer planning layer operates a master control computer to periodically trigger task scheduling, a joint controller of a lower computer execution layer receives an instruction through a redundant bus to perform servo control, hand-eye camera data triggers a visual assembly line through an interrupt event, and a visual assembly line is controlled through a control interface. According to the method, pressure is calculated through visual processing, motion planning and joint control in a load sharing mode, a double-buffering mechanism is adopted to enable current frame visual processing and previous frame motion control to be executed in an overlapping mode, real-time scheduling and parallel computing of tasks are completed, hot data are cached in a memory database, cold data are archived to an HDFS and migrated through an LRU strategy, and the real-time scheduling and parallel computing of the tasks are completed. Data type conversion is automatically derived and performed based on a feature type registry, and conditional branch execution is triggered according to real-time sensor data. According to the control system, cross-hardware plug and play, algorithm flow configurability and data flow real-time sharing are achieved, and the requirement for improving the efficiency of complex operation tasks is met.

Computer-readable media, methods, and systems are disclosed for improving performance when dropping database snapshots by linking converter streams associated with adjacent snapshots in a database system. A request is received to drop a snapshot. The database system iterates a first converter tree associated with a first converter stream to identify a first physical page number to be loaded. A physical page corresponding to the first physical page number is loaded. The database system iterates a second converter tree associated with a second converter stream to identify a second physical page number to be loaded. The second physical page number is queried from an iterator associated with the first converter tree. Responsive to determining that the physical page associated with the second physical page number is not present in main memory, the in-memory database system loads from disk, the physical page associated with the second physical page number.

The invention relates to the field of data storage, and provides a crop phenotype data storage and retrieval method and device, equipment and a storage medium, and the method comprises the steps: obtaining multi-modal data comprising crop phenotype data and metadata information; storing the structured data and the metadata information to a distributed database, and storing the unstructured data to a distributed file system; synchronously storing a retrieval field of the structured data, a retrieval field of the small file data and metadata information of the large file data into a distributed search analysis engine; and under the condition that the data matched with the user retrieval condition does not exist in the memory database, determining a retrieval result based on the user retrieval condition and the distributed search analysis engine. According to the method, the multi-modal crop phenotype data are classified and stored according to different data types, so that the storage efficiency and the retrieval rate of the multi-modal crop phenotype data are improved.

The invention discloses a failure recovery method, device and equipment of a security transaction system and a medium, and belongs to the technical field of application system failure recovery. The method comprises the steps that when it is monitored that a transaction node cannot run, the transaction node is restarted, or a standby node is started to become a new transaction node; setting the newly started transaction node as a fault state; acquiring operation data of the transaction node by using the snapshot service and writing the operation data into a message queue; the transaction node pulls the operation data from the message queue, obtains return data during the abnormal period of the transaction node from a downstream node of the transaction node, and updates the data to the memory database; and setting the transaction node to be in a normal service state. The fault recovery method is high in speed and universality, is not influenced by development languages, development frameworks, business logics and the like, can ensure the data consistency of fault recovery, can support a plurality of transaction systems by one set of snapshot service, and saves server resources.

The invention provides a multi-task multi-satellite telemetry data efficient processing method and system and electronic equipment, and relates to the technical field of satellite ground telemetry data processing. Comprising the following steps: dynamically allocating a task plug-in to an optimal computing node based on a resource state evaluation result of each computing node in a server cluster; uploading task registration information to a configuration center, and storing telemetering parameter configuration information contained in the information in a memory database in a key-value pair form; according to the distributed task plug-in, task telemetering data is led from a specified information source, and step-by-step subdivision is carried out on the data based on a data shunt model; and extracting a telemetering source packet based on the obtained telemetering data flow of each frequency band, and distributing a telemetering source packet analysis task through a thread pool. And on the basis of the telemetering parameter configuration information, utilizing the parameter resolving virtual thread to resolve all parameters in the source packet in parallel. The defect that high-code-rate telemetry data cannot be efficiently processed in real time in a multi-task and multi-satellite scene is overcome.

The invention discloses a data processing method and device, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps: receiving real-time data streams sent by a plurality of clients through a target server cluster; wherein the target server cluster comprises an asynchronous socket server based on a non-blocking network, and the target server cluster realizes event-driven network data processing based on a reactor mode; according to the real-time load of each partition in a target message queue, distributing the received real-time data flow to the corresponding partition in the target message queue; cleaning the real-time data flow of each partition in the target message queue by using a target rule engine with a hot loading function to obtain cleaned data; and storing the cleaned data to a distributed memory database cluster. Therefore, resource tension and response delay caused by connection congestion are reduced, and data consistency and service continuity are guaranteed.

The invention discloses a multi-mode automatic deployment starting method and system for redis, and belongs to the technical field of memory databases, and the method comprises the following steps: an early-stage preparation work part: making a redis binary executable program compatible under different CPU (Central Processing Unit) architectures; the resource verification part is used for verifying node information, a node memory and node disk information configured in the configuration file; the storage configuration part is used for configuring a storage directory according to the storage information configured in the configuration file, and automatically creating and mounting a data disk for a redis working directory; and the self-deployment part is used for installing a binary executable program into each node according to node information configured in the configuration file, modifying necessary parameters of the redis configuration file, and pushing the redis configuration file to a data directory of each node. According to the method and the device, the redis can be automatically deployed and started according to different CPU architectures and deployment modes under the condition of different CPU architectures and deployment modes.

The inventors have implemented in a columnar in-memory database and studied access patterns of a large production enterprise system. To obtain accurate cost estimates for a configuration, the inventors have used the what-if capabilities of modern query optimizers. What-if calls, however, are the major bottleneck for most index selection approaches. Hence, a major constraint is to limit the number of what-if optimizer calls. And even though the inventive approach does not limit the index candidate set, it decreases the number of what-if calls because in each iteration step the number of possible (index) extensions is comparably small which results in a limited number of what-if calls.

Disclosed is a training system performing training on a plurality of neural network models in parallel. The training system includes a first job proxy that receives a training request for a first neural network model and partitions a first training job corresponding to the first neural network model into first microservices, a second job proxy that receives a training request for a second neural network model and partitions a second training job corresponding to the second neural network model into second microservices, a scheduler that dynamically schedules the first microservices and the second microservices so as to be executed by heterogeneous processing units, a cluster that includes the heterogeneous processing units, sequentially executes the first microservices and sequentially executes the second microservices, and a distributed in-memory database that stores parameters generated in response to the execution of the first microservices and the second micro services.

The invention relates to the technical field of power acquisition terminal data storage, and discloses a data hybrid storage method and device and computer equipment, which are applied to a memory database, and the method comprises the following steps: when electric energy meter reading data is written into a main database of a power acquisition terminal, receiving a data change notification sent by the main database; recording the meter reading data into a data cache by adopting a key value mode based on the data change notification; and preferentially querying the corresponding meter reading data in the data cache based on a preset data retrieval requirement, and if the corresponding meter reading data is not queried, switching to the main database to continue data retrieval. According to the method, a large amount of data is prevented from simultaneously accessing the main database at the same moment, so that the load of the main database is reduced, load balancing is realized, the memory database is preferentially queried during data retrieval, and the main database is switched to retrieve when the memory database cannot be queried, so that the access frequency of the main database is greatly reduced, and the data retrieval efficiency is improved. And the performance bottleneck during batch data query of the relational database is solved.

The invention provides a method for performing data processing on a plurality of list sets by using SQL statements. The method comprises the following steps of S1, receiving request parameters and verifying the request parameters; s2, storing the data in the list set into a user-defined memory database; s3, checking the SQL statement; s4, the SQL statement is analyzed, and SELECT information is obtained; s5, a SELECT statement in the SELECT information is analyzed, and SQL grammar information is obtained; s6, processing the data according to the SQL grammar information; s7, returning final result data; the problem that data processing of multiple List list sets is difficult is solved, connection with a database server is not needed based on a flexible data processing mode of the SQL language and a mode of creating the user-defined memory database, the database security problem does not exist, high concurrency is supported, efficiency is improved, complex data processing is achieved, and the method is suitable for large-scale popularization and application. SQL statements are modified and updated at any time according to requirements, and time and labor are

The invention provides an intelligent legal case processing system based on a multi-level cache architecture, and the system comprises a case pool management module which is used for uploading case data to a distributed storage server, storing case metadata into a relational database, and caching key case information into a memory database; the intelligent case division module is used for obtaining data from the memory database based on a case division algorithm to calculate scores and allocating cases to appropriate processing personnel; a table module; a litigation data management module; an electronic signature module; a task monitoring module; and the data synchronization module is used for synchronizing the memory database and the relational database so as to keep the data consistency between the memory database and the relational database. The efficient, reliable and intelligent legal case intelligent processing system is constructed through deep application of a memory database multi-data structure in combination with distributed storage and a case division algorithm.

The invention discloses a database system construction method and system based on new energy centralized control, and the method comprises the steps: employing a MySQL / PostgreSQL cluster to construct a historical library, constructing a real-time library through Redis, and achieving the cooperation of historical data storage and real-time data processing; the method comprises the following steps of: dividing data according to applications, and defining a multi-table structure; (2) realizing data hierarchical organization by adopting a multi-level name domain; a historical library and real-time library conversion service is configured, and single-table / multi-table data import is supported; an I / IV region security partition architecture is established, and data security interaction is realized through a bidirectional isolation synchronization mechanism; according to the system, through collaborative design of a relational database and a memory database, mass data storage and real-time response requirements are considered, a structured alias domain is adopted to realize accurate data management, and the requirements of a new energy centralized control system on data processing efficiency and security are met.

The invention provides an anti-caching method supporting a nonvolatile memory database, which comprises the following steps of: constructing a storage hierarchical architecture comprising a nonvolatile memory layer and a solid state disk layer, and introducing a data dynamic migration mechanism for migrating cold data from the nonvolatile memory layer to the solid state disk layer; a cache management algorithm is adopted, a hash table and an LRU linked list are combined, distribution of data between a nonvolatile memory layer and a solid state disk layer is determined according to the access frequency of the data, and the data are tracked and managed by using a globally unique identifier; a space management mechanism is implemented, and idle pages are managed by using an idle page linked list in a non-volatile memory layer, so that the released non-volatile memory space is quickly reutilized. According to the method, not only is a large performance fall not generated when a large amount of data is stored in the nonvolatile database, but also a remarkable effect is achieved in the aspect of optimizing storage resources, and a new solution is provided for the field of storage of the nonvolatile memory database.

The invention discloses a data processing method and device applied to a high-concurrency scene and electronic equipment, and the method comprises the steps: responding to a received data obtaining request, if configuration type data is obtained, obtaining target configuration type data corresponding to the data obtaining request based on a first data obtaining mode, the first data acquisition mode represents that the configuration data is acquired according to the access sequence of the local cache, the memory database and the relational database. If the service class data is acquired, target service class data corresponding to the data acquisition request is acquired based on a second data acquisition mode, and the second data acquisition mode represents that the service class data is acquired according to the access sequence of the slave database and the master database. According to the method, the configuration type data is applied through the local cache and the memory database, and the business type data is read through the master-slave database, so that the reading pressure of the database is reduced, the transaction response speed is improved, and the system service capability in a high-concurrency scene is further improved.

The invention provides a ticket query method and device, equipment, a storage medium and a program product, and relates to the technical field of ticket query. The method comprises the following steps: based on a ticket query request, determining query parameters corresponding to a query task; based on the memory database and the query parameters, determining a query quantity corresponding to the query task; wherein the memory database stores telephone traffic key value pairs which are counted based on identification information, service categories and account periods of users; based on the query quantity, determining the number N of query threads, and distributing the query task to the N query threads; querying from a distributed database based on the N query threads, and obtaining a detailed list query result of each query thread; and based on the ticket query request, summarizing the detailed ticket query results of the N query threads through a summarizing thread to obtain a ticket query result corresponding to the ticket query request. According to the invention, the response speed of ticket query can be effectively improved.

The application discloses a hot ticket card blacklist generation method and device and a medium, relates to the technical field of computers, and comprises the following steps: determining a quantity threshold of a hot ticket card blacklist; sorting each initial ticket card according to the time attribute of each initial ticket card to obtain a first sorting result, and screening a first candidate ticket card from the first sorting result; if the first ticket card quantity of the first candidate ticket card is not less than the quantity threshold, determining a target ticket card from the first candidate ticket card according to a priority rule; if the first ticket card quantity is less than the quantity threshold, sorting each initial ticket card according to the balance information in each initial ticket card to obtain a second sorting result, and screening a second candidate ticket card from the second sorting result; screening the target ticket card from the first candidate ticket card and the second candidate ticket card, and taking the target ticket card as the hot ticket card blacklist. The application can accurately obtain a hot ticket card blacklist with higher activity, and can reduce the memory resource demand on the memory database of a gate.

An accelerator is disclosed. The accelerator may include an on-chip memory to store data from a database. The on-chip memory may include a first memory group and a second memory group. The first memory group may store data, which may include a first value and a second value. A computing engine may execute a command on a first value of the data in the on-chip memory and execute a command on a second value of the data in the on-chip memory in parallel. The on-chip memory may be configured to load second data from the database into the second memory group in parallel with the computing engine executing the command on the first value of the data and executing the command on the second value of the data.

The present invention discloses a device modeling method, a storage medium and an electronic device for an intelligent rail transit system. The method includes: setting a plurality of metadata templates, where the metadata templates include a general metadata template, a reference metadata template, a measurement point metadata template and a device metadata template; generating a metadata template group according to each metadata template; constructing a device list, and generating device data, sub-device data, measurement point data, corresponding relationship data and subordinate relationship data between the data according to the metadata template group and the device list to generate device model data; storing the device model data in an in-memory database, so that during the operation of the device model, the device model data is obtained from the in-memory database and loaded into the device list. The present invention can improve the design flexibility of the real-time data platform, improve the efficiency of the data preparation stage during device modeling, and enrich the application scenarios of the real-time data platform.

The invention discloses a service orchestration interruption execution method and system based on trusted data space, and belongs to the technical field of trusted data space, and the method comprises the steps: service orchestration interruption detection: carrying out the real-time monitoring of each node in a service orchestration canvas, so as to ensure that the abnormality of the node state can be captured in time; the method comprises the following steps of: interrupting access of interrupted data, and when interruption occurs, storing related execution state data and node service data into a memory database by a system according to a currently requested transaction ID (Identity); the interruption recovery mechanism is used for positioning problems when interruption occurs, recovering the state and ensuring smooth and continuous execution of the service; and continuing execution after interruption, and in the execution process of the service orchestration system, generating a unique request transaction ID for each request. According to the method, the service orchestration interruption processing capability can be improved, the data credibility is improved, and the dynamic adjustment capability is improved.

The application relates to the technical field of flexible direct current power transmission, and particularly provides a lightweight real-time memory database transaction concurrent control method and device, which comprises the following steps: S101. Classifying to-be-processed transactions through cluster analysis to determine the cluster to which the to-be-processed transactions belong; S102. Obtaining a transaction lock corresponding to the to-be-processed transaction in the cluster to which the to-be-processed transaction belongs and a data lock called by the to-be-processed transaction, and executing S103 under the condition that the acquisition state is successful; S103. Conflict verification is performed on the to-be-processed transaction, if there is a conflict, the to-be-processed transaction is prevented, otherwise, the to-be-processed transaction is executed. The technical scheme provided by the application meets the demand of high-efficiency access of Internet of Things data and improves the transaction execution performance of the real-time memory database.