203 results about "In-memory database" patented technology

The invention relates to a method and a system for storing and backing up electric power data of a data-in-data station based on use popularity, and belongs to the technical field of data-in-data stations, and the method comprises the following steps: classifying electric power business data into hot data, temperature data and cold data based on preset access frequency, time window and data volume index; according to a classification result and a business importance degree, storing hot data in an SSD cloud disk or a memory database, storing temperature data in a distributed column database, and compressing cold data and then archiving the cold data to an object storage or a tape library; dynamically configuring differentiated backup strategies for different types of data; automatic scheduling of storage and backup is realized through a cloud platform component, and hot switching of master and slave databases, OSS archived data recovery by a DataWorks tool and a remote multi-copy recovery mechanism are supported; and migrating data in advance in combination with a historical access prediction popularity trend, and triggering cold data compression and archiving when storage resources exceed 80%. The method is suitable for power industry scenes, the storage cost is optimized, and the data calling efficiency is improved.

The invention discloses a real-time interaction and intelligent management method for a multi-dimensional data space, and relates to the technical field of data interaction management, and the method comprises the following steps: I, collecting multi-source data, including structured data, unstructured data and spatio-temporal data, in real time through a diversified collection terminal, and dynamically adjusting the sampling frequency based on a data entropy value, carrying out cross-device space-time synchronization calibration and edge end lightweight processing on the acquired data; iI, preprocessing the data flow in real time, fusing the multi-modal data, and the like. According to the method, the sampling frequency is dynamically adjusted based on the data entropy value, and in combination with edge end lightweight processing, invalid data transmission is reduced, and the key data acquisition efficiency is improved; hot data is stored in a time sequence database and a memory database, real-time monitoring of an emergency scene and second-level response of emergency command are supported, dynamic capacity expansion and shrinkage of computing resources are matched, edge-cloud end collaboratively distributes tasks, network delay is reduced, and system stability and response speed in a high-concurrency scene are guaranteed.

The invention provides a final consistency processing system and method for transaction messages. The method comprises the following steps: acquiring transaction state metadata and a service message body about a transaction message in a distributed system, storing the service message body into a distributed message queue, and storing the transaction state metadata into a memory database; constructing a transaction state index structure; determining a physical offset based on the business message body, creating a transaction state record in the memory database based on the physical offset, and generating a to-be-rechecked transaction task according to the transaction state record; based on the transaction creation time and the system load factor, determining a rechecking interval, and executing the to-be-rechecked transaction task; when it is detected that the transaction state is changed into a transaction confirmation state from a to-be-confirmed state, pulling a service message body according to the physical offset; and performing double confirmation on the pulled service message body. The problems of performance bottleneck and low recovery efficiency caused by strong coupling of the transaction state and the message body are solved.

The invention relates to the field of financial science and technology, and particularly discloses a cross-platform financial data integration and real-time analysis system and method. The system is characterized in that a distributed heterogeneous data acquisition module accesses multi-source data in parallel through a pluggable protocol adapter; the streaming data cleaning engine realizes field-level normalization based on a dynamic template and semantic mapping; the hybrid storage architecture forms a three-level system by a memory database, a time sequence database and distributed file storage; the stream processing core engine adopts an elastic sliding window to execute CEP complex event processing; and the dynamic analysis decision module integrates online machine learning to generate a real-time transaction signal. The method comprises the steps of multi-source concurrent acquisition and microsecond-level timestamp injection, context sensing data standardization, dual-channel data storage, event time window aggregation, dynamic risk calculation and block chain audit evidence storage. The defects of low data fusion efficiency, high real-time analysis delay and the like in the prior art are overcome, and cross-platform data processing timeliness and decision accuracy are remarkably improved.

The invention relates to the field of data processing, and discloses a cloud computing-based data processing method, which comprises the following steps of: constructing a reference count management micro-service cluster at a cloud end, receiving a reference count request carrying a count type through an API (Application Program Interface) gateway, and analyzing the reference count request into a structured operation instruction; reading a first state identifier of the referenced object and a second state identifier of the reference object from the distributed cache system according to the metadata snapshot; according to the method, through dynamic scheduling of a reference counting management micro-service cluster and a container arrangement engine, efficient allocation and scheduling of computing resources can be realized, maximization of resource utilization is ensured, resource waste is avoided, and high-speed query capability of a distributed cache system and a memory database is utilized; the state identification of the object can be retrieved at a millisecond level, the system response speed is improved, and the performance requirement under a high-concurrency scene is met.

The invention provides a concurrent debugging method and system based on sandbox-LLM (Language Model) session bidirectional binding. The method comprises the following steps: distributing a global unique session identifier for each LLM session; writing the globally unique session identifier and the sandbox instance identifier into a distributed memory database to form a bidirectional binding table, and marking the initial state as a binding state; realizing unvarnished transmission of a full-link global unique session identifier through an environment variable and an HTTP / gRPC header field; during sandbox thermal migration, the old instance atomically updates a binding table entry state into a migration state through a distributed mutual exclusion lock, and the new instance is written into the binding state after being preheated within a preset time; the log agent identifies an aggregation index according to a sandbox instance, collects average generation delay according to a preset frequency, accumulates a data collection number and GPU video memory occupation; and periodically polling the binding table, and cleaning entries which exceed a preset threshold value of the distributed mutual exclusion lock and are in a recovery state.

The invention provides a memory database starting method, system and device and a medium, and belongs to the technical field of computers. The method comprises the following steps: detecting a current project scale and a performance index of an original server, and determining whether to trigger a memory processing mechanism; if a memory processing mechanism is triggered, selecting a memory server according to high-performance computing and expansibility requirements, accessing an original server by utilizing a hot plug technology, configuring a memory database and initializing a memory computing cluster; configuring a to-be-synchronized database table by utilizing a metadata management function, loading original data to a memory database through a full-amount synchronization and incremental updating mechanism, and starting real-time data synchronization; starting a memory computing cluster, automatically detecting a memory processing state on an original server when a service task is triggered, and routing a large quantity of data tasks meeting conditions to a memory server to execute parallel computing. According to the method, the memory can be dynamically started for data processing, the operation efficiency is improved, and the user experience is improved.

The invention discloses a hand-eye cooperative robot control system and method based on dynamic operator arrangement, an upper computer planning layer operates a master control computer to periodically trigger task scheduling, a joint controller of a lower computer execution layer receives an instruction through a redundant bus to perform servo control, hand-eye camera data triggers a visual assembly line through an interrupt event, and a visual assembly line is controlled through a control interface. According to the method, pressure is calculated through visual processing, motion planning and joint control in a load sharing mode, a double-buffering mechanism is adopted to enable current frame visual processing and previous frame motion control to be executed in an overlapping mode, real-time scheduling and parallel computing of tasks are completed, hot data are cached in a memory database, cold data are archived to an HDFS and migrated through an LRU strategy, and the real-time scheduling and parallel computing of the tasks are completed. Data type conversion is automatically derived and performed based on a feature type registry, and conditional branch execution is triggered according to real-time sensor data. According to the control system, cross-hardware plug and play, algorithm flow configurability and data flow real-time sharing are achieved, and the requirement for improving the efficiency of complex operation tasks is met.

Computer-readable media, methods, and systems are disclosed for improving performance when dropping database snapshots by linking converter streams associated with adjacent snapshots in a database system. A request is received to drop a snapshot. The database system iterates a first converter tree associated with a first converter stream to identify a first physical page number to be loaded. A physical page corresponding to the first physical page number is loaded. The database system iterates a second converter tree associated with a second converter stream to identify a second physical page number to be loaded. The second physical page number is queried from an iterator associated with the first converter tree. Responsive to determining that the physical page associated with the second physical page number is not present in main memory, the in-memory database system loads from disk, the physical page associated with the second physical page number.

The invention relates to the field of data storage, and provides a crop phenotype data storage and retrieval method and device, equipment and a storage medium, and the method comprises the steps: obtaining multi-modal data comprising crop phenotype data and metadata information; storing the structured data and the metadata information to a distributed database, and storing the unstructured data to a distributed file system; synchronously storing a retrieval field of the structured data, a retrieval field of the small file data and metadata information of the large file data into a distributed search analysis engine; and under the condition that the data matched with the user retrieval condition does not exist in the memory database, determining a retrieval result based on the user retrieval condition and the distributed search analysis engine. According to the method, the multi-modal crop phenotype data are classified and stored according to different data types, so that the storage efficiency and the retrieval rate of the multi-modal crop phenotype data are improved.

The invention provides a multi-task multi-satellite telemetry data efficient processing method and system and electronic equipment, and relates to the technical field of satellite ground telemetry data processing. Comprising the following steps: dynamically allocating a task plug-in to an optimal computing node based on a resource state evaluation result of each computing node in a server cluster; uploading task registration information to a configuration center, and storing telemetering parameter configuration information contained in the information in a memory database in a key-value pair form; according to the distributed task plug-in, task telemetering data is led from a specified information source, and step-by-step subdivision is carried out on the data based on a data shunt model; and extracting a telemetering source packet based on the obtained telemetering data flow of each frequency band, and distributing a telemetering source packet analysis task through a thread pool. And on the basis of the telemetering parameter configuration information, utilizing the parameter resolving virtual thread to resolve all parameters in the source packet in parallel. The defect that high-code-rate telemetry data cannot be efficiently processed in real time in a multi-task and multi-satellite scene is overcome.

The invention discloses a data processing method and device, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps: receiving real-time data streams sent by a plurality of clients through a target server cluster; wherein the target server cluster comprises an asynchronous socket server based on a non-blocking network, and the target server cluster realizes event-driven network data processing based on a reactor mode; according to the real-time load of each partition in a target message queue, distributing the received real-time data flow to the corresponding partition in the target message queue; cleaning the real-time data flow of each partition in the target message queue by using a target rule engine with a hot loading function to obtain cleaned data; and storing the cleaned data to a distributed memory database cluster. Therefore, resource tension and response delay caused by connection congestion are reduced, and data consistency and service continuity are guaranteed.

The invention provides a method for performing data processing on a plurality of list sets by using SQL statements. The method comprises the following steps of S1, receiving request parameters and verifying the request parameters; s2, storing the data in the list set into a user-defined memory database; s3, checking the SQL statement; s4, the SQL statement is analyzed, and SELECT information is obtained; s5, a SELECT statement in the SELECT information is analyzed, and SQL grammar information is obtained; s6, processing the data according to the SQL grammar information; s7, returning final result data; the problem that data processing of multiple List list sets is difficult is solved, connection with a database server is not needed based on a flexible data processing mode of the SQL language and a mode of creating the user-defined memory database, the database security problem does not exist, high concurrency is supported, efficiency is improved, complex data processing is achieved, and the method is suitable for large-scale popularization and application. SQL statements are modified and updated at any time according to requirements, and time and labor are

The invention provides an intelligent legal case processing system based on a multi-level cache architecture, and the system comprises a case pool management module which is used for uploading case data to a distributed storage server, storing case metadata into a relational database, and caching key case information into a memory database; the intelligent case division module is used for obtaining data from the memory database based on a case division algorithm to calculate scores and allocating cases to appropriate processing personnel; a table module; a litigation data management module; an electronic signature module; a task monitoring module; and the data synchronization module is used for synchronizing the memory database and the relational database so as to keep the data consistency between the memory database and the relational database. The efficient, reliable and intelligent legal case intelligent processing system is constructed through deep application of a memory database multi-data structure in combination with distributed storage and a case division algorithm.

The invention discloses a database system construction method and system based on new energy centralized control, and the method comprises the steps: employing a MySQL / PostgreSQL cluster to construct a historical library, constructing a real-time library through Redis, and achieving the cooperation of historical data storage and real-time data processing; the method comprises the following steps of: dividing data according to applications, and defining a multi-table structure; (2) realizing data hierarchical organization by adopting a multi-level name domain; a historical library and real-time library conversion service is configured, and single-table / multi-table data import is supported; an I / IV region security partition architecture is established, and data security interaction is realized through a bidirectional isolation synchronization mechanism; according to the system, through collaborative design of a relational database and a memory database, mass data storage and real-time response requirements are considered, a structured alias domain is adopted to realize accurate data management, and the requirements of a new energy centralized control system on data processing efficiency and security are met.

The invention provides a ticket query method and device, equipment, a storage medium and a program product, and relates to the technical field of ticket query. The method comprises the following steps: based on a ticket query request, determining query parameters corresponding to a query task; based on the memory database and the query parameters, determining a query quantity corresponding to the query task; wherein the memory database stores telephone traffic key value pairs which are counted based on identification information, service categories and account periods of users; based on the query quantity, determining the number N of query threads, and distributing the query task to the N query threads; querying from a distributed database based on the N query threads, and obtaining a detailed list query result of each query thread; and based on the ticket query request, summarizing the detailed ticket query results of the N query threads through a summarizing thread to obtain a ticket query result corresponding to the ticket query request. According to the invention, the response speed of ticket query can be effectively improved.

The application discloses a hot ticket card blacklist generation method and device and a medium, relates to the technical field of computers, and comprises the following steps: determining a quantity threshold of a hot ticket card blacklist; sorting each initial ticket card according to the time attribute of each initial ticket card to obtain a first sorting result, and screening a first candidate ticket card from the first sorting result; if the first ticket card quantity of the first candidate ticket card is not less than the quantity threshold, determining a target ticket card from the first candidate ticket card according to a priority rule; if the first ticket card quantity is less than the quantity threshold, sorting each initial ticket card according to the balance information in each initial ticket card to obtain a second sorting result, and screening a second candidate ticket card from the second sorting result; screening the target ticket card from the first candidate ticket card and the second candidate ticket card, and taking the target ticket card as the hot ticket card blacklist. The application can accurately obtain a hot ticket card blacklist with higher activity, and can reduce the memory resource demand on the memory database of a gate.

The present invention discloses a device modeling method, a storage medium and an electronic device for an intelligent rail transit system. The method includes: setting a plurality of metadata templates, where the metadata templates include a general metadata template, a reference metadata template, a measurement point metadata template and a device metadata template; generating a metadata template group according to each metadata template; constructing a device list, and generating device data, sub-device data, measurement point data, corresponding relationship data and subordinate relationship data between the data according to the metadata template group and the device list to generate device model data; storing the device model data in an in-memory database, so that during the operation of the device model, the device model data is obtained from the in-memory database and loaded into the device list. The present invention can improve the design flexibility of the real-time data platform, improve the efficiency of the data preparation stage during device modeling, and enrich the application scenarios of the real-time data platform.

The invention discloses a service orchestration interruption execution method and system based on trusted data space, and belongs to the technical field of trusted data space, and the method comprises the steps: service orchestration interruption detection: carrying out the real-time monitoring of each node in a service orchestration canvas, so as to ensure that the abnormality of the node state can be captured in time; the method comprises the following steps of: interrupting access of interrupted data, and when interruption occurs, storing related execution state data and node service data into a memory database by a system according to a currently requested transaction ID (Identity); the interruption recovery mechanism is used for positioning problems when interruption occurs, recovering the state and ensuring smooth and continuous execution of the service; and continuing execution after interruption, and in the execution process of the service orchestration system, generating a unique request transaction ID for each request. According to the method, the service orchestration interruption processing capability can be improved, the data credibility is improved, and the dynamic adjustment capability is improved.

The application relates to the technical field of flexible direct current power transmission, and particularly provides a lightweight real-time memory database transaction concurrent control method and device, which comprises the following steps: S101. Classifying to-be-processed transactions through cluster analysis to determine the cluster to which the to-be-processed transactions belong; S102. Obtaining a transaction lock corresponding to the to-be-processed transaction in the cluster to which the to-be-processed transaction belongs and a data lock called by the to-be-processed transaction, and executing S103 under the condition that the acquisition state is successful; S103. Conflict verification is performed on the to-be-processed transaction, if there is a conflict, the to-be-processed transaction is prevented, otherwise, the to-be-processed transaction is executed. The technical scheme provided by the application meets the demand of high-efficiency access of Internet of Things data and improves the transaction execution performance of the real-time memory database.

In an example embodiment, a data warehousing environment, or a similar architecture, is extended to permit data storage in either an in-memory database or a Lakehouse architecture, which leverages one or more hyperscalers for the underlying storage. More specifically, a single artifact is defined in Datasphere that stores data in either the in-memory database or the Lakehouse architecture, and does so in a storage-agnostic manner.

This invention discloses a method and system for accurate electricity consumption data supplementation, relating to the field of electricity consumption data acquisition technology. It includes: the master station dynamically initializes and records the missing curve data information for each electricity meter based on a memory database; when accurate supplementation is triggered, using a single acquisition terminal as the smallest unit, it extracts only the missing data point information from the memory database to generate the missing data point sequence for each electricity meter; through an extended communication protocol for accurate supplementation, it generates an accurate supplementation notification instruction carrying only the missing data point information and sends it to the acquisition terminal; the acquisition terminal extracts the missing data point sequence, retrieves only the missing data corresponding to the missing data point sequence from its local cache, and reports it in batches to the master station; after receiving the missing data, the master station compares and verifies it based on locally maintained missing data status information; after successful verification, it completes the supplementation and updates the missing data marker. This invention effectively improves the efficiency of electricity consumption data supplementation and reduces the resource consumption across the entire link.

The invention discloses an abnormity monitoring method, system, equipment and medium based on a plant station gateway metering point, and relates to the technical field of gateway metering point abnormity detection, and the method comprises the steps: collecting the real-time data of the plant station gateway metering point through an industrial protocol interface, loading a protocol stack in a collection gateway, carrying out the preprocessing of the preliminarily collected data, and carrying out the detection of the abnormity of the gateway metering point; data transmission is carried out on the preprocessed data through a real-time stream processing layer by utilizing a transmission protocol, the real-time data is stored in a memory database, historical data is stored in a distributed file system, data organization and management are carried out, and multi-mode anomaly detection is carried out on the stored data through an anomaly detection service deployed in a containerization mode. And identifying various abnormities, packaging and outputting a detection result, and dynamically rendering through a responsive front-end interface to display the detection result in real time. According to the invention, the multi-mode intelligent detection and containerized elastic architecture are fused, so that the leap-type improvement of the metering abnormity of the plant station from passive alarm to active early warning and deep diagnosis is realized.

A computer-implemented method for regulating delivery frequency of outbound communications. The method includes receiving message context data comprising one or more of a user identifier, a client identifier, a campaign identifier, a channel identifier, a label identifier, and a trigger identifier. A counter-field identifier is determined that encodes the channel identifier, label identifier, trigger identifier, campaign identifier, and a temporal-interval indicator. A counter field identified by the counter-field identifier is created. A value stored in the counter field is incremented through an atomic operation. The value is compared with a limit stored in a rule set. An allow signal is output when the value does not exceed the limit and a suppress signal is output when the value exceeds the limit. An engagement-counter record is updated to the in-memory database.

The invention discloses a real-time processing method, system and equipment for high-frequency time sequence data and a storage medium, and relates to the technical field of data processing in the steel industry, and the method comprises the steps: collecting the high-frequency time sequence data of equipment in a steel production process, and uploading the high-frequency time sequence data to a target data bus; based on a trigger signal, sequentially acquiring the high-frequency time sequence data from the target data bus; preprocessing the obtained high-frequency time sequence data to obtain a standard data stream; configuring a calculation process based on a zero code editor, and calculating the standard data stream through the calculation process to obtain calculation result data; respectively storing the calculation result data into a memory database and a disk time sequence database according to a time dimension; and outputting the calculation result data through an application program interface.

An electric vehicle charging and discharging operation regulation platform, comprising: a data collection cluster, which is configured to receive and parse collected data sent by an external system, wherein the external system comprises multiple terminals, an Internet-of-Vehicles platform and a power grid dispatching platform; a data processing cluster, which is configured to perform processing analysis and periodic sampling storage on real-time remote signaling / telemetering data; a data storage cluster, which has a distributed memory database constructed therein, and is configured to implement the large-capacity storage of historical data and high-concurrency access to the historical data; and a micro-service cluster, which is configured to implement, on the basis of data stored in the data storage cluster, a service application micro-service and a platform micro-service by means of a micro-service architecture.

The present invention discloses a data isolation method, device, equipment and storage medium for a stress testing scenario. The method includes: dividing and routing all traffic flowing into a system gateway to obtain processed traffic; identifying the processed traffic through middleware, wherein traffic with a stress testing identifier is stress testing traffic, and traffic without a stress testing identifier is real traffic; inputting the identified real traffic into the main database of the business, and inputting the data generated by the identified stress testing traffic into the main database in the in-memory database. The present invention adopts the method to achieve complete data isolation, will not generate dirty data, will not affect normal online users, can be used directly in a real online environment, obtain real performance indicator data, and provide an accurate and reliable reference for system expansion and operation and maintenance.

The present invention can debug field devices in a control system by performing the following steps: (i) detecting a connection event including a field device and an I / O port communicatively coupled to a server through an interface; (ii) extracting a first field device identifier corresponding to the field device from a memory of the field device interface-connected to the I / O port; (iii) extracting field device configuration data associated with the I / O port from a non-transitory memory database communicatively coupled to the server; (iv) generating one of a transformed first field device identifier and a transformed second field device identifier based on predefined transformation data; (v) comparing the transformed first field device identifier with the second field device identifier, or comparing the first field device identifier with the transformed second field device identifier; (vi) generating a connection check output decision based on the comparison output; and (vii) optionally, triggering a connection check output event based on the connection check output decision.

The present application relates to data processing technology for the financial field, and discloses a data extraction method, comprising: data encoding of business vector data according to the data format of the business vector data, obtaining a plurality of encoded data and storing into an in-memory database, request type analysis of a data extraction request, obtaining a type analysis result, calling a synchronous connector to connect a sender and the in-memory database when the type analysis result meets a first preset condition, calling an asynchronous connector to connect the sender and the in-memory database when the type analysis result meets a second preset condition, data decoding of the extracted data according to the memory return value of the in-memory database when the reference connector is connected with the in-memory database, and obtaining standard extraction data. The present application also provides a data extraction device, an electronic device and a storage medium. The present application can improve the efficiency of data extraction in the financial field such as insurance transactions.

The invention relates to the technical field of data driving, and particularly provides a hierarchical data driving method, system and device for digital twin archives and a medium, and the method comprises the steps: dividing physical entity data into three processing levels according to real-time requirements, the first level being the highest in real-time performance, and the third level being the lowest in real-time performance; establishing a parallel and resource-isolated data processing pipeline for each level; first-level data is directly written into a memory database, and a digital twin engine actively queries through a high-frequency timer to drive a virtual model to realize millisecond-level synchronization; performing state aggregation on the second-level data through stream calculation, and pushing the second-level data to an engine to realize second-level state updating; and persistently storing the third-level data in various databases, and accessing the third-level data by an engine when receiving a query request to realize post analysis and backtracking. According to the method, the hierarchical parallel processing assembly line is established, so that accurate optimization of data driving is realized.