32 results about "Eventual consistency" patented technology

The invention provides a final consistency processing system and method for transaction messages. The method comprises the following steps: acquiring transaction state metadata and a service message body about a transaction message in a distributed system, storing the service message body into a distributed message queue, and storing the transaction state metadata into a memory database; constructing a transaction state index structure; determining a physical offset based on the business message body, creating a transaction state record in the memory database based on the physical offset, and generating a to-be-rechecked transaction task according to the transaction state record; based on the transaction creation time and the system load factor, determining a rechecking interval, and executing the to-be-rechecked transaction task; when it is detected that the transaction state is changed into a transaction confirmation state from a to-be-confirmed state, pulling a service message body according to the physical offset; and performing double confirmation on the pulled service message body. The problems of performance bottleneck and low recovery efficiency caused by strong coupling of the transaction state and the message body are solved.

The invention provides a multi-database automatic synchronization method under an AI platform, and belongs to the technical field of database automatic synchronization. The captured change records are converted into synchronous messages in a unified format, the synchronous messages are input into an intelligent scheduling optimization model for dependency analysis and transaction reordering based on dynamic programming, data conversion rules are dynamically generated according to mode version differences, and data consistency is verified through a Merkle tree hierarchical structure; the multi-stage self-adaptive compression transmission and data writing are completed after the target node executes conflict detection and circular dependency processing, and finally the final consistency among the multiple databases is guaranteed through a periodic full-amount account checking task, so that the technical problem of synchronization performance bottleneck caused by low transaction scheduling efficiency in the automatic synchronization process of the multiple databases is solved.

The invention relates to an inventory data synchronization and consistency verification system supporting network disconnection fault tolerance, and discloses an inventory data synchronization and consistency verification system which realizes continuous recording of inventory operation during a network disconnection period through network state detection and local cache, performs batch transmission after network recovery through a synchronous uploading and verification mechanism, and ensures data integrity. According to the inventory data synchronization and consistency verification system supporting network disconnection fault tolerance, data conflicts are automatically solved through a conflict detection and strategy processing module, a difference report is generated through consistency verification, and secondary synchronization is executed, so that safe synchronization and final consistency guarantee of inventory data are achieved, and it is ensured that the service process is continuous and stable. The system is characterized by comprising a network state detection module, a local cache module, a transaction log management module, a synchronous uploading module, a conflict detection module, a conflict resolution strategy module, a consistency verification module and an abnormity alarm and manual intervention interface module.

The invention discloses a distributed transaction coordination system for a high-concurrency scene, and particularly relates to the technical field of distributed computing and transaction management. The system comprises a transaction manager, a resource agent group and a log storage module, the transaction manager receives a transaction request, generates a global transaction identifier and maintains a state machine, and an internal fragment coordinator routes the transaction to different processing fragments according to a consistent Hash algorithm for parallel coordination; the resource agent group is connected with each resource node, executes local operation and reports a state through an asynchronous batch mechanism; and the log storage module adopts a multi-copy distributed structure to persist transaction logs. Through fragment parallel processing, asynchronous state reporting and multi-copy log cooperation, the system throughput is effectively improved, the transaction delay is reduced, and the final consistency and high availability of distributed transactions are guaranteed.

The invention discloses a financial industry distributed system data consistency automatic test platform and method, and the method comprises the steps: a test data distribution module is mainly used for generating a test data set with financial transaction characteristics, and distributing the data to a plurality of service nodes of a distributed system; the automatic test module is mainly used for testing the data consistency of the distributed system in four aspects of read-write, transaction, abnormity and final consistency; the node agent module is mainly used for collecting data states, operation logs and abnormal events of each service node in real time, and is synchronized with the central control end; and the result analysis module is mainly used for performing comprehensive evaluation analysis based on the test result of each service node to obtain a consistency analysis result. The method has the advantages that through intelligent data distribution, comprehensive consistency testing and real-time monitoring analysis, the stability and reliability of the financial transaction system in a high-concurrency environment are ensured.

This application relates to the field of database backup technology, specifically disclosing a method, apparatus, computer device, and storage medium for backup of a distributed database. This application pre-builds a Citus cluster to be written to on the backup end during database backup, solving the problem of missing metadata when performing logical backups of CN and DN nodes separately. Incremental data synchronization is performed through the starting point of incremental data synchronization, enabling eventual consistency of data from multiple nodes on the source end, thus improving the reliability of database backup. Synchronization relationships and sharding instructions can achieve backups between instances of unequal specifications, improving the flexibility of distributed database backup.

Some embodiments provide or utilize code-quality-focused replicated data type functionality for collaborative source code editing support. Some embodiments combine replicated data types with automated quality assessments of potential resolutions of source code update conflicts. The replicated data types support strong eventual consistency of copies of source code, even without a repository or other centralized facility. The quality assessments apply source code quality criteria, such as syntactic correctness, consistency with other source code, programming style criteria, and other measures of source code quality. Deterministic resolution rules are applied, such as last-write-wins, grow-only, or observed-remove, with various replicated data types. Some embodiments prompt a machine learning model for snippet quality assessments, for recommended snippets, or for both.

The present invention provides a cache management method and system for a union welfare platform in high-concurrency scenarios, belonging to the field of electronic digital data processing. The method comprises: collecting union member request metadata, performing burst mode detection based on a spectral clustering algorithm, marking potential high-demand welfare benefits, and dynamically sharding the inventory keys of potential high-demand welfare benefits; constructing a three-level collaborative cache based on the dynamically sharded inventory keys, and implementing distributed inventory deductions through lease locks and atomic operations; establishing a multi-level asynchronous buffer queue, including edge queues, regional queues, and global queues, aggregating and batching inventory change requests, and combining incremental synchronization with blockchain operation hashing to ensure eventual consistency; dynamically adjusting routing and circuit breaker thresholds based on potential high-demand welfare benefits, assigning different routes to high-concurrency requests, and implementing millisecond-level traffic switching in the event of regional node failures, thereby ensuring the stability of union welfare distribution activities and the priority of member rights.

Generally disclosed herein is an approach to migrate data from a first type of distributed system to a second type of distributed system without locking data, where transactional dual writes are not available across the two systems. The approach starts by setting up a bi-directional replication between the first system and the second system. The first system will initially operate as a primary system, where the primary system receives and serves write requests from clients or other devices. For each write to the first system, the second system is updated with an asynchronous write. When the second system is caught up to the first system, such that both the first and second systems reflect approximately the same data, the second system can be switched over to serve as the primary system. The second system can now directly receive and serve all future read and write requests.

The invention relates to an AI task asynchronous processing method and device, electronic equipment and a readable storage medium. The method comprises the steps of generating a unique task ID in response to a task request from a client, and persisting task metadata to a preset result database; returning an acceptance response containing the task ID to the client, and pushing a task message containing the task ID to an asynchronous message queue; acquiring corresponding input data from a result database based on the task ID; calling an AI reasoning model to perform reasoning calculation on the preprocessed data to obtain a reasoning result; writing the reasoning result and the updated task state back to a result database; and in response to the query request, returning a corresponding task state and a processing result according to the task ID. According to the scheme provided by the invention, the problem of high delay caused by synchronous blocking can be effectively solved, the traceability and the final consistency of the processing process are ensured, and the maintainability and the expandability of the system are effectively improved.

The application relates to the technical field of database backup and synchronization, in particular to a database master-slave file adaptive fast synchronization method, which comprises the following steps: performing difference evaluation based on the meta information of the database master-slave files, directly adopting a record-by-record synchronization mode if the overall difference is too large, or entering an incremental synchronization mode; the incremental synchronization identifies inconsistent blocks by comparing the check values of the data blocks and counts the number of the inconsistent blocks; the system respectively estimates the total time consumption of the two synchronization modes according to performance parameters, the total block number, the inconsistent block number and the total record number; a secondary decision is made by comparing the estimated time consumption, and the scheme with shorter time consumption is selected for execution; and the final consistency verification is performed on the master database file and the backup database file. Through the multi-layer threshold value judgment and the dynamic time consumption estimation model, the application intelligently selects the record-by-record synchronization or the block incremental synchronization strategy, guarantees the data consistency, and significantly improves the efficiency of the database master-slave file synchronization.

Systems and methods directed to establishing a Proxy Remote Blockchain Node (PRBP) between a Local Sensor Fusion Blockchain (LSFBC) network and an industrial supply chain as a service blockchain, wherein the PRBP is connected to a remote blockchain node of the LSFBC network and configured to participate in a distributed workflow of a Supply-Chain-as-a-Service (SCaaS) blockchain; and for the PRBP detecting a connection failure for the remote blockchain node, issuing a Connection Failed Non-Fungible Token (CF-NFT) to partner peer nodes of the SCaaS blockchain.

The application is suitable for the field of information technology, and provides a double distributed micro-service architecture, a scheduling method and an edge cloud data sharing system, wherein the micro-service architecture formulates the edge cloud data sharing system problem as a cooperative cache optimization problem in a completely connected edge network under the condition of a semi-heterogeneous cost model, utilizes the concept of expected cache on a strong consistency distributed structure, and further studies the design of reducing cost expenditure. Meanwhile, the scheduling method adopted in the micro-service architecture maintains the global data volume based on the server node by using a strong consistency distributed algorithm, and maintains the effective data address information based on the client node by using a final consistency distributed algorithm, so that the micro-service architecture is more in line with the cost-driven data cache under the actual distributed environment, and solves the problems of distributed node exchange and data consistency maintenance. The edge cloud data sharing system adopting the architecture also has the above technical effects.

This disclosure provides a transaction execution method, system, electronic device, and readable medium. The method includes: receiving an execution instruction for a target transaction; operating on a first database according to the target transaction to obtain an execution success message and sending the execution success message to a message table; sending the execution success message to a first queue; listening to feedback messages in a second queue, wherein the feedback messages are generated by a second node in the transaction execution system after operating on a second database based on the execution success message in the first queue, and then sending the feedback messages to the second queue; and changing the status of the execution success messages in the message table according to the feedback messages, thereby managing the target transaction based on the status of the execution success messages in the message table. The transaction execution method, system, electronic device, and readable medium provided by this disclosure can guarantee the eventual consistency of a distributed transaction system.

The invention relates to a method for realizing adjustable consistency of a storage system adopting a Raft protocol, and belongs to the technical field of storage systems. The method comprises the steps that a write-in return point and a read return point are introduced for each db, the write-in return point controls the position, returned to a user, of a write-in command in the whole execution process, when write-in operation is executed to the position corresponding to the write-in return point, a state machine is notified to execute through prepply, and an execution result is returned to the user after execution is completed; the reading return point controls the reading command to read from the target copy and the reading mode; the consistency of all read-write commands on the db is controlled by combining values of different write-in return points and values of different read return points; the data applied in advance is rolled back when conflicting so as to ensure that the system data is finally recovered to be consistent. According to the method and the device, the difficulty of realizing the adjustable consistency under the condition of using the Raft protocol is reduced, the additional prepply is introduced, the adjustable consistency is realized, the retention of the onpply also ensures the correctness of the Raft, and the proof of the final consistency is simplified.

This invention relates to the field of digital asset trading technology and discloses a high-concurrency method and system for pre-deducting digital assets based on multi-level caching. The system includes a request reception and verification module, an inventory pre-deduction module, an order locking module, and an asynchronous order placement and rollback module. After receiving a user's purchase request, the system first performs eligibility verification, and then performs pre-deduction based on a digital asset inventory pool stored in a distributed cache, directly allocating a specific asset identifier. Upon successful deduction, an order lock record associated with the user and the asset identifier is created in the cache. This record is processed asynchronously through a message queue to generate a persistent order, completing the final binding. If the asynchronous process fails, an inventory rollback operation is automatically performed based on the asset identifier in the order lock record, adding it back to the inventory pool. This invention achieves high-concurrency, accurate, and secure allocation of unique digital assets and ensures the eventual consistency of the transaction process.

The invention relates to the technical field of gateways, in particular to an API gateway migration verification method and system based on a flow mirror image, a medium and equipment, and the method comprises the steps: recording the flow information of an API gateway called by a production environment; recovering the recorded traffic information to the original traffic; the restored original flow is forwarded to a credential environment in a multi-thread mode; and monitoring and comparing the performance indexes of the API gateway in the creative environment and the original production environment, and comparing the final consistency of each business index of the creative environment and the original production environment under the same flow. According to the method, flow mirroring and final consistency verification are adopted, the existing production environment is not affected, the correctness of the whole business process can be grasped from the system level, the interaction problem between interfaces can be found, the verification efficiency is improved, and the actual requirement of API gateway information creation environment migration verification can be better met.

Techniques are described for enabling uninterrupted services by computing resources on nodes of a consistency hash ring (CHR) while adding or removing nodes (i.e., making changes) of the CHR. In some embodiments, a duplicate of the existing CHR (i.e., old version) is created to become a new version for performing the changes. Two versions of consistent hash rings (CHRs) co-exist during the transition period of making changes. In some embodiments, computing resources on the nodes of these CHRs perform version upgrades and data migration while continuing to service client requests without interruption.

The invention discloses a distributed architecture-oriented time sequence library high-concurrency writing method, which relates to the technical field of computer databases, and comprises the following steps of: constructing a self-adaptive multi-stage writing path of time sequence data, and dynamically selecting the writing path of the time sequence data and adjusting a load in real time by embedding an intelligent scheduling layer and a cross-layer feedback control mechanism; based on a sliding time window and an additional recording queue mechanism, performing isolation and combination on out-of-order data and historical complement data written in the distributed time sequence database; and based on version conflict detection of a vector clock and a latest version priority merging strategy, merging and writing confirmation are carried out on the time sequence data after out-of-order writing and historical additional recording processing. According to the method, a layered write-in path design and a lightweight consistency protocol are adopted, high-concurrency write-in and final consistency guarantee of data are realized aiming at complex scenes such as out-of-order write-in, historical compensation and network jitter, and the write-in performance and reliability of a system are improved.

The application provides a message-driven data center consistency control method based on low-code rule evolution, comprising: constructing a low-code rule model, establishing a rule version evolution mechanism, implementing a message-driven event flow, constructing a rule consistency control mechanism, and executing state synchronization through a data center. The rule conditions, rule actions and event relationships are configured visually in a low-code manner. Rule dynamic evolution control is achieved through a rule state space model, a rule evolution model, a message consistency model and a double rule window model. The rule engine performs message routing, field parsing and rule processing according to the associated rule version of the message, and combines idempotent checking, rule conflict detection, event traceability and compensation recalculation mechanism to realize data consistency control between distributed nodes, thereby improving the stability, reliability and eventual consistency control capability of the data center in a high-concurrency and dynamic rule scenario.

The invention provides a multi-terminal data synchronization and consistency verification method and system based on a message queue, and relates to the technical field of data processing. According to the method, firstly, a service data change event sent from multi-terminal data is received, and the service data change event corresponds to core service data; secondly, releasing the business data change event to a RabbitMQ message queue and distributing the business data change event to a target terminal so as to realize data synchronization among multiple terminals; next, a preset lightweight consistency verification engine is called to execute periodic snapshot comparison on the core service data; and finally, when the data is detected to be inconsistent through the periodic snapshot comparison, triggering an abnormal automatic repair mechanism to maintain the consistency of the multi-terminal data. According to the technical scheme provided by the invention, high reliability and self-healing capability of a multi-terminal data synchronization process are realized, and final consistency of core service data is also ensured.

The application discloses a problem positioning method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: obtaining a training set according to a labeled and unlabeled log sample set, performing first training on a problem classification model; taking logs of a preset number of problem scenarios as a test set, performing consistency evaluation on the problem classification model after the first training, performing second training on the problem classification model after the first training according to the consistency evaluation result, repeating the consistency evaluation and the second training process until the final consistency evaluation result meets a preset condition to obtain a trained problem classification model, and determining a problem category in log data of a problem to be positioned. The embodiment can realize automatic and accurate positioning and classification of bugs, help testers avoid low-level problems, enhance the independence of testers, save the cost of repairing problems for developers, has low requirements on test resources, has low computation cost, and has strong self-adaptability.

This invention provides a method for automatic synchronization of multiple databases under an AI platform, belonging to the field of automatic database synchronization technology. This invention initializes the multi-database synchronization topology and deploys a change data capture module to monitor data changes in real time. The captured change records are converted into unified format synchronization messages and input into an intelligent scheduling optimization model for dependency analysis and transaction reordering based on dynamic programming. Data conversion rules are dynamically generated based on mode version differences, and Merkle tree hierarchical structure is used to verify data consistency. After conflict detection and circular dependency handling are performed at the target node, multi-level adaptive compression transmission and data writing are completed. Finally, periodic full reconciliation tasks ensure eventual consistency among the multiple databases, solving the technical problem of low transaction scheduling efficiency leading to synchronization performance bottlenecks during automatic multi-database synchronization.

The invention provides an electric power data consistency evaluation method and system based on deep learning, and the method comprises the steps: carrying out the scene analysis based on the operation state information of an electric power system in a current operation state, and determining a target electric power scene type in the current operation state; matching in a pre-established power model library based on the target power scene category to obtain a target power model; inputting power data of the power system in the current operation state into the target power model to obtain a preliminary consistency evaluation result output by the target power model; and determining a final consistency evaluation result of the power data based on correlation and difference analysis between the initial consistency evaluation results under different target power scene categories. According to the method, the problem of evaluation deviation caused by insufficient scene adaptability is avoided, and a reliable basis is provided for stable operation and optimization decision making of a power system.

The invention provides a distributed real-time scheduling instruction integrity tracking system. The distributed real-time scheduling instruction integrity tracking system comprises a hardware architecture, a software module and a data model, the software module comprises a scheduling instruction generation module, a distributed data storage and management module and a real-time monitoring and tracking module, and the scheduling instruction generation module is used for generating high-priority scheduling instructions through a machine learning algorithm based on real-time data flow and task requirements. The system provided by the invention also solves the problem that the final consistency in the distributed environment cannot be completely ensured. By introducing the consistency control model based on the Paxos or Raft and other distributed consensus algorithms, the consistency and synchronization of data among multiple nodes are ensured, the instruction execution states are always kept consistent, execution errors or inconsistency caused by data collision or update delay are avoided, and thus the accuracy of instruction integrity tracking is improved.

The application belongs to the technical field of Internet of Things, and discloses a cloud edge synchronization method and system of a distributed Internet of Things rule engine, wherein a state of a rule set is constructed as a verifiable Merkle-CRDT rule state tree, efficient and auditable final consistency is realized through synchronization of a state root hash; before a new rule state is activated, the system performs dynamic behavior verification on the new rule set in a pre-execution simulation sandbox based on digital twinning to find logical conflicts that cannot be identified by static detection; after the rule is activated, a causal anomaly detection engine driven by a large language model continuously monitors system operation, and once a causal anomaly is found that is contrary to physical common sense, the relevant rule is immediately dynamically placed in a runtime isolation sandbox to deprive it of control over the physical executor. Through the three-layer defense system of pre-verification, pre-rehearsal and in-process controllability, the application realizes cloud edge rule collaboration with high reliability and high resilience.

There is provide a method and system for eventual consistency of data types in geo-distributed active-active database systems. A multi-type conflict-free replicated data type (CRDT) structure is provided. The multi-type CRDT structure and method allow a geo-distributed active-active database system to reach eventual consistency on data type conflicts by supporting multiple incompatible data types.

The present invention proposes a method that takes into account availability and partition tolerance while improving the consistency of streaming computing, quantitatively describes the CAP problem, and then optimizes it from a holistic level. First, tensors are used as a mathematical tool to quantitatively express the consistency of non-fixed redundant distributed storage systems. Then, based on the streaming computing architecture and update process of the eventual consistency model, a quantitative constraint relationship between C, A, and P is derived. After determining the properties and cost constraints of different parameters in the constraint relationship model, a three-objective optimization problem is formed. Finally, the improved NSGA2 genetic algorithm is used to solve the problem, and after iteration, a frontier image depicting the Pareto optimal solution set can be obtained. This method realizes the quantitative representation of the CAP problem that can only be analyzed qualitatively in the traditional way, and uses a genetic algorithm to obtain the optimal solution set, providing support for decision makers to select a satisfactory solution as a solution to the streaming computing consistency problem.

The invention discloses an Elasticsearch intelligent search method based on Spring Boot, and the method comprises the steps: converting a business peak value QPS into a thread pool and connection pool parameter through an area-geometry model, and achieving the zero-manual configuration; the uniqueness of the index name is checked through geometric coverage of a degradation circle, and the name duplication conflict is eradicated; a fom + size or search after paging path is judged and adaptively locked by using a'rectangular-square 'area, and deep paging linear increment pulling is completed through a'semi-plane vernier'; metadata is cached after a special field is reflected for the first time, subsequent query is directly reused, and repeated reflection is avoided; a three-level quantization formula of'change density D-sub-batch number B-consistency score Cidx 'is introduced, incremental updating is split into a minimum difference set, index back-off retry is matched, and finally Cidx is larger than or equal to 99%. According to the method, full-link intelligent closed loop from configuration, query and mapping to incremental synchronization is realized without manual intervention, the query performance of large data volume is remarkably improved, the development complexity is reduced, and the final consistency is ensured.

The invention provides a method and a system for realizing final consistency of data types in a remote multi-activity database system. According to the method, a multi-type conflict-free replicated data type (CRDT) structure is provided, and the method is applied to the field of multi-type conflict-free replicated data types (CRDT) structures and can be applied to the field of multi-type conflict-free replicated data types (CRDT) structures. According to the multi-type CRDT structure and the multi-type CRDT method, a remote multi-activity database system achieves final consistency in data type conflicts by supporting multiple incompatible data types.