9results about How to "Improve data synchronization efficiency" patented technology

The present application discloses a load balancing method and device based on an improved consistent hash ring, which is applied to a distributed cluster, wherein the distributed cluster includes at least two server nodes. The method includes: obtaining data to be stored and determining a hash value of the data to be stored; determining the first node that is closest to the hash value of the data to be stored in a clockwise direction on a created target hash ring as a candidate node, wherein the target hash ring is obtained by creating an initial hash ring based on the hash value corresponding to each server node, and creating a corresponding virtual node for each server node at a symmetric point on the initial hash ring; if the candidate node is determined to be any server node, the data to be stored is stored in the server node; if the candidate node is determined to be a virtual node, the data to be stored is stored in the server node corresponding to the virtual node.

This invention provides a data synchronization method and apparatus. Through a visual interface, the data source and target cluster index parameters are configured. A data synchronization task is created based on the data source and index parameters. The task's time period and triggering rules are set. The task is scheduled according to these rules. An incremental watermark algorithm is used to read data from the data source. Write nodes are dynamically allocated based on load. A search interface combining HTTP / Socket dual protocols and local caching is provided to query data for business scenarios. This improves development efficiency, ensures data consistency and reliability, enhances resource utilization, and reduces data synchronization time. It significantly improves the data synchronization efficiency and system stability of distributed systems and is suitable for high-concurrency, multi-source, heterogeneous data synchronization scenarios.

The present invention relates to the field of distributed system control technology, and specifically provides a data synchronization method, device and computer-readable storage medium for a distributed system, aiming to solve the technical problem of how to efficiently and reliably synchronize data for a distributed system. To this end, the method according to an embodiment of the present invention can quickly locate a service engine to be synchronized that needs to be synchronized and determine the original data to be synchronized based on the working status of the service engine and the abnormal timestamp, and can accurately synchronize data for each service engine to be synchronized based on the determined original data to be synchronized, thereby overcoming the defects of low efficiency and easy data loss caused by manual adjustment of data synchronization in the prior art. In addition, when data synchronization needs to be performed on a large number of service engines to be synchronized, data synchronization can also be performed on each service engine to be synchronized at the same time, further improving the data synchronization efficiency of the distributed system.

The present disclosure provides a data synchronization method, which can be applied to the fields of computer software and financial technology. The method includes: transmitting a business model code from a data provider to a data receiver, the business model code including data to be synchronized; creating an event processing class according to the business model code; obtaining a standard development kit; using the event processing class and the standard development kit to publish the data provider and monitor the data receiver to synchronize the data; wherein the data to be synchronized is published and monitored through an event bus. The present disclosure also provides a data synchronization device, equipment, storage medium and program product.

The invention is suitable for the technical field of electric power, and provides an electric power data increment synchronization method and device, and the method comprises the steps: obtaining network delay and the size of a to-be-synchronized file; judging whether the to-be-synchronized file meets a preset first synchronization triggering condition or not according to the network delay and the file size; and if the first synchronization triggering condition is met, performing incremental synchronization on the to-be-synchronized file. According to the invention, the problems of high power data synchronization delay and low efficiency in the prior art can be solved.

The invention provides a vehicle-mounted data processing method, a vehicle control method and a control system. The vehicle-mounted data processing method comprises the following steps: generating a plurality of target data blocks based on compressed vehicle-mounted data; the first root hash of each target data block is sent to the cloud and registered, and after the target data block is uploaded to the cloud, the registered first root hash is used to carry out integrity verification on the target data block; when the verification fails, recursively comparing the sub-tree Hash based on the first root Hash which fails in verification to obtain an error data unit in the target data block, and performing retransmission processing; integrity verification is carried out again after each retransmission, and the number of retransmission times is provided with an upper limit; and writing the verified target data block into the target storage layer. According to the invention, the bandwidth utilization rate can be obviously improved in a weak network environment, the transmission energy consumption of the edge node is reduced, the transmission and processing speed of the vehicle-mounted data from the vehicle end to the cloud end is accelerated, and the data synchronization efficiency of the whole vehicle networking system is improved.

The application provides a data synchronization method, a storage system, equipment and a storage medium, and relates to the field of storage. The storage system comprises a first computing node, a second computing node and a shared storage device. When the first computing node flushes the obtained primary copy into a data segment in the memory, the first computing node copies a first part of data of the data segment to the second computing node through network transmission, and stores the data segment into the shared storage device when preset storage conditions are met. The second computing node processes and stores the received first part of data, and reads and loads a second part of data of the data segment from the shared storage device when it is determined that the second part of data is not stored. The first part of data and the second part of data are secondary copies of the primary copy. In the technical scheme, in the storage system architecture of separation of computing and storage, the primary and secondary copy synchronization is realized by using two synchronization links of the shared storage device and the network transmission, and the data synchronization efficiency is improved.

The embodiments of this specification provide a data synchronization method and device. The data synchronization method includes: obtaining a set of pre-write log data recorded for data processing on a database master node; performing a read operation on each pre-write log data in the pre-write log data set based on multiple parallel read threads; performing a parsing operation on each pre-write log data based on multiple parallel parsing threads; and applying each pre-write log data to a database slave node based on multiple parallel application threads according to the generation time of each pre-write log data, to obtain a database slave node after data synchronization. In the case of reading, parsing and applying operations on different pre-write log data, the reading thread, parsing thread and application thread run in parallel. This solution can improve the efficiency of data synchronization processing based on parallelization and pipeline processing between reading operations, parsing operations and application operations.

The application discloses a kind of trisomy nomadic AI ark full industry chain cloud general control integrated edge platform system, belong to MiC modularization cluster multi-subsystem unified scheduling, edge cloud fusion processing, multi-protocol data intercommunication, travel energy storage carbon asset business cooperation technical field.Each functional subsystem is independently deployed to form data island in the prior art, lacks unified top edge general control hardware carrier, multi-business data cannot be fused to judge, and there is room for improvement in the efficiency of multi-scenario cooperation of industry operation, energy storage transaction and emergency dispatching.The application takes the whole cloud general control edge gateway as the entity bearing, builds seven-layer general control architecture, and standardizes the interface of all bottom hardware subsystems from 1 to 7;Integrates seven cooperative modules of protocol adaptation, data fusion, multi-business analysis, virtual power plant transaction, carbon asset statistics, planning and emergency coordination and industry report, synchronously collects land / sea / air ark full-dimensional operation data in the whole world and generates integrated operation and dispatching strategy.The scheme is suitable for the operation of grassland travel, offshore camp and low-altitude station full industry chain, and the synchronization time delay of cross-system data is reduced to 120 ms, the workload of multi-business artificial comprehensive disposal is reduced by 66%, and the cluster comprehensive operation income is increased by 24%.