29results about How to "Improve synchronization efficiency" patented technology

This invention discloses a data synchronization method and system based on front-end orchestration management, belonging to the field of data synchronization. The method includes the following steps: selecting the data table and field information to be monitored through the front-end interface, generating configuration information, and orchestrating the synchronization process logic according to business needs; storing the configuration information in real time to the configuration center to generate real-time monitoring rules; capturing changed data of the data table and field information, encapsulating it into standard data units and sending it to a message queue; preprocessing the standard data units in the message queue; the synchronization adaptation module matching the corresponding driver type according to the synchronization process logic to generate a synchronization instruction set, and executing data synchronization operations based on the preprocessing results; performing full-link monitoring of the synchronization process, and performing task lifecycle management based on the lifecycle rules configured on the front end. This invention improves synchronization efficiency through dynamic threshold verification, multi-source field fusion weighting, and dynamic encryption strategies, and the full-link monitoring further enhances front-end manageability.

This application discloses a method, apparatus, and electronic device for synchronizing Hive data tables between clusters. Relating to the field of big data technology, the method includes: acquiring source cluster information, wherein the source cluster information includes at least: the name of the source cluster, the name of the Hive database in the source cluster, and the name of the Hive data table to be synchronized in the Hive database of the source cluster; determining target information of the Hive data table to be synchronized based on the source cluster information, wherein the target information includes at least: table structure information and authorization information, wherein the authorization information includes at least: the authorized object to which the permission belongs and the permission name; processing the target information to obtain processed target information; and synchronizing the processed target information to the target Hive data table in the Hive database of the target cluster. This application solves the problem of low synchronization efficiency of Hive tables across clusters in related technologies.

A multi-terminal data synchronization and secure storage method, device, program product and medium, relating to the technical field of mobile Internet, by dividing learning data into dynamic interactive data and static resource data, and managing the change history of dynamic interactive data through version index tree, so that the system can accurately locate the data version that needs to be synchronized. When the second terminal requests data synchronization, the system only transmits the necessary target data, and realizes on-demand access of static resources through access token mechanism, avoiding repeated transmission of large resource files. This data classification and incremental synchronization based method significantly reduces network transmission overhead and improves the efficiency of multi-terminal data synchronization. At the same time, by separating the access control of static resource data and the synchronization transmission of dynamic interactive data, the safe use of electronic resources is guaranteed, and the needs of users to flexibly switch learning among multiple terminals are met.

Embodiments of the present application disclose a process recipe updating method and device and a computer device. Embodiments of the present application can display a process recipe updating page of a process recipe management system, the process recipe updating page comprising a plurality of object identifiers, the plurality of objects comprising the process recipe management system and at least one semiconductor device in communication connection with the process recipe management system, and the objects storing a process recipe set. In response to a selection operation on a first object identifier, a plurality of process recipe identifiers contained in the first object are displayed. In response to a selection operation on at least one first target process recipe identifier of the first object, a first target process recipe to be transmitted is determined. In response to an updating operation on the first target process recipe, the first target process recipe is transmitted to at least one second object, and updating state information is displayed on the process recipe updating page, thereby efficiently synchronizing the process recipe among the plurality of semiconductor devices.

The invention discloses a key distribution method based on a hierarchical hash chain. The method is used for a password management service platform, and comprises the following steps: generating a session key and a key identifier according to a received session key generation request sent by a first terminal, and sending the session key and the key identifier to the first terminal; receiving a session key acquisition request sent by the second terminal, and verifying the validity of the identity of the second terminal according to verification information submitted by the second terminal; and under the condition that the verification is passed, issuing the session key to the second terminal. Thus, the service platform generates the first Hash chain fragment bound with the terminal permission, so that the password management service platform not only can verify the authenticity of the terminal identity during verification, but also can accurately verify whether the terminal has the legal key application permission, thereby effectively blocking the path of applying for the session key by stealing the key identifier by an illegal user, and improving the user experience. And the risk that the session key is illegally stolen is reduced, so that the security and reliability of encrypted communication between the terminals are ensured.

The application provides a kind of data offline synchronization method based on MQTT protocol client and server, it is related to information technology field, including that client and server establish MQTT long connection;Client encapsulates message synchronization interface, and judges current network state in interface, if network is unobstructed, go online mode, if network is not connected, go offline mode;In online mode, data is directly synchronized to server through MQTT connection;In offline mode, client first caches synchronization data, and checks network state regularly, when network recovers, immediately start synchronization task;Server receives message, and parses data according to message type and handles it.The application can be applied in places with good network environment such as shopping mall, office building, street, etc., and also can be applied in places with poor network environment such as field, mountainous area, jungle, etc., with wider application scenarios.Using the characteristics of MQTT publishing and subscribing, when a large amount of data is used, the server can process data on demand, reduce the pressure on the service, and improve the stability of the service.

This invention belongs to the field of blockchain technology, specifically relating to a method for synchronizing the activity state between blockchains. The method executes the following steps: Step 1: Grouping the target blockchain into several sub-blockchains; each blockchain contains at least two nodes. It achieves state and data synchronization between blockchains by adding routers and synchronizers to the blockchains; routers ensure real-time data collection and generate identifier codes; synchronizers control the routers to achieve synchronization. During synchronization, the synchronizer uses an algorithm to determine whether synchronization is necessary, based on data collected by the routers, thus improving synchronization efficiency. Compared to existing technologies that require continuous acquisition of data from each node for synchronization, this method significantly reduces complexity and effectively avoids the situation where synchronization is performed when it is not needed, thus consuming system resources.

This invention proposes a global parameter configuration method based on parallel discrete event multi-process simulation. This method achieves bidirectional handshake interaction between the host configuration software and the simulation platform software through a Socket transmission interface, mutually verifying the initialization status and automatically restarting the corresponding software initialization process in case of initialization errors. The simulation platform software aggregates all process local parameters and reports them to the host configuration software, which then generates a unified format global parameter file. After the simulation platform software sends a distribution request, the host configuration software distributes the global parameter files sequentially according to the process number. The simulation platform software collects the file reception identifier and process parameter configuration identifier of each process for hierarchical judgment, distinguishing between file reception failure and parameter configuration abnormality, and performs targeted retries for repair. After all processes pass verification, the parallel discrete event multi-process simulation is started. This invention solves the problems of complex global parameter configuration, difficult synchronization, and cumbersome fault handling in existing parallel simulations.

The present application relates to the technical field of wireless communication, and in particular to a smart water meter supporting dual-mode communication and a data synchronization method thereof. The method comprises: obtaining communication jam values of a plurality of smart water meters, wherein the communication jam value is used to represent the communication jam degree between the corresponding smart water meter and the management platform; performing screening processing based on the communication jam values of each smart water meter to determine a first smart water meter and a second smart water meter; controlling the first smart water meter to transmit data to the management platform in a first communication mode; determining a first coordination water meter among the plurality of second smart water meters, and controlling the first coordination water meter to transmit data of all second smart water meters to the management platform, wherein the first coordination water meter is one of the plurality of second smart water meters, and the data of the second smart water meter is reported to the first coordination water meter in a second communication mode. The present application can improve the reliability of data synchronization.

The invention provides a data synchronization method and device, equipment, a medium and a program product, and relates to the field of big data. Receiving remote log transmission requests of a plurality of synchronization services through a log agent, requesting a log file from the source end database through the log agent based on the log analysis breakpoint, receiving a target log file from the source end database, and storing the target log file in a local log storage unit, the plurality of synchronization services share and analyze; according to the method, the log requests of the multiple synchronization services are received in a centralized mode through the log agent, the target log files are uniformly obtained from the source end database based on respective analysis breakpoints and locally cached, multi-service efficient sharing and collaborative analysis are achieved, repeated pulling is avoided, the database load is reduced, and the synchronization efficiency is improved.

The invention discloses a log analysis method, system and equipment of a database and a computer readable storage medium. According to the method, a log set is analyzed, if no abnormal log exists in the log set, a check point is generated based on the log with the latest generation time in the log set, logs included in the log set are marked as analyzed logs, a new log set is determined from to-be-analyzed logs, the new log set serves as the log set, and the log set is analyzed according to the new log set. Analyzing the log set; if the abnormal log exists in the log set, determining a target check point from the generated check points according to the error type of the abnormal log; the new log set is obtained according to the target log and the error type of the abnormal log, the new log set is used as the log set, the log set is analyzed, and by generating and dynamically selecting the check points, different check points can be selected for backspacing when analysis abnormity occurs in the data synchronization process, so that the load of a source library is reduced, and the data synchronization efficiency is improved. And the data synchronization efficiency is improved.

The application relates to the technical field of data processing, and discloses a data synchronization method and system, electronic equipment and a storable medium, wherein a log analyzer reads redo logs of different source databases and obtains json files, the json files are sequentially stored in different queues of a first buffer pool according to different source databases and acquisition sequences, and then the json files are sequentially stored in a rocketmq middleware; a writer sequentially acquires the json files, the json files are sequentially stored in different queues of a second buffer pool, SQL statements corresponding to different data table names are obtained according to the json files, and the SQL statements are written into corresponding data tables. The application solves the problem that when an exception occurs in the writer, the synchronization needs to be restarted from the beginning, but enables the writer to start from the interrupted place, improves the synchronization efficiency, and through the increase of the buffer pool, the changes of different databases can be sequentially recorded and written, and writing errors are prevented.

This application relates to a data synchronization method, apparatus, computer device, storage medium, and program product. The method includes: receiving a data synchronization request from a target storage node, the data synchronization request carrying a first data version tree, the first data version tree being constructed by the target storage node based on the version numbers of the stored data; matching the first data version tree with a local second data version tree to obtain a data path where the first data version tree and the second data version tree differ, the second data version tree being constructed based on the version numbers of the locally stored data; determining, based on the data path, data to be synchronized with the target storage node; and performing inter-node data synchronization processing on the data to be synchronized, the first data version tree, and the second data version tree. This method can improve data synchronization efficiency.

The invention discloses an ERP system data synchronization processing batch optimization adjustment method, device and equipment and a storage medium, and the method comprises the steps: monitoring a network performance parameter, a system load parameter and a data complexity parameter of an ERP system in a current processing period, and carrying out the optimization adjustment of the ERP system data synchronization processing batch based on the network performance parameter, the system load parameter and the data complexity parameter; and calculating to obtain a batch adjustment network factor, a batch adjustment load factor and a batch adjustment complexity factor, and optimizing the reference batch amount of the current processing period through the batch adjustment network factor, the batch adjustment load factor and the batch adjustment complexity factor to obtain the data synchronization processing batch amount of the current processing period. Therefore, by monitoring the network, the system load and the data complexity parameter in real time, calculating the corresponding adjustment factor optimization batch quantity and dynamically adapting the operation state, the fixed batch stiffness is avoided, the data synchronization efficiency is improved, and the system overload is avoided, so that the optimal utilization of resources is realized.

The invention discloses a MySQL data detection and synchronization method and electronic equipment, and belongs to the technical field of software development. The method comprises the following steps: starting a global transaction function of MySQL nodes, wherein a transaction corresponding to each MySQL node contains a unique identifier of each transaction; the monitoring period of the MySQL nodes is determined, and the MySQL nodes are monitored in the monitoring period; automatically determining a target monitoring period, and obtaining an update transaction corresponding to the MySQL node in the target monitoring period; determining a master-slave relationship of the MySQL node based on the information related to the update transaction; according to the master-slave relationship, detecting the data synchronization integrity between the MySQL nodes; and executing a data synchronization integrity repairing step. According to the method and the device, the integrity and the data consistency of data synchronization between the MySQL master and slave nodes are ensured, the synchronization efficiency is improved, and the synchronization difficulty is reduced.

The application provides an artificial intelligence-based data synchronization method and device, electronic equipment and storage medium. The artificial intelligence-based data synchronization method comprises the following steps: constructing a data synchronization template; classifying the data synchronization template according to the type of a source database and the type of a target database to obtain a first category, and constructing a training data set according to the category to train a template classification model; replacing variable parameters in the data synchronization template with parameter placeholders; constructing feature data to be classified according to database information in a data synchronization requirement; inputting the feature data to be classified into the template classification model to obtain a second category to screen a candidate template; replacing the parameter placeholders in the candidate template with variable parameters in the data synchronization requirement to obtain an executable candidate template, screening a target template from the executable candidate template, and executing the target template to complete data synchronization. The method can efficiently query a data synchronization template meeting a requirement, thereby improving the efficiency of data synchronization.

The application discloses a method and device for realizing efficient data synchronization based on a HetuEngine read plug-in of DataX. The method comprises the steps of configuration analysis, connection verification, task splitting, task distribution and execution, data extraction, type conversion and data transmission. Through a standard JDBC interface of HetuEngine, a query is directly connected and executed, a large-scale query task is intelligently split into multiple sub-query tasks according to configured parallel parameters and executed in parallel, and a query result set is read in a streaming mode, and after data type mapping, the query result set is written into a DataX transmission channel. The application avoids an intermediate storage link, fully utilizes distributed parallel processing capacity, significantly improves data synchronization efficiency, reduces system resource consumption and operation and maintenance complexity, has good universality and expansibility, and is suitable for data synchronization scenes of HetuEngine and similar Presto / Trino engines.

Embodiments of the present specification provide a data stream processing method and device based on a merge tree model and a storage medium. The data stream processing method comprises: obtaining source data; obtaining a vertex set and an edge data set according to the source data; obtaining an extreme point set and a rule point set according to the vertex set; obtaining a candidate saddle point set according to the extreme point set, the rule point set and the edge data set; obtaining an extreme point-candidate saddle point pair set according to the extreme point set and the candidate saddle point set; obtaining an extreme point-saddle point pair set according to the extreme point-candidate saddle point pair set and a preset vertex distribution rule; adding the extreme point-saddle point pair set to a preset merge tree; and the communication strategy comprises pushing, pulling and publishing / subscription. The technical solution provided by the present application is used to solve the problems in the prior art that the data volume is increasing, the data operation amount is large, the efficiency is low when the distributed nodes update the data, and it is difficult to achieve good performance when the data volume is large or the control flow is complex.

This application discloses a method, apparatus, device, and medium for synchronizing command formats in a settings interface, relating to the field of server technology. The method includes: in response to the server powering on and entering the Basic Input / Output System (BMC) settings interface, editing the command format interaction page; saving the edited command format interaction page and restarting the server; the BMC configuring commands based on the edited command format interaction page; and uploading the configured BMC file to the Baseboard Management Controller (BMC) web server, so that the BMC web server can update the BMC firmware and the command format space of the BMC settings interface based on the configured BMC file. This application enables synchronized modification by modifying the interface and uploading the file to the BMC, providing convenient operation and improved synchronization efficiency.

The application discloses an intelligent marketing document generation device and method based on multi-source data fusion, and the method comprises four processes: user instruction analysis, regular cleaning, model extraction parameter, Neo4j knowledge graph verification, and output of structured parameters; external data acquisition, parameter calling of a Baidu large model to collect data, de-duplication, semantic enhancement, and weighted screening of high-quality data; internal case processing, regular slicing of case storage, filtering, propagation calculation, multi-dimensional scoring, and generation of a CoT reasoning chain report; multi-modal output, format adaptation, data integration by a BART model, and output of a metadata document. The device and the method are coordinated, multi-source fusion, knowledge graph association and multi-modal conversion are realized, high-quality marketing documents suitable for multiple industries are efficiently generated, decision-making is supported, and the market response and output efficiency of enterprises are improved.

The invention relates to the technical field of electric digital data processing, in particular to a multi-source heterogeneous data synchronization method based on an internet platform, which comprises the following steps of: determining initial data synchronization pressure of a target function plate according to multi-source heterogeneous data, determining the overall data synchronization pressure of the target function plate in combination with the initial data synchronization pressure of the adjacent function plates; determining the local synchronization pressure of the target synchronization link by using the overall data synchronization pressure and the data processing amount corresponding to the target synchronization link; determining the data synchronization interference degree of the front function plate to the target synchronization link by using the data application amount of the front synchronization data set in the target synchronization link; and determining the comprehensive data synchronization degree of the target function plate in combination with the local synchronization pressure and the data synchronization interference degree so as to adjust the data synchronization operation. Through the technical scheme of the invention, the whole data processing flow of the human resource management platform is more coherent, and the synchronization efficiency is higher.

The invention relates to the technical field of data processing, in particular to a data synchronization method, data synchronization equipment and a data synchronization system. The method comprises the following steps: when a data change event of a relational database is monitored, generating a first target identifier based on first configuration information corresponding to the data change event; wherein one data change event corresponds to one first target identifier, and one first target identifier corresponds to one storage unit in the memory key value database; based on a pre-configured synchronization rule, writing the first target identifier and change data corresponding to the data change event into a memory key value database; wherein the synchronization rule comprises any one of full-amount synchronization and selective synchronization.

The embodiment of the invention discloses a data synchronization method and device, electronic equipment and a storage medium. In the application, for any target column in a to-be-synchronized target table in a database, a candidate compression algorithm matched with the data feature of the target column is firstly selected; and when the number of the candidate compression algorithms is greater than 1, the compression efficiency of each candidate compression algorithm is predicted, and a target compression algorithm which is most adaptive to the target column and has an optimal compression effect is selected from the candidate compression algorithms. And finally, carrying out data compression on each target column by using a target compression algorithm which is adaptive to each target column and has an optimal compression effect so as to carry out data synchronization. The optimal compression algorithm with the optimal compression effect is dynamically and adaptively selected for each target column, so that each target column achieves the optimal compression effect, the problems of insufficient compression and low synchronization efficiency caused by one-step compression are avoided, namely, the compression efficiency of the to-be-synchronized data is improved, and the data synchronization efficiency is improved.

The application relates to the technical field of database synchronization, and provides a data synchronization method and device with grouping execution. In the application, multi-version grouping management is introduced, the grouping rules can be adjusted at any time in the data synchronization process on the basis of guaranteeing the consistency of the synchronized data and the execution granularity of the split transaction, the convenience and fault tolerance of the synchronization program are improved, and the data synchronization efficiency is improved. By supporting the setting of the execution priority of each group in the grouping rules, the adaptability of the data synchronization to the business demand is improved, and the manual designation of the synchronization sequence is supported in the case of avoiding the running error of the synchronization service.

This invention discloses a method for dynamic monitoring of forest resources and automatic database updating based on multi-source remote sensing data, belonging to the field of forestry informatization. This method addresses the inconsistencies in resolution, imaging time, and band coverage of multi-source remote sensing data by dynamically generating multi-scale grids to achieve spatial alignment and fusion of data. Utilizing temporal residual discriminative convolutional networks and sparse attention mechanisms, it intelligently detects changes in forest resources, achieving efficient and automatic attribute inversion. The system also includes automatic differential database updates and multi-version management to ensure information synchronization and reliability. Combined with external climate and fire data, it provides alarms for abnormal changes and suggestions for auxiliary intervention, forming an intelligent closed loop from monitoring to management. This method improves the automation, intelligence, and precision of forest resource monitoring and is suitable for large-scale, dynamic forest resource management scenarios.

This invention relates to a method, system, and electronic device for synchronizing vehicle-mounted electronic maps. The method includes: acquiring multiple vehicle-mounted electronic maps organized by route; storing the multiple vehicle-mounted electronic maps in a vehicle-mounted host; the vehicle-mounted host selecting the corresponding vehicle-mounted electronic map based on the locomotive's location; the vehicle-mounted host and the dispatch center exchanging data wirelessly, and notifying and downloading the vehicle-mounted electronic maps; and triggering the synchronization of the vehicle-mounted electronic maps based on the locomotive's scenario. Specifically, selecting the corresponding vehicle-mounted electronic map based on the locomotive's location ensures the correspondence between the locomotive's location and the vehicle-mounted electronic map. Notifying and downloading the vehicle-mounted electronic maps allows them to be handled by different servers, avoiding disruption to normal locomotive control and dispatching operations. Furthermore, triggering the synchronization of the vehicle-mounted electronic maps based on the locomotive's scenario enables synchronization to be triggered in multiple application environments.

The application relates to an FTP unstructured file transmission breakpoint continuation implementation method and device based on DataX. The method comprises the following steps: integrating a progress tracking module into a DataX Writer, creating a transmission progress metadata file (TPMF) containing information such as the number of uploaded bytes and the state for each file and dynamically updating; deploying an independent process monitor to monitor task interruption and trigger progress persistence to the TPFM; a user selects a file through an independent FTP continuation scheduler, reads the TPFM and the server-side target file size to determine the breakpoint, calls an FTP REST command to continue from the breakpoint, updates the state after completion, and the like. The method realizes accurate progress recording, active saving in abnormal interruption and user-controllable continuation, avoids redundant retransmission, improves transmission efficiency and resource utilization, enhances data synchronization robustness and reduces operation and maintenance complexity. The method has small invasiveness to the DataX core, is strong in universality and is suitable for high-reliability synchronization scenes of large-scale unstructured files.

This application belongs to the field of blockchain technology, specifically relating to a block data synchronization method, a block data synchronization device, a computer-readable medium, an electronic device, and a computer program product. The block data synchronization method includes: acquiring multiple data nodes in a communication connection state, wherein the data nodes are blockchain nodes used to provide block data reading services; respectively calculating the round-trip latency and packet loss rate of data communication between each of the data nodes; determining evaluation information for each data node based on the round-trip latency and packet loss rate, wherein the evaluation information represents the real-time communication quality of the data node; selecting a target node from the multiple data nodes based on the evaluation information, and sending a block data synchronization request to the target node according to the block height. This application can improve the synchronization efficiency and accuracy of block data.

The invention provides a data synchronization method, device and system of a time sequence database and a storage medium, events generated by the time sequence database are monitored, and the events comprise an operation log writing event and a data file writing event; distributing a file life cycle for each data file, obtaining the current state of the file life cycle, and selecting to add the operation log write-in event or the data file write-in event into a waiting queue according to the current state of the file life cycle; the event is obtained from the waiting queue, and the operation log writing event or the data file writing event is selected to be sent to the receiving end according to the state of the file life cycle corresponding to the event. The log and the file are dynamically and mixedly transmitted, so that the problems of large network flow and high synchronization delay of a traditional scheme are effectively solved; the optimal balance between high throughput and low delay is realized, and in addition, the storage structure of the time sequence database is adapted, so that the transmission pressure and the calculation load of the receiving end in the edge weak network environment are reduced.