Self-adaptive dynamic adjustment method and device of data synchronization system

A data synchronization and dynamic adjustment technology, applied in database management systems, special data processing applications, structured data retrieval, etc., can solve problems affecting performance, unable to adjust adaptively, and insufficient processing capacity, so as to improve overall performance and enhance The effect of process adaptive adjustment of processing capacity

Active Publication Date: 2019-03-19
WUHAN DAMENG DATABASE +1
View PDF7 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] For each synchronization task, the first-level thread needs to allocate the total available memory of the synchronization process and the available memory of each functional component through calculation when starting. Each functional component cannot adaptively adjust the maximum memory required for this execution, and too much memory is allocated. Resources are wasted, and too little memory will directly affect performance. The repeated timing call execution of the process lacks self-learning ability and cannot effectively use memory; the synchronization process allocates the maximum amount of available memory during initialization, but it may not be able to fully utilize it during execution. Used, even if it is not used, the remaining memory cannot be used by other synchronous processes executed;
[0006] The nu

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Self-adaptive dynamic adjustment method and device of data synchronization system
  • Self-adaptive dynamic adjustment method and device of data synchronization system
  • Self-adaptive dynamic adjustment method and device of data synchronization system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0054] An embodiment of the present invention provides a data synchronization system adaptive dynamic adjustment method,

[0055] For the convenience of description, the letters a-e are used to represent different cache amounts, among which:

[0056] The value of a indicates the maximum amount of cache that the functional component can apply to the system during this execution, which is recorded as the maximum available memory: maxCacheSize;

[0057] The b value indicates the maximum cache size that the functional component has obtained from the system according to the maximum cache size it may need during this execution, which is recorded as the current available cache: currentAvailableCacheSize;

[0058] The c value indicates the amount of cache actually used by the functional component during this execution, which is recorded as the currently used cache: currentUsingCacheSize;

[0059] The d value indicates the maximum cache amount actually used in the life cycle of the fu...

Embodiment 2

[0076] On the basis of Embodiment 1 above, the embodiment of the present invention provides a specific implementation method for dynamically adding primary threads and secondary threads during the synchronization process. It can be seen from Embodiment 1 that in the step 30, when the functional component adds data to its own message queue, if c≥b*present.1 is found, the functional component notifies its downstream component to add a secondary thread . In addition, the functional component can also notify its upstream component to add a secondary thread, specifically, when the functional component reads data from the message queue of its upstream component, if it finds that the message queue of the upstream component is empty or the message If the amount of cached data in the queue is less than the preset threshold, it means that the processing capability of the upstream component to read data or convert data is too low to keep up with the processing speed of the functional com...

Embodiment 3

[0088] On the basis of a data synchronization system adaptive dynamic adjustment method provided in Embodiment 1 and Embodiment 2, the present invention also provides a data synchronization system adaptive dynamic adjustment device that can be used to implement the above method, such as Image 6 Shown is a schematic diagram of the device architecture of the embodiment of the present invention. The data synchronization system adaptive dynamic adjustment device of this embodiment includes one or more processors 21 and memory 22 . in, Image 6 A processor 21 is taken as an example.

[0089] The processor 21 and the memory 22 may be connected via a bus or in other ways, Image 6 Take connection via bus as an example.

[0090] The memory 22, as a non-volatile computer-readable storage medium of a data synchronization system adaptive dynamic adjustment method, can be used to store non-volatile software programs, non-volatile computer-executable programs and modules, as in Embodim...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of data synchronization, in particular to a self-adaptive dynamic adjustment method and a device for a data synchronization system. The method comprises the following steps: determining a maximum available memory a that a functional component can apply to the system when synchronization is executed this time; according to the history log record, determining the maximum amount of delay memory b required for this execution, obtaining the corresponding memory from the system and starting the functional components; when the functional component appendsdata to its own message queue, updating the currently used cache c in real time. If c >= b*present.1 and b < a, it applies to the system for increasing memory. At that end of this execution, the maximum c value is log as the actual maximum usage cache for the next use. By increasing the maximum amount of slow memory for historical execution in the log, the functional component can determine the amount of cache usage required for this execution according to the log record, and adjust the memory adaptively at run time; at the same time, it can dynamically increase the first-level thread and thesecond-level thread according to the actual operation, and improve the performance of data synchronization.

Description

【Technical field】 [0001] The invention relates to the technical field of data synchronization, in particular to an adaptive dynamic adjustment method and device for a data synchronization system. 【Background technique】 [0002] In the data synchronization system, the synchronization process refers to the data synchronization task of data extraction, conversion, and loading composed of data reading components, data conversion components, and data loading components, including one or more data reading components, Zero or more data conversion components, one or more data loading components, and a synchronization process constitute a synchronization task. When a data synchronization task is executed, multiple execution threads are started. First, a first-level thread is started for the entire synchronization process to start the synchronization task described in the synchronization process, and second, one or more second-level threads are started for each functional component in...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/2455G06F16/25G06F16/27
Inventor 付铨袁松彪梅纲
Owner WUHAN DAMENG DATABASE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap