Client database incremental migration method
Through incremental data capture, block processing, version control and migration decision optimization, it solves the problems of high network bandwidth consumption, serious resource occupation and data inconsistency in traditional database migration methods, and realizes efficient and reliable client-side incremental database migration to meet the migration needs of databases of different sizes and types.
Patent Information
- Application Number
- CN202510770572.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-10
- Publication Date
- 2025-09-23
- Estimated Expiration
- 2045-06-10
AI Technical Summary
Traditional full database migration methods consume high network bandwidth, severely occupy resources, and have low migration efficiency during large-scale data migration. They are also unable to effectively process incremental data, leading to data inconsistencies and system failures. Existing incremental migration technologies lack efficient data capture and processing methods.
Through incremental data capture, block processing, version control, data synchronization and migration decision optimization, principal component analysis and genetic algorithm are used to extract migration feature parameters and generate migration decision classifiers to achieve efficient and reliable client database incremental migration.
Significantly improve migration efficiency, reduce system downtime, ensure data consistency and reliability, reduce resource consumption, and adapt to database migration needs of different sizes and types.
Smart Images

Figure CN120687435A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of database migration, and in particular to a method for incremental migration of a client database. Background Art
[0002] With the rapid development of information technology, the ever-expanding data volumes, and the evolving business needs, client database migration has become a critical task in data management. Traditional database migration methods typically employ a full migration approach, requiring the entire database to be retransmitted and stored for each migration. However, this approach has significant drawbacks when dealing with large-scale data.
[0003] Full migrations require transferring large amounts of data, which consumes significant network bandwidth. In unstable networks or with limited bandwidth, the migration process can be extremely slow or even interrupted, severely impacting business continuity. For example, for a client database with tens of GB or more of data, a full migration can take hours or even days to complete. During this time, related business systems may need to be suspended, resulting in significant financial losses and a degraded user experience for the enterprise.
[0004] Full migrations are also very resource-intensive. During the migration process, the servers hosting the source and target databases must perform extensive data read and write operations, which can dramatically increase server CPU, memory, and other resource utilization. This can impact the performance of other running business systems and even cause server failures.
[0005] Full migrations cannot effectively handle incremental data. In real-world applications, database data is constantly updated and changing, especially in systems with active business operations, where incremental data is generated very quickly. Using a full migration requires reprocessing all data for each migration, which not only wastes time and resources but can also lead to data inconsistencies. For example, if the source database receives new data updates during the migration process and the full migration fails to capture these incremental data in a timely manner, the target database will become inconsistent with the source database, impacting business operations.
[0006] To address the challenges of traditional full migration methods, incremental migration technology has emerged. Incremental migration only migrates the incremental data that has changed in the database, significantly reducing data transfer and system resource usage. However, existing incremental migration technologies still face challenges in practical applications. Efficiently capturing and processing incremental data is a key issue. Different database management systems may use different methods to record data changes, making incremental data capture complex and diverse. Furthermore, incremental data may contain version conflicts and complex dependencies, requiring effective methods to detect and process them to ensure the accuracy and consistency of the migrated data. Prioritizing migrated data and optimizing migration strategies are also challenging. During incremental migration, different data blocks may have different characteristics, such as importance, size, and version span. These characteristics must be used to determine the migration order and method to improve migration efficiency and success rate. Existing migration strategies often fail to comprehensively consider these characteristics, resulting in low migration efficiency and even migration failure.
[0007] Therefore, there is an urgent need for an efficient and reliable client database incremental migration method that can effectively capture and process incremental data, resolve data conflicts and dependency issues, reasonably determine migration priorities, and improve migration efficiency and data consistency to meet the needs of modern data management. Summary of the Invention
[0008] The purpose of the present invention is to provide a client database incremental migration method to solve the problems raised in the above background technology.
[0009] To achieve the above-mentioned object, the present invention provides the following technical solution: a method for incremental migration of a client database, the method comprising:
[0010] S1. Acquire incremental data from a client database to generate a to-be-processed incremental data set. Use an incremental data capture method to extract data blocks to be migrated. Perform data segmentation to obtain a segmented incremental data set. Use a version control algorithm to determine the version identifier of the segmented incremental data set. Locate and extract the migration data blocks to obtain a migration data block set.
[0011] S2. Perform data synchronization processing on the migration data block set to obtain a synchronized migration data block set, then generate a synchronized migration data block matrix, set data block associations, mark the migration data blocks, and obtain a marked migration data block matrix;
[0012] S3. Extract the migration feature parameters of the marked migration data block matrix, calculate the migration priority after optimizing the synchronization parameters in the feature parameters, determine the main migration parameters according to the migration priority, use the main migration parameters to train the migration decision model to obtain a migration decision classifier, output the migration execution result, and realize the incremental migration of the client database.
[0013] Preferably, the S1 comprises the following steps:
[0014] S11. Acquire incremental data from the client database to generate a set of incremental data to be processed, and use an incremental data capture method to extract data blocks to be migrated to obtain a set of data blocks to be migrated;
[0015] S12, dividing the data block set to be migrated by data block processing to obtain a block-based incremental data set;
[0016] S13. Determine the final version identifier of the incremental data set after segmentation using a version control algorithm;
[0017] S14. Extract the migration data block using the final version identifier to obtain a migration data block set.
[0018] Preferably, the S13 includes the following steps:
[0019] S131. Calculate the modification time and data block hash value of each data block in the block-based incremental data set to obtain a data block normalization weight; divide the version interval according to the data block normalization weight to generate a set of candidate version identifiers;
[0020] S132: Calculate the time intervals between the candidate version identifiers in the candidate version identifier set, merge the candidate version identifiers with the same time interval, and determine the final version identifier of the incremental data set after segmentation.
[0021] Preferably, S2 comprises the following steps:
[0022] S21, performing data block merging, conflict detection, and redundancy cleaning on the migration data block set to complete data synchronization and obtain a synchronized migration data block set;
[0023] S22 . Generate a synchronized migration data block matrix based on the synchronized migration data block set, and perform migration marking on the synchronized migration data block matrix using the data block association relationship to obtain a marked migration data block matrix.
[0024] Preferably, the step of using the data block association relationship to mark the synchronized migration data block matrix includes:
[0025] Mark the associated data block of the first data block in the upper left corner of the synchronized migration data block matrix, generate a migration sequence number, traverse the data block association relationship to the right, and update the migration sequence number until reaching the upper right corner of the synchronized migration data block matrix. Then traverse the entire migration data block matrix in sequence to complete the migration marking.
[0026] Preferably, S3 includes the following steps:
[0027] S31, using a principal component analysis algorithm to extract migration characteristic parameters of the marked migration data block matrix, and using a genetic algorithm to optimize synchronization parameters in the characteristic parameters to obtain a migration characteristic parameter set;
[0028] S32. Calculate the priority weight of the migration feature parameter set and determine the main migration parameters, which include a data block size parameter, a version span parameter, and a dependency parameter.
[0029] S33. Train the migration decision model to obtain a migration decision classifier, input the data block size parameter, version span parameter, and dependency parameter into the migration decision classifier, and output the migration execution result.
[0030] Preferably, the optimization of synchronization parameters in the characteristic parameters using a genetic algorithm includes:
[0031] The fitness function is set as a linear combination of synchronization parameters. In the genetic population, the kth individual moves along the parameter space with a mutation probability of p. When the individual encounters a parameter boundary during movement, if the random number r∈[0,1] is greater than p, the mutation direction is positive, otherwise it is negative. The fitness value is calculated after the individual position is updated. When the current number of iterations reaches the maximum number of iterations, the optimization is stopped and the optimal synchronization parameters are output.
[0032] Preferably, the S33 includes the following steps:
[0033] S331. Acquire new incremental data from the client database, extract new data block size parameters, version span parameters, and dependency parameters to form a new migration sample set. Divide the new migration sample set into a migration test set and a migration verification set. Set the migration decision model input node to 3 and the output node to 1. Input the migration test set into the migration decision model. Iterate until the maximum number of times has been reached, and then stop to obtain a trained migration decision model.
[0034] S332. Input the migration verification set into the trained migration decision model. If the model error is less than the preset threshold, it is determined to be a migration decision classifier; otherwise, adjust the model parameters until the error meets the requirements.
[0035] Preferably, the conflict detection process in S21 includes the following steps:
[0036] S211, traverse the synchronized migration data block set, identify data blocks with version overlap or dependency conflicts, and generate a conflicting data block list;
[0037] S212 , prioritizing the conflicting data blocks based on the version span parameter, preferentially retaining the data blocks with smaller version spans, and marking the migration data block matrix after the conflict is resolved.
[0038] Preferably, the migration marking step further comprises:
[0039] A migration dependency graph is generated based on the data block association relationships. If a circular dependency structure is detected, the dependency link is forcibly disconnected based on the version span parameter, and the broken link location is recorded in the migration log for subsequent manual review.
[0040] Compared with the prior art, the present invention has the following beneficial effects:
[0041] In terms of data processing efficiency, this method uses incremental data capture technology to accurately extract the data blocks to be migrated, avoiding full processing of the entire database. Compared with the traditional full migration that needs to process massive amounts of unchanged data, this method only operates on incremental data, greatly reducing the amount of data processing. For example, in a scenario where data is frequently updated, each migration only needs to process a small number of changed data blocks, which can shorten the migration time to a few tenths of the traditional method or even less, greatly improving migration efficiency, reducing system downtime, and ensuring business continuity. At the same time, data block processing technology divides the data to be migrated into reasonable data blocks, and combines the version control algorithm to determine the version identifier of the data block, so that the migration process can be carried out in an orderly and efficient manner. Through operations such as version interval division and candidate version identifier merging, it is possible to accurately track version changes of data blocks, avoid migration errors caused by version confusion, and ensure that the latest and most accurate data version is migrated.
[0042] The conflict detection and redundancy cleanup mechanisms in the data synchronization process play a key role in ensuring data consistency and reliability. By traversing the synchronized set of migrated data blocks, data blocks with overlapping versions or dependency conflicts can be promptly identified and prioritized based on the version span parameter, with data blocks with smaller version spans being retained first. This effectively resolves data conflicts and ensures that the migrated data remains consistent with the source data. For example, in a scenario where multiple clients are updating the same data block simultaneously, this mechanism can accurately determine the priority of the data and avoid data inconsistencies. In addition, the migration dependency graph generated during the migration marking process can clearly demonstrate the dependency relationships between data blocks. When a circular dependency structure is detected, the dependency link is forcibly disconnected based on the version span parameter and the location of the broken link is recorded. This not only resolves the dependency conflict problem but also provides a basis for subsequent manual review, further improving the reliability of data migration.
[0043] In terms of intelligent decision-making and optimization, the application of principal component analysis (PCA) and genetic algorithms makes the extraction and optimization of migration feature parameters more scientific and efficient. PCA can extract key migration feature parameters from complex migration data, reducing data dimensionality while retaining essential information and reducing model complexity. As an intelligent optimization algorithm, the genetic algorithm searches for optimal synchronization parameters in the parameter space. By setting mechanisms such as fitness functions and mutation probabilities, it continuously optimizes synchronization parameters, improving the accuracy and adaptability of the migration decision model. For example, under varying network environments and data characteristics, the genetic algorithm can automatically adjust synchronization parameters to ensure the migration process remains optimal. The migration decision classifier, derived from training, intelligently determines migration priorities and execution strategies based on key migration parameters such as data block size, version span, and dependency relationships. This automates and intelligently makes decisions during the migration process, reduces manual intervention, and improves migration accuracy and efficiency.
[0044] In terms of resource consumption control, since only incremental data is processed, this method significantly reduces data transfer and storage resource usage. Compared to traditional full migrations, which require transferring and storing the entire database, this method only transfers and stores the changed data blocks, reducing network bandwidth and storage requirements. This advantage is particularly significant for resource-constrained client devices, as it can avoid system crashes caused by insufficient resources, while also reducing hardware and operating costs for enterprises.
[0045] This method also offers excellent scalability and adaptability. Through flexible data partitioning and versioning mechanisms, it can adapt to the migration needs of client databases of varying sizes and types. Whether it's a client database for a small application or a large-scale enterprise-level client database, efficient and reliable incremental migration can be achieved by adjusting the partitioning strategy and versioning parameters. Furthermore, the training and optimization mechanisms of the migration decision model enable this method to evolve and improve as data characteristics and business needs change, maintaining its long-term applicability and advancement. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 This is a working principle diagram of the client database incremental migration method according to the present invention;
[0047] Figure 2 A flow chart established for incremental data capture and versioning;
[0048] Figure 3 Flowchart for data synchronization and tagging processing;
[0049] Figure 4Flowchart for training and validating the migration decision model. DETAILED DESCRIPTION
[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0051] See also Figure 1-Figure 4 The present invention relates to a client database incremental migration method that achieves efficient and accurate database incremental migration through incremental data segmentation, version control, data synchronization, and migration decision optimization. The specific implementation steps are as follows:
[0052] S1. Incremental Data Acquisition and Migration Data Block Extraction: Incremental data from the client database is acquired through database log monitoring or change capture mechanisms (such as triggers or CDC technology) to form a set of incremental data to be processed. Using incremental data capture methods, data blocks to be migrated are extracted from the incremental data set according to pre-defined rules (such as table partitioning and timestamp ranges) to obtain the set of data blocks to be migrated.
[0053] Subsequently, the data block set to be migrated is partitioned into several sub-blocks based on data size, business relevance, and other dimensions to form a partitioned incremental data set. For example, data blocks can be divided into 1024 blocks based on the hash modulus of the data table row ID, or data blocks can be divided into blocks based on time windows (such as every hour).
[0054] A version control algorithm is used to determine the version identifier for the incremental data set after segmentation. The algorithm calculates the modification timestamp and hash value of each data block, generates a standardized weight that incorporates both time and content characteristics, and then divides the version intervals based on the weights to generate a set of candidate version identifiers. The time intervals between candidate version identifiers are further calculated, and version identifiers with the same intervals are merged to obtain a unique final version identifier. This version identifier is used to locate and extract the target migration data block, forming a migration data block set.
[0055] S2. Migration data block synchronization and marking: Perform data synchronization processing on the migration data block set: first merge the data blocks to merge the related data blocks of the same business entity into a logical unit; then use conflict detection to identify data blocks with overlapping versions or dependency conflicts, sort the conflicting data blocks based on the version span parameter, retain the data blocks with smaller version spans, and clean up redundant data to obtain the synchronized migration data block set.
[0056] Generate a two-dimensional matrix based on the synchronized set of migration data blocks, with the rows and columns representing the logical coordinates of the data blocks. Set the data block relationships (such as foreign key relationships and business process sequence) and begin marking the first data block in the upper left corner of the matrix. First, mark its associated data blocks to generate the initial migration sequence number. Then, traverse rightward through the data blocks in the same row, updating the sequence numbers based on the relationships, until you reach the upper right corner of the matrix. Then, traverse downwards row by row to complete the migration marking of the entire matrix, forming a marked migration data block matrix.
[0057] S3. Migration feature optimization and decision execution: Use the principal component analysis (PCA) algorithm to extract migration feature parameters from the labeled migration data block matrix, including data block size, version span, dependency complexity, etc. Optimize the synchronization parameters in the feature parameters using a genetic algorithm: Set the fitness function as a linear combination of the synchronization parameters. In the genetic population, the kth individual moves along the parameter space, and the mutation probability is set to p (e.g., 0.1). When the individual moves to the parameter boundary, if the random number r∈[0,1] is greater than p, the mutation direction is positive, otherwise it is negative. Calculate the fitness value after each update of the individual position. Stop optimization when the maximum number of iterations (e.g., 100 generations) is reached to obtain the optimal synchronization parameters.
[0058] The migration priority is calculated based on the optimized feature parameters, and the weight distribution is determined using the Analytic Hierarchy Process (AHP). The data block size parameter, version span parameter, and dependency parameter are used as the primary migration parameters. The migration decision model is trained using these primary migration parameters: The corresponding parameters are extracted from the newly acquired incremental data to form a migration sample set, which is divided into a test set and a validation set. The model input nodes are set to 3 (corresponding to the three primary parameters) and the output node is set to 1 (the migration execution result). The model is iterated through the backpropagation algorithm to the maximum number of times (e.g., 5000). The validation set is input into the model. If the error is less than a preset threshold (e.g., 0.01), it is determined to be a migration decision classifier. Finally, the migration execution result is output, completing the incremental database migration.
[0059] The present invention will be further described below in conjunction with Examples 1 to 5:
[0060] Example 1: In step S1, after the incremental data set to be processed is generated, the data blocks to be migrated are extracted according to the primary key range of the data table through the incremental data capture method to form a data block set to be migrated. The data block processing stage adopts a dynamic block strategy, which is specifically implemented as follows: First, a data block size threshold is set, for example, 1MB is used as the maximum capacity of a single data block. For files that exceed the threshold, they are split into several small data blocks according to the physical storage order or logical association relationship of the data records, ensuring that the number of records contained in each data block does not exceed the preset value (such as 1000). During the splitting process, the logical association between data blocks is retained. For example, records within a continuous primary key range in the same data table are divided into the same data block, or fields with strong associations are combined in the same data block according to the business scenario, so as to reduce dependency conflicts in the subsequent migration process.
[0061] The specific implementation of the version control algorithm includes the following steps: For each data block in the incremental data set after block division, its modification time and data block hash value are calculated. The modification time is obtained with millisecond accuracy through the database system's timestamp field or log records; the data block hash value is generated using the SHA-256 algorithm, and the integrity of the data block content is verified to ensure that the data has not been tampered with or damaged. Through normalization, the modification time and hash value are converted into the data block's standardized weight. The weight calculation formula is:
[0062]
[0063] The "maximum timestamp" is the maximum modification timestamp of all data blocks in the incremental data set after segmentation, and the "maximum checksum" is the maximum checksum of the hash values of all data blocks. This formula quantifies and integrates the time dimension and content characteristics, giving each data block a unique standardized weight value that reflects its comprehensive properties in version control.
[0064] Based on the standardized weights of data blocks, version intervals are divided. The specific operation is to sort the weight values in chronological order and divide the version intervals in units of natural days, that is, data blocks generated or modified on the same natural day belong to the same version interval. Each version interval generates a corresponding candidate version identifier in the format of "V+year+month+date+serial number". For example, the first version interval identifier of May 28, 2025 is "V20250528_01". After generating the set of candidate version identifiers, the candidate version identifiers need to be time interval calculated and merged. The time interval corresponding to any two adjacent candidate version identifiers is calculated. If the time interval is less than the preset threshold (such as 1 hour), the two version identifiers are considered to belong to the same logical version and are merged into a final version identifier. For example, if multiple candidate version identifiers are generated on May 28, 2025, and the time intervals between adjacent identifiers are less than 1 hour, they are merged into a unique final version identifier "V20250528".
[0065] The process for extracting migration data blocks using the final version identifier is as follows: First, the corresponding version interval is determined based on the final version identifier, and all data blocks within this interval are filtered out. Next, the integrity and consistency of each data block are checked to eliminate any abnormal data blocks that may have appeared during the block or version division process. Finally, the eligible data blocks are sorted in order of standardized weight to form the migration data block set. During the extraction process, if a data block's hash value is found to be inconsistent with the original data or its modification time is outside the version interval, the exception handling process is triggered, an error log is recorded, and the data block is skipped to ensure the accuracy of the migration data.
[0066] During data segmentation, dynamic segmentation strategies must also consider the business relevance of data segments. For example, for order data, the order header and detail information corresponding to the same order number can be grouped into the same data segment to avoid data dependency disruption during the migration process. Furthermore, different segmentation rules can be used for historical archived data and real-time data. Historical data can be segmented into fixed segments based on time windows (e.g., monthly), while real-time data can be segmented based on dynamic thresholds to accommodate different data processing requirements.
[0067] The time interval merging mechanism in the version control algorithm effectively reduces the number of version identifiers, avoiding inefficient migration due to version fragmentation. For example, in high-concurrency scenarios, multiple small incremental data blocks may be generated within a short period of time. Although these blocks were modified at different times, they represent continuous updates of the same business operation. By merging version identifiers with smaller time intervals, these blocks can be migrated as a whole, improving migration efficiency and data consistency.
[0068] After extracting the migration data block set, a pre-verification step is required, including counting the number of data blocks, verifying the data size, and checking initial dependencies. This pre-verification ensures that the migration data block set meets the requirements of subsequent processing flows. For example, the number of data blocks does not exceed the system's processing capacity, the data size is within the network bandwidth, and the dependencies between data blocks are clearly identifiable. If pre-verification fails, adjustments are made to the data segmentation or version control process until the requirements are met.
[0069] Example 2: In step S2, the specific implementation method of data synchronization processing and migration marking is as follows: perform data block merging processing on the migration data block set. The merging rule is based on the business relevance and physical storage characteristics of the data block. For example, for the incremental data blocks of the same data table, adjacent data blocks are merged into continuous logical units in the order of the primary key (such as auto-increment ID, timestamp) to reduce the number of data blocks and improve migration efficiency. During the merging process, it is necessary to ensure that the merged data blocks do not exceed the preset maximum capacity (such as 2MB). If it exceeds, the original blocks are retained to avoid transmission or processing delays due to excessively large data blocks. For example, for the incremental data of the user table, the hash value of the user ID is segmented, and multiple small data blocks in the same hash segment are merged into one logical data block to ensure that the relevant data of the same user are processed as a whole during the migration process.
[0070] Conflict detection and processing is a key step in data synchronization. The specific steps are as follows: By traversing the synchronized migration data block set, check the metadata information of each data block one by one, including version number, dependency fields (such as foreign key constraints, business process identifiers), etc. For the version number, if the version number ranges of the two data blocks overlap (such as the version number of data block A is V3.1-V3.3, and the version number of data block B is V3.2-V3.4), it is determined to be a version conflict; for the dependency, if the primary key of the parent data block referenced by the child data block does not exist in the current migration data block set, or the version number of the parent data block is lower than that of the child data block, it is determined to be a dependency conflict. After the conflict detection is completed, a conflict data block list is generated, which includes the conflict data block ID, conflict type (version conflict / dependency conflict), version number, and dependency details.
[0071] Conflicting data blocks are prioritized based on the version span parameter, which is defined as the absolute value of the difference in data block version numbers. For data blocks with version conflicts, priority is given to retaining data blocks with smaller version spans, that is, data blocks with version numbers closer to the current migration target version. For example, when data block C (version number V3.2, span 1) conflicts with data block D (version number V3.5, span 4), data block C is retained and marked as valid, and data block D is marked as a conflict to be processed. For data blocks with dependency conflicts, they are sorted according to the dependency hierarchy, and the migration of the parent data block is processed first, and then the child data block. If the parent data block is not included in the current migration set, the exception handling process is triggered, the missing parent data block information is recorded, and the migration is suspended until manual confirmation is required before continuing.
[0072] Redundancy cleanup targets duplicate or invalid data blocks, identifying identical data blocks through hash value comparison. Only one of these blocks is retained, while the remaining blocks are marked as redundant and removed from the synchronized migration data block set. For example, if the same business operation generates multiple data blocks with identical content (such as repeatedly submitted order data), only the first block is retained after hash value verification confirms redundancy, reducing data transmission and storage overhead.
[0073] After data synchronization is complete, the migration marking phase begins. First, a two-dimensional matrix is generated based on the synchronized set of migration data blocks. The rows and columns of the matrix correspond to the logical groupings of data blocks (e.g., rows divided by business module, columns divided by data update time). Each matrix cell stores metadata such as the data block's unique identifier, version number, and dependencies. Migration marking begins with the first data block in the upper left corner of the matrix (coordinates 0,0). The specific process is as follows:
[0074] Initial Marking: Read the associated data blocks of the data block at coordinate (0,0). Associated data blocks are determined by foreign key fields in the data block metadata or business process definitions. For example, if the data block is an order header data block and its associated data block is the corresponding order detail data block, then mark the associated data block as the first node in the migration path and generate the initial migration sequence number "1."
[0075] In-row traversal: Starting from coordinate (0,0) and moving rightward, the subsequent data blocks in the same row are processed sequentially. For each data block, its association with the currently marked data block is checked. If a direct association exists (such as different details of the same order), the migration sequence number is updated to the current maximum sequence number plus 1, and the direction of the association is marked (such as rightward or downward). If no direct association exists, the default sequence number (initially 0) is retained for subsequent traversal processing. For example, order detail data blocks in the same row are arranged in chronological order. Each time you move one column to the right, the migration sequence number increases by 1, forming a continuous migration sequence.
[0076] Cross-row and column traversal: After completing the first row traversal, move down row by row to the next row (e.g., coordinate 1,0). The starting migration sequence number of this row is the last sequence number of the previous row plus 1. Repeat the same row traversal steps until all data blocks in the matrix have been processed. During the traversal process, if a marked associated data block is encountered, it is skipped to avoid duplicate marking. If a circular dependency is encountered (e.g., data block E depends on F, and F depends on E), the dependency is recorded but not marked for the time being, and will be processed uniformly after the migration dependency graph is generated.
[0077] During the migration marking process, migration numbers are assigned according to the principle of "association first, ascending order" to ensure that data blocks with strong business dependencies are migrated first, avoiding migration failures caused by dependency inversion. For example, in an e-commerce scenario, the order header data block (sequence number 1) is migrated first, followed by the corresponding order detail data blocks (sequence numbers 2 and 3), and finally the logistics information data block (sequence number 4), forming a migration sequence that conforms to the business process.
[0078] After migration marking is complete, a matrix of marked migration data blocks is generated. Each data block in the matrix contains a unique migration sequence number, an association identifier, and a status flag (such as "marked," "pending migration," or "conflict resolved"). This matrix serves as the basis for subsequent migration feature extraction and priority calculation, ensuring that the migration process is executed in an orderly manner according to the preset logic.
[0079] In the specific implementation of conflict detection and processing, the consistency of database transactions must also be considered. For example, for a transaction operation across multiple data blocks, if a conflict occurs in one of the data blocks, all data blocks related to the entire transaction must be rolled back to their pre-conflict state and re-blocked or re-versioned to ensure transactional integrity of the data. In addition, after conflict resolution, the migration data block matrix must record a conflict handling log, including information such as the time of conflict occurrence, data block ID, handling method (such as retention / deletion), and operator (if manual intervention was required) to facilitate subsequent audits and problem tracing.
[0080] During redundancy cleanup, different redundancy determination strategies can be applied to data blocks in different business modules. For example, a certain degree of redundancy is permitted for log data blocks (e.g., retaining the three most recent identical logs), while redundancy is strictly prohibited for transaction data blocks to ensure data uniqueness. Deletion of redundant data blocks must be performed within a transaction to avoid data inconsistencies caused by partial deletion.
[0081] Example 3: In the feature extraction and parameter optimization link of step S3, the specific implementation method is as follows: the principal component analysis algorithm (PCA) is used to reduce the dimension of the marked migration data block matrix to extract key migration feature parameters. The core of principal component analysis is to convert the original high-dimensional features into low-dimensional principal components through linear transformation, where the first k principal components can retain the main information of the original data. In this embodiment, the first three principal components are selected, and the cumulative variance contribution rate must reach a preset threshold (such as 95%) to ensure that the extracted features can fully reflect the essential attributes of the migration data. The original features include but are not limited to data block size, modification time, version number, dependency complexity, data block hash value, synchronization delay, etc. After principal component analysis, the three principal components are finally determined to be:
[0082] Principal component 1: data block size parameter (unit: KB), reflecting the physical storage scale of the data block;
[0083] Principal component 2: version span parameter (dimensionless), defined as the absolute value of the difference between the current version number of the data block and the target migration version number;
[0084] Principal component 3: dependency parameter (dimensionless), which is calculated by the in-degree and out-degree in the dependency graph and represents the complexity of the data block in the migration dependency network.
[0085] After feature extraction is complete, a genetic algorithm is used to optimize the synchronization parameters within the feature parameters. A genetic algorithm is a global optimization algorithm that simulates the biological evolution process, iteratively searching for the optimal solution through operations such as selection, crossover, and mutation. In this embodiment, the synchronization parameters include a synchronization delay threshold (x1), a conflict resolution priority coefficient (x2), and a data block merge threshold (x3). x1 represents the upper limit of the allowed data block synchronization delay (in milliseconds), which is used to control the time tolerance range during data synchronization. x2 is a coefficient between 0 and 1 that adjusts the priority weight when resolving conflicting data blocks. A larger value indicates a higher weight for the version span parameter. x3 represents the maximum capacity threshold for data block merges (in MB), which is used to control the size of the merged data block to prevent migration efficiency from being affected by overly large data blocks.
[0086] The specific implementation steps of the genetic algorithm are as follows: Population initialization: Generate an initial population containing N individuals, each individual is a three-dimensional vector (x1, x2, x3), where the value range of x1 is [x 1min ,x 1max ] (e.g., 50ms to 100ms), the value range of x2 is [0,1], and the value range of x3 is [1MB,3MB]. The initial values of the individuals are generated through uniform random distribution to ensure the coverage of the search space.
[0087] Fitness function definition: Set the fitness function as a linear combination of synchronization parameters, the formula is:
[0088] f(x)=ω1x1+ω2x2+ω3x3
[0089] Wherein, ω1, ω2, and ω3 are preset weight coefficients corresponding to the importance of synchronization delay, conflict resolution priority, and data block merging, respectively. In this embodiment, the weight coefficients are set to ω1 = 0.3, ω2 = 0.5, and ω3 = 0.2, indicating that conflict resolution priority is the core optimization goal, and synchronization delay and data block merging are secondary goals.
[0090] Selection: Roulette wheel selection is used to select individuals from the current population. Individuals with higher fitness values have a greater probability of being selected. The specific operation is to calculate the proportion of each individual's fitness to the total fitness, generate a cumulative probability distribution, and select the corresponding individual to enter the next generation of the population through random numbers.
[0091] Crossover operation: with crossover probability P c (e.g., 0.8) Crossover is performed on each of the selected individuals, exchanging some of their genes (parameter values) to generate new individuals. The crossover point is randomly chosen between the dimensions of the three-dimensional vector. For example, for individuals A = (a1, a2, a3) and B = (b1, b2, b3), if crossover is chosen in the second dimension, the new individuals after crossover are A′ = (a1, b2, a3) and B′ = (b1, a2, b3).
[0092] Mutation operation: Each gene of each individual has a mutation probability P m (such as 0.05) to mutate. During the mutation process, the individual moves in the parameter space. If the current parameter value reaches the value boundary (such as x1 = x 1min Or x1=x 1max ), then according to the random number r(r∈[0,1]) and the mutation probability P m The comparison result determines the direction of mutation: if r>P m , the mutation direction is positive (i.e., moving in the direction of parameter increase); otherwise, it is negative (moving in the direction of parameter decrease). For example, when x1 reaches the minimum value of 50ms and r=0.7>P m =0.05, x1 varies in the direction of increase, and the value increases by one step (such as 1ms); if r = 0.03 <P m , then keep the current value or process according to the preset rules.
[0093] Iterative optimization: After the individual position is updated, the fitness value is recalculated and the optimal individual of the current generation (i.e. the individual with the largest fitness value) is recorded. When the number of iterations reaches the preset maximum number of iterations (e.g. 200 generations), the optimization is stopped and the optimal synchronization parameter combination is output.
[0094] When processing mutation boundaries, the step size should be determined based on the parameter's range and precision requirements. For example, for x1 (unit: milliseconds), the step size is set to 1 millisecond to ensure precise parameter adjustments; for x3 (unit: megabytes), the step size is set to 0.1 megabytes to balance optimization accuracy and computational efficiency. Furthermore, to avoid falling into local optimal solutions, an elite retention strategy can be introduced during the iteration process. This strategy retains the best individuals from each generation and directly advances them to the next generation, ensuring that the optimal solution is not destroyed by crossover or mutation operations.
[0095] The specific implementation of the principal component analysis algorithm requires a data standardization preprocessing step, that is, the original features are zero-mean and unit-varianced to avoid principal component calculation deviations due to different feature dimensions. The standardization formula is:
[0096]
[0097] Where μ is the mean of the feature and σ is the standard deviation of the feature. The preprocessed data is used to calculate the covariance matrix, and then the principal component vectors are obtained through eigenvalue decomposition or singular value decomposition.
[0098] The parameter settings of genetic algorithm (such as population size N = 50, crossover probability P c =0.8, mutation probability P m =0.05) needs to be adjusted based on the complexity of the actual problem and computing resources. For example, if the migration data is large and the feature space is complex, the population size and number of iterations can be appropriately increased to increase the probability of finding the global optimal solution. If computing resources are limited, the population size and number of iterations can be reduced to achieve a balance between accuracy and efficiency.
[0099] The optimized synchronization parameters will be used for subsequent migration priority calculation and migration decision model training. For example, the optimal synchronization delay threshold It can be used as a basis for judging whether the delay exceeds the standard during data synchronization, and the conflict resolution priority coefficient Can be directly used for priority sorting of conflicting data blocks and data block merging thresholds It can guide the adjustment of data block merging strategies to ensure that the merged data blocks meet business relevance requirements while not exceeding the system processing capacity.
[0100] Example 4: In the migration decision model training and conflict detection process in step S3, the specific implementation method is as follows:
[0101] During the migration decision model training phase, we first acquire new incremental data from the client database. For example, taking user order data from an e-commerce platform, this new data might include incremental records generated by operations such as creating an order, modifying the delivery address, and canceling an order. For each piece of incremental data, we extract three key migration parameters:
[0102] Data block size parameter: Take the order header information and detail information as one data block. Assuming it contains 50 order details, the data block size is 800KB (calculated based on the number of bytes of serialized data);
[0103] Version span parameter: The current database version is V4.2, the version number of the data block to be migrated is V4.1, and the version span is 1 (i.e. |4.2-4.1|);
[0104] Dependency parameters: The order data block depends on the user basic information data block (foreign key associated with the user ID). If the user data block is included in the migration set, the dependency level is 1 (indicating a first-level dependency); otherwise, it is 0 (indicating no dependency or missing dependency).
[0105] Combine the extracted parameters into a new migration sample set, for example, containing 1000 sample data points. Divide the sample set into a migration test set (700 data points) and a migration validation set (300 data points) in a 7:3 ratio. The test set is used to train the model, and the validation set is used to evaluate model performance.
[0106] The migration decision model adopts a three-layer neural network structure:
[0107] Input layer: Set up 3 nodes, corresponding to data block size, version span, and dependency parameters respectively;
[0108] Hidden layer: 10 nodes are set and ReLU activation function is used to extract nonlinear features;
[0109] Output layer: Set 1 node, and the output value is 0 or 1, where 0 means not to migrate the data block for the time being, and 1 means to migrate it immediately.
[0110] The training process is as follows: Sample data from the transfer test set is fed into the model batch by batch, using the mean squared error (MSE) as the loss function and backpropagation to update the model weights and biases. The maximum number of iterations is set to 3000, and the learning rate is 0.001. In each iteration, the model predicts the input sample based on the current parameters, calculates the error between the predicted value and the actual label (such as the manually labeled transfer priority), and adjusts the network parameters to reduce the error.
[0111] After training is complete, the migration validation set is input into the trained model. For example, a validation sample with a data block size of 1.2MB, a version span of 2, a dependency level of 0 (independent data block), and a model output value of 0.8, close to the threshold of 1, is considered for immediate migration. If the overall error (e.g., root mean square error) of the model output is less than a preset threshold (e.g., 0.05), the model is considered a qualified migration decision classifier and can be used for actual migration decisions. If the error does not meet the standard, the model parameters are adjusted (e.g., increasing the number of hidden layer nodes to 15, or reducing the learning rate to 0.0005), and training is repeated until the error meets the requirements.
[0112] The specific implementation of conflict detection processing uses order data migration as an example: by traversing the synchronized set of migrated data blocks, the version number and dependency relationships of each data block are checked. For example, if data block A (order ID = 1001, version number V4.1) and data block B (order ID = 1001, version number V4.3) are found to have overlapping versions (both fall within the scope of the current migration version V4.2), a list of conflicting data blocks is generated, recording the conflict type as "version conflict", including the data block ID, version number, and conflict timestamp.
[0113] Prioritize conflicting data blocks based on the version span parameter: Data block A has a version span of 1 (|4.2-4.1|), and data block B has a version span of 1 (|4.2-4.3|). Given the same span, block A can be sorted by modification time, retaining block B with the more recent modification time. Block A is marked as "Conflict Resolved - Deleted" in the matrix, and block B is marked as "Valid." If the conflicting data blocks are dependencies (for example, an order details data block references a non-existent product data block), migration is suspended, the missing product data block ID is recorded, and a manual review process is triggered. Conflict detection is then repeated after the parent data block is added.
[0114] After a conflict is resolved, the migration block matrix must update the block status. For example, find the coordinates of block A in the matrix (row = order module, column = column 5), mark its status as "deleted," and adjust the migration sequence numbers of subsequent blocks to ensure sequence continuity. A conflict resolution log is also kept, including the resolution time, action type (automatic resolution / manual intervention), and resolution results, to facilitate subsequent audits.
[0115] When dividing the sample sets, ensure that the data distribution of the test and validation sets is consistent to avoid insufficient model generalization due to data bias. For example, if 80% of the incremental data consists of data blocks with a low version span (≤1) and 20% consists of data blocks with a high version span (>1), the ratio of high and low version span samples in the test and validation sets should also be maintained at 8:2 to ensure balanced model training.
[0116] The input parameters of the neural network model need to be normalized. For example, the data block size is converted from KB to MB and scaled to the [0, 1] interval. The version span and dependency parameters are directly mapped to the [0, 1] interval to avoid the influence of different feature dimensions on the model training effect. The normalization method uses the minimum-maximum scaling method, and the formula is:
[0117]
[0118] Among them, x min and x max are the minimum and maximum values of the feature, respectively.
[0119] During conflict detection, for complex dependencies across multiple data tables (such as a three-level dependency: order → user → address), the integrity of the dependency relationships must be checked layer by layer. If the user data block exists but the address data block is missing, the dependency level of the order data block is marked as 2 (secondary dependency), and the missing address data block is recorded in the conflict list, prompting operations personnel to supplement the relevant data before migration.
[0120] Example 5: In the migration marking step of step S2, the specific implementation method involves the generation of the migration dependency graph and the processing of circular dependencies, which is described in detail below in conjunction with specific business scenarios:
[0121] Taking the database migration of an enterprise resource planning (ERP) system as an example, assume there are three data blocks: a purchase order header block (D101), a purchase order detail block (D102), and a supplier information block (D103). D101 is linked to D103 via a supplier ID foreign key, and D102 is linked to D101 via an order ID foreign key, forming a linear dependency chain: D103 → D101 → D102. Normally, migration markers are numbered according to the dependency order: D103 (sequence number 1) → D101 (sequence number 2) → D102 (sequence number 3).
[0122] However, if there are historical data errors in the system, causing D101 to also depend on D102 (such as temporary data in the order header reference details), a circular dependency structure is formed. At this time, when traversing the data block association relationship during the migration marking process, it is detected that the dependency edge of D101 points to D102, and the dependency edge of D102 points in the opposite direction to D101, forming a ring dependency (ring length is 2).
[0123] The migration dependency graph is generated based on the association field in the data block metadata. For example, each data block stores a list of parent data block IDs on which it depends. D101's dependency list is [D103, D102], D102's dependency list is [D101], and D103 has no dependencies. A directed graph is constructed using a graph traversal algorithm (such as depth-first search (DFS)). Nodes are data block IDs, and directed edges are dependency relationships (e.g., D101→D103 indicates that D101 depends on D103). During the graph construction process, if duplicate nodes are detected in the path starting from the current node, a circular dependency is determined.
[0124] After detecting a circular dependency, the system performs a forced disconnect based on the version span parameter. The version span parameter is the absolute difference between the current version number of the data block and the target migration version number. Assuming the target migration version is V5.0, the version number of D101 is V4.8 (span 0.2), and the version number of D102 is V4.5 (span 0.5). Based on the principle of preferential disconnection of data blocks with larger version spans, D102 is selected as the disconnection point, and its dependency edge pointing to D101 is deleted, so that the dependency graph becomes D103 → D101 and D102 (independent nodes).
[0125] After the link is broken, D101's dependency list is updated to [D103], and D102's dependency list is cleared. Migration continues: D103 is marked as sequence 1, D101 as sequence 2, and D102 as sequence 3 (since there are no dependencies, they will be migrated as independent data blocks). At the same time, the system generates a migration log, recording the location of the link break: data block ID is D102, dependency relationship is D102 → D101, link break time is May 28, 2025, 14:30:05, and the reason for the link break is "circular dependency, large version gap."
[0126] Take another scenario as an example: In a customer relationship management (CRM) system, data blocks D201 (customer order), D202 (customer contact), and D203 (order product) form a triangular dependency loop: D201 depends on D202 (customer ID) and D203 (product ID), while D203 depends on D201 (order ID). A circular path D201 → D203 → D201 is detected, with a loop length of 2. The version spans of each data block are: D201 (V3.6, span 0.4) and D203 (V3.3, span 0.7). Based on the version spans, D203 is selected as the breakpoint, and the dependency edge D203 → D201 is deleted, making D203 an independent data block. The migration mark order is D202 (sequence number 1) → D201 (sequence number 2) → D203 (sequence number 3), and a break log is recorded for manual review.
[0127] Independent data blocks after a chain break are handled as follows: If the broken block is a leaf node (no downstream dependencies), it can be migrated independently. If it is an intermediate node (with downstream dependencies), its downstream blocks must have their dependencies adjusted or be marked as independent blocks. For example, if D203 is still dependent on D204 (order logistics) after a chain break, D204's dependencies must also be adjusted, or D204 and D203 must be migrated together as independent blocks to ensure that the migration order does not disrupt business logic.
[0128] The storage structure of the migration log includes: log ID, disconnection time, data block ID, dependency description, disconnection method (version span / manually specified), and processing status (pending review / processed). Operations and maintenance personnel can manually verify the rationality of the disconnection by querying the log. For example, in a CRM scenario, a disconnection from D203 to D201 may cause inconsistency issues between order product data and order header data. Manual confirmation is required to determine whether there are business logic errors or whether additional constraints (such as transaction locking mechanisms) are needed to avoid data inconsistencies.
[0129] During the migration marking process, circular dependencies are detected after generating the migration data block matrix and before extracting migration feature parameters. This marking-first, then-detection process ensures that the relationships between all data blocks are incorporated into matrix management, preventing the omission of implicit dependency paths. The detection algorithm uses a stack-based DFS, recording the access status of each node (not accessed / accessed / accessed). When a node in the "accessing" state is accessed, a circular dependency is determined to exist, and the path is backtracked to determine the specific node in the loop.
[0130] For multi-layered circular dependencies (such as dependency graphs containing multiple rings) that may arise in complex business scenarios, the system handles them according to the following strategy: Prioritize detection of the shortest ring (the one with the fewest nodes), and disconnect nodes within the shortest ring by sorting by version span. If multiple rings of equal length exist, the ring containing the data block with the largest version span is prioritized. For example, if there are both rings of length 2 and length 3, the ring with length 2 is prioritized. If both rings are of length 2, the sum of the version spans of the data blocks within the rings is compared, and the ring with the larger span is disconnected.
[0131] The essence of the chain breaking operation is to convert unstructured circular dependencies into a directed acyclic graph (DAG) to ensure the determinability of the migration order. Although chain breaking may cause a temporary break in data dependencies, through logging and manual review mechanisms, design flaws in the underlying data model can be repaired in subsequent version iterations to prevent the recurrence of similar problems. For example, in an ERP system, manual review revealed that the circular dependency between D101 and D102 stems from a business process design error (the order header should not rely on temporary data in the details). The data model can be modified during the system upgrade to delete D101's dependency on D102, eliminating the circular dependency at its root.
[0132] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.
[0133] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.
Claims
1. A client database incremental migration method, characterized in that: The steps include: S1. Acquire incremental data from a client database to generate a to-be-processed incremental data set. Use an incremental data capture method to extract data blocks to be migrated. Perform data segmentation to obtain a segmented incremental data set. Use a version control algorithm to determine the version identifier of the segmented incremental data set. Locate and extract the migration data blocks to obtain a migration data block set. S2. Perform data synchronization processing on the migration data block set to obtain a synchronized migration data block set, then generate a synchronized migration data block matrix, set data block associations, mark the migration data blocks, and obtain a marked migration data block matrix; S3. Extract the migration feature parameters of the marked migration data block matrix, calculate the migration priority after optimizing the synchronization parameters in the feature parameters, determine the main migration parameters according to the migration priority, use the main migration parameters to train the migration decision model to obtain a migration decision classifier, output the migration execution result, and realize the incremental migration of the client database.
2. The client database incremental migration method according to claim 1, characterized in that: The S1 comprises the following steps: S11. Acquire incremental data from the client database to generate a set of incremental data to be processed, and use an incremental data capture method to extract data blocks to be migrated to obtain a set of data blocks to be migrated; S12, dividing the set of data blocks to be migrated by data block processing to obtain a block-based incremental data set; S13. Determine the final version identifier of the incremental data set after segmentation using a version control algorithm; S14. Extract the migration data block using the final version identifier to obtain a migration data block set.
3. The client database incremental migration method according to claim 2, characterized in that: The S13 comprises the following steps: S131. Calculate the modification time and data block hash value of each data block in the block-based incremental data set to obtain a data block normalization weight; divide the version interval according to the data block normalization weight to generate a set of candidate version identifiers; S132: Calculate the time intervals between the candidate version identifiers in the candidate version identifier set, merge the candidate version identifiers with the same time interval, and determine the final version identifier of the incremental data set after segmentation.
4. The client database incremental migration method according to claim 3, characterized in that: The S2 comprises the following steps: S21, performing data block merging, conflict detection, and redundancy cleaning on the migration data block set to complete data synchronization and obtain a synchronized migration data block set; S22 . Generate a synchronized migration data block matrix based on the synchronized migration data block set, and perform migration marking on the synchronized migration data block matrix using the data block association relationship to obtain a marked migration data block matrix.
5. The client database incremental migration method according to claim 4, characterized in that: The step of using the data block association relationship to mark the synchronized migration data block matrix includes: Mark the associated data block of the first data block in the upper left corner of the synchronized migration data block matrix, generate a migration sequence number, traverse the data block association relationship to the right, and update the migration sequence number until reaching the upper right corner of the synchronized migration data block matrix. Then traverse the entire migration data block matrix in sequence to complete the migration marking.
6. The client database incremental migration method according to claim 5, characterized in that: The S3 comprises the following steps: S31, using a principal component analysis algorithm to extract migration characteristic parameters of the marked migration data block matrix, and using a genetic algorithm to optimize synchronization parameters in the characteristic parameters to obtain a migration characteristic parameter set; S32. Calculate the priority weight of the migration feature parameter set and determine the main migration parameters, which include a data block size parameter, a version span parameter, and a dependency parameter. S33. Train the migration decision model to obtain a migration decision classifier, input the data block size parameter, version span parameter, and dependency parameter into the migration decision classifier, and output the migration execution result.
7. The client database incremental migration method according to claim 6, characterized in that: The method of using a genetic algorithm to optimize synchronization parameters in characteristic parameters includes: The fitness function is set as a linear combination of synchronization parameters. In the genetic population, the kth individual moves along the parameter space with a mutation probability of p. When the individual encounters a parameter boundary during movement, if the random number r∈[0,1] is greater than p, the mutation direction is positive, otherwise it is negative. The fitness value is calculated after the individual position is updated. When the current number of iterations reaches the maximum number of iterations, the optimization is stopped and the optimal synchronization parameters are output.
8. The client database incremental migration method according to claim 6, characterized in that: The S33 includes the following steps: S331. Acquire new incremental data from the client database, extract new data block size parameters, version span parameters, and dependency parameters to form a new migration sample set. Divide the new migration sample set into a migration test set and a migration verification set. Set the migration decision model input node to 3 and the output node to 1. Input the migration test set into the migration decision model. Iterate until the maximum number of times has been reached, and then stop to obtain a trained migration decision model. S332. Input the migration verification set into the trained migration decision model. If the model error is less than the preset threshold, it is determined to be a migration decision classifier; otherwise, adjust the model parameters until the error meets the requirements.
9. The client database incremental migration method according to claim 4, characterized in that: The conflict detection process in S21 includes the following steps: S211, traverse the synchronized migration data block set, identify data blocks with version overlap or dependency conflicts, and generate a conflicting data block list; S212 , prioritizing the conflicting data blocks based on the version span parameter, preferentially retaining the data blocks with smaller version spans, and marking the migration data block matrix after the conflict is resolved.
10. The client database incremental migration method according to claim 5, characterized in that: The migration marking step further includes: A migration dependency graph is generated based on the data block association relationships. If a circular dependency structure is detected, the dependency link is forcibly disconnected based on the version span parameter, and the broken link location is recorded in the migration log for subsequent manual review.
Citation Information
Patent Citations
Data flash-back query method and device based on dot database
CN117033398A
Time sequence database multi-objective optimization method based on genetic algorithm parameter adjustment
CN117743294A
Database migration method and device, electronic equipment and storage medium
CN117971805A
Data storage method based on large model
CN119292524A
Cross-cloud resource migration method and system
CN119440855A
Cited By
Timing task-driven submitted data acquisition quality monitoring method and system
CN121051104A
Timed task driven data submission collection quality monitoring method and system
CN121051104B
Breakpoint continuous migration method and device for distributed migration of heterogeneous database and medium
CN121979866A