Data table processing method and apparatus, computer device, and readable storage medium

Abstract

A data table processing method, comprising: acquiring source table structure information of a source data table, changing the source table structure information, and, according to the changed table structure information, creating a target data table (S202); determining a plurality of source primary key values in the source data table, each source primary key value being used for identifying one piece of source data in the source data table (S204); sequentially migrating the source primary key values and the source data in the source data table to the target data table, each migration involving at least one source primary key value and the source data identified by the at least one source primary key value, and each migration being started after the previous migration succeeds (S206); for the source primary key values in each migration, generating a migration progress record (S208); when migration interruption occurs, determining a migration progress record at the time of the migration interruption, and determining, on the basis of the migration progress record at the time of the migration interruption, the source primary key value in the source data table where the migration has been interrupted (S210); and beginning from the source primary key value in the source data table where the migration has been interrupted, continuing to sequentially migrate to the target data table the source primary key values and the source data in the source data table which haven't been migrated yet (S212).

Description

Data table processing method, device, computer equipment and readable storage medium

[0001] Related applications

[0002] This application claims priority to Chinese patent application number 2023113816337, filed on October 24, 2023, entitled “Data table processing method, device, computer equipment and readable storage medium”, the entire text of which is hereby incorporated by reference. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a data table processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0004] Changing the table structure is an essential step in database operations. However, for tables with very large data volumes, performing a table structure change is a very time-consuming operation. A single table structure change typically takes hours or even days to complete. During the process, the change can fail due to software bugs, hardware failures, or human interference.

[0005] According to the current database processing logic, if the execution fails, all data will be rolled back and the database system will be restored to a state where the table structure has not been changed. If the user wants to continue to complete the table structure change, they need to re-execute, which means that the migrated data will be invalid, resulting in inefficient table structure changes.

[0006] Summary of the Invention

[0007] According to various embodiments provided in the present application, a data table processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product are provided.

[0008] In one aspect, the present application provides a data table processing method. The method comprises:

[0009] Acquiring source table structure information of a source data table, changing the source table structure information, and creating a target data table according to the changed table structure information;

[0010] Determine a plurality of source primary key values ​​in the source data table, each of the source primary key values ​​being used to identify a piece of source data in the source data table;

[0011] Migrating the source primary key values ​​and source data in the source data table to the target data table one by one, wherein each time at least one source primary key value and the source data identified by the at least one source primary key value are migrated, and each migration starts after the previous migration succeeds;

[0012] Generate migration progress records for each source primary key value of the migration;

[0013] When a migration interruption occurs, determining a migration progress record at the time of the migration interruption, and determining a source primary key value of the interrupted migration in the source data table based on the migration progress record at the time of the migration interruption; and

[0014] Starting from the source primary key value whose migration is interrupted in the source data table, the source primary key values ​​and source data that have not been migrated in the source data table continue to be migrated to the target data table one by one.

[0015] On the other hand, the present application also provides a data table processing device. The device includes:

[0016] The change module is used to obtain source table structure information of the source data table, change the source table structure information, and create a target data table of the target table structure according to the changed table structure information;

[0017] A first determining module is configured to determine a plurality of source primary key values ​​in the source data table, each of the source primary key values ​​being used to identify a piece of source data in the source data table;

[0018] a first migration module, configured to migrate source primary key values ​​and source data in the source data table to the target data table one by one, wherein each migration includes at least one source primary key value and the source data identified by the at least one source primary key value, and each migration starts after the previous migration succeeds;

[0019] A generation module, used to generate migration progress records based on the source primary key value of each migration;

[0020] A second determining module is configured to, when a migration interruption occurs, determine a migration progress record at the time of the migration interruption, and determine a source primary key value of the interrupted migration in the source data table based on the migration progress record at the time of the migration interruption; and

[0021] The second migration module is configured to continue migrating the unmigrated source primary key values ​​and source data in the source data table to the target data table one by one, starting from the source primary key value whose migration was interrupted in the source data table.

[0022] On the other hand, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the method embodiments of the present application when executing the computer program.

[0023] In another aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of each method embodiment of the present application.

[0024] On the other hand, the present application further provides a computer program product, which includes a computer program that, when executed by a processor, performs the steps of each method embodiment of the present application.

[0025] The details of one or more embodiments of the present application are set forth in the accompanying drawings and the description below. Other features and advantages of the present application will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the disclosed drawings without any creative work.

[0027] FIG1 is an application environment diagram of a data table processing method according to an embodiment;

[0028] FIG2 is a schematic flow chart of a data table processing method in one embodiment;

[0029] FIG3 is a schematic diagram of source primary key values ​​and source data in a source data table according to one embodiment;

[0030] FIG4 is a schematic diagram of a target data table in one embodiment;

[0031] FIG5 is a schematic diagram of updating a migration progress record in one embodiment;

[0032] FIG6 is a schematic diagram of a progress data table constructed in one embodiment;

[0033] FIG7 is a schematic diagram of adding a migration progress record to the location of the source primary key value in the source data table in one embodiment;

[0034] FIG8 is a schematic diagram showing an embodiment of using migration progress records as index value data corresponding to index key data;

[0035] FIG9 is a schematic diagram of adding a migration progress record to the index key data to which the source data belongs in one embodiment;

[0036] FIG10 is a schematic diagram of synchronizing the processing operation of a data processing request in a source data table to a target data table based on an association relationship in one embodiment;

[0037] FIG11 is a block diagram of a data table processing device according to an embodiment;

[0038] FIG12 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0040] The data table involved in the data table processing method provided in the embodiment of the present application may be a data table in a big data warehouse. Big data refers to a data set that cannot be captured, managed, and processed by conventional software tools within a certain time frame. It is a massive, high-growth, and diversified information asset that requires a new processing model to have stronger decision-making power, insight discovery, and process optimization capabilities. With the advent of the cloud era, big data has also attracted more and more attention. Big data requires special technologies to effectively process large amounts of data within a tolerable time. Technologies applicable to big data include large-scale parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems.

[0041] A database, in short, can be thought of as a digital filing cabinet—a place where electronic files are stored, allowing users to add, query, update, and delete data. A database is a collection of data stored in a specific way, shared by multiple users, with minimal redundancy, and independent of applications.

[0042] A database management system (DBMS) is a computer software system designed for managing databases, typically providing basic functions such as storage, retrieval, security, and backup. DBMSs can be categorized by the database model they support, such as relational or XML (Extensible Markup Language); by the type of computer they support, such as server clusters or mobile phones; by the query language they use, such as SQL (Structured Query Language) or XQuery; by performance priorities, such as maximum scale or maximum speed; or by other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, supporting multiple query languages ​​simultaneously.

[0043] The data table processing method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other servers. The terminal 102 and the server 104 can both execute the data table processing method provided in the embodiment of the present application independently. The terminal 102 and the server 104 can also be used in conjunction to execute the data table processing method provided in the embodiment of the present application. When the terminal 102 and the server 104 are used in conjunction to execute the data table processing method provided in the embodiment of the present application, the terminal 102 obtains the source data table and sends the source data table to the server 104. The server 104 obtains the source table structure information of the source data table, changes the source table structure information, and creates a target data table according to the changed table structure information. The server 104 determines multiple source primary key values ​​in the source data table, and each source primary key value is used to identify a source data in the source data table. The server 104 migrates the source primary key values ​​and source data in the source data table to the target data table one by one, wherein each time at least one source primary key value and at least one source data identified by the source primary key value are migrated, and each migration starts after the previous migration is successful. The server 104 generates a migration progress record for each source primary key value migrated. When a migration is interrupted, the server 104 determines the migration progress record at the time of the migration interruption, and determines the source primary key value in the source data table where the migration was interrupted based on the migration progress record at the time of the migration interruption. The server 104 continues to migrate the unmigrated source primary key values ​​and source data in the source data table to the target data table one by one, starting from the source primary key value in the source data table where the migration was interrupted.

[0044] Terminal 102 may be, but is not limited to, various desktop computers, laptops, smartphones, tablet computers, intelligent voice interaction devices, smart home appliances, in-vehicle terminals, aircraft, etc. Applications may run on terminal 102, such as communication applications, audio and video applications, and image processing applications. Server 104 may be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0045] In one embodiment, as shown in FIG2 , a data table processing method is provided. The method is described by taking a computer device (the computer device may be a terminal or a server in FIG1 ) as an example, and includes the following steps:

[0046] Step S202 : obtaining source table structure information of a source data table, changing the source table structure information, and creating a target data table according to the changed table structure information.

[0047] Among them, the source data table refers to the data table that needs to be migrated. The "columns" of the data table are called "fields", and a field can contain one type of information. The field name can be the column name of each column in the data table, which is used to describe the common attributes of the data in the column. The specific attributes can be at least one of the meaning or purpose of the data. For example, the "Address Book" data table includes field names such as "Name" and "Contact Number". Field content refers to the specific data in each column in the data table. For example, each data in a column of data with the field name "Name" is regarded as field content, for example, including "Zhang XX", "Li XX", etc.

[0048] Source table structure information describes the structure of the source data table. The source data table structure, or data structure, is the way a computer stores and organizes data. A data structure is a collection of data that has one or more specific relationships between them. Source table structure information includes at least one of the following: the source data table name, number of fields, field names, field types, data length, primary keys, foreign keys, or indexes.

[0049] The target data table is the data table that receives the migrated data from the source data table. The modified table structure information is referred to as the target table structure information. The target data table is the data table with the target table structure. The target table structure refers to the data structure of the target data table. The modified table structure information includes at least one of the following information: the table name, number of fields, field name, field type, data length, primary key, foreign key, or index of the target data table.

[0050] In this embodiment, the source table structure information is different from the changed table structure information. Specifically, the source table structure information is different from the changed table structure information, and specifically, at least one of the table name, number of fields, field name, field type, data length, primary key, foreign key, or index is different.

[0051] In this embodiment, the computer device may determine a source data table requiring data migration and obtain source table structure information of the source data table. The computer device may modify the source table structure information to obtain modified table structure information. The computer device may create a target data table based on the modified table structure information. The modified table structure information may be referred to as target table structure information. The created target data table then has the target table structure.

[0052] For example, the source table structure information defines that the source data table includes 3 fields, and the field names are: Field 1, Field 2, and Field 3. The computer device changes the source table structure information to add a new field. The obtained target table structure information includes 4 fields, and the field names are: Field 1, Field 2, Field 3, and Field 4.

[0053] In one embodiment, a computer device modifies source table structure information to obtain modified table structure information. The computer device may create an empty target data table according to the modified table structure information. The empty target data table has a target table structure.

[0054] In one embodiment, when there are multiple source data tables that require data migration, the computer device can determine the source table structure of each source data table; when the source table structures of multiple source data tables are the same, a target data table with a target table structure is generated based on the source table structures of the multiple source data tables.

[0055] In this embodiment, when the source table structures of the multiple source data tables are different, the source table structures of the multiple source data tables are adjusted to generate multiple data tables with the same structure, and then a target data table with the target table structure is generated based on the multiple data tables with the same structure.

[0056] Step S204: determining multiple source primary key values ​​in the source data table, where each source primary key value is used to identify a piece of source data in the source data table.

[0057] A data table has a primary key, which can have multiple primary key values. Each primary key value uniquely identifies a record in the data table. The primary key of the source data table is called the source primary key, which can have multiple source primary key values. These multiple source primary key values ​​are distinct, meaning they are not the same value. For example, the three source primary key values ​​could be 1, 2, and 3.

[0058] Multiple source primary key values ​​are used to identify different source data in the source data table. Each source primary key value uniquely identifies a piece of source data in the source data table. Source data is a record in the source data table. For example, source data refers to the data in a row of the source data table excluding the source primary key value for that row, as shown in Figure 3.

[0059] In this embodiment, the computer device may determine a source primary key in the source data table and determine multiple source primary key values ​​of the source primary key. A source primary key value is used to identify a piece of source data in the source data table.

[0060] For example, a source data table has five columns of data, forming four rows of data. Column 1 is the primary key column, so the elements in each row of column 1 are the source primary key values. The source primary key values ​​in column 1 are used to identify the source data in row 1. Row 1 of source data refers to the elements in row 1 of columns 2-5.

[0061] Step S206 , migrating the source primary key values ​​and source data in the source data table to the target data table one by one, wherein each time at least one source primary key value and at least one source data identified by the source primary key value are migrated, and each migration starts after the previous migration is successful.

[0062] In this embodiment, the computer device may migrate the source primary key values ​​and the source data identified by the source primary key values ​​in the source data table to the target data table one by one, using a migration method in which at least one source primary key value and the source data identified by the at least one source primary key value are migrated each time. A single migration may migrate at least one source primary key value and the source data identified by the at least one source primary key value. Furthermore, each migration begins after the previous migration succeeds.

[0063] For example, each time a source primary key value and the source data identified by the source primary key value are migrated, and after the current migration is successful, the next migration is performed. Alternatively, the computer device migrates a preset number of source primary key values ​​and the source data identified by the preset number of source primary key values ​​each time.

[0064] Step S208: Generate a migration progress record for each source primary key value of the migration.

[0065] Among them, the migration progress record is used to record the progress of data migration in the source data table. The migration progress record may include at least one of the following: the source primary key value of the migration, the location of the source primary key value of the migration in the source data table, the migration time, the migration status, etc. The migration time may specifically include at least one of the following: the initiation time of the migration, the end time of the migration. The migration status is used to indicate whether the migration is successful or failed. The migration status can be represented by a preset status bit, and different preset status bits can represent different migration states. For example, 1 represents a successful migration, and 0 represents a failed migration. The specific setting can be based on the needs.

[0066] The migration progress record generated for each migration can record the migration status of each source primary key value. For example, if 50 rows of data are migrated from the source data table each time, the migration progress record generated for migrating rows 1-50 can record the migration status of the source primary key value for each row. For example: Row 1 migrated successfully, Row 2 migrated successfully, ..., Row 50 migrated successfully. If, during the migration of rows 51-100, rows 51-60 migrated successfully but row 61 and above failed, the migration progress record would be: Row 51 migrated successfully, Row 52 migrated successfully, ..., Row 61 failed.

[0067] The migration progress record generated for each migration can record only one migration status. For example, if the migration progress record generated for migrating rows 1-50 is "Migration Successful," it indicates that all rows 1-50 were successfully migrated. If the migration progress record generated for migrating rows 51-100 is "Migration Failed," it indicates that the migration of rows 51-100 was interrupted. You can subsequently determine the specific row of data that was interrupted to determine the source primary key value for the interrupted migration. Alternatively, you can use the source primary key value of row 51 as the source primary key value for the interrupted migration.

[0068] In other embodiments, the migration progress record may specifically record the source primary key value and the migration progress of the source data for each migration.

[0069] In this embodiment, during each data migration, the computer device may generate a migration progress record for each data migration. The migration progress record may be generated after each data migration to the target data table, or before each data migration to the target data table.

[0070] A migration progress record is generated each time data is migrated to the target table. For example, if the source primary key value of the first row of data is "1," a migration progress record such as "1 Migration Successful" will be generated after the first row of data is migrated to the target table.

[0071] When a migration progress record is generated after each data migration to the target data table, each migration progress record may record the source primary key value of the successfully migrated source data, or the position of the source primary key value in the source data table.

[0072] Generated before each data migration to the target data table, this can be after the data to be migrated has been determined but before it is migrated to the target data table. The migration progress record can record the source primary key value of the source data to be migrated, or the location of the source primary key value in the source data table. For example, if the computer device determines that rows 1-3 of data need to be migrated, it will copy rows 1-3, generate a migration progress record for "Migrating rows 1-3," and then migrate them to the target data table.

[0073] Step S210 : When a migration interruption occurs, the migration progress record at the time of the migration interruption is determined, and the source primary key value of the interrupted migration in the source data table is determined based on the migration progress record at the time of the migration interruption.

[0074] A migration interruption refers to an interruption that occurs during the migration of source primary key values ​​and source data from the source table to the target table. Migration interruptions can be caused by hardware or software failures, human interference, and other factors.

[0075] In this embodiment, when a migration interruption occurs during the data migration process of the source data table, the computer device can determine the migration progress record at the time of the migration interruption from the migration progress record of each migration, and determine the source primary key value of the interrupted migration in the source data table based on the migration progress record at the time of the migration interruption.

[0076] In this embodiment, each migration progress record records the source primary key value of each migration. The computer device can determine the migration progress record at the time of the migration interruption and determine the source primary key value recorded in the migration progress record. The recorded source primary key value is used as the source primary key value in the source data table for the interrupted migration.

[0077] In this embodiment, when the migration progress record when the migration was interrupted records multiple source primary key values, the respective positions of the multiple source primary key values ​​in the source data table can be determined. The source primary key value of the interrupted migration in the source data table is determined based on the respective positions.

[0078] If the migration progress record at the time of the interruption contains multiple source primary key values ​​and multiple migration states, with each source primary key value corresponding to each migration state, the migration state indicating the failed migration can be determined from the migration progress record. The source primary key value corresponding to the migration state indicating the failed migration is used as the source primary key value for the interrupted migration.

[0079] In one of the embodiments, when the migration progress record at the time of migration interruption records the migration status of multiple source primary key values, and the migration status of the multiple source primary key values ​​all indicate successful migration, the next source primary key value of the source primary key value with the smallest position in the source data table can be determined based on the positions of the multiple source primary key values ​​in the source data table, and the next source primary key value can be used as the source primary key value for the interrupted migration.

[0080] For example, if the migration progress record at the time of an interruption contains three source primary key values, located in rows 51, 52, and 53 of the source data table, the source primary key value in row 51 can be used as the source primary key value for the interrupted migration. If the migration progress record also contains the migration status corresponding to the three source primary key values, such as row 51 successfully migrated, row 52 successfully migrated, and row 53 failed to migrate, the source primary key value in row 53 can be used as the source primary key value for the interrupted migration.

[0081] If the migration progress record when the migration was interrupted is: migration successful on row 51, migration successful on row 52, ​​and migration successful on row 53, the source primary key value on row 54 can be used as the source primary key value for the interrupted migration.

[0082] Step S212 , starting from the source primary key value whose migration was interrupted in the source data table, continue to migrate the unmigrated source primary key values ​​and source data in the source data table to the target data table one by one.

[0083] In this embodiment, after determining the source primary key value in the source data table where migration was interrupted, the computer device continues to migrate the unmigrated source primary key values ​​and the source data identified by the source primary key values ​​in the source data table to the target data table, starting from the source primary key value where migration was interrupted, in a migration manner that migrates at least one source primary key value and the source data identified by the at least one source primary key value at a time. Similarly, a single migration can migrate at least one source primary key value and the source data identified by the at least one source primary key value.

[0084] In this embodiment, after determining the source primary key value in the source data table where the migration is interrupted, the computer device continues to migrate the source primary key values ​​that have not been migrated and the source data identified by the source primary key values ​​in the source data table to the target data table one by one, starting from the source primary key value in the source data table where the migration is interrupted, in a manner that migrates a preset number of source primary key values ​​and the source data identified by a preset number of source primary key values ​​each time.

[0085] In this embodiment, in the process of continuing the migration from the source primary key value where the migration was interrupted, each migration is also started after the previous migration is successful.

[0086] In the above-described data table processing method, source table structure information of a source data table is obtained and modified to obtain new table structure information. A target data table is created based on the modified table structure information to obtain a new data table with a different table structure from the source data table, thereby determining the source of the data migration and the data recipient. Multiple source primary key values ​​are determined in the source data table, each of which identifies a piece of source data in the source data table, to achieve migration of different source data. The source primary key values ​​and source data in the source data table are sequentially migrated to the target data table in a migration manner, migrating at least one source primary key value and at least one piece of source data identified by the source primary key value each time, such that at least one source primary key value and at least one piece of source data are migrated each time. Furthermore, each migration begins after the previous migration is successful, ensuring that each migration is executed after the previous migration is successful, thereby ensuring the consistency and accuracy of the data migration. A migration progress record is generated for each source primary key value migrated. In the event of a migration interruption, the migration progress record at the time of the interruption can be retrieved to determine which data in the source data table has been successfully migrated and which data has not been successfully migrated. The source primary key value of the interrupted migration in the source data table is determined based on the migration progress record when the migration is interrupted. Therefore, the unmigrated source primary key values ​​and source data in the source data table can be migrated to the target data table one by one starting from the source primary key value of the interrupted migration in the source data table, without having to migrate the data from the beginning. This solves the problem of low data migration efficiency caused by the need to migrate data from the beginning when the table structure change fails in the traditional solution, and effectively improves the efficiency of data migration.

[0087] In one embodiment, the source primary key value in the source data table is used as the target primary key value in the target data table; when a migration interruption occurs, the migration progress record at the time of the migration interruption is determined, and the source primary key value of the interrupted migration in the source data table is determined based on the migration progress record at the time of the migration interruption, including:

[0088] When a migration interruption occurs, determine the migration progress record at the time of the interruption, and query the target primary key value in the target data table at the time of the interruption; update the migration progress record at the time of the interruption based on the target primary key value; and determine the source primary key value of the interrupted migration in the source data table based on the updated migration progress record.

[0089] In this embodiment, the computer device uses the source primary key in the source data table as the target primary key in the target data table, uses the source primary key value in the source data table as the target primary key value in the target data table, and uses the source data in the source data table as the target data in the target data table. The target primary key has multiple target primary key values, each of which is used to identify different target data in the target data table. One target primary key value is used to identify a piece of target data in the target data table.

[0090] In one embodiment, the target primary key in the target data table is an explicit primary key.

[0091] In one embodiment, a computer device migrates the source primary key values ​​and source data in a source data table to a target data table one by one in a migration manner of migrating at least one source primary key value and at least one source data identified by the source primary key value each time, and uses the source primary key value of each migration as the target primary key value in the target data table.

[0092] If a migration interruption occurs during the migration of data from a source data table, the computer device can determine the migration progress record at the time of the interruption from the migration progress record for each migration, and query the target primary key value in the target data table at the time of the interruption. Based on the target primary key value, the migration progress record at the time of the interruption is updated to obtain an updated migration progress record.

[0093] In this embodiment, the migration progress record is updated when the migration is interrupted. Specifically, the source primary key value in the migration progress record is updated, or the location of the record in the migration progress record is updated. The location of the record in the migration progress record refers to the location of the migrated source primary key value in the source data table.

[0094] In this embodiment, each generated migration progress record records the source primary key value for each migration. The computer device queries the target primary key value in the target data table at the time of the migration interruption. If the target primary key value in the target data table at the time of the migration interruption is inconsistent with the source primary key value in the migration progress record at the time of the migration interruption, the source primary key value in the migration progress record is replaced with the target primary key value in the target data table, thereby obtaining an updated migration progress record. The computer device searches the source data table for the updated source primary key value and uses the updated source primary key value as the source primary key value in the source data table for the interrupted migration.

[0095] In this embodiment, the computer device can determine the maximum target primary key value in the target data table at the time of the migration interruption, and replace the source primary key value in the migration progress record at the time of the migration interruption with the maximum target primary key value, thereby obtaining an updated migration progress record. As shown in Figure 4, the maximum target primary key value in the target data table at the time of the migration interruption was 150, indicating that the migration was to row 150. As shown in Figure 5, the source primary key value of the interrupted migration recorded in the migration progress record is 100. In this case, the source primary key value in the migration progress record can be updated from 100 to 150, allowing subsequent migration to resume from row 150.

[0096] In one embodiment, when a migration is interrupted, the migration progress record records the location of the source primary key value of the interrupted migration in the source data table. The computer device updates the location in the migration progress record based on the target primary key value. The computer device searches the source data table for the updated location in the migration progress record and uses the source primary key value at the updated location as the source primary key value of the interrupted migration.

[0097] In this embodiment, the source primary key value in the source data table is used as the target primary key value in the target data table. When a migration interruption occurs, the migration progress record at the time of the migration interruption is determined, and the target primary key value in the target data table at the time of the migration interruption is queried. The target primary key value is the dividing point between successful and failed data migration. The migration progress record at the time of the migration interruption is updated based on the target primary key value, so that the source primary key value of the interrupted migration recorded in the migration progress record is more accurate, so that data migration can be continued from the source primary key value of the interrupted migration in the source data table, and the migration initiation position can be accurately determined when migrating again, avoiding the need to migrate some successfully migrated data again, which is beneficial to improving the accuracy and speed of data migration.

[0098] In one embodiment, the migration progress record is stored in a progress data table; and determining the source primary key value of the interrupted migration in the source data table according to the updated migration progress record includes:

[0099] Determine the composite primary key in the progress data table. The composite primary key is a combination of the source data table name and the database name. The database is used to store the source data table. Based on the composite primary key and the updated migration progress record, determine the source primary key value in the source data table for the interrupted migration.

[0100] The progress data table is used to store migration progress records. A joint primary key is the primary key of the progress data table, which is composed of multiple fields within the table. For example, by configuring both the table field name and the library field name in the progress data table as the primary key, you can obtain the joint primary key for the progress data table. Configuring a joint primary key with a table field name and a library field name means that the table field content under the table field name and the library field content under the library field name serve as the joint primary key.

[0101] In this embodiment, the computer device writes the migration progress record generated for each migration into the progress data table, or the computer device generates the migration progress record in the progress data table based on the source primary key value of the source data for each migration.

[0102] When a migration interruption occurs, the computer device determines the joint primary key in the progress data table, and queries the source data table causing the migration interruption and the database in which the source data table is stored based on the joint primary key.

[0103] The computer device queries the migration progress record when the migration is interrupted from the progress data table to determine the target primary key value in the target data table when the migration is interrupted, updates the migration progress record when the migration is interrupted based on the target primary key value, and obtains an updated migration progress record.

[0104] The computer device queries the source data table in the database based on the joint primary key, and thereby determines the source primary key value of the interrupted migration in the source data table based on the updated migration progress record, so as to continue data migration from the source primary key value of the interrupted migration in the source data table of the database.

[0105] In this embodiment, a joint primary key in the progress data table is determined. The joint primary key is a combination of the table name of the source data table and the library name of the database. The database is used to store the source data table, so that it is possible to determine which data table caused the migration interruption phenomenon and in which database the data table causing the migration interruption phenomenon is stored, thereby quickly locating the data table with the interrupted migration in a specific database. After determining the source primary key value of the interrupted migration in the source data table based on the joint primary key and the updated migration progress record, the data migration of the data table of the database can be continued to ensure the accuracy of the data migration.

[0106] In one embodiment, the method further comprises:

[0107] Obtain the table name of the source data table and determine the library name of the database that stores the source data table; construct a progress data table, which includes the table field name, library field name, and record field name, and uses the migration progress record as the record field content of the record field name; use the table name of the source data table as the table field content under the table field name, and use the library name of the database as the library field content under the library field name; configure the table field content and the library field content as the joint primary key of the progress data table.

[0108] In this embodiment, the computer device obtains the table name of the source data table, determines the database storing the source data table, and obtains the library name of the database. The computer device constructs a progress data table, which includes table field names, library field names, and record field names. The table name of the source data table is used as the table field content under the table field name, and the library name of the database is used as the library field content under the library field name. The computer device configures the table field name and the library field name as the joint primary key of the progress data table, that is, configures the table field content and the library field content as the joint primary key of the progress data table.

[0109] Each time data is migrated from the source data table, the generated migration progress record is written to the record field name of the progress data table, that is, the migration progress record is used as the record field content of the record field name.

[0110] For example, the progress data table t_ddl_break_point is used to persistently record breakpoint K, which is the source primary key value where the migration is interrupted. The definition of t_ddl_break_point is as follows:

[0111] CREATE TABLE t_ddl_break_point(

[0112] db_name VARCHAR(256), / / Execute ddl library name

[0113] table_name VARCHAR(256), / / table name for executing DDL

[0114] break_point VARCHAR(256), / / Record breakpoints, which may be of various data types;

[0115] PRIMARY KEY (db_name, table_name) / / The database table name forms a joint primary key to quickly identify the table that needs to be resumed

[0116] );

[0117] The generated progress data table is shown in FIG6 , which includes table field names, library field names, and record field names.

[0118] In this embodiment, the table name of the source data table is obtained, and the library name of the database storing the source data table is determined to construct a progress data table. The progress data table includes table field names, library field names, and record field names. The migration progress record is used as the record field content of the record field name, so that the migration progress record can be queried from the progress data table. The table name of the source data table is used as the table field content under the table field name, and the library name of the database is used as the library field content under the library field name. The table field content and the library field content are configured as the joint primary key of the progress data table, so that the data table that needs to be resumed can be quickly identified.

[0119] In one embodiment, migrating source primary key values ​​and source data in a source data table to a target data table one by one includes:

[0120] Based on the positions of the multiple source primary key values ​​in the source data table, the migration order of the multiple source primary key values ​​is determined; according to the migration order of the multiple source primary key values, the source primary key values ​​and source data in the source data table are migrated to the target data table one by one.

[0121] In this embodiment, the computer device may determine the position of each source primary key value in the source data table, and determine a migration order for each of the multiple source primary key values ​​based on their positions in the source data table. The source primary key values ​​and source data in the source data table are sequentially migrated to the target data table according to the migration order of the multiple source primary key values.

[0122] In this embodiment, based on the positions of multiple source primary key values ​​in the source data table, the migration order of the multiple source primary key values ​​is determined, so that the source primary key values ​​and source data in the source data table are migrated to the target data table one by one according to the migration order of the multiple source primary key values. This can ensure that the order of the source data after migration will not be disrupted, thereby ensuring the accuracy of the data.

[0123] In one embodiment, the method further includes: determining the location of the source primary key value of each migration in the source data table, and adding a migration progress record of each migration at the location;

[0124] When a migration interruption occurs, the migration progress record at the time of the migration interruption is determined, and based on the migration progress record at the time of the migration interruption, the source primary key value of the interrupted migration in the source data table is determined, including: when a migration interruption occurs, determining the location of the migration progress record at the time of the migration interruption from the source data table; and using the source primary key value at the location of the migration progress record at the time of the migration interruption as the source primary key value of the interrupted migration in the source data table.

[0125] In this embodiment, the computer device can determine the location of the source primary key value of the source data of each migration in the source data table, and after generating the migration progress record, add the migration progress record at that location in the source data table to indicate the source primary key value of each data migration.

[0126] As shown in FIG7 , after the source data identified by the source primary key value 1 is successfully migrated, a migration progress record “Success” is added at the location of the source primary key value 1.

[0127] In this embodiment, the migration progress record can be represented by a preset status bit. The migration status of the source primary key value and source data can be represented by the preset status bit. For example, adding a preset status bit "1" at the position of the source primary key value 1 indicates that the migration of the source primary key value and source data is successful. Adding a preset status bit "0" at the position of the source primary key value 1 indicates that the migration of the source primary key value and source data has failed.

[0128] In this embodiment, before or after each migration, a migration progress record may be added to the location where the corresponding source primary key value is located.

[0129] In this embodiment, when a migration interruption occurs, the computer device queries the source primary key value in the source data table to determine the source primary key value where the migration progress record exists, determines the migration progress record at the time of the migration interruption from these migration progress records, and determines the location of the migration progress record at the time of the migration interruption.

[0130] In this embodiment, the source primary key value at the location of the migration progress record when the migration is interrupted is determined, and the source primary key value is used as the source primary key value of the interrupted migration in the source data table.

[0131] In this embodiment, the location of the source primary key value of the source data for each migration in the source data table is determined, and a migration progress record is added at that location, thereby directly marking which data in the source data table has been migrated and which has not. Furthermore, if a migration is interrupted, the location of the migration progress record at the time of the interruption can be quickly and accurately determined from the source data table. The source primary key value at the location of the migration progress record at the time of the interruption is used as the source primary key value for the interrupted migration, allowing data migration to continue from that location, avoiding the situation where the entire data needs to be re-migrated due to the migration interruption.

[0132] In one embodiment, the source primary key value and the source data in the source data table are stored in the form of a key-value pair, with the source primary key value serving as the key and the source data identified by the source primary key value serving as the value corresponding to the key; the method further comprises: determining index data for each key-value pair in the source data table, the index data comprising the source primary key value and the source data in the key-value pair; and adding a migration progress record generated for each migrated source primary key value to the index data of the key-value pair to which the source data for each migration belongs;

[0133] When a migration interruption occurs, the migration progress record at the time of the migration interruption is determined, and based on the migration progress record at the time of the migration interruption, the source primary key value of the interrupted migration in the source data table is determined, including: when a migration interruption occurs, querying the index data corresponding to each key-value pair, and obtaining the index data to which the migration progress record at the time of the migration interruption belongs; and using the source primary key value included in the obtained index data as the source primary key value of the interrupted migration in the source data table.

[0134] The source primary key value and source data in the source data table are stored as key-value pairs. The source primary key value serves as the key, and the source data identified by the source primary key value serves as the value corresponding to the key. Index data is used to query the source primary key value and source data in the source data table. An index data item includes the source primary key value and source data in a key-value pair. Each key-value pair corresponds to different index data.

[0135] In this embodiment, the index data is also stored in the form of key-value pairs. The index data includes index key data and index value data. The key is the index key data, and the value is the index value data.

[0136] In this embodiment, the computer device determines each key-value pair in the source data table. For each key-value pair, the computer device determines the index data corresponding to the key-value pair. The index data corresponding to the key-value pair includes the key and value in the key-value pair, that is, the source primary key value as the key and the source data as the value in the key-value pair.

[0137] In this embodiment, for each migration, the index data of the key-value pair to which the source data for the migration belongs is determined. After a migration progress record is generated based on the source primary key value for the migration, the migration progress record is added to the index data of the key-value pair to which the source data for the migration belongs.

[0138] In this embodiment, the index data includes index key data and index value data, and the computer device adds the migration progress record generated for the migration to the corresponding index data including the index key data or the index value data.

[0139] When a migration interruption occurs, the computer device queries the index data of each key-value pair to obtain the index data to which the migration progress record at the time of the migration interruption belongs.

[0140] Furthermore, when a migration interruption occurs, the computer device traverses the index data of each key-value pair in sequence. When there is a migration progress record in the currently traversed index data and there is no migration progress record in the index data after the currently traversed index data, the currently traversed index data is used as the index data to which the migration progress record when the migration was interrupted belongs.

[0141] In this embodiment, the computer device determines the index data to which the migration progress record belongs when the migration is interrupted, determines the source primary key value included in the index data, and uses the source primary key value as the source primary key value for interrupting the migration.

[0142] In this embodiment, the source primary key value and source data in the source data table are stored in the form of a key-value pair, with the source primary key value serving as the key and the source data identified by the source primary key value serving as the value corresponding to the key, so that the corresponding value can be obtained through each key. The index data of each key-value pair in the source data table is determined, and the index data includes the source primary key value and the source data in the key-value pair. The migration progress record generated for the source primary key value of each migration is added to the index data of the key-value pair to which the source data of each migration belongs, so that the migration progress record can be added to the index data to directly mark which data has been migrated. In the event of a migration interruption, the index data of each key-value pair can be queried to quickly and accurately determine the index data to which the migration progress record at the time of the migration interruption belongs, thereby quickly and accurately locating the source primary key value in the source data table where the migration was interrupted.

[0143] In one embodiment, the index data includes index key data, and the index key data includes a source primary key value and source data in a key-value pair; adding a migration progress record generated for the source primary key value of each migration to the index data of the key-value pair to which the source data of each migration belongs, includes: determining the index key data to which the source data of each migration belongs; using the migration progress record generated for the source primary key value of each migration as index value data of the index key data to which the source data of each migration belongs;

[0144] When a migration interruption occurs, query the index data corresponding to each key-value pair to obtain the index data to which the migration progress record belongs when the migration is interrupted, including: when a migration interruption occurs, query each index value data based on each index key data to obtain the index value data when the migration is interrupted;

[0145] Using the source primary key value included in the obtained index data as the source primary key value for interrupting migration in the source data table includes: using the source primary key value included in the index key data of the obtained index value data as the source primary key value for interrupting migration in the source data table.

[0146] The index data includes index key data, and the index key data includes the source primary key value and the source data in the key-value pair.

[0147] In this embodiment, in each migration, the computer device determines the index key data to which the source data of each migration belongs, generates a migration progress record based on the source primary key value of the source data of each migration, and obtains the migration progress record of each migration.

[0148] The computer device uses the migration progress record of each migration as the index value data corresponding to the index key data of the source data of each migration, so that each index data includes index key data and index value data, and the index value data is the migration progress record.

[0149] As shown in Figure 8, if source primary key value 1 identifies source data 1, then index key data K1 is: source primary key value 1 + source data 1. After migrating this data, migration progress record 1 is used as index value data V1 corresponding to index data K1. If source primary key value 2 identifies source data 2, then index key data K2 is: source primary key value 2 + source data 2. After migrating this data, migration progress record 2 is used as index value data V2 corresponding to index data K2.

[0150] The source primary key value 3 identifies the source data 3, and the index key data K3 is: source primary key value 3 + source data 3. After migrating the data, the migration progress record 3 is used as the index value data V3 corresponding to the index data K3.

[0151] The source primary key value 4 identifies the source data 4, and the index key data K4 is: source primary key value 4+source data 4. After migrating the data, the migration progress record 4 is used as the index value data V4 corresponding to the index data K4.

[0152] In this embodiment, when a migration interruption occurs, the computer device queries each index value data based on each index key data to obtain the index value data at the time of the migration interruption. Furthermore, when a migration interruption occurs, the computer device sequentially traverses each index value data in the order of each index key data. If index value data exists for the currently traversed index key data and no corresponding index value data exists for the index key data subsequent to the currently traversed index key data, the index value data corresponding to the currently traversed index key data is used as the index value data at the time of the migration interruption. The index value data at the time of the migration interruption is thus a record of the migration progress at the time of the migration interruption.

[0153] The computer device determines the index key data corresponding to the index value data when the migration is interrupted, and determines the source primary key value included in the index key data as the source primary key value of the interrupted migration.

[0154] In this embodiment, the index key data to which the source data of each migration belongs is determined, and the migration progress record generated based on the source primary key value of the source data of each migration is used as the index value data corresponding to the index key data to which the source data of each migration belongs, so that it is possible to directly query which data in the source data table has been migrated through the index information, so that when a migration interruption occurs, the index value data can be queried based on the index key data to quickly obtain the index value data at the time of the migration interruption, thereby quickly determining the source primary key value of the interrupted migration.

[0155] In one embodiment, the method further comprises:

[0156] The migration progress record generated for the source primary key value of each migration is added to the index key data belonging to the source data of each migration; when a migration interruption occurs, the index key data is queried to obtain the index key data belonging to the migration progress record at the time of the migration interruption; the source primary key value included in the obtained index key data is determined as the source primary key value of the interrupted migration in the source data table.

[0157] In this embodiment, the computer device generates a migration progress record for each migration based on the source primary key value of the source data of each migration, and may also add the migration progress record for each migration to the index key data belonging to the source data of each migration.

[0158] As shown in Figure 9, source primary key value 1 identifies source data 1. Before data migration, index key data K1 is: source primary key value 1 + source data 1. After the data is migrated, migration progress record 1 is added to index key data K1, forming index key data K1: source primary key value 1 + source data 1 + migration progress record 1. Similarly, index key data K2 is: source primary key value 2 + source data 2 + migration progress record 2, index key data K3 is: source primary key value 3 + source data 3 + migration progress record 3, and index key data K4 is: source primary key value 4 + source data 4 + migration progress record 4.

[0159] When a migration interruption occurs, the computer device queries each index key data to determine which data has been migrated and which has not been migrated, thereby obtaining the index key data to which the migration progress record at the time of the migration interruption belongs.

[0160] Furthermore, the computer device queries each index key data. When the index key data includes a migration progress record, it indicates that the migration has been executed to the location or the source data indicated by the index key data has been migrated. When the index key data does not include a migration progress record, it indicates that the migration has not been executed to the location or has been interrupted when it was executed at the location. When the index key data currently being queried includes a migration progress record, and the next index key data of the currently queried index key data does not include a migration progress record, the source primary key value included in the currently queried index key data is used as the source primary key value for the interrupted migration.

[0161] In this embodiment, the migration progress record generated based on the source primary key value of the source data of each migration is added to the index key data belonging to the source data of each migration, so that the migration progress record can be added to the index key data of the index data to directly mark which data has been migrated. In the event of a migration interruption, the index key data at the time of the migration interruption can be quickly determined by querying the index key data of the index data, thereby accurately determining the source primary key value of the interrupted migration.

[0162] In one embodiment, the method further comprises:

[0163] Establish an association relationship between the source data table and the target data table; record the migration initiation time of the source data table; when a data processing request for the source data table is received after the migration initiation time, perform the operation indicated by the data processing request on the source data table, and synchronize the operation indicated by the data processing request to the target data table based on the association relationship.

[0164] Among them, the association relationship includes at least one of the following: a one-to-one mapping relationship between multiple source primary key values ​​in the source data table and multiple target primary key values ​​in the target data table, a one-to-one mapping relationship between multiple source data and multiple target data, a one-to-one mapping relationship between the position of the source primary key value in the source data table and the position of the target primary key value in the target data table, and a one-to-one mapping relationship between the position of the source data in the source data table and the position of the target data in the target data table.

[0165] In this embodiment, the computer device may establish an association relationship between the source data table and the target data table. When initiating data migration of the source data table, the migration initiation time is recorded. The migration initiation time may be the time when the data migration of the source data table is first initiated.

[0166] When a data processing request is received for a source data table after the migration is initiated, the computer device performs the operation indicated by the data processing request on the source data table and synchronizes the operation indicated by the data processing request to the target data table based on the association relationship. In other words, the processing operation is performed on both the source data table and the target data table.

[0167] As shown in Figure 10, before data is copied, an association is established between source table t1 and target table t1_new. Data from source table t1 is migrated to target table t1_new using Thomas Write. During this migration, access requests to the source table may occur, such as inserting, updating, or deleting data. Because the association has been established between source table t1 and target table t1_new, operations on source table t1 can be directly synchronized to t1_new.

[0168] For example, if the access request occurs at time T_ddl, the data already in the source data table t1 before T_ddl is migrated to the target data table t1_new through data copy backfill, and the new data generated in the source data table after T_ddl is synchronized to the target data table t1_new through write forwarding. This ensures that the data generated during the access period will not affect data migration, and the new data generated will be synchronized to the target data table, without any data loss.

[0169] As shown in Figure 10, during a data copy operation using DDL (data definition language), for example, the source data has been copied to the location where the source primary key value is 4. If a DML (data manipulation language) command modifies b at primary key value 2 to 3, according to traditional copy operations, the data modification performed by DML at this point is no longer detectable by DDL. That is, the value of b at primary key value a = 2 in the target data table t1_new remains 2 and is not changed, resulting in data loss. However, in this embodiment, an association is established between the source data table t1_old and the target data table t1_new. Therefore, data modification operations on the source data table t1_old can be forwarded to the target data table t1_new. Based on the Thomas write rule, the data modification operations will also take effect on t1_new, thus ensuring that DML data is not lost during the DDL operation.

[0170] In this embodiment, an association is established between the source data table and the target data table, and the migration initiation time of the source data table is recorded. Data in the source data table before the migration initiation time is treated as existing data, and data generated after the migration initiation time is treated as new data. When a data processing request is received for the source data table after the migration initiation time, the operation indicated by the data processing request is executed on the source data table and synchronized to the target data table based on the association, allowing normal access to the source data table during the data migration. Furthermore, this ensures that data generated during access will not affect data migration, and that new data generated will be synchronized to the target data table, preventing data loss.

[0171] In one embodiment, when a data processing request for a source data table is received after the migration is initiated, the operation indicated by the data processing request is performed on the source data table, and the operation indicated by the data processing request is synchronized to the target data table based on the association relationship, including:

[0172] When a data processing request is received for the source data table after the migration is initiated, the type of the data processing request is determined; based on the association relationship, the trigger that matches the type is determined; the operation indicated by the data processing request is performed on the source data table, and the operation indicated by the data processing request is synchronized to the target data table through the trigger that matches the type.

[0173] In this embodiment, upon receiving a data processing request for a source data table, the computer device may determine the time the data processing request was received. If the time is after the migration initiation time, the computer device determines the type of data processing request and, based on the association relationship, determines a trigger that matches the type of data processing request. Using the trigger that matches the type of data processing request, the processing operation indicated by the data processing request is executed in the source data table and then executed in the target data table.

[0174] In this embodiment, at least one of an insert trigger, an update trigger, or a delete trigger is pre-set for the source data table; the insert trigger is used to synchronously insert the data inserted in the source data table into the target data table; the update trigger is used to synchronously update the updated data in the source data table to the target data table; the delete trigger is used to synchronously delete the data deleted in the source data table from the target data table.

[0175] In this embodiment, when a data processing request is received for a source data table after the migration is initiated, the type of the data processing request is determined. Based on the association relationship, a trigger that matches the type of data processing request is identified. Using this trigger, the processing operations of the data processing request in the source data table are synchronized to the target data table, allowing normal access to the source data table during the data migration. Furthermore, data generated during access does not affect data migration, and new data generated is synchronized to the target data table, preventing data loss.

[0176] In one embodiment, a data table processing method is provided, which is applied to a computer device and includes:

[0177] Obtain source table structure information of a source data table, change the source table structure information, and create a target data table with a target table structure according to the changed table structure information.

[0178] Establish an association relationship between a source data table and a target data table, the association relationship including at least one of the following: a one-to-one mapping relationship between multiple source primary key values ​​in the source data table and multiple target primary key values ​​in the target data table, a one-to-one mapping relationship between multiple source data and multiple target data, a one-to-one mapping relationship between the positions of source primary key values ​​in the source data table and the positions of target primary key values ​​in the target data table, and a one-to-one mapping relationship between the positions of source data in the source data table and the positions of target data in the target data table.

[0179] Obtain the source data table's name and determine the database name where the source data table is stored. Build a progress data table, which includes table field names, library field names, and record field names. Use the source data table's name as the table field content under the table field name, and the database library name as the library field content under the library field name. Configure the table field content and the library field content as the joint primary key of the progress data table.

[0180] Determine multiple source primary key values ​​in the source data table, where each source primary key value is used to identify a piece of source data in the source data table.

[0181] Based on the positions of multiple source primary key values ​​in the source data table, determine the migration order of each of the multiple source primary key values. Then, migrate the source primary key values ​​and source data in the source data table to the target data table one by one according to the migration order. Each migration includes at least one source primary key value and the source data identified by it, and each migration begins after the previous migration succeeds. Record the migration initiation time of the first migration.

[0182] A migration progress record is generated based on the source primary key value of the source data of each migration, and the migration progress record of each migration is written under the record field name in the progress data table as the record field content of the record field name.

[0183] When a migration interruption occurs, the migration progress record at the time of the interruption is determined from the progress data table, and the source primary key value of the interrupted migration is determined based on the migration progress record at the time of the interruption.

[0184] Based on the joint primary key in the progress data table, the source data table stored in the database is determined, and starting from the source primary key value where migration was interrupted, the unmigrated source primary key values, source primary key values, and source data in the source data table are successively migrated to the target data table. Each migration involves migrating at least one source primary key value and at least one source data identified by the source primary key value, and each migration begins after the previous migration succeeds.

[0185] When a data processing request for a source data table is received after the migration is initiated, the type of the data processing request is determined; and based on the association relationship, a trigger matching the type of the data processing request is determined.

[0186] The operation indicated by the data processing request is executed on the source data table, and the operation indicated by the data processing request is synchronized to the target data table through the trigger.

[0187] In this embodiment, a target data table is obtained by structurally changing the source data table to determine the source of the data migration and the data recipient. The table name of the source data table is obtained, and the library name of the database storing the source data table is determined to construct a progress data table, which includes a table field name, a library field name, and a record field name. The migration progress record is used as the record field content of the record field name, so that the migration progress record can be queried from the progress data table. The table name of the source data table is used as the table field content under the table field name, and the library name of the database is used as the library field content under the library field name. The table field content and the library field content are configured as the joint primary key of the progress data table, so that the data table that needs to be resumed can be quickly identified.

[0188] Identify multiple source primary key values ​​in the source data table. These multiple source primary key values ​​are used to identify different source data in the source data table, enabling migration of different source data. Based on their positions in the source data table, determine the migration order corresponding to the multiple source primary key values. Then, in accordance with the migration order corresponding to the multiple source primary key values, migrate the source primary key values ​​and source data in the source data table to the target data table one by one. This ensures that the order of the migrated source data is not disrupted, thereby ensuring data accuracy.

[0189] Furthermore, each migration starts after the previous migration is successful, ensuring that each migration is executed after the previous migration is successful, thereby ensuring the consistency and accuracy of data migration.

[0190] A migration progress record is generated based on the source primary key value of the source data of each migration, so that in the event of a migration interruption, the migration progress record at the time of the interruption can be obtained to determine which data in the source data table has been successfully migrated and which data has not been successfully migrated.

[0191] Determine the joint primary key in the progress data table. The joint primary key is a combination of the table name of the source data table and the database name. The database is used to store the source data table, so that it is possible to determine which data table caused the migration interruption phenomenon and in which database the data table causing the migration interruption phenomenon is stored. Based on the joint primary key and the updated migration progress record, after determining the source primary key value of the interrupted migration in the source data table, the data migration of the data table of the database can be continued without starting the migration from the beginning. This effectively solves the negative impact of the failure of the table structure change and improves the efficiency of data migration.

[0192] In one embodiment, an application scenario of a data table processing method is provided, including:

[0193] In a database, each table has an explicit or implicit primary key (PRIMARY KEY), which can uniquely identify the fields or union of fields in a data row in a data table. The general DDL execution method is to create a new data table (t_new) based on the new table definition, and then write the data in the source data table (t_old) into the new data table in sequence. In this embodiment, the source data table is stored in a preset storage engine based on a key-value data structure, and the multiple source primary key values ​​of the source primary key are stored in order and arranged in the lexicographical order of the data, and the scanning phase of data copying is also performed in the order of the source primary key values. Data copying is data migration, which migrates the data in the source data table to the target data table.

[0194] For example, if the source data table t1_old has multiple source primary key values ​​ranging from 1 to 10, then during the data migration to the target data table t1_new, the data is also migrated in the order of 1 to 10. If the migration fails at a certain stage, a source primary key value K will exist, which separates the data that has been migrated from the data that has not yet been migrated. This source primary key value K can be called a breakpoint.

[0195] During the process of migrating data in the source data table t1_old to the target data table t1_new, the source primary key value currently scanned is recorded periodically and recorded in the progress record table t_ddl_break_point.

[0196] The breakpoint storage period is set to update the progress record t_ddl_break_point at regular intervals. The update method is as follows:

[0197] Each time a row of data is scanned from the source data table, a migration progress record is generated and t_ddl_break_point is updated, so that breakpoint accuracy can be controlled at the row level.

[0198] Alternatively, set a time period to generate a migration progress record each time a time period passes, and update t_ddl_break_point.

[0199] Alternatively, set a data row range and generate a migration progress record for each data row range, and update t_ddl_break_point once; for example, generate a migration progress record every time 50 rows of data are migrated.

[0200] Alternatively, set an explicit primary key, that is, a composite primary key, in the target data table t1_new. You can then determine the row currently being migrated by querying the maximum value of the target primary key in the target data table t1_new. Then, you can update the latest breakpoint value in t_ddl_break_point based on the maximum value of the target primary key.

[0201] When migration is interrupted, the client receives a failure notification. The distributed database system then queries t_ddl_break_point based on the library table name to determine whether a corresponding breakpoint exists. If so, all data rows before the breakpoint are skipped and data migration continues from the breakpoint.

[0202] This distributed database system consists of two main functional modules: MetaCluster (MC), responsible for global information management, and HyperNode (HN), responsible for computing and data storage. HN uses the self-developed, efficient storage engine TDStore (TS) for data storage, and data is stored in a distributed manner. Breakpoint information provided by HN (HyperNode) can be directly located from the data layer, allowing data breakpoints to be found in a very short time.

[0203] Furthermore, in this embodiment, DML is allowed during data migration:

[0204] Before copying data, establish an association between source table t1 and target table t1_new. Data from source table t1 is migrated to target table t1_new using Thomas Write. During this migration, access requests to the source table may occur, such as inserting, updating, or deleting data. Because the association between source table t1 and target table t1_new has been established, operations on source table t1 can be directly synchronized to t1_new.

[0205] For example, if the access request occurs at time T_ddl, the data already in the source data table t1 before T_ddl is migrated to the target data table t1_new through data copy backfill, and the new data generated in the source data table after T_ddl is synchronized to the target data table t1_new through write forwarding. This ensures that the data generated during the access period will not affect data migration, and the new data generated will be synchronized to the target data table, without any data loss.

[0206] Traditional table structure changes can affect online services and potentially cause data anomalies. Failure to change the structure of large tables can also lead to significant time loss. The data table processing method in this embodiment, however, employs a breakpoint-resume mechanism that allows for failures during the table structure change process, allowing the database to remember the progress of the change. If an error occurs during the table structure change process and the process exits, the same table structure change command can be executed again. The database can then find the location where the previous execution was interrupted and continue the table structure change operation based on that location. This eliminates the need for users to worry about the negative impact of failed table structure changes or the significant amount of waiting time wasted, thereby improving the efficiency of table structure change processing.

[0207] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0208] Based on the same inventive concept, embodiments of the present application further provide a data table processing device for implementing the aforementioned data table processing method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations in one or more data table processing device embodiments provided below can be found in the above-described limitations on the data table processing method and will not be further elaborated here.

[0209] In one embodiment, as shown in FIG11 , a data table processing apparatus 1100 is provided, comprising a change module 1102 , a first determination module 1104 , a first migration module 1106 , a generation module 1108 , a second determination module 1110 , and a second migration module 1112 , wherein:

[0210] The changing module 1102 is used to obtain source table structure information of a source data table, change the source table structure information, and create a target data table according to the changed table structure information.

[0211] The first determining module 1104 is configured to determine a plurality of source primary key values ​​in the source data table, where each source primary key value is used to identify a piece of source data in the source data table.

[0212] The first migration module 1106 is used to migrate the source primary key values ​​and source data in the source data table to the target data table one by one, wherein at least one source primary key value and at least one source data identified by the source primary key value are migrated each time, and each migration starts after the previous migration is successful.

[0213] The generating module 1108 is configured to generate a migration progress record for each source primary key value of the migration.

[0214] The second determining module 1110 is configured to determine the migration progress record when migration is interrupted, and determine the source primary key value of the interrupted migration in the source data table according to the migration progress record when migration is interrupted.

[0215] The second migration module 1112 is configured to continue migrating the unmigrated source primary key values ​​and source data in the source data table to the target data table one by one, starting from the source primary key value whose migration was interrupted in the source data table.

[0216] In this embodiment, the source table structure information of the source data table is obtained, and a target data table with a target table structure is created based on the source table structure information, so as to create a new data table based on the structure information of the source data table, and determine the source of the data migration and the data recipient. Multiple source primary key values ​​in the source data table are determined, and the multiple source primary key values ​​are used to respectively identify different source data in the source data table to achieve the migration of different source data. Based on the source primary key values ​​in the source data table, the source data in the source data table are migrated to the target data table one by one, and each migration starts after the previous migration is successful, so as to ensure that each migration is executed after the previous migration is successful, thereby ensuring the consistency and accuracy of the data migration. A migration progress record is generated based on the source primary key value of the source data of each migration, so that in the event of a migration interruption, the migration progress record at the time of the migration interruption can be obtained to determine which data in the source data table has been successfully migrated and which data has not been successfully migrated. The source primary key value of the interrupted migration is determined based on the migration progress record when the migration was interrupted. This allows the unmigrated data in the source data table to be migrated to the target data table starting from the source primary key value of the interrupted migration without having to start the migration from scratch. This effectively solves the negative impact of failed table structure changes and improves the efficiency of data migration.

[0217] In one embodiment, the source primary key value in the source data table is used as the target primary key value in the target data table; the second determination module 1110 is also used to determine the migration progress record when the migration is interrupted, and query the target primary key value in the target data table when the migration is interrupted; update the migration progress record when the migration is interrupted based on the target primary key value; and determine the source primary key value of the interrupted migration in the source data table according to the updated migration progress record.

[0218] In this embodiment, the source primary key value in the source data table is used as the target primary key value in the target data table. When a migration interruption occurs, the migration progress record at the time of the migration interruption is determined, and the target primary key value in the target data table at the time of the migration interruption is queried. The target primary key value is the dividing point between successful and failed data migration. The migration progress record at the time of the migration interruption is updated based on the target primary key value, so that the source primary key value of the interrupted migration recorded in the migration progress record is more accurate, so that data migration can be continued from the source primary key value of the interrupted migration, and the migration initiation position at the time of re-migration can be accurately determined, avoiding the need to migrate some successfully migrated data again, which is beneficial to improving the accuracy and speed of data migration.

[0219] In one embodiment, the migration progress record is stored in a progress data table; the second determination module 1110 is also used to determine the joint primary key in the progress data table, where the joint primary key is a combination of the table name of the source data table and the library name of the database, which is used to store the source data table; based on the joint primary key and the updated migration progress record, the source primary key value of the interrupted migration in the source data table is determined.

[0220] In this embodiment, the joint primary key in the progress data table is determined. The joint primary key is a combination of the table name of the source data table and the library name of the database. The database is used to store the source data table, so that it is possible to determine which data table caused the migration interruption phenomenon and in which database the data table causing the migration interruption phenomenon is stored. After determining the source primary key value of the interrupted migration in the source data table based on the joint primary key and the updated migration progress record, the data migration of the data table of the database is continued to ensure the accuracy of the data migration.

[0221] In one embodiment, the apparatus further comprises:

[0222] A construction module is used to obtain the table name of the source data table and determine the library name of the database that stores the source data table; construct a progress data table, which includes table field names, library field names, and record field names; migrate progress records as record field contents under the record field names; use the table name of the source data table as the table field content under the table field name, and use the library name of the database as the library field content under the library field name; configure the table field content and the library field content as the joint primary key of the progress data table.

[0223] In this embodiment, the table name of the source data table is obtained, and the library name of the database storing the source data table is determined to construct a progress data table. The progress data table includes table field names, library field names, and record field names. The migration progress record is used as the record field content of the record field name, so that the migration progress record can be queried from the progress data table. The table name of the source data table is used as the table field content under the table field name, and the library name of the database is used as the library field content under the library field name. The table field content and the library field content are configured as the joint primary key of the progress data table, so that the data table that needs to be resumed can be quickly identified.

[0224] In one embodiment, the first migration module 1106 is also used to determine the migration order of multiple source primary key values ​​based on their positions in the source data table; and migrate the source primary key values ​​and source data in the source data table to the target data table one by one according to the migration order of the multiple source primary key values.

[0225] In this embodiment, based on the positions of multiple source primary key values ​​in the source data table, the migration order corresponding to the multiple source primary key values ​​is determined, so that the source primary key values ​​and source data in the source data table are migrated to the target data table one by one according to the migration order corresponding to the multiple source primary key values. This can ensure that the order of the source data after migration will not be disrupted, thereby ensuring the accuracy of the data.

[0226] In one embodiment, the apparatus further comprises:

[0227] Add a module to determine the location of the source primary key value of each migration in the source data table, and add a migration progress record for each migration at the location;

[0228] The second determination module 1110 is also used to determine the location of the migration progress record at the time of migration interruption from the source data table when migration interruption occurs; and use the source primary key value at the location of the migration progress record at the time of migration interruption as the source primary key value of the interrupted migration in the source data table.

[0229] In this embodiment, the location of the source primary key value of the source data for each migration in the source data table is determined, and a migration progress record is added at that location, thereby directly marking which data in the source data table has been migrated and which has not. Furthermore, if a migration is interrupted, the location of the migration progress record at the time of the interruption can be quickly and accurately determined from the source data table. The source primary key value at the location of the migration progress record at the time of the interruption is used as the source primary key value for the interrupted migration, allowing data migration to continue from that location, avoiding the situation where the entire data needs to be re-migrated due to the migration interruption.

[0230] In one embodiment, the source primary key value and the source data in the source data table are stored in the form of a key-value pair, with the source primary key value serving as the key and the source data identified by the source primary key value serving as the value corresponding to the key; the apparatus further comprises:

[0231] Add a module to determine the index data of each key-value pair in the source data table, where the index data includes the source primary key value and the source data in the key-value pair; add the migration progress record generated for each source primary key value to the index data of the key-value pair to which the source data of each migration belongs;

[0232] The second determination module 1110 is also used to query the index data of each key-value pair when a migration interruption occurs, and obtain the index data belonging to the migration progress record when the migration is interrupted; and use the source primary key value included in the obtained index data as the source primary key value of the interrupted migration in the source data table.

[0233] In this embodiment, the source primary key value and source data in the source data table are stored in the form of a key-value pair, with the source primary key value serving as the key and the source data identified by the source primary key value serving as the value corresponding to the key, so that the corresponding value can be obtained through each key. The index data corresponding to each key-value pair in the source data table is determined, and the index data includes the source primary key value and source data in the key-value pair. The migration progress record generated based on the source primary key value of the source data of each migration is added to the index data corresponding to the key-value pair to which the source data of each migration belongs, so that the migration progress record can be added to the index data to directly mark which data has been migrated. In the event of a migration interruption, the index data corresponding to each key-value pair can be queried, and the index data to which the migration progress record at the time of the migration interruption belongs can be quickly and accurately determined, thereby accurately determining the source primary key value that interrupted the migration.

[0234] In one embodiment, the index data includes index key data, which includes a source primary key value and source data in a key-value pair; an additional module is further configured to determine the index key data to which the source data of each migration belongs; and to use the migration progress record generated for the source primary key value of each migration as the index value data of the index key data to which the source data of each migration belongs;

[0235] The second determination module 1110 is also used to query each index value data based on each index key data when migration interruption occurs, and obtain the index value data at the time of migration interruption; and use the source primary key value included in the index key data of the obtained index value data as the source primary key value of the interrupted migration in the source data table.

[0236] In this embodiment, the index key data to which the source data of each migration belongs is determined, and the migration progress record generated based on the source primary key value of the source data of each migration is used as the index value data corresponding to the index key data to which the source data of each migration belongs, so that it is possible to directly query which data in the source data table has been migrated through the index information, so that when a migration interruption occurs, the index value data can be queried based on the index key data to quickly obtain the index value data at the time of the migration interruption, thereby quickly determining the source primary key value of the interrupted migration.

[0237] In one embodiment, the adding module is further configured to add a migration progress record generated for each source primary key value of the migration to the index key data to which the source data of the migration belongs;

[0238] The second determination module 1110 is also used to query each index key data when a migration interruption occurs, obtain the index key data belonging to the migration progress record when the migration is interrupted, and use the source primary key value included in the obtained index key data as the source primary key value of the interrupted migration in the source data table.

[0239] In this embodiment, the migration progress record generated based on the source primary key value of the source data of each migration is added to the index key data belonging to the source data of each migration, so that the migration progress record can be added to the index key data of the index data to directly mark which data has been migrated. In the event of a migration interruption, the index key data at the time of the migration interruption can be quickly determined by querying the index key data of the index data, thereby accurately determining the source primary key value of the interrupted migration.

[0240] In one embodiment, the apparatus further comprises:

[0241] The synchronization module is used to establish an association relationship between the source data table and the target data table; record the migration initiation time of the source data table; when a data processing request for the source data table is received after the migration initiation time, perform the operation indicated by the data processing request on the source data table, and synchronize the operation indicated by the data processing request to the target data table based on the association relationship.

[0242] In this embodiment, an association is established between the source data table and the target data table, and the migration initiation time of the source data table is recorded. Data in the source data table before the migration initiation time is treated as existing data, and data generated after the migration initiation time is treated as new data. When a data processing request for the source data table is received after the migration initiation time, the processing operations of the data processing request in the source data table are synchronized to the target data table based on the association, allowing normal access to the source data table during the data migration. Furthermore, this ensures that data generated during access will not affect data migration, and that new data generated will be synchronized to the target data table, preventing data loss.

[0243] In one embodiment, the synchronization module is also used to determine the type of data processing request when a data processing request for the source data table is received after the migration is initiated; determine the trigger that matches the type based on the association relationship; execute the operation indicated by the data processing request on the source data table, and synchronize the operation indicated by the data processing request to the target data table through the trigger that matches the type.

[0244] In this embodiment, when a data processing request is received for a source data table after the migration is initiated, the type of the data processing request is determined. Based on the association relationship, a trigger that matches the type of data processing request is identified. Using this trigger, the processing operations of the data processing request in the source data table are synchronized to the target data table, allowing normal access to the source data table during the data migration. Furthermore, data generated during access does not affect data migration, and new data generated is synchronized to the target data table, preventing data loss.

[0245] Each module in the above-mentioned data table processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0246] In one embodiment, a computer device is provided, which may be a terminal or a server. Taking a server as an example, its internal structure diagram may be shown in FIG12 . The computer device includes a processor, a memory, an input / output (I / O) interface, and a communication interface. The processor, the memory, and the I / O interface are connected via a system bus, and the communication interface is connected to the system bus via the I / O interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is configured to store data table processing data. The I / O interface of the computer device is configured to exchange information between the processor and an external device. The communication interface of the computer device is configured to communicate with an external terminal via a network connection. When executed by the processor, the computer program implements a data table processing method.

[0247] Those skilled in the art will understand that the structure shown in FIG12 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0248] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0249] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0250] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0251] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0252] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0253] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0254] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A data table processing method, the method comprising: Acquire source table structure information of a source data table, change the source table structure information, and create a target data table according to the changed table structure information; Determine a plurality of source primary key values ​​in the source data table, each of the source primary key values ​​being used to identify a piece of source data in the source data table; Migrating source primary key values ​​and source data in the source data table to the target data table one by one, wherein at least one source primary key value and the source data identified by the at least one source primary key value are migrated each time, and each migration starts after the previous migration succeeds; Generate migration progress records for each source primary key value of migration; When a migration interruption occurs, determining a migration progress record at the time of the migration interruption, and determining a source primary key value of the interrupted migration in the source data table according to the migration progress record at the time of the migration interruption; and Starting from the source primary key value whose migration is interrupted in the source data table, the source primary key values ​​and source data that have not been migrated in the source data table continue to be migrated to the target data table one by one.

2. The method according to claim 1, wherein the source primary key value in the source data table is used as the target primary key value in the target data table; when migration is interrupted, determining the migration progress record when the migration is interrupted, and determining the source primary key value of the interrupted migration in the source data table according to the migration progress record when the migration is interrupted, comprising: When a migration interruption occurs, determine the migration progress record at the time of the migration interruption, and query the target primary key value in the target data table at the time of the migration interruption; updating the migration progress record when the migration is interrupted based on the target primary key value; According to the updated migration progress record, the source primary key value of the interrupted migration in the source data table is determined.

3. The method according to claim 2, characterized in that The migration progress record is stored in a progress data table; and determining the source primary key value of the interrupted migration in the source data table according to the updated migration progress record comprises: Determine a joint primary key in the progress data table, where the joint primary key is a combination of a table name of the source data table and a database name, and the database is used to store the source data table; Based on the combined primary key and the updated migration progress record, a source primary key value for interrupting migration in the source data table is determined.

4. The method according to claim 3, further comprising: Obtain the table name of the source data table, and determine the library name of the database storing the source data table; Constructing a progress data table, the progress data table including table field names, library field names and record field names, the migration progress record being the record field content of the record field name; Using the table name of the source data table as the table field content under the table field name, and using the library name of the database as the library field content under the library field name; and The table field content and the library field content are configured as a joint primary key of the progress data table.

5. The method according to any one of claims 1 to 4, wherein the step of successively migrating the source primary key values ​​and source data in the source data table to the target data table comprises: Determine the migration order of each of the plurality of source primary key values ​​based on the positions of the plurality of source primary key values ​​in the source data table; and The source primary key values ​​and source data in the source data table are migrated to the target data table one by one according to the migration order of the plurality of source primary key values.

6. The method according to any one of claims 1 to 5, further comprising: Determine the position of the source primary key value of each migration in the source data table, and add a migration progress record of each migration at the position; When a migration interruption occurs, determining a migration progress record when the migration is interrupted, and determining a source primary key value of the interrupted migration in the source data table according to the migration progress record when the migration is interrupted, includes: When a migration interruption occurs, the location of the migration progress record at the time of the migration interruption is determined from the source data table. Place; and The source primary key value at the location of the migration progress record when the migration is interrupted is used as the source primary key value of the interrupted migration in the source data table.

7. The method according to any one of claims 1 to 5, wherein the source primary key value and the source data in the source data table are stored in the form of a key-value pair, the source primary key value is used as a key, and the source data identified by the source primary key value is used as a value corresponding to the key; The method further comprises: Determine index data for each key-value pair in the source data table, wherein the index data includes a source primary key value and source data in the key-value pair; Add the migration progress record generated for each source primary key value of the migration to the index data of the key-value pair to which the source data of each migration belongs; and When a migration interruption occurs, determining a migration progress record when the migration is interrupted, and determining a source primary key value of the interrupted migration in the source data table according to the migration progress record when the migration is interrupted, includes: When a migration interruption occurs, query the index data of each key-value pair to obtain the index data to which the migration progress record at the time of the migration interruption belongs; The source primary key value included in the obtained index data is used as the source primary key value for interrupting migration in the source data table.

8. The method according to claim 7, wherein the index data comprises index key data, and the index key data comprises a source primary key value and source data in the key-value pair; The step of adding the migration progress record generated for the source primary key value of each migration to the index data of the key-value pair to which the source data of each migration belongs includes: Determine the index key data to which the source data for each migration belongs; The migration progress record generated for the source primary key value of each migration is used as the index value data of the index key data of the source data of each migration; When the migration is interrupted, querying the index data of each key-value pair to obtain the index data of the migration progress record when the migration is interrupted includes: When a migration interruption occurs, querying each of the index value data based on each of the index key data to obtain the index value data at the time of the migration interruption; and The source primary key value included in the index data to be obtained, as the source primary key value of the interrupted migration in the source data table, includes: The source primary key value included in the index key data of the obtained index value data is used as the source primary key value of the interrupted migration in the source data table.

9. The method according to claim 8, further comprising: Add the migration progress record generated for each source primary key value of migration to the index key data of the source data of each migration; When a migration interruption occurs, query the index key data to obtain the index key data to which the migration progress record at the time of the migration interruption belongs; and The source primary key value included in the obtained index key data is determined as the source primary key value of the interrupted migration in the source data table.

10. The method according to any one of claims 1 to 9, further comprising: Establishing an association relationship between the source data table and the target data table; Record the migration initiation time of the source data table; and When a data processing request for the source data table is received after the migration initiation time, an operation indicated by the data processing request is performed on the source data table, and the operation indicated by the data processing request is synchronized to the target data table based on the association relationship.

11. The method according to claim 10, wherein when a data processing request for the source data table is received after the migration initiation time, the operation indicated by the data processing request is performed on the source data table, and the operation indicated by the data processing request is synchronized to the target data table based on the association relationship, comprising: When a data processing request for the source data table is received after the migration initiation time, determining a type of the data processing request; Based on the association relationship, determining a trigger matching the type; and The operation indicated by the data processing request is executed on the source data table, and the operation indicated by the data processing request is synchronized to the target data table through a trigger matching the type.

12. A data table processing device, the device comprising: The change module is used to obtain source table structure information of a source data table, change the source table structure information, and create a target data table according to the changed table structure information; A first determining module is used to determine a plurality of source primary key values ​​in the source data table, each of which is used to identify a piece of source data in the source data table; A first migration module is used to migrate source primary key values ​​and source data in the source data table to the target data table one by one, wherein at least one source primary key value and the source data identified by the at least one source primary key value are migrated each time, and each migration starts after the previous migration succeeds; A generation module is used to generate a migration progress record for each source primary key value of the migration; A second determination module is used to determine the migration progress record when the migration is interrupted, and determine the source primary key value of the interrupted migration in the source data table according to the migration progress record when the migration is interrupted; and The second migration module is used to continue to migrate the unmigrated source primary key values ​​and source data in the source data table to the target data table one by one, starting from the source primary key value whose migration is interrupted in the source data table.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images