A method, device and readable storage medium for data migration of a credit card system

Abstract

The application provides a method and device for data migration of a credit card system, electronic equipment and a readable storage medium, and is characterized in that the method comprises the following steps: S1, obtaining a data migration instruction; S2, obtaining first customer information of a first database and first customer mark information of a second database; S3, judging whether the first customer mark information has corresponding first customer information, and if not, marking the corresponding first customer mark information as first difference customer mark information; S4, generating first difference customer information based on the first difference customer mark information, and generating second customer mark information according to the first difference customer information and the first customer information; and S5, storing the first customer information and the first difference customer information to a third database based on the second customer mark information. The application can improve the data migration efficiency and save the data test time.

Description

Technical Field

[0001] This invention relates to the technical field of data migration, and in particular to a method, apparatus and readable storage medium for data migration of a credit card system. Background Technology

[0002] Due to regulatory or outsourcing reasons, banking data is stored in other companies or locations. The outgoing customer information system stores customer details such as name, ID type, and ID number. The outgoing credit card system stores corresponding tagging information, such as customer number. Due to historical changes, there are data differences between the old customer information system and the old credit card system, especially since the old credit card system stores tagging information without corresponding customer details. During the data migration between the old and new bank card systems, the outgoing customer information and the outgoing customer tagging information will form a first pair, while the incoming customer information system and the incoming customer tagging information will form a second pair. These first and second pairs will form a unique mapping. Since database sharding is based on the tagging information of the new credit card system, the incoming customer tagging information is an essential component of this application. However, due to data differences, especially the lack of outgoing customer information, a unique mapping cannot be constructed for data migration, the incoming customer tagging information cannot be formed, and the data migration cannot be completed. The lack of customer tagging information at the receiving end will lead to a series of serious data problems, including the inability of this customer data to be used normally in the new system and discrepancies in the daily reconciliation results of the general ledger and individual ledgers. Current technical solutions for data migration employ a cleanup and supplementation method, which involves first supplementing and correcting discrepancies in the customer information database at the sending end before data migration. However, because the supplementation process is time-consuming, business data remains in an incomplete migration state from the discovery of discrepancies to the completion of supplementation, making data testing impossible and wasting limited testing time and resources.

[0003] Therefore, this application addresses the above technical issues by first identifying discrepancies in customer information, then automatically generating virtual discrepancies, and subsequently constructing a new credit card system's tagging information based on existing customer information and the virtual discrepancies. This establishes a unique mapping between the first and second pairs, and performs database sharding and table partitioning based on the new credit card system's tagging information. Since no data cleanup or supplementation is required, valuable testing time and resources are saved. Data testing can begin once data migration is complete. Finally, discrepancies can be replaced or deleted using methods such as substitution.

[0004] It is worth noting that the data collection and migration involved in this application have been with the user's consent and are necessary for the applicant's normal business operations. The applicant has anonymized and desensitized the collected data and set corresponding access permissions to ensure that user privacy is not leaked. The data compliance measures involved in this application are only listed here; the applicant has also implemented other relevant data compliance measures. The data processing activities carried out by the applicant comply with the provisions of the Data Security Law and the Personal Information Protection Law, and other relevant laws and regulations. Summary of the Invention

[0005] To address the shortcomings of the existing system, this invention proposes a method, apparatus, and readable storage medium for data migration in a credit card system, taking into account the deficiencies of the aforementioned background technology.

[0006] The first aspect of this application proposes a method for data migration in a credit card system, characterized by comprising:

[0007] S1. Obtain data migration instructions, including full migration, incremental migration, and full-incremental migration;

[0008] S2. According to the data migration instruction, obtain the first customer information of the first database and the first customer tag information of the second database;

[0009] S3. Determine whether there is corresponding first customer information for the first customer tag information. If there is no corresponding first customer information for the first customer tag information, then mark the first customer tag information without corresponding first customer information as the first differential customer tag information.

[0010] S4. Generate first-differentiated customer information based on the first-differentiated customer tagging information, and generate second-differentiated customer tagging information based on the first-differentiated customer information and the first-customer information;

[0011] S5. Based on the second customer tag information, store the first customer information and the first differential customer information in separate databases and tables in the third database, and save the mapping relationship between the second customer tag information and the first customer information and the first differential customer information.

[0012] Furthermore, the first database includes a credit card customer information database for the data migration originating end, the second database includes a credit card tagging information database for the data migration originating end, and the third database includes a credit card customer information database for the data migration in end. The customer information includes the customer's name, document type, and document number, and the customer tagging information is a unique tagging information generated based on the customer information.

[0013] Furthermore, S2 also includes:

[0014] S21. Detect the data migration environment according to the data migration instructions;

[0015] S22. If the data migration environment test passes, determine the data migration time split point between the first database and the second database.

[0016] S23. Determine the full data of the first database and the second database based on the time segmentation point, wherein the full data is the data generated before the time segmentation point of the first database and the second database;

[0017] S24. Determine the augmented data based on the time segmentation point, wherein the incremental data is the data generated after the time segmentation point until the data migration is completed;

[0018] S25. Obtain the full data and the incremental data in chronological order.

[0019] Furthermore, step S4, generating the first differentiated customer information based on the first differentiated customer tagging information, further includes:

[0020] S41. Generate first automatically differentiated customer information according to the first preset rule and the first differentiated customer tag information, wherein the first preset rule is an automatically generated data rule that is consistent with the format of the first customer information;

[0021] S42. Traverse the content information of the first customer information and determine whether the first automatically differentiated customer information and the first customer information have the same content information. If there is no same content information, directly mark the first automatically differentiated customer information as the first differentiated customer information.

[0022] S43. If there is identical content information, then based on the first preset rule, generate automatic difference information that is different from the content information of the first customer information again, replace the identical information in the first automatic difference customer information, and mark the first automatic difference information that has no identical content information with the first customer information as the first difference customer information.

[0023] S44. Mark the generated first difference customer information.

[0024] Furthermore, step S5 also includes:

[0025] S51. Obtain and load table task information, wherein the table task information includes first customer information, first differential customer information, splitting field and database field information;

[0026] S52. Divide the first customer information and the first differential customer information into a first number of slices according to the segmentation field in the task information of the table, and request the distributed system to establish a mapping between each slice and the second customer tag information;

[0027] S53. Based on the database sharding field in the task information of the table, shard each slice to obtain a second number of database sharding tables, and request the distributed system to establish the mapping between each database sharding table and the second customer tag information.

[0028] S54. Write the sharded database tables into the distributed database.

[0029] Furthermore, after step S5, the method further includes storing the second customer tag information in the fourth database, storing the first customer information and the first differential customer information in the third database, comparing the third database with the first database, and checking whether there are any missing or incomplete data during the data migration process.

[0030] Furthermore, after step S5, the method further includes replacing or deleting the first differential customer information and / or the second tag information corresponding to the first differential customer information.

[0031] Furthermore, the method also includes connecting to the database of the data provider via a link address to obtain data information from the database.

[0032] A second aspect of this application discloses a data migration apparatus for a credit card system, characterized in that it comprises:

[0033] The first acquisition module is used to acquire data migration instructions, which include full migration, incremental migration, and full-incremental migration.

[0034] The second acquisition module is used to acquire the first customer information in the first database and the first customer tag information in the second database according to the data migration instruction.

[0035] The marking module is used to determine whether there is corresponding first customer information for the first customer marking information. If there is no corresponding first customer information for the first customer marking information, the first customer marking information without corresponding first customer information is marked as first differential customer marking information.

[0036] The generation module is used to generate first differentiated customer information based on first differentiated customer tagging information, and to generate second customer tagging information based on first differentiated customer information and first customer information.

[0037] The storage module is used to store the first customer information and the first differential customer information in separate databases and tables in the third database based on the second customer tag information, and to save the mapping relationship between the second customer tag information and the first customer information and the first differential customer information.

[0038] A third aspect of this application provides an electronic device, characterized in that it comprises: a processor, and a memory communicatively connected to the processor;

[0039] The memory stores computer-executed instructions;

[0040] The processor executes computer execution instructions stored in the memory to implement the data migration method of the credit card system described above.

[0041] A fourth aspect of this application discloses a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the data migration method of the aforementioned credit card system.

[0042] The fifth aspect of this application discloses a computer program product, characterized in that it includes a computer program that, when executed by a processor, implements the data migration method of the aforementioned credit card system.

[0043] The beneficial effects of this invention are as follows: Since no cleaning or supplementary recording is required, data testing can be carried out immediately after the data migration is completed, saving valuable testing time. Finally, the automatically generated difference data can be replaced by methods such as replacement and deletion for the difference information, thus completing the data migration task more efficiently. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the data migration process for the credit card system of the present invention.

[0045] Figure 2 This is a structural block diagram of the data migration device for the credit card system of the present invention.

[0046] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0047] To better understand the content of this invention, it will be described in detail with reference to the accompanying drawings and embodiments.

[0048] However, it should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention. The terminology used herein is solely for describing particular embodiments and is not intended to limit the invention. The words “a,” “an,” and “the,” as used herein, should also include the meanings of “a plurality” and “multiple,” unless the context clearly indicates otherwise. Furthermore, the terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0049] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid manner. This invention is an improvement upon prior art; therefore, any parts not described in this application are implemented using existing technology.

[0050] Terminology Explanation:

[0051] ETL is an abbreviation for Extract-Transform-Load, used to describe the process of extracting, transforming, and loading data from a source to a destination.

[0052] Data mapping is an essential component of large-scale data migration and data integration processes. It involves establishing a correspondence between data elements given two data models. This process can be divided into manual coding and visualization operations.

[0053] Due to regulatory or outsourcing reasons, banking data is stored in other companies or locations. The outgoing customer information system stores customer element information, such as name, document type, and document number. The outgoing credit card system stores corresponding tag information, such as customer number. Due to historical changes, there are data differences between the old customer information system and the old credit card system, especially since the old credit card system stores tag information without corresponding customer element information. During the data migration process between the old and new bank card systems, the outgoing customer information and the outgoing customer tag information will form a first pair, while the incoming customer information system and the customer tag information will form a second pair. The first and second pairs will form a unique mapping. Since database sharding is based on the tag information of the new credit card system, the incoming customer tag information is an essential component of this application. Due to data differences, especially the lack of outgoing customer information, a unique mapping cannot be constructed for data migration, the incoming customer tag information cannot be formed, and the data migration cannot be completed. The lack of customer identification information at the importing end will lead to a series of serious data problems, including the inability of this customer data to be used normally in the new system and discrepancies in the daily reconciliation results of the general ledger and individual ledgers. In existing technologies, the old credit card customer information identification system at the data exporting end differs from the new credit card customer information system in its generation rules or format requirements. Therefore, it needs to generate second customer identification information instead of using the customer identification information from the importing end. Of course, if the new credit card system at the importing end is compatible with the first customer identification information, then the first customer identification information can be used as the data input for the second customer identification information. However, due to the need to establish a unique mapping, the first customer identification information can be simply processed.

[0054] Therefore, this application addresses the above technical issues by first identifying discrepancies in customer information, then automatically generating virtual discrepancies, and subsequently constructing tagging information for a new credit card system (the customer tagging information system for the migrating end) based on existing customer information and the virtual discrepancies. Finally, it performs database sharding and table partitioning based on the tagging information of the new credit card system. Since no data cleanup or supplementation is required, valuable testing time and resources are saved. Data testing can begin immediately after data migration is complete. Finally, discrepancies can be replaced or deleted using methods such as substitution.

[0055] The first aspect of this application proposes a method for data migration in a credit card system, characterized by comprising:

[0056] S1. Obtain data migration instructions, including full migration, incremental migration, and full-incremental migration;

[0057] Furthermore, the first database includes a credit card customer information database for the data migration originating end, the second database includes a credit card tagging information database for the data migration originating end, and the third database includes a credit card customer information database for the data migration inbound end. The customer information includes the customer's name, document type, and document number. The customer tagging information is a unique tagging information generated based on the customer information. The first customer tagging information in the second database of this application is a unique tagging information generated based on the original complete customer information, such as the customer number. The second customer identification information in the credit card tagging information database for the data migration inbound end is a unique tagging information generated based on a different customer information and the first customer information.

[0058] In the process of bank card data migration, the data being migrated out consists of the old credit card system and the old customer information system, while the data being migrated in consists of the new credit card system and the new customer information system. The system mentioned in this application includes a database system. Therefore, the system in this application can realize the functions of a database. Since the content information of both the database at the migration-in end and the database at the migration-out end is stored in the form of data tables, but this application does not focus on the data storage method, this application uses data tables as an example but is not limited to data tables.

[0059] For example, according to the data migration instruction, the table information of the data to be migrated is first determined, and the data to be migrated is divided into full synchronization tables and / or incremental synchronization tables; when the data to be migrated is divided into full synchronization tables, the migration type of the data to be migrated is determined to be full migration.

[0060] When the data to be migrated is divided into incremental synchronization tables, the migration type of the data to be migrated is determined to be incremental migration;

[0061] When the data to be migrated is divided into a full synchronization table and an incremental synchronization table, the migration type of the data to be migrated is determined to be full-incremental migration.

[0062] In one embodiment, migrating the data to be migrated according to the migration type includes:

[0063] When it is determined that the migration type of the data to be migrated is full migration, configure the migration conditions of the full synchronization table;

[0064] Establish the migration link for the full synchronization table;

[0065] The data to be migrated is migrated based on the migration conditions and the migration link.

[0066] In one embodiment, migrating the data to be migrated according to the migration type includes:

[0067] When it is determined that the migration type of the data to be migrated is full-incremental migration, the migration conditions of the full synchronization table and the incremental synchronization table are configured.

[0068] Establish the migration link between the full synchronization table and the incremental synchronization table;

[0069] The data to be migrated is migrated based on the migration conditions and the migration link.

[0070] S2. According to the data migration instruction, obtain the first customer information of the first database and the first customer tag information of the second database;

[0071] Furthermore, S2 also includes:

[0072] S21. Detect the data migration environment according to the data migration instructions;

[0073] S22. If the data migration environment test passes, determine the data migration time split point between the first database and the second database.

[0074] S23. Determine the full data of the first database and the second database based on the time segmentation point, wherein the full data is the data generated before the time segmentation point of the first database and the second database;

[0075] S24. Determine the augmented data based on the time segmentation point, wherein the incremental data is the data generated after the time segmentation point until the data migration is completed;

[0076] S25. Obtain the full data and the incremental data in chronological order.

[0077] In this embodiment, after the data provider determines the databases to be migrated, it can send a data migration request to the data receiver. The data receiver can determine the time segmentation point for data migration upon receiving the data migration request from the data provider. The data migration request includes information about each database to be migrated. In another embodiment, the time segmentation point can also be determined based on the time of the data migration request sent by the data migration provider.

[0078] In practice, database data migration can be the migration of all data up to a certain point in time (e.g., the current time), or it can be the migration of all data up to a certain point in time, data added after that point in time, and data added during the migration process. The aforementioned point in time is the time segmentation point for data migration.

[0079] Information such as which type of migration the database to be migrated belongs to can be recorded in the data migration request sent by the data provider. Information such as the time segmentation point of the data migration can be recorded in the data migration request sent by the data provider, or it can be determined by the data receiver based on the data migration request. For example, the data receiver can determine the current time point when the data migration request is sent as the time segmentation point of the data migration, or it can determine the size of the data in the database. If the data size is greater than a preset data size threshold (such as 1TB, where TB is an abbreviation for Terabyte, representing one megabyte), then the time when the data size exceeds the preset data size threshold is generated will be used as the time segmentation point of the data migration, and so on.

[0080] In some optional implementations of this embodiment, when the data receiving end receives a data migration request from the data providing end, it can also check the migration environment for the database migration. If the migration environment check passes, the time segmentation point for data migration is determined, and the database migration is executed; otherwise, the database migration is not executed. Here, the migration environment can be the migration environment of the data receiving end, the migration environment of the data providing end, or a migration environment in which the data receiving end and the data providing end cooperate.

[0081] S3. Determine whether there is corresponding first customer information for the first customer tag information. If there is no corresponding first customer information for the first customer tag information, then mark the first customer tag information without corresponding first customer information as the first differential customer tag information.

[0082] S4. Generate first-differentiated customer information based on the first-differentiated customer tagging information, and generate second-differentiated customer tagging information based on the first-differentiated customer information and the first-customer information;

[0083] Furthermore, step S4, generating the first differentiated customer information based on the first differentiated customer tagging information, further includes:

[0084] S41. Generate first automatically differentiated customer information according to the first preset rule and the first differentiated customer tag information, wherein the first preset rule is an automatically generated data rule that is consistent with the format of the first customer information;

[0085] S42. Traverse the content information of the first customer information and determine whether the first automatically differentiated customer information and the first customer information have the same content information. If there is no same content information, directly mark the first automatically differentiated customer information as the first differentiated customer information.

[0086] S43. If there is identical content information, then based on the first preset rule, generate automatic difference information that is different from the content information of the first customer information again, replace the identical information in the first automatic difference customer information, and mark the first automatic difference information that has no identical content information with the first customer information as the first difference customer information.

[0087] S44. Mark the generated first difference customer information.

[0088] In this application, a unique mapping between the first pair and the second pair is constructed during the data migration process. However, during the historical data change or data migration process, customer information may be deleted or lost due to replacement or other reasons. This will inevitably result in the absence of customer information in the customer tag information database (old credit card customer tag information system) at the migration end. Since the different customer information data is indispensable for constructing a unique mapping, this application first finds the customer tag information corresponding to the absence of customer information. The specific search method can be based on traversing the customer tag information at the migration end to confirm which customer tag information has no customer information mapping or corresponding.

[0089] After confirming the identification information for customers without corresponding customer information, this application generates virtual customer information that differs from existing customer information based on a format consistent with the existing customer information. This virtual customer information is marked as the first differentiated customer information. For example, the generation of customer ID card types can be restricted by limiting the number of digits and the format of the generated data. For names, restrictions can be placed on the maximum number of characters, such as 10. This application can restrict data format information such as fields for the generated customer information. Similarly, this application can also generate first automatically differentiated customer information based on existing customer information, specifying the corresponding region and surname. After the first automatically differentiated customer information is generated, it is checked whether there is any duplicate customer information. For example, if the generated customer name contains "Zhang San" and there is also a duplicate customer name in the existing customer information, then the duplicate customer information is replaced according to the first preset rule. In this application, "difference" can refer to one or multiple identical customer information. Accordingly, the data replacement condition can be set to replace the first automatically differentiated customer information if there is one identical item, or it can be set to replace the first automatically differentiated customer information if multiple identical customer information items are found, to ensure that no errors occur during the data migration and mapping process between the generated first automatically differentiated customer information and the existing customer information.

[0090] S5. Based on the second customer tag information, store the first customer information and the first differential customer information in separate databases and tables in the third database, and save the mapping relationship between the second customer tag information and the first customer information and the first differential customer information.

[0091] Furthermore, step S5 also includes:

[0092] S51. Obtain and load table task information, wherein the table task information includes first customer information, first differential customer information, splitting field and database field information;

[0093] S52. Divide the first customer information and the first differential customer information into a first number of slices according to the segmentation field in the task information of the table, and request the distributed system to establish a mapping between each slice and the second customer tag information;

[0094] S53. Based on the database sharding field in the task information of the table, shard each slice to obtain a second number of database sharding tables, and request the distributed system to establish the mapping between each database sharding table and the second customer tag information.

[0095] S54. Write the sharded database tables into the distributed database.

[0096] The customer tagging information at the migrating end of this application can generate second customer tagging information based on the first difference customer information, the first customer information, and the first customer tagging information, or it can be generated automatically. However, the automatically generated second customer tagging information needs to establish a mapping during the data migration process to satisfy the mapping relationship with the first difference customer information and the first customer information.

[0097] Database sharding refers to the process of dividing the data of a source data table into multiple database shard tables using a database sharding algorithm. The second customer tag information in this application can be obtained using a database sharding algorithm during the data migration mapping process, or it can be obtained after mapping is completed using a database sharding algorithm, such as a hash algorithm. Based on the sharding field in the table task information, the source data table to be migrated is divided into a first number of slices, and a first mapping task is requested from the distributed system for each slice, i.e., a first number of first mapping tasks are formed, mapping each slice to the second customer tag information. Each first mapping task connects to the first database and reads a slice of the source data table. A slice is defined as the data of the data table to be processed being divided into multiple parts according to the sharding field provided in the table task information; each part is a slice. Each slice, after being read, can be stored in the cache of a processor on a computer in the distributed system. By using a distributed system, each slice of a source data table can be distributed to a regular computer for processing. Since the data volume of each slice is small, it is not necessary to use a server with high processing power. Furthermore, the system is requested to perform database sharding on each slice read by the first mapping task in each first mapping task according to the database sharding field in the table task information to obtain a second number of database sharding tables, and the distributed system is requested to establish a mapping between each database sharding table obtained after database sharding and the second customer tag information.

[0098] Furthermore, after step S5, the method further includes storing the second customer tag information in the fourth database, storing the first customer information and the first differential customer information in the third database, comparing the third database with the first database, and checking whether there are any missing or incomplete data during the data migration process.

[0099] Specific verification can be performed by creating an intermediate data table to store all data information. The intermediate data table verifies the integrity of the data after data verification. The intermediate data table can be deleted, cached, or saved to the database after verification.

[0100] Furthermore, after step S5, the method further includes replacing or deleting the first differential customer information and / or the second tag information corresponding to the first differential customer information.

[0101] Since this application has already completed the data migration using the virtually generated customer information, the time for cleaning and supplementing is omitted. The imported data can be tested and supplemented immediately. On the other hand, since the generated virtual first difference customer information is marked, the first difference customer information can be replaced with real customer information without replacing the corresponding second mark information. Alternatively, the first difference customer information and the corresponding second mark information can be deleted to perform data operations after data migration.

[0102] Furthermore, the method also includes connecting to the database of the data provider via a link address to obtain data information from the database.

[0103] In this embodiment, the data receiving end can connect to the data platform of the data providing end through a pre-acquired connection address, and obtain database information in the data platform so that the providing end can determine the database to be migrated from the database information.

[0104] Here, a data platform can be an application, a website, or even a file, etc., and can be used for one or more operations such as data input, output, processing, storage, querying, retrieval, and analysis. Examples include online data platforms like "Wanfang Data Knowledge Service Platform," "Patent Information Service Platform," and "XX Company Internal Data Management Platform." A data platform can include one database or multiple databases. When the data platform is a file, it can be stored on one electronic device or a cluster of electronic devices. When the data platform is an application or website, it can have a supporting backend server. The data platform can be accessed through the access path on its own electronic device or remotely through other electronic devices. In this case, other electronic devices can access the data platform from the aforementioned backend server or the electronic device where the data platform is located via wired or wireless connections. These wireless connection methods include, but are not limited to, 3G / 4G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future wireless connection methods. The aforementioned backend server or the electronic device where the data platform is located can serve as the data provider in this embodiment.

[0105] When remotely accessing the data platform, the data receiving end can connect to the data providing end via a connection address. This connection address can include the access path of the data platform. For example, this connection address can be a domain name, the IP (Internet Protocol) address of the network accessed by the data providing end, or, if the data providing end has one IP address and can provide multiple services simultaneously, each service can be distinguished by a port number. In this case, the connection address can also be an IP address plus a port number, where the port number corresponds to the service provided by the data platform within the data providing end.

[0106] The data receiving end can obtain the connection address of the data platform from the data provider in advance, or it can receive the connection address entered by the user and send a connection request containing the connection address to the gateway. The gateway then connects to the data platform of the data provider by parsing the connection request.

[0107] After the data receiving end establishes a connection with the data platform of the data providing end, it can obtain database information from the data platform, such as reading a list of database information in the data platform, or detecting database information in the data platform according to predetermined fields of the database.

[0108] In some optional implementations of this embodiment, the data platform can set different access permissions to limit user access rights. For example, some users can only access database A, some users can only access database B, and some users can access databases A and B, as well as database C, which is used to manage databases A and B, etc. Access permissions can be determined by access permission information (such as account, password, etc.) carried by the electronic device when accessing the data platform. In this case, the data receiving end can first send the access permission information to the data providing end, so that the data providing end can match the access permission information with preset permission information.

[0109] A second aspect of this application discloses a data migration apparatus for a credit card system, characterized in that it comprises:

[0110] The first acquisition module is used to acquire data migration instructions, which include full migration, incremental migration, and full-incremental migration.

[0111] The second acquisition module is used to acquire the first customer information in the first database and the first customer tag information in the second database according to the data migration instruction.

[0112] The marking module is used to determine whether there is corresponding first customer information for the first customer marking information. If there is no corresponding first customer information for the first customer marking information, the first customer marking information without corresponding first customer information is marked as first differential customer marking information.

[0113] The generation module is used to generate first differentiated customer information based on first differentiated customer tagging information, and to generate second customer tagging information based on first differentiated customer information and first customer information.

[0114] The storage module is used to store the first customer information and the first differential customer information in separate databases and tables in the third database based on the second customer tag information, and to save the mapping relationship between the second customer tag information and the first customer information and the first differential customer information.

[0115] The device further includes a data information confirmation module, used to confirm that the first database includes a data migration origin credit card customer information database, the second database includes a data migration origin credit card tag information database, and the third database includes a migration destination credit card customer information database. The customer information includes the customer's name, document type, and document number, and the customer tag information is a unique tag information generated based on the customer information.

[0116] Furthermore, the device includes a migration acquisition module for acquiring migration data between the first database and the second database, including:

[0117] S21. Detect the data migration environment according to the data migration instructions;

[0118] S22. If the data migration environment test passes, determine the data migration time split point between the first database and the second database.

[0119] S23. Determine the full data of the first database and the second database based on the time segmentation point, wherein the full data is the data generated before the time segmentation point of the first database and the second database;

[0120] S24. Determine the augmented data based on the time segmentation point, wherein the incremental data is the data generated after the time segmentation point until the data migration is completed;

[0121] S25. Obtain the full data and the incremental data in chronological order.

[0122] Furthermore, the device also includes a difference information confirmation generation module: used to generate first difference customer information based on first difference customer tagging information, including:

[0123] S41. Generate first automatically differentiated customer information according to the first preset rule and the first differentiated customer tag information, wherein the first preset rule is an automatically generated data rule that is consistent with the format of the first customer information;

[0124] S42. Traverse the content information of the first customer information and determine whether the first automatically differentiated customer information and the first customer information have the same content information. If there is no same content information, directly mark the first automatically differentiated customer information as the first differentiated customer information.

[0125] S43. If there is identical content information, then based on the first preset rule, generate automatic difference information that is different from the content information of the first customer information again, replace the identical information in the first automatic difference customer information, and mark the first automatic difference information that has no identical content information with the first customer information as the first difference customer information.

[0126] S44. Mark the generated first difference customer information.

[0127] Furthermore, the device also includes a database sharding and table creation module, used for:

[0128] S51. Obtain and load table task information, wherein the table task information includes first customer information, first differential customer information, splitting field and database field information;

[0129] S52. Divide the first customer information and the first differential customer information into a first number of slices according to the segmentation field in the task information of the table, and request the distributed system to establish a mapping between each slice and the second customer tag information;

[0130] S53. Based on the database sharding field in the task information of the table, shard each slice to obtain a second number of database sharding tables, and request the distributed system to establish the mapping between each database sharding table and the second customer tag information.

[0131] S54. Write the sharded database tables into the distributed database.

[0132] Furthermore, the device also includes a data storage module, which is used to store the second customer tag information in a fourth database, store the first customer information and the first differential customer information in a third database, compare the third database with the first database, and check whether there are any missing or incomplete data during the data migration process.

[0133] Furthermore, the device also includes an information replacement module for replacing or deleting the first differential customer information and / or the second tag information corresponding to the first differential customer information.

[0134] Furthermore, the device also includes a data connection module for connecting to the database of the data provider via a link address and obtaining data information from the database.

[0135] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 3 As shown, the electronic device may include: transceiver 121, processor 122, and memory 123.

[0136] Processor 122 executes computer execution instructions stored in memory, causing processor 122 to perform the scheme in the above embodiments. Processor 122 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0137] The memory 123 is connected to the processor 122 via the system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0138] Transceiver 121 can be used to acquire data migration instructions and first customer information of the first database and first customer tag information of the second database.

[0139] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0140] The electronic device provided in this application embodiment can be the terminal device described in the above embodiments.

[0141] This application also provides a chip for executing instructions, which is used to execute the data migration method of the credit card system described in the above embodiments.

[0142] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the data migration method of the credit card system described in the above embodiments.

[0143] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solution of the data migration method of the credit card system described in the above embodiments.

[0144] The beneficial effects of the present invention are as follows: Since no cleaning and supplementation are required, the present application performs cleaning and supplementation after the data migration is completed, saving valuable testing time. Subsequently, the difference information can be replaced by methods such as replacement and deletion, so as to complete the data migration task more efficiently.

[0145] It is worth noting that the data collection and migration involved in this application have been with the user's consent and are necessary for the applicant's normal business operations. The applicant has anonymized and desensitized the collected data and set corresponding access permissions to ensure that user privacy is not leaked. The data compliance measures involved in this application are only listed here; the applicant has also implemented other relevant data compliance measures. The data processing activities carried out by the applicant comply with the provisions of the Data Security Law and the Personal Information Protection Law, and other relevant laws and regulations.

[0146] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0147] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

[0148] Those skilled in the art will further recognize that the algorithm steps in connection with the embodiments herein can be implemented in electronic hardware, computer software, or a combination of both. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0149] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0150] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for data migration in a credit card system, characterized in that, include: S1. Obtain data migration instructions, including full migration, incremental migration, and full-incremental migration; Specifically, based on the data migration instruction, the table information of the data to be migrated is determined, and the data to be migrated is divided into a full synchronization table and / or an incremental synchronization table. When the data to be migrated is divided into a full synchronization table and an incremental synchronization table, the migration type of the data to be migrated is determined to be full-incremental migration; when the migration type of the data to be migrated is determined to be full-incremental migration, the migration conditions of the full synchronization table and the incremental synchronization table are configured. Establish the migration link between the full synchronization table and the incremental synchronization table; The data to be migrated is migrated based on the migration conditions and the migration link; S2. According to the data migration instruction, obtain the first customer information of the first database and the first customer tag information of the second database; S3. Determine whether there is corresponding first customer information for the first customer tag information. If there is no corresponding first customer information for the first customer tag information, then mark the first customer tag information without corresponding first customer information as the first differential customer tag information. S4. Generate first-differentiated customer information based on the first-differentiated customer tagging information, and generate second-differentiated customer tagging information based on the first-differentiated customer information and the first-customer information; S5. Based on the second customer tag information, store the first customer information and the first differential customer information in separate databases and tables in the third database, and save the mapping relationship between the second customer tag information and the first customer information and the first differential customer information; The first database includes a credit card customer information database for data migration originating from the ...

2. The method according to claim 1, characterized in that, The customer information includes the customer's name, document type, and document number.

3. The method according to claim 2, characterized in that, S2 further includes: S21. Detect the data migration environment according to the data migration instructions; S22. If the data migration environment test passes, determine the data migration time split point between the first database and the second database. S23. Determine the full data of the first database and the second database based on the time segmentation point, wherein the full data is the data generated before the time segmentation point of the first database and the second database; S24. Determine incremental data based on the time segmentation point, wherein the incremental data is the data generated after the time segmentation point until the data migration is completed; S25. Obtain the full data and the incremental data in chronological order.

4. The method according to claim 2, characterized in that, The step S4 of generating the first differential customer information based on the first differential customer tagging information further includes: S41. Generate first automatically differentiated customer information according to the first preset rule and the first differentiated customer tag information, wherein the first preset rule is an automatically generated data rule that is consistent with the format of the first customer information; S42. Traverse the content information of the first customer information and determine whether the first automatically differentiated customer information and the first customer information have the same content information. If there is no same content information, directly mark the first automatically differentiated customer information as the first differentiated customer information. S43. If there is identical content information, then based on the first preset rule, generate automatic difference information that is different from the content information of the first customer information again, replace the identical content information in the first automatic difference customer information, and mark the first automatic difference information that has no identical content information with the first customer information as the first difference customer information. S44. Mark the generated first difference customer information.

5. The method according to claim 2, characterized in that, Step S5 further includes: S51. Obtain and load table task information, wherein the table task information includes first customer information, first differential customer information, splitting field and database field information; S52. Divide the first customer information and the first differential customer information into a first number of slices according to the segmentation field in the task information of the table, and request the distributed system to establish a mapping between each slice and the second customer tag information; S53. Based on the database sharding field in the task information of the table, shard each slice to obtain a second number of database sharding tables, and request the distributed system to establish the mapping between each database sharding table and the second customer tag information. S54. Write the sharded database tables into the distributed database.

6. The method according to claim 2, characterized in that, After step S5, the method further includes storing the second customer tag information in the fourth database, storing the first customer information and the first differential customer information in the third database, comparing the third database with the first database, and checking whether there is any missing or incomplete data during the data migration process.

7. The method according to claim 2, characterized in that, Step S5 is followed by replacing or deleting the first difference customer information and / or the second tag information corresponding to the first difference customer information.

8. The method according to claim 1, characterized in that, The method also includes connecting to the database of the data provider via a link address to obtain data information from the database.

9. A data migration device for a credit card system, characterized in that, include: The first acquisition module is used to acquire data migration instructions, which include full migration, incremental migration, and full-incremental migration. Specifically, based on the data migration instruction, the table information of the data to be migrated is determined, and the data to be migrated is divided into a full synchronization table and / or an incremental synchronization table. When the data to be migrated is divided into a full synchronization table and an incremental synchronization table, the migration type of the data to be migrated is determined to be full-incremental migration; when the migration type of the data to be migrated is determined to be full-incremental migration, the migration conditions of the full synchronization table and the incremental synchronization table are configured. Establish the migration link between the full synchronization table and the incremental synchronization table; The data to be migrated is migrated based on the migration conditions and the migration link; The second acquisition module is used to acquire the first customer information in the first database and the first customer tag information in the second database according to the data migration instruction. The marking module is used to determine whether there is corresponding first customer information for the first customer marking information. If there is no corresponding first customer information for the first customer marking information, the first customer marking information without corresponding first customer information is marked as first differential customer marking information. The generation module is used to generate first differentiated customer information based on first differentiated customer tagging information, and to generate second customer tagging information based on first differentiated customer information and first customer information. The storage module is used to store the first customer information and the first differential customer information in separate databases and tables in the third database based on the second customer tag information, and to save the mapping relationship between the second customer tag information and the first customer information and the first differential customer information. The first database includes a credit card customer information database for data migration originating from the ...

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-8.

Images