Production data partition migration method and device

Abstract

The invention provides a production data partition migration method and device, relates to the technical field of big data, and can be applied to the financial field, the intelligent operation and maintenance field or other technical fields. The method comprises the following steps: determining target production data needing to be subjected to partition migration according to production data subjected to day-end batch processing; importing the target production data into a shared volume file by executing partition data migration tasks corresponding to the parallel threads respectively; and performing partition migration on the target production data from the shared volume file, and migrating the target production data from a source database partition to a target database partition. The device executes the method. According to the production data partition migration method and device provided by the embodiment of the invention, the data partition storage space can be saved, and the time consumption in the production data migration process is reduced.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a method and device for migrating production data partitions. Background Art

[0002] Large commercial organizations or other data operation organizations that use mainframe systems often have a wide range of actual business operations, or their operations are distributed across multiple countries or regions, resulting in a large amount of data. To ensure the transaction response of the core system and the efficiency of database operation and maintenance, the data center to which it belongs uses a mainframe architecture system for data storage, and generally chooses a DB2 multi-partition database architecture, which collaboratively supports the operation of the headquarters' business in all regions through multi-partition data storage.

[0003] Correspondingly, during the preparation of the functional development and testing environment in the R&D department of the headquarters of the organization, in order to maintain consistency with the production data distribution, a large amount of hardware disk capacity used by the host DB2 database is required for data partition storage, and the production data migration process takes too long. Summary of the invention

[0004] In view of the problems in the prior art, an embodiment of the present invention provides a method and device for migrating production data partitions, which can at least partially solve the problems in the prior art.

[0005] In one aspect, the present invention provides a method for migrating production data partitions, comprising:

[0006] Determine the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing;

[0007] Importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread;

[0008] The target production data is partition-migrated from the shared volume file, and the target production data is migrated from the source database partition to the destination database partition.

[0009] The step of determining the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing includes:

[0010] The production data is deformed, data is extracted from the deformed production data, and the extracted production data is used as the target production data.

[0011] The step of extracting data from the deformed production data includes:

[0012] Extract data from the production data after deformation processing according to the preset extraction principle;

[0013] The preset extraction principles include at least one of a first extraction principle based on the region to which the business belongs, a second extraction principle based on the business product category number, and a third extraction principle based on a designated customer group.

[0014] The step of importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread includes:

[0015] The target production data is imported into the shared volume file by executing the monitoring and prediction target partition capacity task corresponding to the first parallel thread.

[0016] The step of importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread includes:

[0017] The target production data is imported into the shared volume file by executing the table data monitoring and data integration tasks corresponding to the second parallel thread.

[0018] The executing of the table data monitoring and data integration tasks corresponding to the second parallel thread includes:

[0019] Acquire partial target production data with the same key value content but different non-key value information;

[0020] Acquire the data field type of the partial target production data;

[0021] If it is determined that the data field type is associated with a business keyword reflecting a business function, the portion of the target production data is imported into a shared volume file.

[0022] The production data partition migration method further includes:

[0023] If it is determined that there is no association relationship between the data field type and the business keyword reflecting the business function, the part of the target production data is not imported into the shared volume file.

[0024] In one aspect, the present invention provides a production data partition migration device, comprising:

[0025] A determination unit, used to determine target production data that needs to be partitioned and migrated according to production data that has completed end-of-day batch processing;

[0026] An import unit, configured to import the target production data into a shared volume file by executing partition data migration tasks corresponding to each parallel thread;

[0027] The migration unit is used to perform partition migration on the target production data from the shared volume file, and migrate the target production data from the source database partition to the destination database partition.

[0028] In another aspect, an embodiment of the present invention provides an electronic device, including: a processor, a memory, and a bus, wherein:

[0029] The processor and the memory communicate with each other via the bus;

[0030] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the following method:

[0031] Determine the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing;

[0032] Importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread;

[0033] The target production data is partition-migrated from the shared volume file, and the target production data is migrated from the source database partition to the destination database partition.

[0034] An embodiment of the present invention provides a non-transitory computer-readable storage medium, including:

[0035] The non-transitory computer-readable storage medium stores computer instructions, which cause the computer to execute the following method:

[0036] Determine the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing;

[0037] Importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread;

[0038] The target production data is partition-migrated from the shared volume file, and the target production data is migrated from the source database partition to the destination database partition.

[0039] The production data partition migration method and device provided in the embodiment of the present invention determine the target production data that needs to be partitioned migrated according to the production data that has completed the end-of-day batch processing; import the target production data into a shared volume file by executing the partition data migration tasks corresponding to each parallel thread; partition migrate the target production data from the shared volume file, and migrate the target production data from the source database partition to the destination database partition, which can save data partition storage space and reduce the time consumed in the production data migration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0041] Figure 1 It is a flowchart of a production data partition migration method provided by an embodiment of the present invention.

[0042] Figure 2 It is a flowchart of a production data partition migration method provided by another embodiment of the present invention.

[0043] Figure 3 It is a schematic diagram of the structure of a production data partition migration device provided by an embodiment of the present invention.

[0044] Figure 4 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other arbitrarily.

[0046] Figure 1 FIG. 1 is a flow chart of a method for migrating production data partitions provided by an embodiment of the present invention. Figure 1 As shown, the production data partition migration method provided by the embodiment of the present invention includes:

[0047] Step S1: Determine the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing.

[0048] Step S2: importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread.

[0049] Step S3: performing partition migration on the target production data from the shared volume file, migrating the target production data from the source database partition to the destination database partition.

[0050] In the above step S1, the device determines the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing. The device can be a computer device that executes the method, for example, it can include a server. It should be noted that the information collected in the embodiments of the present invention is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0051] Provide users with corresponding operation portals for them to choose to agree or reject the automated decision results; if the user chooses to reject, the expert decision process will be entered. Production data refers to the data in the production environment database. Partition migration refers to migrating the production environment database from the source database partition to the destination database partition.

[0052] Determining the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing includes:

[0053] The production data is deformed, data is extracted from the deformed production data, and the extracted production data is used as the target production data.

[0054] The deforming the production data comprises:

[0055] Production data contains a large amount of sensitive customer information. If the data is used for testing, there may be a risk of sensitive information leakage. Sensitive information needs to be transformed before being provided to the test stage, including customer-related sensitive information (such as name, contact number, ID number, etc.), system-related sensitive information (including IP, domain name, etc.), and in-bank user-related sensitive information (in-bank personnel such as account manager information, EMAIL address, etc.). Data desensitization and deformation processing needs to be carried out according to certain rules, so that the actual production data cannot be directly displayed, and the deformed data can be used for testing. For example, the name "AAA" is changed to "BBB" according to the rules, and the contact number or ID number is reorganized into another set of data according to certain rules, and the rules are stored in a secret key manner.

[0056] The step of extracting data from the deformed production data includes:

[0057] Extract data from the production data after deformation processing according to the preset extraction principle;

[0058] The preset extraction principle includes at least one of a first extraction principle based on the business region, a second extraction principle based on the business product category number, and a third extraction principle based on the designated customer group. For different extraction rules, extraction rules that meet the current data distribution status and extraction procedures can be specified to achieve automatic and rapid data extraction.

[0059] like Figure 2 As shown, before the step of deforming the production data, the method further includes:

[0060] The production data is backed up and stored.

[0061] Generally speaking, large enterprises have deployed global or regional centralized data centers, which continuously operate and maintain the production environment and access business data through online and batch methods. Business data is centrally deployed in a multi-partition DB2 database environment. In order to ensure business continuity and reliable data recovery solutions in case of sudden failures, the automatic backup of the core DB2 data of the host will generally be started after the batch processing at the end of the day. The backed-up data is stored in the form of "XXXX-20240305".

[0062] In the above step S2, the device imports the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread. Applications in the Pod can also access the shared volume, which is part of the Pod definition and can be used to mount to the file system of each application.

[0063] In Kubernetes, you can use shared Kubernetes volumes as a simple and effective way to share data between containers in a Pod. In most cases, it is sufficient to share a directory on the host that all containers in a Pod use.

[0064] Kubernetes Volumes enable data to be restored after a container restart, but these volumes have the same lifetime as the Pod. This means that the volume (and the data it stores) exists exactly as long as the Pod exists. If the Pod is deleted for any reason, the shared volume will be destroyed and created from scratch, even if an identical replacement is created.

[0065] The standard use case for a multi-container Pod with a shared volume is when one container writes logs or other files to a shared directory and another container reads from the shared directory.

[0066] The step of importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread includes:

[0067] The target production data is imported into the shared volume file by executing the task of monitoring and predicting the target partition capacity corresponding to the first parallel thread. The task of monitoring and predicting the target partition capacity can be understood as follows: if the target partition capacity is insufficient, the table space needs to be quickly expanded in advance, and if necessary, the existing data of the original partition can be compressed and reorganized first. The specific instructions are as follows:

[0068] The current target partition capacity (i.e., the storage space of the destination database partition) can be monitored and the target partition capacity for a period of time in the future can be predicted. If it is determined that the target partition capacity is less than the preset capacity threshold, the target partition capacity is first expanded, and it is predicted whether the target partition capacity after the expansion can accommodate the data to be migrated. Usually, there is a limit on the expanded capacity. If it is predicted within the limit that the target partition capacity after the expansion cannot accommodate the data to be migrated, it is necessary to compress all or part of the data to be migrated so that the compressed data to be migrated can be migrated to the destination database partition after the expansion.

[0069] The step of importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread includes:

[0070] The target production data is imported into the shared volume file by executing the table data monitoring and data integration tasks corresponding to the second parallel thread.

[0071] The executing of the table data monitoring and data integration tasks corresponding to the second parallel thread includes:

[0072] Acquire partial target production data with the same key value content but different non-key value information;

[0073] The data field type of the partial target production data is obtained; for example, a remark information field.

[0074] If it is determined that the data field type is associated with a business keyword reflecting a business function, the part of the target production data is imported into the shared volume file. Business keywords can be defined independently, and the configured tool can be used to determine whether the data field type is associated with the business keyword. The above tool can include a knowledge graph structure, that is, if there are nodes connected by edges, it is considered that there is an association relationship between the nodes; if there are no nodes connected by edges, it is considered that there is no association relationship between the nodes. Nodes can include data field types and business keywords.

[0075] The production data partition migration method also includes:

[0076] If it is determined that there is no association relationship between the data field type and the business keyword reflecting the business function, the part of the target production data is not imported into the shared volume file.

[0077] The instructions are as follows:

[0078] For the case where the key value content is the same but the non-key value information is different and does not need to be moved in:

[0079] Due to the different opening times of overseas branches and the different establishment times of different host partitions, some data information is not unified. For example, the interbank number table (for overseas regional institutions designated to receive funds for cross-border remittances) has some identical bank numbers but different remarks. This information does not affect the main business functions and can be based on the data of the destination database partition. The transferred data will not be processed. This situation can be included in the automatic detection and processing tools and included in the fixed list for processing.

[0080] For situations where the key value content is the same but the non-key value information is different and needs to be moved in:

[0081] Referring to the above example, if the information affects the main business functions, it is necessary to update the destination database partition data based on the data to be moved in. This situation can be included in the automatic detection and processing tools and included in the fixed list processing.

[0082] The executing of the table data monitoring and data integration tasks corresponding to the second parallel thread includes:

[0083] Monitor the data consistency in different data partitions, and do not import data in data partitions with consistent data. The instructions are as follows:

[0084] For example, completely consistent data information is stored in different data partitions (Asia Pacific, Europe and Africa, and Americas) of commercial banks operating globally, such as bank product categories (current deposits, time deposits, personal loans, etc.), or global national financial work calendars (such as Chinese holidays, American holidays, Japanese holidays, etc., used for cross-border remittances to determine holiday extensions, etc.). For this type of information, automatic detection and processing tools can be developed to include existing situations in the whitelist and use fixed logic to eliminate them, and further, they can be manually reviewed without the need for import.

[0085] The executing of the table data monitoring and data integration tasks corresponding to the second parallel thread includes:

[0086] If it is determined that the table structure does not have a unique key value, delete at least one piece of data that does not have a unique key value. If the table structure does not have a unique key value, it means that there are duplicate records, which can be deleted after manual confirmation, that is, data deduplication, just delete one or more records. System and application evaluations are based on different types of differentiated processing strategies to ensure smooth joint testing after migration. This situation can be included in the automatic detection and processing tools and included in the fixed list for processing.

[0087] like Figure 2 As shown, the target production data can be partition-migrated from "forming the extracted data" via the path of "importing the source data from the original DB2 partition to the shared volume file".

[0088] The target production data can also be migrated from partitions through the parallel paths of "forming the extracted data" through the parallel "importing the source data from the original DB2 partition into the shared volume file", "statistical registration and organization of table information" (table data monitoring and data integration tasks) and "monitoring and predicting the target partition capacity, and implementing early expansion when necessary". Through parallel thread task processing, on the one hand, it can dynamically save data partition storage space through real-time execution of table data monitoring and data integration tasks; on the other hand, it can reduce the time spent in the production data migration process.

[0089] In the above step S3, the device performs partition migration on the target production data from the shared volume file, and migrates the target production data from the source database partition to the destination database partition. Figure 2 As shown, after the partition migration is completed, the record data in the target table can also be statistically checked, so as to achieve complete and effective data migration across DB2 partitions into the target partition database.

[0090] The production data partition migration method provided in the embodiment of the present invention determines the target production data that needs to be partitioned migrated according to the production data that has completed the end-of-day batch processing; imports the target production data into a shared volume file by executing the partition data migration tasks corresponding to each parallel thread; partitions the target production data from the shared volume file, and migrates the target production data from the source database partition to the destination database partition, which can save data partition storage space and reduce the time consumed in the production data migration process.

[0091] Furthermore, the step of determining the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing includes:

[0092] The production data is deformed, data is extracted from the deformed production data, and the extracted production data is used as the target production data. The above embodiments can be referred to for explanation, and will not be described in detail.

[0093] The production data partition migration method provided by the embodiment of the present invention can obtain target production data as needed and ensure data security.

[0094] Furthermore, the extracting of data from the production data after deformation processing includes:

[0095] Data extraction is performed on the production data after deformation processing according to the preset extraction principle; the description can be made with reference to the above embodiment and will not be repeated here.

[0096] The preset extraction principle includes at least one of a first extraction principle based on the business region, a second extraction principle based on the business product category number, and a third extraction principle based on a designated customer group.

[0097] The production data partition migration method provided by the embodiment of the present invention can further achieve the acquisition of target production data as needed.

[0098] Furthermore, the step of importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to the parallel threads respectively includes:

[0099] The target production data is imported into the shared volume file by executing the monitoring and prediction target partition capacity task corresponding to the first parallel thread.

[0100] The production data partition migration method provided by the embodiment of the present invention can obtain the target partition capacity in real time and accurately, so as to integrate the target production data in advance, that is, not importing part of the target production data into the shared volume file.

[0101] Furthermore, the step of importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to the parallel threads respectively includes:

[0102] The target production data is imported into the shared volume file by executing the table data monitoring and data integration tasks corresponding to the second parallel thread. The above embodiments can be referred to for explanation and will not be described in detail.

[0103] The production data partition migration method provided by the embodiment of the present invention can further save data partition storage space and reduce the time consumption in the production data migration process.

[0104] Furthermore, the execution of the table data monitoring and data integration tasks corresponding to the second parallel thread includes:

[0105] Acquire partial target production data with the same key value content but different non-key value information; refer to the above embodiment for description and no further details will be given.

[0106] The data field type of the partial target production data is obtained; this may be described with reference to the above embodiment and will not be described in detail.

[0107] If it is determined that the data field type is associated with a business keyword reflecting a business function, the part of the target production data is imported into the shared volume file. The above-mentioned embodiment can be referred to for explanation, and no further description is given.

[0108] The production data partition migration method provided by the embodiment of the present invention can ensure the effective availability of target production data.

[0109] Furthermore, the production data partition migration method also includes:

[0110] If it is determined that the data field type has no association with the business keyword reflecting the business function, the part of the target production data is not imported into the shared volume file. The above embodiment can be referred to for explanation and will not be described in detail.

[0111] The production data partition migration method provided by the embodiment of the present invention can further save data partition storage space.

[0112] It should be noted that the production data partition migration method provided in the embodiment of the present invention can be used in the financial field, and can also be used in any technical field other than the financial field. The embodiment of the present invention does not limit the application field of the production data partition migration method.

[0113] Figure 3 FIG. 1 is a schematic diagram of the structure of a production data partition migration device provided by an embodiment of the present invention. Figure 3 As shown, the production data partition migration device provided by the embodiment of the present invention includes a determination unit 301, an import unit 302 and a migration unit 303, wherein:

[0114] The determination unit 301 is used to determine the target production data that needs to be partitioned according to the production data that has completed the end-of-day batch processing; the import unit 302 is used to import the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread; the migration unit 303 is used to partition the target production data from the shared volume file, and migrate the target production data from the source database partition to the destination database partition.

[0115] Specifically, the determination unit 301 in the device is used to determine the target production data that needs to be partitioned according to the production data that has completed the end-of-day batch processing; the import unit 302 is used to import the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread; the migration unit 303 is used to partition the target production data from the shared volume file, and migrate the target production data from the source database partition to the destination database partition.

[0116] The production data partition migration device provided in the embodiment of the present invention determines the target production data that needs to be partitioned migrated according to the production data that has completed the end-of-day batch processing; imports the target production data into a shared volume file by executing the partition data migration tasks corresponding to each parallel thread; partitions the target production data from the shared volume file, and migrates the target production data from the source database partition to the destination database partition, which can save data partition storage space and reduce the time consumed in the production data migration process.

[0117] Further, the determining unit 301 is specifically configured to:

[0118] The production data is deformed, data is extracted from the deformed production data, and the extracted production data is used as the target production data.

[0119] The production data partition migration device provided by the embodiment of the present invention can obtain target production data as needed and ensure data security.

[0120] Furthermore, the determining unit 301 is further specifically configured to:

[0121] Extract data from the production data after deformation processing according to the preset extraction principle;

[0122] The preset extraction principles include at least one of a first extraction principle based on the region to which the business belongs, a second extraction principle based on the business product category number, and a third extraction principle based on a designated customer group.

[0123] The production data partition migration device provided by the embodiment of the present invention can further achieve the acquisition of target production data as needed.

[0124] Furthermore, the import unit 302 is specifically used for:

[0125] The target production data is imported into the shared volume file by executing the monitoring and prediction target partition capacity task corresponding to the first parallel thread.

[0126] The production data partition migration device provided by the embodiment of the present invention can obtain the target partition capacity in real time and accurately, so as to integrate the target production data in advance, that is, not import part of the target production data into the shared volume file.

[0127] Furthermore, the import unit 302 is specifically used for:

[0128] The target production data is imported into the shared volume file by executing the table data monitoring and data integration tasks corresponding to the second parallel thread.

[0129] The production data partition migration device provided by the embodiment of the present invention can further save data partition storage space and reduce the time consumption in the production data migration process.

[0130] Furthermore, the import unit 302 is also specifically used for:

[0131] Acquire partial target production data with the same key value content but different non-key value information;

[0132] Acquire the data field type of the partial target production data;

[0133] If it is determined that the data field type is associated with a business keyword reflecting a business function, the portion of the target production data is imported into a shared volume file.

[0134] The production data partition migration device provided by the embodiment of the present invention can ensure the effective availability of target production data.

[0135] Furthermore, the production data partition migration device is also used for:

[0136] If it is determined that there is no association relationship between the data field type and the business keyword reflecting the business function, the part of the target production data is not imported into the shared volume file.

[0137] The production data partition migration device provided by the embodiment of the present invention can further save data partition storage space.

[0138] The embodiment of the production data partition migration device provided in the embodiment of the present invention can be specifically used to execute the processing flow of the above-mentioned method embodiments. Its functions are not repeated here, and reference can be made to the detailed description of the above-mentioned method embodiments.

[0139] Figure 4 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as Figure 4 As shown, the electronic device includes: a processor (processor) 401, a memory (memory) 402 and a bus 403;

[0140] The processor 401 and the memory 402 communicate with each other via a bus 403;

[0141] The processor 401 is used to call the program instructions in the memory 402 to execute the methods provided by the above method embodiments, for example, including:

[0142] Determine the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing;

[0143] Importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread;

[0144] The target production data is partition-migrated from the shared volume file, and the target production data is migrated from the source database partition to the destination database partition.

[0145] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can perform the methods provided by the above method embodiments, for example, including:

[0146] Determine the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing;

[0147] Importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread;

[0148] The target production data is partition-migrated from the shared volume file, and the target production data is migrated from the source database partition to the destination database partition.

[0149] This embodiment provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program enables the computer to execute the methods provided by the above method embodiments, for example, including:

[0150] Determine the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing;

[0151] Importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread;

[0152] The target production data is partition-migrated from the shared volume file, and the target production data is migrated from the source database partition to the destination database partition.

[0153] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0157] In the description of this specification, the description with reference to the terms "one embodiment", "a specific embodiment", "some embodiments", "for example", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0158] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for migrating production data partitions, characterized in that: include: Determine the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing; Importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread; The target production data is partition-migrated from the shared volume file, and the target production data is migrated from the source database partition to the destination database partition.

2. The method for migrating production data partitions according to claim 1, characterized in that: Determining the target production data that needs to be partitioned and migrated based on the production data that has completed the end-of-day batch processing includes: The production data is deformed, data is extracted from the deformed production data, and the extracted production data is used as the target production data.

3. The method for migrating production data partitions according to claim 2, characterized in that: The step of extracting data from the deformed production data includes: Extract data from the production data after deformation processing according to the preset extraction principle; The preset extraction principles include at least one of a first extraction principle based on the region to which the business belongs, a second extraction principle based on the business product category number, and a third extraction principle based on a designated customer group.

4. The method for migrating production data partitions according to any one of claims 1 to 3, characterized in that: The step of importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread includes: The target production data is imported into the shared volume file by executing the monitoring and prediction target partition capacity task corresponding to the first parallel thread.

5. The method for migrating production data partitions according to claim 4, characterized in that: The step of importing the target production data into the shared volume file by executing the partition data migration tasks corresponding to each parallel thread includes: The target production data is imported into the shared volume file by executing the table data monitoring and data integration tasks corresponding to the second parallel thread.

6. The method for migrating production data partitions according to claim 5, characterized in that: The executing of the table data monitoring and data integration tasks corresponding to the second parallel thread includes: Acquire partial target production data with the same key value content but different non-key value information; Acquire the data field type of the partial target production data; If it is determined that the data field type is associated with a business keyword reflecting a business function, the portion of the target production data is imported into a shared volume file.

7. The method for migrating production data partitions according to claim 6, characterized in that: The production data partition migration method also includes: If it is determined that there is no association relationship between the data field type and the business keyword reflecting the business function, the part of the target production data is not imported into the shared volume file.

8. A production data partition migration device, characterized in that: include: A determination unit, used to determine target production data that needs to be partitioned and migrated according to production data that has completed end-of-day batch processing; An import unit, configured to import the target production data into a shared volume file by executing partition data migration tasks corresponding to each parallel thread; The migration unit is used to perform partition migration on the target production data from the shared volume file, and migrate the target production data from the source database partition to the destination database partition.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images