Data processing method and device in system migration scene, equipment and storage medium

Abstract

The embodiment of the invention provides a data processing method and device in a system migration scene, equipment and a storage medium. The method can be used in the distributed technology field. The method comprises the steps that after a first processing request sent by a front end is received, fragmentation key information contained in data requested to be processed by the first processing request is determined; based on the fragment key information, determining a data migration state and data position information corresponding to data requested to be processed by the first processing request; based on the data migration state, after a target system where data requested to be processed by the first processing request is located is determined, the first processing request is routed to the target system based on the data position information, so that the data requested to be processed by the first processing request is processed based on the target system, and a data processing result is obtained. According to the method, the data can be simply and efficiently positioned in the transition period of system transformation.

Description

Technical Field

[0001] The present application relates to the field of distributed technology, and in particular to a data processing method, device, equipment and storage medium in a system migration scenario. Background Art

[0002] With the digital transformation of banking systems and the localization of information technology, the business systems of various enterprises are gradually shifting from traditional monolithic architecture to distributed architecture. In the process of system transformation, there will be coexistence of new and old systems. Therefore, during the transition period of system transformation, how to accurately locate and process business data in the new and old systems has become a technical problem that needs to be solved urgently.

[0003] In the related art, data processing is mainly completed by setting data location rules in the front-end system and locating the storage location of the business data in the new system or the old system according to the data location rules.

[0004] This implementation method not only requires setting corresponding data location rules on each front end, but also requires setting separate judgment logic for various types of business data. Not only is the data location method complicated, it also causes a large workload, thereby affecting data processing efficiency. Summary of the invention

[0005] The embodiments of the present application provide a data processing method, apparatus, device and storage medium in a system migration scenario, so as to achieve the effect of simply and efficiently locating data from the old and new systems during the transition period of system transformation.

[0006] In a first aspect, an embodiment of the present application provides a data processing method in a system migration scenario, which is applied to a computer device, including:

[0007] After receiving the first processing request sent by the front end, determining the shard key information of the data requested to be processed by the first processing request; wherein the first processing request indicates a request to process the data;

[0008] Based on the shard key information, determine the data migration status and data location information corresponding to the data requested to be processed by the first processing request; wherein the data migration status is used to indicate whether the data requested to be processed is migrated;

[0009] Based on the data migration status, after determining the target system where the data requested to be processed by the first processing request is located, the first processing request is routed to the target system based on the data location information, so as to process the data requested to be processed by the first processing request based on the target system to obtain a data processing result.

[0010] In a possible implementation, determining, based on the shard key information, data migration status and data location information corresponding to the data requested to be processed by the first processing request includes:

[0011] Determine, based on the first shard key included in the shard key information, a storage unit corresponding to the data requested to be processed by the first processing request, and determine the data location information based on the storage unit;

[0012] If it is determined that the shard key information includes the second shard key, it is determined that the data migration status indicates that the requested data has been migrated; otherwise, it is determined that the data migration status indicates that the requested data has not been migrated.

[0013] In a possible implementation, before receiving the first processing request sent by the front end, the method further includes:

[0014] Get preset migration dimension information;

[0015] Based on the preset migration dimension information, acquiring the data to be migrated from the first system;

[0016] After determining the first sharding key based on the data key information of the data to be migrated and determining the second sharding key based on the preset migration dimension information, the data to be migrated is migrated from the first system to the second system.

[0017] In a possible implementation, determining, based on the data migration status, a target system where the data requested to be processed by the first processing request is located includes:

[0018] If the data migration status indicates that the data requested for processing is migrated, determining the second system as the target system;

[0019] If the data migration status indicates that the data requested for processing has not been migrated, the first system is determined as the target system.

[0020] In a possible implementation, determining the shard key information of the data requested to be processed by the first processing request includes:

[0021] Obtain a preset mapping relationship table; wherein the preset mapping relationship table includes a correspondence between data and shard key information;

[0022] Based on the preset mapping relationship table, determine the shard key information of the data requested to be processed by the first processing request.

[0023] In a possible implementation manner, after receiving a second processing request sent by the target system and obtaining a data processing result, the method further includes:

[0024] receiving a second processing request sent by the target system; wherein the second processing request includes a third sharding key; the third sharding key indicates a sharding key updated based on the first processing request;

[0025] Based on the third shard key included in the second processing request, shard key information of the data requested to be processed by the first processing request is updated.

[0026] In a possible implementation, the method further includes:

[0027] Receiving a third processing request sent by the target system at a preset time interval; wherein the third processing request includes a fourth sharding key; the fourth sharding key indicates a sharding key that is updated within the preset time interval;

[0028] Based on the fourth shard key, update processing is performed on the shard key information that has not been updated within the preset time interval.

[0029] In a second aspect, an embodiment of the present application provides a data processing device in a system migration scenario, including:

[0030] A first determining unit is configured to determine, after receiving a first processing request sent by a front end, shard key information of data requested to be processed by the first processing request; wherein the first processing request indicates a request to process data;

[0031] A second determining unit is used to determine, based on the shard key information, data migration status and data location information corresponding to the data requested to be processed by the first processing request; wherein the data migration status is used to indicate whether the data requested to be processed is migrated;

[0032] A processing unit is used to determine, based on the data migration status, a target system where the data requested to be processed by the first processing request is located, and then, based on the data location information, route the first processing request to the target system to process the data requested to be processed by the first processing request based on the target system to obtain a data processing result.

[0033] In a possible implementation manner, the second determining unit is configured to:

[0034] Determine, based on the first shard key included in the shard key information, a storage unit corresponding to the data requested to be processed by the first processing request, and determine the data location information based on the storage unit;

[0035] If it is determined that the shard key information includes the second shard key, it is determined that the data migration status indicates that the requested data has been migrated; otherwise, it is determined that the data migration status indicates that the requested data has not been migrated.

[0036] In a possible implementation manner, the device further includes an initialization unit, configured to:

[0037] Before receiving the first processing request sent by the front end, obtaining preset migration dimension information;

[0038] Based on the preset migration dimension information, acquiring the data to be migrated from the first system;

[0039] After determining the first sharding key based on the data key information of the data to be migrated and determining the second sharding key based on the preset migration dimension information, the data to be migrated is migrated from the first system to the second system.

[0040] In a possible implementation, the processing unit is configured to:

[0041] If the data migration status indicates that the data requested for processing is migrated, determining the second system as the target system;

[0042] If the data migration status indicates that the data requested for processing has not been migrated, the first system is determined as the target system.

[0043] In a possible implementation manner, the first determining unit is configured to:

[0044] Obtain a preset mapping relationship table; wherein the preset mapping relationship table includes a correspondence between data and shard key information;

[0045] Based on the preset mapping relationship table, determine the shard key information of the data requested to be processed by the first processing request.

[0046] In a possible implementation manner, when the data migration status indicates that the data requested for processing is migrated, the device further includes a data synchronization unit, which is configured to:

[0047] After obtaining the data processing result, receiving a second processing request sent by the target system; wherein the second processing request includes a third sharding key; the third sharding key indicates a sharding key updated based on the first processing request;

[0048] Based on the third shard key included in the second processing request, shard key information of the data requested to be processed by the first processing request is updated.

[0049] In a possible implementation manner, the data synchronization unit is further configured to:

[0050] Receiving a third processing request sent by the target system at a preset time interval; wherein the third processing request includes a fourth sharding key; the fourth sharding key indicates a sharding key that is updated within the preset time interval;

[0051] Based on the fourth shard key, update processing is performed on the shard key information that has not been updated within the preset time interval.

[0052] In a third aspect, an embodiment of the present application provides a computer device, including: a memory, a processor;

[0053] The memory stores computer-executable instructions;

[0054] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0055] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0056] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0057] The data processing method, device, equipment and storage medium in the system migration scenario provided by the embodiment of the present application can modify the shard key information of the data, so that the shard key information of the data can be located from the system dimension and the unit dimension. In this case, after receiving the first processing request sent by the front end, the shard key information of the data requested to be processed by the first processing request can be determined first, and then, according to the shard key information, the data migration state and data location information corresponding to the data requested to be processed by the first processing request can be determined. At this time, the target system where the data requested to be processed by the first processing request is located can be determined based on the data migration state, so that when the system is in a transition period, the target system where the data requested to be processed can be located can be simply, accurately and quickly located, thereby improving the data positioning speed and accuracy, thereby improving the speed and accuracy of data processing. Afterwards, according to the data location information, the first processing request is routed to the target system, so as to process the data requested to be processed by the first processing request based on the target system, and obtain the data processing result. This implementation method can complete the positioning of data in a single node, thereby reducing the probability of error, further improving the accuracy and speed of data positioning, and further improving the speed and efficiency of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0059] Figure 1 A schematic diagram of a system migration scenario provided for this application;

[0060] Figure 2 A flowchart of a data processing method in a system migration scenario provided by this application;

[0061] Figure 3 A flowchart of another data processing method in a system migration scenario provided by this application;

[0062] Figure 4 A schematic diagram of an implementation flow of a data processing method in a system migration scenario provided in this application;

[0063] Figure 5 A schematic diagram of the structure of a data processing device in a system migration scenario provided by the present application;

[0064] Figure 6 A schematic diagram of the structure of a data processing device in another system migration scenario provided by the present application;

[0065] Figure 7 A schematic diagram of the structure of a computer device provided in this application.

[0066] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0067] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0068] It should be noted that the data processing methods, devices, equipment and storage media in the system migration scenario provided by the present application can be used in the field of distributed technology, in the field of big data technology, or in any field other than the above-mentioned technical fields. The application fields of the data processing methods, devices, equipment and storage media in the system migration scenario provided by the present application are not limited here.

[0069] First, the terms involved in this application are explained:

[0070] Unit positioning: Provides routing mapping service capabilities for unit routing under the unitized architecture;

[0071] Unit: In a unitized architecture, a unit is a self-contained collection that can complete all business operations. This collection contains all services required by all businesses and the data assigned to this unit.

[0072] Unit positioning module: that is, the T-UPC module shown below, responsible for completing the unit positioning route mapping action;

[0073] Route mapping: that is, finding the mapping relationship of the corresponding unit through the shard key in a distributed, unitized architecture;

[0074] Hash: The process of transforming input information into an output of fixed length through a series of algorithms. The purpose is to quickly find or establish a corresponding relationship. Common algorithms include square method, random method, modulus method, etc.

[0075] SDK: Software Development Kit, a software package with certain business functions after compilation and debugging, which can be migrated to other programs that need to reference this function to avoid repeated development and reduce development costs;

[0076] Sharding key: the key value data used to quickly locate the data location in the sharding strategy;

[0077] Sharding: The global business data is divided according to certain rules to facilitate distributed storage. After the table is divided, the data location can be quickly located through the shard key;

[0078] Online real-time synchronization: a data synchronization method that ensures data consistency, which means informing the target party of data synchronization through online calls;

[0079] Asynchronous message synchronization is another data synchronization method that ensures data consistency. It refers to sending the data to be synchronized to the target party through a queue through message middleware such as MQ (MessageQueue).

[0080] End-of-day verification: Verify the incremental data changes of the day based on the end-of-day batch text, which can make up for the data inconsistency caused by online real-time synchronization or asynchronous message synchronization;

[0081] Front-end system: a human-computer interaction system provided to external customers, such as ATM (Automated Teller Machine), mobile banking APP (Application), bank teller operation terminal system, etc.

[0082] Backend system: performs calculations, storage, and other processing on business data, and provides feedback to the frontend system.

[0083] With the digital transformation of banking systems and the localization of information technology, the business systems of various enterprises are gradually shifting from traditional monolithic architecture to distributed architecture. For large-scale business systems with large data volumes, the old and new systems will coexist during the system transformation process.

[0084] See also Figure 1 , Figure 1 A schematic diagram of a system migration scenario provided for this application, such as Figure 1 As shown, in the system migration scenario, part of the data can be migrated from the first system (i.e., the old system) to the second system (i.e., the new system) according to certain data migration rules (e.g., migration by region, migration by card number, etc.). At this time, part of the data is stored in the first system, and the other part of the data is stored in the second system. At this time, after receiving the data processing request sent by the front end, in order to prevent the location data from being searched in both systems, it is necessary to first determine whether to route the data processing request to the first system (see the solid line shown in body 1) or the second system (see the dotted line shown in body 1) to perform data processing.

[0085] Therefore, during the transition period of system transformation, how to accurately locate and process business data in the old and new systems has become a technical problem that needs to be solved urgently.

[0086] In the related art, data processing is mainly completed by setting data location rules in the front-end system (for example, determining the processing logic of data in the new and old systems), and locating the storage location of the business data in the new system or the old system according to the data location rules.

[0087] This implementation method not only requires setting corresponding data location rules on each front end, but also requires adding judgment logic for different types of business data, which leads to high development costs and a large workload. In addition, this data location method has complex logic, which can easily affect the data location speed and thus the data processing efficiency. At the same time, when business data increases, it is also necessary to synchronously update the data location rules corresponding to the newly added business data on each front end. When there are a large number of front ends, synchronization inconsistency is prone to occur.

[0088] The data processing method in the system migration scenario provided in the present application determines the data migration status and data location information through shard key information, thereby determining the system where the data is located according to the data migration status, and determining the storage location of the data in the system according to the data location information, thereby realizing rapid positioning and processing of the data to solve the above-mentioned technical problems.

[0089] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0090] See also Figure 2 , Figure 2 The present application provides a flow chart of a data processing method in a system migration scenario. The method can be applied to a computer device. In this case, the computer device may include a unit positioning module (T-UPC, Tracking Unit Positioning Component), so as to complete the positioning and routing of data based on the unit positioning module, thereby completing the data processing. Figure 2 As shown, the method includes:

[0091] S201. After receiving a first processing request sent by a front end, determine shard key information of data requested to be processed by the first processing request.

[0092] The first processing request indicates a request to process data.

[0093] In an example, the first processing request may be a transaction request, a deposit request, a withdrawal request, a card application request, etc. The specific content of the first processing request is not limited here, and is based on meeting actual needs.

[0094] In one example, the first processing request includes at least the data requested to be processed, the details of the processing requested, etc. For example, when the first processing request is a deposit request for a bank card, the data requested to be processed by the first processing request is the card number, and the details of the processing requested are a deposit of XX yuan, etc.

[0095] In one example, the shard key information of the data requested to be processed by the first processing request can be used to locate the storage location of the data requested to be processed in the system.

[0096] S202: Based on the shard key information, determine the data migration status and data location information corresponding to the data requested to be processed by the first processing request.

[0097] The data migration status is used to indicate whether the requested data is migrated.

[0098] In one example, the shard key information may include at least one shard key. In this case, the data migration status and data location information corresponding to the data requested to be processed by the first processing request may be determined according to the shard key included in the shard key information.

[0099] In one example, each shard key included in the shard key information may be any one of text, numbers, and letters, and the representation form of the shard key is not limited here.

[0100] S203. After determining the target system where the data requested to be processed by the first processing request is located based on the data migration status, the first processing request is routed to the target system based on the data location information, so as to process the data requested to be processed by the first processing request based on the target system to obtain a data processing result.

[0101] In one example, the target system may indicate an old system before the migration, or may indicate a new system after the migration.

[0102] It can be known from the above description that the embodiment of the present application can modify the shard key information of the data, so that the shard key information of the data can be used to locate the data from the system dimension and the unit dimension. In this case, after receiving the first processing request sent by the front end, the shard key information of the data requested to be processed by the first processing request can be determined first, and then, according to the shard key information, the data migration status and data location information corresponding to the data requested to be processed by the first processing request can be determined. At this time, the target system where the data requested to be processed by the first processing request is located can be determined based on the data migration status, so that when the system is in a transition period, the target system where the data requested to be processed can be located can be simply, accurately and quickly located, thereby improving the data positioning speed and accuracy, thereby improving the speed and accuracy of data processing. Afterwards, based on the data location information, the first processing request is routed to the target system, so as to process the data requested to be processed by the first processing request based on the target system, and obtain the data processing result. This implementation method can complete the positioning of data in a single node, thereby reducing the probability of error, further improving the accuracy and speed of data positioning, and further improving the speed and efficiency of data processing.

[0103] See also Figure 3 , Figure 3 This is a flow chart of another data processing method in a system migration scenario provided by the present application. The method can be applied to a computer device, which may include a unit positioning module. Figure 3 As shown, the method includes:

[0104] S301: Obtain preset migration dimension information.

[0105] In one example, the preset migration dimension information may correspond to the preset data migration rules, for example, the preset migration dimension information may indicate the card number dimension, the region dimension, the customer whitelist dimension, etc. Among them, the card number dimension may indicate the range of card numbers to be migrated, for example, the card number dimension may indicate the migration of data corresponding to card numbers ending in 0001 to 0999; the region dimension may indicate the region range to be migrated, for example, the region dimension may indicate the migration of data under "YY Province"; the customer whitelist dimension may indicate the migration of data of customers included in the whitelist, etc. There is no limitation on the preset migration dimension information here, and it shall be based on meeting actual needs.

[0106] S302: Based on preset migration dimension information, obtain data to be migrated from the first system.

[0107] In one example, based on the migration range indicated by the preset migration dimension information, data within the migration range may be acquired from the first system, thereby acquiring the data to be migrated.

[0108] S303: After determining a first sharding key based on data key information of the data to be migrated and determining a second sharding key based on preset migration dimension information, migrate the data to be migrated from the first system to the second system.

[0109] In one example, the data key information may indicate information that uniquely identifies the data to be migrated. For example, if the data to be migrated is card number data, the data key information may be the card suffix, or if the data to be migrated is customer data, the data key information may be a customer identification number, etc. The data key information of the data to be migrated is not limited here. At this time, the data key information may be determined as the first sharding key.

[0110] In one example, the second shard key may correspond to a preset migration dimension. For example, when the preset migration dimension information is a regional dimension, it can be determined that the second shard key may be the "YY province" indicated by the regional dimension.

[0111] This implementation can add a second shard key indicating preset migration dimension information to the data, so that the system in the transition period can be located according to the second shard key. The implementation is simple and error-prone, which not only reduces the cost of data positioning, but also ensures the accuracy and efficiency of data positioning.

[0112] S304: After receiving the first processing request sent by the front end, determine the shard key information of the data requested to be processed by the first processing request.

[0113] The first processing request indicates a request to process data.

[0114] In specific implementation, after receiving the first processing request sent by the front end, a preset mapping relationship table can be obtained first; wherein the preset mapping relationship table includes the correspondence between data and sharding key information; then, based on the preset mapping relationship table, the sharding key information of the data requested to be processed by the first processing request is determined.

[0115] At this time, the speed of determining the shard key information of the data can be increased by presetting the comparison relationship between the data and its shard key information, that is, presetting the mapping relationship, thereby improving efficiency.

[0116] S305: Based on the shard key information, determine the data migration status and data location information corresponding to the data requested to be processed by the first processing request.

[0117] The data migration status is used to indicate whether the requested data is migrated.

[0118] In a specific implementation, the storage unit corresponding to the data requested to be processed by the first processing request can be determined based on the first shard key included in the shard key information, and the data location information can be determined based on the storage unit. Then, it is determined whether the shard key information includes the second shard key. If it is determined that the shard key information includes the second shard key, it is determined that the data migration status indicates that the data requested to be processed has been migrated; otherwise, it is determined that the data migration status indicates that the data requested to be processed has not been migrated.

[0119] In one example, when determining the storage unit corresponding to the data requested for processing based on the first shard key, the first shard key may be hashed to obtain a hash result, and the storage unit corresponding to the first shard key may be determined based on the hash result. Alternatively, the storage unit corresponding to the first shard key may be determined based on the value of the first shard key, etc. The method of determining the storage unit based on the first shard key is not limited here, and is subject to implementation.

[0120] S306: If the data migration status indicates that the requested data is to be processed, the second system is determined as the target system.

[0121] S307: If the data migration status indicates that the requested data has not been migrated, the first system is determined as the target system.

[0122] S308. Based on the data location information, the first processing request is routed to the target system, so that the data requested to be processed by the first processing request is processed based on the target system to obtain a data processing result.

[0123] In one example, the first processing request can be routed to a location in the target system corresponding to the data location information, so that the data at the location can be processed based on the target system, thereby processing the data requested by the first processing request and obtaining a data processing result.

[0124] Further, if it is determined that the data migration status indicates that the requested processed data has been migrated, after obtaining the data processing result according to the steps described above, data synchronization processing can be performed according to the process described below to ensure data consistency.

[0125] Method 1: Data synchronization is performed for each first processing request. At this time, data synchronization can be performed through online real-time synchronization or asynchronous message synchronization.

[0126] In specific implementation, a second processing request sent by the target system may be received, and then, based on the third sharding key included in the second processing request, sharding key information of the data requested to be processed by the first processing request may be updated.

[0127] The second processing request includes a third sharding key, and the third sharding key indicates a sharding key updated based on the first processing request.

[0128] In one embodiment, if the target system determines that the request type corresponding to the first processing request is the target type, a second processing request is generated and sent to the computer device (i.e., the unit positioning module), so that the computer device performs data synchronization based on the second processing request. At this time, after data processing based on the first processing request whose request type is the target type, the shard key of the data requested for processing needs to be updated. For example, the target type can indicate a newly added data type, a migrated data type, or a change in the second shard key, etc.

[0129] This implementation method can timely update the shard key information of the data, thereby ensuring the accuracy of data processing based on the next processing request.

[0130] Method 2: Data synchronization is performed on the first processing request of the batch.

[0131] In specific implementation, a third processing request sent by the target system at a preset time interval may be received, and then, based on the fourth sharding key, update processing may be performed on the sharding key information that has not been updated within the preset time interval.

[0132] The third processing request includes a fourth sharding key, and the fourth sharding key indicates a sharding key that is updated within a preset time interval.

[0133] In one example, the second method may correspond to the above-mentioned end-of-day verification method. In this case, the preset time interval is one day.

[0134] In an example, the preset time interval may also be two days, one week, etc., and the value of the preset time interval is not limited here, and is based on meeting actual needs.

[0135] In a possible implementation, when performing data synchronization on the first processing request of the batch, in addition to receiving the third processing request sent by the target system at a preset time interval, the fourth processing request sent by the target system according to the preset request processing volume can also be received, so that the unupdated shard key information is updated according to the fifth shard key included in the fourth processing request. The triggering conditions for batch data synchronization are not limited here, and are subject to what can be achieved.

[0136] See also Figure 4 , Figure 4 A schematic diagram of the implementation flow of a data processing method in a system migration scenario provided in this application, such as Figure 4As shown, when the system is switched for the first time, the data to be migrated can be obtained from the old system (i.e., the first system) according to the preset migration dimension information, and the first sharding key is determined based on the data key information of the data to be migrated, and the second sharding key is determined based on the preset migration dimension information, and then the data to be migrated is migrated from the first system to the new system (i.e., the second system). At the same time, the data to be migrated and the sharding key information of the data to be migrated are passed to the unit positioning module.

[0137] At this time, the unit location module can receive the first processing request sent by the front end, and determine the shard key information of the data requested to be processed by the first processing request. Figure 4 If the shard key information of the data requested to be processed in the request 1) does not include the second shard key, the first processing request is routed to the old system to complete data processing according to the first shard key included in the shard key information of the data requested to be processed. Figure 4 In the request 2 shown, if the shard key information of the data requested for processing includes the second shard key, the first processing request is routed to the new system to complete the data processing according to the first shard key included in the shard key information of the data requested for processing.

[0138] In one example, if the new system determines that the first processing request will change the shard key information of the data requested for processing, the updated shard key will be synchronized to the unit positioning module to perform data synchronization.

[0139] See also Figure 5 , Figure 5 A structural diagram of a data processing device in a system migration scenario provided by this application, such as Figure 5 As shown, the data processing device 500 in the system migration scenario provided by this embodiment includes:

[0140] The first determining unit 501 is used to determine the shard key information of the data requested to be processed by the first processing request after receiving the first processing request sent by the front end; wherein the first processing request indicates a request to process the data.

[0141] The second determination unit 502 is used to determine the data migration status and data location information corresponding to the data requested to be processed by the first processing request based on the shard key information; wherein the data migration status is used to indicate whether the data requested to be processed is migrated.

[0142] The processing unit 503 is used to determine the target system where the data requested to be processed by the first processing request is located based on the data migration status, and then route the first processing request to the target system based on the data location information, so as to process the data requested to be processed by the first processing request based on the target system to obtain a data processing result.

[0143] See also Figure 6 , Figure 6 A structural diagram of a data processing device in another system migration scenario provided by the present application, such as Figure 6 As shown, the data processing device 600 in the system migration scenario provided by this embodiment includes:

[0144] The first determining unit 601 is used to determine the sharding key information of the data requested to be processed by the first processing request after receiving the first processing request sent by the front end; wherein the first processing request indicates a request to process the data.

[0145] The second determination unit 602 is used to determine the data migration status and data location information corresponding to the data requested to be processed by the first processing request based on the shard key information; wherein the data migration status is used to indicate whether the data requested to be processed is migrated.

[0146] The processing unit 603 is used to determine the target system where the data requested to be processed by the first processing request is located based on the data migration status, and then route the first processing request to the target system based on the data location information, so as to process the data requested to be processed by the first processing request based on the target system to obtain a data processing result.

[0147] In a possible implementation manner, the second determining unit 602 is configured to:

[0148] Based on the first shard key included in the shard key information, determine the storage unit corresponding to the data requested to be processed by the first processing request, and determine the data location information based on the storage unit;

[0149] If it is determined that the shard key information includes the second shard key, it is determined that the data migration status indicates that the data requested for processing has been migrated; otherwise, it is determined that the data migration status indicates that the data requested for processing has not been migrated.

[0150] In a possible implementation manner, the device further includes an initialization unit 604, configured to:

[0151] Before receiving the first processing request sent by the front end, obtaining preset migration dimension information;

[0152] Based on the preset migration dimension information, the data to be migrated is obtained from the first system;

[0153] After determining a first sharding key based on data key information of the data to be migrated and determining a second sharding key based on preset migration dimension information, the data to be migrated is migrated from the first system to the second system.

[0154] In a possible implementation, the processing unit 603 is configured to:

[0155] If the data migration status indicates that the data requested for processing is migrated, determining the second system as the target system;

[0156] If the data migration status indicates that the data requested for processing has not been migrated, the first system is determined as the target system.

[0157] In a possible implementation manner, the first determining unit 601 is configured to:

[0158] Obtain a preset mapping relationship table; wherein the preset mapping relationship table includes a correspondence between data and shard key information;

[0159] Based on the preset mapping relationship table, shard key information of the data requested to be processed by the first processing request is determined.

[0160] In a possible implementation manner, when the data migration status indicates that the data requested for processing is migrated, the device further includes a data synchronization unit 605, which is configured to:

[0161] After obtaining the data processing result, receiving a second processing request sent by the target system; wherein the second processing request includes a third sharding key; the third sharding key indicates a sharding key updated based on the first processing request;

[0162] Based on the third shard key included in the second processing request, shard key information of the data requested to be processed by the first processing request is updated.

[0163] In a possible implementation manner, the data synchronization unit 605 is further configured to:

[0164] Receiving a third processing request sent by the target system at a preset time interval; wherein the third processing request includes a fourth sharding key; the fourth sharding key indicates a sharding key that is updated within the preset time interval;

[0165] Based on the fourth shard key, update processing is performed on the shard key information that has not been updated within a preset time interval.

[0166] The data processing device in the system migration scenario provided in this embodiment can execute the data processing method in the system migration scenario provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.

[0167] See also Figure 7, Figure 7 This is a schematic diagram of the structure of a computer device provided in this application. Figure 7 As shown, the computer device 700 provided in this embodiment includes: at least one processor 701 and a memory 702. Optionally, the device 700 also includes a communication component 703. The processor 701, the memory 702 and the communication component 703 are connected via a bus 704.

[0168] In a specific implementation process, at least one processor 701 executes the computer-executable instructions stored in the memory 702, so that at least one processor 701 executes the above method.

[0169] The specific implementation process of the processor 701 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0170] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0171] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0172] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0173] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0174] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0175] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0176] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0177] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0180] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0181] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0182] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A data processing method in a system migration scenario, characterized in that: Applicable to computer equipment, including: After receiving the first processing request sent by the front end, determining the shard key information of the data requested to be processed by the first processing request; wherein the first processing request indicates a request to process the data; Based on the shard key information, determine the data migration status and data location information corresponding to the data requested to be processed by the first processing request; wherein the data migration status is used to indicate whether the data requested to be processed is migrated; Based on the data migration status, after determining the target system where the data requested to be processed by the first processing request is located, the first processing request is routed to the target system based on the data location information, so as to process the data requested to be processed by the first processing request based on the target system to obtain a data processing result.

2. The method according to claim 1, characterized in that Determining, based on the shard key information, data migration status and data location information corresponding to the data requested to be processed by the first processing request, includes: Determine, based on the first shard key included in the shard key information, a storage unit corresponding to the data requested to be processed by the first processing request, and determine the data location information based on the storage unit; If it is determined that the shard key information includes the second shard key, it is determined that the data migration status indicates that the requested data has been migrated; otherwise, it is determined that the data migration status indicates that the requested data has not been migrated.

3. The method according to claim 2, characterized in that Before receiving the first processing request sent by the front end, the method further includes: Get preset migration dimension information; Based on the preset migration dimension information, acquiring the data to be migrated from the first system; After determining the first sharding key based on the data key information of the data to be migrated and determining the second sharding key based on the preset migration dimension information, the data to be migrated is migrated from the first system to the second system.

4. The method according to claim 3, characterized in that Determining, based on the data migration status, a target system where the data requested to be processed by the first processing request is located includes: If the data migration status indicates that the data requested for processing is migrated, determining the second system as the target system; If the data migration status indicates that the data requested for processing has not been migrated, the first system is determined as the target system.

5. The method according to claim 1, characterized in that Determining shard key information of data to be processed by the first processing request includes: Obtain a preset mapping relationship table; wherein the preset mapping relationship table includes a correspondence between data and shard key information; Based on the preset mapping relationship table, determine the shard key information of the data requested to be processed by the first processing request.

6. The method according to any one of claims 1 to 5, characterized in that The data migration status indicates that the data requested for processing is migrated; After obtaining the data processing result, the method further includes: receiving a second processing request sent by the target system; wherein the second processing request includes a third sharding key; the third sharding key indicates a sharding key updated based on the first processing request; Based on the third shard key included in the second processing request, shard key information of the data requested to be processed by the first processing request is updated.

7. The method according to claim 6, characterized in that The method further comprises: Receiving a third processing request sent by the target system at a preset time interval; wherein the third processing request includes a fourth sharding key; the fourth sharding key indicates a sharding key that is updated within the preset time interval; Based on the fourth shard key, update processing is performed on the shard key information that has not been updated within the preset time interval.

8. A data processing device in a system migration scenario, characterized in that: include: A first determining unit is configured to determine, after receiving a first processing request sent by a front end, shard key information of data requested to be processed by the first processing request; wherein the first processing request indicates a request to process data; A second determining unit is used to determine, based on the shard key information, data migration status and data location information corresponding to the data requested to be processed by the first processing request; wherein the data migration status is used to indicate whether the data requested to be processed is migrated; A processing unit is used to determine, based on the data migration status, a target system where the data requested to be processed by the first processing request is located, and then, based on the data location information, route the first processing request to the target system to process the data requested to be processed by the first processing request based on the target system to obtain a data processing result.

9. A computer device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images