A data fusion method, device, equipment and storage medium
By employing a data fusion method based on data transaction status in a multi-site active-active architecture, the problem of data inconsistency during data center switching is solved, ensuring the integrity and continuity of the data center.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNIONPAY
- Filing Date
- 2022-08-19
- Publication Date
- 2026-05-12
Smart Images
Figure CN115438723B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] Embodiments of the present application relate to the technical field of data processing, and in particular, to a data fusion method, device, equipment and storage medium. BACKGROUND
[0002] With the rapid development of Internet technology, the scale of business systems is becoming larger and larger, and the loss caused by each technical failure is immeasurable. In order to improve the disaster recovery capability of the business system, the current business system generally adopts a multi-active architecture in different geographic locations, that is, data centers are set up in different geographic locations, and different data centers can provide business services to the outside. The data stored in different data centers are backups of each other, and since there is a synchronization time delay when different data centers backup data, at any time point, the data stored in different data centers are not completely consistent.
[0003] For a certain business service, it is set that A data center provides the business service to the outside, when A data center has a device failure, it is often switched to other data centers to continue to provide the business service. However, due to the synchronization time delay between different data centers, the other data centers may not have the data corresponding to the business service, or the data of the other data centers is inconsistent with the data of A data center, which will cause the business service provided by the other data centers to be wrong.
[0004] Currently, a preset time period including a switching time point is generally determined, data in the preset time period is obtained from A data center as source data, the source data is copied to other data centers, and when the source data is inconsistent with the data of other data centers, the data corresponding to the later update time is selected for updating according to the update time of the data. This method may have data omission and other problems, and cannot guarantee the data integrity and continuity of the data center. SUMMARY
[0005] Embodiments of the present application provide a data fusion method, device, equipment and storage medium for guaranteeing the data integrity and continuity of the data center.
[0006] In one aspect, the present application provides a data fusion method, which comprises:
[0007] obtaining a plurality of first comparison data in a preset time period from a first data center, and obtaining a plurality of second comparison data in the preset time period from a second data center; the preset time period is determined based on a data center switching time point;
[0008] determining the same data unique identifier in the plurality of first comparison data and the plurality of second comparison data as a first data identifier;
[0009] For any first data identifier, the first data identifier corresponding first comparison data is obtained from the plurality of first comparison data as the first target data, and the first data identifier corresponding second comparison data is obtained from the plurality of second comparison data as the second target data; based on the first transaction state of the first target data and the second transaction state of the second target data, the second target data is updated.
[0010] Optionally, the second target data is updated based on the first transaction state of the first target data and the second transaction state of the second target data, comprising:
[0011] If the first transaction state in the first target data is not empty, the second transaction state in the second target data is not empty, and the first transaction state and the second transaction state are different, the first state position corresponding to the first transaction state and the second state position corresponding to the second transaction state are determined based on the preset transaction state machine, respectively.
[0012] If the first state position is located after the second state position, the second target data is updated using the first target data.
[0013] Optionally, further comprising:
[0014] If the first transaction state and the second transaction state are the same, the transaction time point of the first target data and the transaction time point of the second target data are judged.
[0015] If the transaction time point of the first target data is later than the transaction time point of the second target data, the second target data is updated using the first target data.
[0016] Optionally, the second target data is updated based on the first transaction state of the first target data and the second transaction state of the second target data, comprising:
[0017] If the first transaction state in the first target data is empty, the second transaction state in the second target data is empty, the M first subsequent transaction data corresponding to the first target data and the N second subsequent transaction data corresponding to the second target data are determined, respectively; wherein M>=0, N>=0.
[0018] The data unique identifier corresponding to each of the M first subsequent transaction data is determined as the first subsequent identifier, and the data unique identifier corresponding to each of the N second subsequent transaction data is determined as the second subsequent identifier, respectively.
[0019] determine, based on the preset transaction state machine, a first subsequent state position corresponding to each of the M first subsequent transaction data, and a second subsequent state position corresponding to each of the N second subsequent transaction data;
[0020] determine, based on the obtained M first subsequent identifiers and N second subsequent identifiers, and the M first subsequent state positions and the N second subsequent state positions, a target subsequent transaction data chain;
[0021] update the second transaction state in the second target data based on the target subsequent transaction data chain.
[0022] Optionally, the determining, based on the obtained M first subsequent identifiers and N second subsequent identifiers, and the M first subsequent state positions and the N second subsequent state positions, a target subsequent transaction data chain, comprises:
[0023] if there is no same subsequent identifier in the M first subsequent identifiers and the N second subsequent identifiers, then the first subsequent transaction data corresponding to each of the M first subsequent state positions, and the second subsequent transaction data corresponding to each of the N second subsequent state positions, are taken as target subsequent transaction data;
[0024] sort the target subsequent transaction data according to transaction time points to obtain the target subsequent transaction data chain.
[0025] Optionally, the method further comprises:
[0026] if there is a same subsequent identifier in the M first subsequent identifiers and the N second subsequent identifiers, then the first subsequent identifier and the second subsequent identifier with the same subsequent identifier are grouped into a group to obtain at least one identifier matching group; and the first subsequent identifier in the identifier matching group is taken as a first matching identifier, and the second subsequent identifier in the identifier matching group is taken as a second matching identifier;
[0027] for any identifier matching group, determine a first subsequent state position corresponding to the first matching identifier, and a second subsequent state position corresponding to the second matching identifier; and delete the subsequent transaction data corresponding to the latter state position in the first subsequent state position and the second subsequent state position;
[0028] take the first subsequent transaction data corresponding to the remaining P first subsequent state positions and the second subsequent transaction data corresponding to the remaining Q second subsequent state positions as target subsequent transaction data; wherein 0<=P<=M, and 0<=Q<=N;
[0029] sort the target subsequent transaction data according to transaction time points to obtain the target subsequent transaction data chain.
[0030] Optionally, updating the second transaction status in the second target data based on the target subsequent transaction data chain includes:
[0031] For any two adjacent target subsequent transaction data in the target subsequent transaction data chain, a first positional relationship is determined based on the preset transaction state machine;
[0032] Determine the second positional relationship between the two adjacent target subsequent transaction data in the target subsequent transaction data chain;
[0033] If the first positional relationship and the second positional relationship are the same, then the second transaction state in the second target data is determined based on the transaction state corresponding to each target subsequent transaction data in the target subsequent transaction data chain.
[0034] Optionally, the unique data identifier includes an application service unique identifier and a central service unique identifier; it also includes:
[0035] From the plurality of first data to be compared and the plurality of second data to be compared, at least one pair of data to be compared is determined that has different application service unique identifiers but the same central service unique identifier; the pair of data to be compared includes first data to be compared and second data to be compared.
[0036] For the at least one pair of data to be compared, if the transaction time of the first data to be compared in the pair is earlier than the transaction time of the second data to be compared in the pair, the first data to be compared in the pair is used to update the second data to be compared in the pair.
[0037] Optionally, after updating the second target data based on the first transaction status of the first target data and the second transaction status of the second target data, the method further includes:
[0038] For the first attribute identifier in the second target data, determine whether the first attribute value corresponding to the first attribute identifier is within a preset range. If not, add the second target data to the error file.
[0039] For the first attribute identifier in the second target data, determine the second attribute identifier associated with the first attribute identifier, and determine whether the first attribute value corresponding to the first attribute identifier and the second attribute value corresponding to the second attribute identifier satisfy a preset relationship. If so, add the second target data to the exception file; the exception file is used for manual review.
[0040] On one hand, embodiments of this application provide a data fusion apparatus, which includes:
[0041] The acquisition module is used to acquire multiple first comparison data within a preset time period from a first data center, and multiple second comparison data within the preset time period from a second data center; the preset time period is determined based on the data center switching time point;
[0042] The determining module is used to determine a unique identifier for the same data among the plurality of first data to be compared and the plurality of second data to be compared, and use it as the first data identifier;
[0043] The update module is configured to, for any first data identifier, obtain the first data to be compared corresponding to the first data identifier from the plurality of first data to be compared, as the first target data, and obtain the second data to be compared corresponding to the first data identifier from the plurality of second data to be compared, as the second target data; and update the second target data based on the first transaction status of the first target data and the second transaction status of the second target data.
[0044] Optionally, the update module is specifically used for:
[0045] If the first transaction state in the first target data is not empty, the second transaction state in the second target data is not empty, and the first transaction state and the second transaction state are different, then based on the preset transaction state machine, the first state position corresponding to the first transaction state and the second state position corresponding to the second transaction state are determined respectively.
[0046] If the first state position is after the second state position, then the second target data is updated using the first target data.
[0047] Optionally, the update module is further configured to:
[0048] If the first transaction status and the second transaction status are the same, then the transaction time point of the first target data and the transaction time point of the second target data are judged.
[0049] If the transaction time of the first target data is later than the transaction time of the second target data, then the second target data is updated using the first target data.
[0050] Optionally, the update module is specifically used for:
[0051] If the first transaction status in the first target data is empty and the second transaction status in the second target data is empty, then M first subsequent transaction data corresponding to the first target data and N second subsequent transaction data corresponding to the second target data are determined respectively; where M>=0 and N>=0;
[0052] Each of the M first subsequent transaction data is determined to have a unique data identifier as a first subsequent identifier, and each of the N second subsequent transaction data is determined to have a unique data identifier as a second subsequent identifier.
[0053] Based on a preset transaction state machine, the first subsequent state position corresponding to each of the M first subsequent transaction data is determined, and the second subsequent state position corresponding to each of the N second subsequent transaction data is determined.
[0054] Based on the obtained M first subsequent identifiers and N second subsequent identifiers, as well as the M first subsequent state positions and N second subsequent state positions, the target subsequent transaction data chain is determined;
[0055] Based on the target subsequent transaction data chain, the second transaction status in the second target data is updated.
[0056] Optionally, the update module is specifically used for:
[0057] If there is no common subsequent identifier among the M first subsequent identifiers and the N second subsequent identifiers, then the first subsequent transaction data corresponding to each of the M first subsequent state positions and the second subsequent transaction data corresponding to each of the N second subsequent state positions are taken as the target subsequent transaction data.
[0058] The target subsequent transaction data is sorted according to the transaction time point to obtain the target subsequent transaction data chain.
[0059] Optionally, the update module is further configured to:
[0060] If there is a common subsequent identifier among the M first subsequent identifiers and the N second subsequent identifiers, the first and second subsequent identifiers with the same subsequent identifiers are grouped together to obtain at least one identifier matching group; and the first subsequent identifier in the identifier matching group is used as the first matching identifier, and the second subsequent identifier in the identifier matching group is used as the second matching identifier.
[0061] For any identifier matching group, determine the first subsequent state position corresponding to the first matching identifier and the second subsequent state position corresponding to the second matching identifier; delete the subsequent transaction data corresponding to the later state position in the first and second subsequent state positions;
[0062] The remaining P first subsequent state positions and the Q second subsequent state positions are used as the target subsequent transaction data; where 0 <= P <= M, 0 <= Q <= N;
[0063] The target subsequent transaction data is sorted according to the transaction time point to obtain the target subsequent transaction data chain.
[0064] Optionally, the update module is specifically used for:
[0065] For any two adjacent target subsequent transaction data in the target subsequent transaction data chain, a first positional relationship is determined based on the preset transaction state machine;
[0066] Determine the second positional relationship between the two adjacent target subsequent transaction data in the target subsequent transaction data chain;
[0067] If the first positional relationship and the second positional relationship are the same, then the second transaction state in the second target data is determined based on the transaction state corresponding to each target subsequent transaction data in the target subsequent transaction data chain.
[0068] Optionally, the unique data identifier includes an application service unique identifier and a central service unique identifier; the update module is further configured to:
[0069] From the plurality of first data to be compared and the plurality of second data to be compared, at least one pair of data to be compared is determined that has different application service unique identifiers but the same central service unique identifier; the pair of data to be compared includes first data to be compared and second data to be compared.
[0070] For the at least one pair of data to be compared, if the transaction time of the first data to be compared in the pair is earlier than the transaction time of the second data to be compared in the pair, the first data to be compared in the pair is used to update the second data to be compared in the pair.
[0071] Optionally, it also includes a verification module, which is specifically used for:
[0072] After updating the second target data based on the first transaction status of the first target data and the second transaction status of the second target data, for the first attribute identifier in the second target data, it is determined whether the first attribute value corresponding to the first attribute identifier is within a preset range. If not, the second target data is added to the error file.
[0073] For the first attribute identifier in the second target data, determine the second attribute identifier associated with the first attribute identifier, and determine whether the first attribute value corresponding to the first attribute identifier and the second attribute value corresponding to the second attribute identifier satisfy a preset relationship. If so, add the second target data to the exception file; the exception file is used for manual review.
[0074] On one hand, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for determining the loading position of an item described above.
[0075] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the method for determining the loading position of the article described above.
[0076] In this embodiment, multiple first comparison data within a preset time period are obtained from a first data center, and multiple second comparison data within a preset time period are obtained from a second data center. A unique identifier is then determined among the multiple first and second comparison data as a first data identifier. For any first data identifier, the first comparison data corresponding to the first data identifier is obtained from the multiple first comparison data as the first target data, and the second comparison data corresponding to the first data identifier is obtained from the multiple second comparison data as the second target data. The second target data is then updated based on the first transaction state of the first target data and the second transaction state of the second target data. Since this application does not simply judge based on the update time of the first and second target data, but rather updates the second target data based on the first transaction state of the first target data and the second transaction state of the second target data, it fully considers the sequential relationship of various transaction states in the transaction scenario, making the updated second target data more accurate and ensuring the data integrity and continuity of the second data center. Attached Figure Description
[0077] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0078] Figure 1 A schematic diagram of a system architecture provided for an embodiment of this application;
[0079] Figure 2 A flowchart illustrating a data fusion method provided in an embodiment of this application;
[0080] Figure 3 A flowchart illustrating a second target data update method provided in an embodiment of this application;
[0081] Figure 4 A flowchart illustrating a second target data update method provided in an embodiment of this application;
[0082] Figure 5 A schematic diagram of a consumer service state machine provided in an embodiment of this application;
[0083] Figure 6 A flowchart illustrating a second transaction status update method provided in an embodiment of this application;
[0084] Figure 7 A flowchart illustrating a method for determining a target subsequent transaction data chain, provided as an embodiment of this application;
[0085] Figure 8 A flowchart illustrating another data fusion method provided in this application embodiment;
[0086] Figure 9 This is a schematic diagram of the structure of a data fusion device provided in an embodiment of this application;
[0087] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0088] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0089] refer to Figure 1 This is a data fusion system architecture diagram applicable to the embodiments of this application. The data fusion system architecture diagram includes at least a first data center 101 and a second data center 102.
[0090] The first data center 101 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.
[0091] The second data center 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.
[0092] The first data center 101 and the second data center 102 can be directly connected via wired or wireless means, or they can establish a connection through an intermediate server.
[0093] The first data center 101 provides business services to the outside world. When the first data center 101 experiences equipment failure, it switches to the second data center 102, which continues to provide business services. During the data center switchover, the data fusion system 104 in the second data center 102 obtains multiple first comparison data within a preset time period from the first data center and multiple second comparison data within the same preset time period from the second data center. The preset time period is determined based on the data center switchover time. A unique identifier is identified among the multiple first comparison data and multiple second comparison data as a first data identifier. For any first data identifier, the first comparison data corresponding to the first data identifier is obtained from the multiple first comparison data as the first target data, and the second comparison data corresponding to the first data identifier is obtained from the multiple second comparison data as the second target data. Based on the first transaction status of the first target data and the second transaction status of the second target data, the second target data is updated to obtain the second updated data.
[0094] based on Figure 1 The system architecture diagram described above, and the embodiments of this application provide a flow chart of a data fusion method, such as... Figure 2 As shown, the process of this method is as follows: Figure 1 The data fusion system in the second data center 102 shown executes the following steps:
[0095] Step S201: Obtain multiple first comparison data within a preset time period from the first data center, and obtain multiple second comparison data within a preset time period from the second data center.
[0096] Specifically, the preset time period is determined based on the data center switchover time. The preset duration can be the data center synchronization latency, or the sum of the synchronization latency and the specified latency.
[0097] The start point of the preset time period is the data center switchover time minus the preset duration, and the end point of the preset time period is the data center switchover time plus the preset duration.
[0098] Step S202: Determine the unique identifier of the same data among the multiple first data to be compared and the multiple second data to be compared, and use it as the first data identifier.
[0099] Specifically, the unique data identifier includes the application service unique identifier and the central service unique identifier. The application service unique identifier is determined by the service and sent to the data center, and it will not be different from different data centers. The central service unique identifier is determined by the data center and may be different from different data centers.
[0100] In this application, the same data unique identifier means that the application business unique identifier is the same, and the central business unique identifier is the same.
[0101] The first set of data to be compared can be either basic transaction data or subsequent transaction data. For example, in a consumption scenario, there are steps involved in a purchase and return process. The data generated by the purchase process is basic transaction data, while the data generated by the return process is subsequent transaction data.
[0102] When the first data to be compared is the basic transaction data, the first data to be compared includes multiple attribute identifiers, namely, application business unique identifier, central business unique identifier, transaction time point, transaction type, transaction status, transaction amount, payment amount, and cumulative refund amount.
[0103] When the first data to be compared is subsequent transaction data, the first data to be compared includes multiple attribute identifiers, namely, application business unique identifier, central business unique identifier, transaction time point, transaction type, transaction status, transaction amount, payment amount, cumulative refund amount, and also the application business unique identifier of the associated basic transaction data.
[0104] The second set of data to be compared is similar to the first set of data to be compared, and will not be described in detail here.
[0105] Step S203: For any first data identifier, obtain the first data to be compared corresponding to the first data identifier from multiple first data to be compared, as the first target data; and obtain the second data to be compared corresponding to the first data identifier from multiple second data to be compared, as the second target data; and update the second target data based on the first transaction status of the first target data and the second transaction status of the second target data.
[0106] Specifically, the transaction status in the first target data is taken as the first transaction status, and the transaction status in the second target data is taken as the second transaction status.
[0107] Within the same transaction scenario, the transaction status can be different. For example, in a consumption scenario, the transaction status can be order successful, payment successful, order failed, payment failed, etc.
[0108] If both the first transaction state and the second transaction state are not empty, then the transaction sequence relationship between the first and second transaction states is determined. Based on the transaction sequence relationship, the second target data is updated to obtain the second updated data.
[0109] If the first transaction status is empty and the second transaction status is empty, then determine the M first subsequent transaction data corresponding to the first target data and the N second subsequent transaction data corresponding to the second target data. Based on the M first subsequent transaction data and the N second subsequent transaction data, update the second target data to obtain the second updated data.
[0110] In this embodiment, multiple first comparison data within a preset time period are obtained from a first data center, and multiple second comparison data within a preset time period are obtained from a second data center. A unique identifier is then determined among the multiple first and second comparison data as a first data identifier. For any first data identifier, the first comparison data corresponding to the first data identifier is obtained from the multiple first comparison data as the first target data, and the second comparison data corresponding to the first data identifier is obtained from the multiple second comparison data as the second target data. The second target data is then updated based on the first transaction state of the first target data and the second transaction state of the second target data. Since this application does not simply judge based on the update time of the first and second target data, but rather updates the second target data based on the first transaction state of the first target data and the second transaction state of the second target data, it fully considers the sequential relationship of various transaction states in the transaction scenario, making the updated second target data more accurate and ensuring the data integrity and continuity of the second data center.
[0111] Optionally, in step S203 above, the second target data is updated based on the first transaction status of the first target data and the second transaction status of the second target data, including the following two possible implementation methods:
[0112] The first possible implementation, for the case where the first transaction state in the first target data is not empty and the second transaction state in the second target data is not empty, specifically includes the following: Figure 3 The following steps are shown:
[0113] Step S301: Determine whether the first transaction state and the second transaction state are the same. If not, proceed to step S302; otherwise, proceed to step S305.
[0114] Step S302: Based on the preset transaction state machine, determine the first state position corresponding to the first transaction state and the second state position corresponding to the second transaction state.
[0115] Step S303: Determine whether the first state position is after the second state position. If so, proceed to step S304; otherwise, end.
[0116] Step S304: Update the second target data using the first target data, and then end.
[0117] Step S305: Determine whether the transaction time of the first target data is later than the transaction time of the second target data. If yes, proceed to step S304; otherwise, end.
[0118] In this embodiment of the application, for the case where the first transaction state in the first target data is not empty and the second transaction state in the second target data is not empty, based on a preset transaction state machine, the first state position corresponding to the first transaction state and the second state position corresponding to the second transaction state are determined respectively. Based on the positional relationship between the first state position and the second state position, the second target data is updated so that the updated second target data is more accurate.
[0119] The second possible implementation, for the case where the first transaction status in the first target data is empty and the second transaction status in the second target data is empty, specifically includes the following: Figure 4 The following steps are shown:
[0120] Step S401: Determine the M first subsequent transaction data corresponding to the first target data and the N second subsequent transaction data corresponding to the second target data, where M>=0 and N>=0.
[0121] Specifically, since the subsequent transaction data contains unique application business identifiers related to the basic transaction data, based on this, M subsequent transaction data related to the first target data can be determined from multiple subsequent transaction data as M first subsequent transaction data; and N subsequent transaction data related to the second target data can be determined from multiple subsequent transaction data as N second subsequent transaction data.
[0122] Step S402: Determine the unique data identifiers corresponding to each of the M first subsequent transaction data as the first subsequent identifiers, and determine the unique data identifiers corresponding to each of the N second subsequent transaction data as the second subsequent identifiers.
[0123] Step S403: Based on the preset transaction state machine, determine the first subsequent state position corresponding to each of the M first subsequent transaction data, and determine the second subsequent state position corresponding to each of the N second subsequent transaction data.
[0124] Specifically, different business scenarios require different preset transaction state machines. For example, the consumption business state machine corresponds to a consumption business state machine. A preset transaction state machine includes multiple transaction states and state transition paths between these states. These state transition paths are related to the actual transaction order. The order of the transaction states can be determined based on the state transition paths.
[0125] For example, the consumer business state machine, such as Figure 5 As shown, the consumer business state machine includes multiple transaction states: successful order placement S1, successful payment S2, successful cancellation S3, and successful return S4. The consumer business state machine includes multiple state transition paths: order placement V1, payment V2, cancellation V3, and return V4. Specifically, the state transition path between successful order placement S1 and successful payment S2 is payment V2; the state transition path between successful order placement S1 and successful cancellation S3 is cancellation V3; the state transition path between successful payment S2 and successful cancellation S3 is cancellation V3; and the state transition path between successful payment S2 and successful return S4 is return V4.
[0126] Step S404: Based on the obtained M first subsequent identifiers and N second subsequent identifiers, as well as the M first subsequent state positions and N second subsequent state positions, determine the target subsequent transaction data chain.
[0127] Specifically, the target subsequent transaction data chain consists of first subsequent transaction data and second subsequent transaction data.
[0128] The first target data is defined to correspond to two first subsequent transaction data, namely the first payment transaction data and the first return transaction data. The second target data corresponds to one second subsequent transaction data, namely the second payment transaction data.
[0129] The system determines the second payment transaction data and the first return transaction data as target subsequent transaction data based on the first subsequent identifier and the first subsequent status position corresponding to each of the two first subsequent transaction data, and based on the second subsequent identifier and the second subsequent status position corresponding to one second subsequent transaction data.
[0130] Set the transaction time of the second payment transaction data to 10:00:00 and the transaction time of the first return transaction data to 10:00:05. Sort the above two target subsequent transaction data according to the transaction time to obtain the target subsequent transaction data chain. The target subsequent transaction data chain is the second payment transaction data - the first return transaction data.
[0131] Step S405: Update the second transaction status in the second target data based on the target subsequent transaction data chain.
[0132] Specifically, the second transaction status in the second target data is updated, including, for example... Figure 6 The following execution steps are shown:
[0133] Step S601: For any two adjacent target subsequent transaction data in the target subsequent transaction data chain, determine the first positional relationship between the two adjacent target subsequent transaction data based on the preset transaction state machine.
[0134] Specifically, the transaction states corresponding to the subsequent transaction data of two adjacent targets are determined respectively; then, based on the preset transaction state machine, the state positions of the transaction states corresponding to the subsequent transaction data of two adjacent targets are determined; finally, based on the state positions corresponding to the subsequent transaction data of two adjacent targets, the first positional relationship is determined.
[0135] Step S602: Determine the second positional relationship between the two connected target subsequent transaction data in the target subsequent transaction data chain.
[0136] Step S603: If the first positional relationship and the second positional relationship are the same, then the second transaction status in the second target data is determined based on the transaction status corresponding to each target subsequent transaction data in the target subsequent transaction data chain. If the first positional relationship and the second positional relationship are different, then the second target data is subject to manual review.
[0137] In addition, the attribute values corresponding to the attribute identifiers in the second target data can be determined based on the attribute values corresponding to the other attribute identifiers of each target subsequent transaction data in the target subsequent transaction data chain.
[0138] For example, if the target subsequent transaction data chain is set as second payment transaction data - first return transaction data, based on the preset transaction state machine, the transaction state corresponding to the second payment transaction data is determined to be payment successful S2, and the transaction state corresponding to the first return transaction data is return completed S4. Therefore, the first positional relationship between the second payment transaction data and the first return transaction data is: the second payment transaction data precedes the first return transaction data.
[0139] The second positional relationship between the second payment transaction data and the first return transaction data in the target subsequent transaction data chain is determined as follows: the second payment transaction data precedes the first return transaction data.
[0140] Since the first position relationship and the second position relationship are the same, the second transaction status in the second target data is determined according to the transaction status corresponding to the second payment transaction data and the first return transaction data, respectively.
[0141] In this embodiment, for the case where the first transaction state in the first target data is empty and the second transaction state in the second target data is empty, a target subsequent transaction data chain is determined based on M first subsequent transaction data corresponding to the first target data and N second subsequent transaction data corresponding to the second target data. The second transaction state in the second target data is then updated based on this target subsequent transaction data chain. Because this application updates the second transaction state in the second target data based on the target subsequent transaction data chain, the updated second target data is more accurate.
[0142] Optionally, in step S404 above, determining the target subsequent transaction data chain includes the following two possible implementation methods:
[0143] The first possible implementation, for the case where there are no identical subsequent identifiers among the M first subsequent identifiers and N second subsequent identifiers, specifically includes the following execution steps:
[0144] First, take the first subsequent transaction data corresponding to each of the M first subsequent state positions and the second subsequent transaction data corresponding to each of the N second subsequent state positions as the target subsequent transaction data; then sort the target subsequent transaction data according to the transaction time point to obtain the target subsequent transaction data chain.
[0145] In this embodiment, for cases where there are no identical subsequent identifiers among the M first subsequent identifiers and N second subsequent identifiers, the target subsequent transaction data chain is directly determined based on the M first subsequent transaction data and N second subsequent transaction data, thereby improving the efficiency of generating the target subsequent transaction data.
[0146] The second possible implementation addresses the case where there are identical subsequent identifiers among the M first subsequent identifiers and N second subsequent identifiers, specifically including, for example: Figure 7 The following steps are shown:
[0147] Step S701: Group the first and second subsequent identifiers with the same subsequent identifier into one group to obtain at least one identifier matching group; and take the first subsequent identifier in the identifier matching group as the first matching identifier, and take the second subsequent identifier in the identifier matching group as the second matching identifier.
[0148] Step S702: For any identifier matching group, determine the first subsequent state position corresponding to the first matching identifier and the second subsequent state position corresponding to the second matching identifier; delete the subsequent transaction data corresponding to the later state position in the first and second subsequent state positions.
[0149] Step S703: Take the first subsequent transaction data corresponding to the remaining P first subsequent state positions and the second subsequent transaction data corresponding to the remaining Q second subsequent state positions as the target subsequent transaction data.
[0150] Step S704: Sort the target subsequent transaction data according to the transaction time point to obtain the target subsequent transaction data chain.
[0151] In this embodiment of the application, when there is the same subsequent identifier among the M first subsequent identifiers and N second subsequent identifiers, the subsequent transaction data corresponding to the later state position in the first and second subsequent state positions is deleted, which ensures the accuracy of the remaining subsequent transaction data and thus ensures the accuracy of the generated target subsequent transaction data chain.
[0152] Alternatively, this application also provides two other data fusion methods:
[0153] The first type of other data fusion method addresses the situation where the unique identifiers of the first and second data to be compared are identical, specifically including, for example: Figure 8 The following steps are shown:
[0154] Step S801: From multiple first data to be compared and multiple second data to be compared, determine at least one pair of data to be compared that have different application service unique identifiers but the same central service unique identifier.
[0155] The data pairs to be compared include the first data pair to be compared and the second data pair to be compared.
[0156] Step S802: For at least one pair of data to be compared, if the transaction time of the first data to be compared in the pair is earlier than the transaction time of the second data to be compared in the pair, the first data to be compared in the pair is used to update the second data to be compared in the pair.
[0157] In this embodiment of the application, for data pairs to be compared that have different application service unique identifiers but the same central service unique identifier, the above update method ensures the accuracy of the second data pair to be compared.
[0158] The second type of data fusion method, which addresses the case where a unique identifier exists only in multiple sets of first-level comparison data, specifically includes the following steps:
[0159] Use the unique identifier of the data that exists only in multiple first data to be compared as the second data identifier; retrieve the first data to be compared corresponding to the second data identifier from the multiple first data to be compared, and add the first data to be compared corresponding to the second data identifier to the second data center.
[0160] In this embodiment of the application, the first data to be compared, which exists only in the first data data center, is added to the second data data center to ensure the data integrity of the second data center.
[0161] Optionally, in step S203 above, after updating the second target data based on the first transaction state of the first target data and the second transaction state of the second target data, the following two possible verification implementation methods are further included:
[0162] In a first possible verification implementation, for the first attribute identifier in the second target data, it is determined whether the first attribute value corresponding to the first attribute identifier is within a preset range. If not, the second target data is added to the abnormal file.
[0163] For example, the first attribute is the payment amount, and the payment amount value must be greater than or equal to 0.
[0164] In this embodiment of the application, the second target data is verified based on the relationship between the first attribute value corresponding to the first attribute identifier and the preset range, thereby ensuring the accuracy of the second target data.
[0165] The second possible verification implementation method is to determine the second attribute identifier associated with the first attribute identifier in the second target data, and determine whether the first attribute value corresponding to the first attribute identifier and the second attribute value corresponding to the second attribute identifier satisfy a preset relationship. If so, the second target data is added to the abnormal file.
[0166] Among them, abnormal files are used for manual review.
[0167] For example, the first attribute is the payment amount, and the second attribute is the transaction amount. The payment amount and the transaction amount satisfy the preset relationship that the payment amount is less than or equal to the transaction amount.
[0168] In this embodiment of the application, the second target data is verified based on the relationship between the first attribute value corresponding to the first attribute identifier and the second attribute value corresponding to the second attribute identifier, thereby ensuring the accuracy of the second target data.
[0169] Based on the same technical concept, embodiments of this application provide a data fusion device, such as... Figure 9 As shown, the data fusion device 900 includes:
[0170] The acquisition module 901 is used to acquire multiple first comparison data within a preset time period from a first data center, and multiple second comparison data within the preset time period from a second data center; the preset time period is determined based on the data center switching time point;
[0171] The determining module 902 is used to determine a unique identifier of the same data among the plurality of first data to be compared and the plurality of second data to be compared, and use it as the first data identifier;
[0172] The update module 903 is configured to, for any first data identifier, obtain the first data to be compared corresponding to the first data identifier from the plurality of first data to be compared as the first target data, and obtain the second data to be compared corresponding to the first data identifier from the plurality of second data to be compared as the second target data; and update the second target data based on the first transaction status of the first target data and the second transaction status of the second target data.
[0173] Optionally, the update module 903 is specifically used for:
[0174] If the first transaction state in the first target data is not empty, the second transaction state in the second target data is not empty, and the first transaction state and the second transaction state are different, then based on the preset transaction state machine, the first state position corresponding to the first transaction state and the second state position corresponding to the second transaction state are determined respectively.
[0175] If the first state position is after the second state position, then the second target data is updated using the first target data.
[0176] Optionally, the update module 903 is further configured to:
[0177] If the first transaction status and the second transaction status are the same, then the transaction time point of the first target data and the transaction time point of the second target data are judged.
[0178] If the transaction time of the first target data is later than the transaction time of the second target data, then the second target data is updated using the first target data.
[0179] Optionally, the update module 903 is specifically used for:
[0180] If the first transaction status in the first target data is empty and the second transaction status in the second target data is empty, then M first subsequent transaction data corresponding to the first target data and N second subsequent transaction data corresponding to the second target data are determined respectively; where M>=0 and N>=0;
[0181] Each of the M first subsequent transaction data is determined to have a unique data identifier as a first subsequent identifier, and each of the N second subsequent transaction data is determined to have a unique data identifier as a second subsequent identifier.
[0182] Based on a preset transaction state machine, the first subsequent state position corresponding to each of the M first subsequent transaction data is determined, and the second subsequent state position corresponding to each of the N second subsequent transaction data is determined.
[0183] Based on the obtained M first subsequent identifiers and N second subsequent identifiers, as well as the M first subsequent state positions and N second subsequent state positions, the target subsequent transaction data chain is determined;
[0184] Based on the target subsequent transaction data chain, the second transaction status in the second target data is updated.
[0185] Optionally, the update module 903 is specifically used for:
[0186] If there is no common subsequent identifier among the M first subsequent identifiers and the N second subsequent identifiers, then the first subsequent transaction data corresponding to each of the M first subsequent state positions and the second subsequent transaction data corresponding to each of the N second subsequent state positions are taken as the target subsequent transaction data.
[0187] The target subsequent transaction data is sorted according to the transaction time point to obtain the target subsequent transaction data chain.
[0188] Optionally, the update module 903 is further configured to:
[0189] If there is a common subsequent identifier among the M first subsequent identifiers and the N second subsequent identifiers, the first and second subsequent identifiers with the same subsequent identifiers are grouped together to obtain at least one identifier matching group; and the first subsequent identifier in the identifier matching group is used as the first matching identifier, and the second subsequent identifier in the identifier matching group is used as the second matching identifier.
[0190] For any identifier matching group, determine the first subsequent state position corresponding to the first matching identifier and the second subsequent state position corresponding to the second matching identifier; delete the subsequent transaction data corresponding to the later state position in the first and second subsequent state positions;
[0191] The remaining P first subsequent state positions and the Q second subsequent state positions are used as the target subsequent transaction data; where 0 <= P <= M, 0 <= Q <= N;
[0192] The target subsequent transaction data is sorted according to the transaction time point to obtain the target subsequent transaction data chain.
[0193] Optionally, the update module 903 is specifically used for:
[0194] For any two adjacent target subsequent transaction data in the target subsequent transaction data chain, a first positional relationship is determined based on the preset transaction state machine;
[0195] Determine the second positional relationship between the two adjacent target subsequent transaction data in the target subsequent transaction data chain;
[0196] If the first positional relationship and the second positional relationship are the same, then the second transaction state in the second target data is determined based on the transaction state corresponding to each target subsequent transaction data in the target subsequent transaction data chain.
[0197] Optionally, the unique data identifier includes an application service unique identifier and a central service unique identifier; the update module 903 is further configured to:
[0198] From the plurality of first data to be compared and the plurality of second data to be compared, at least one pair of data to be compared is determined that has different application service unique identifiers but the same central service unique identifier; the pair of data to be compared includes first data to be compared and second data to be compared.
[0199] For the at least one pair of data to be compared, if the transaction time of the first data to be compared in the pair is earlier than the transaction time of the second data to be compared in the pair, the first data to be compared in the pair is used to update the second data to be compared in the pair.
[0200] Optionally, it also includes a verification module 904, which is specifically used for:
[0201] After updating the second target data based on the first transaction status of the first target data and the second transaction status of the second target data, for the first attribute identifier in the second target data, it is determined whether the first attribute value corresponding to the first attribute identifier is within a preset range. If not, the second target data is added to the error file.
[0202] For the first attribute identifier in the second target data, determine the second attribute identifier associated with the first attribute identifier, and determine whether the first attribute value corresponding to the first attribute identifier and the second attribute value corresponding to the second attribute identifier satisfy a preset relationship. If so, add the second target data to the exception file; the exception file is used for manual review.
[0203] Based on the same technical concept, embodiments of this application provide a computer device, which may be a terminal or a server, such as... Figure 10As shown, it includes at least one processor 1001 and a memory 1002 connected to at least one processor. In this embodiment, the specific connection medium between the processor 1001 and the memory 1002 is not limited. Figure 10 Taking the connection between processor 1001 and memory 1002 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.
[0204] In this embodiment of the application, the memory 1002 stores instructions that can be executed by at least one processor 1001. By executing the instructions stored in the memory 1002, at least one processor 1001 can perform the steps included in the above-described data fusion method.
[0205] The processor 1001 is the control center of the computer device, capable of connecting various parts of the computer device via various interfaces and lines. It performs data fusion by running or executing instructions stored in the memory 1002 and accessing data stored in the memory 1002. Optionally, the processor 1001 may include one or more processing units. The processor 1001 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1001. In some embodiments, the processor 1001 and the memory 1002 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.
[0206] The processor 1001 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0207] Memory 1002, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 1002 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory 1002 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 1002 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.
[0208] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described data fusion method.
[0209] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0210] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxesFigure 1 A device that provides the functions specified in one or more boxes.
[0211] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0212] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0213] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A data fusion method, characterized in that, include: Obtain multiple first comparison data within a preset time period from a first data center, and obtain multiple second comparison data within the preset time period from a second data center; The preset time period is determined based on the data center switching time point; A unique identifier is identified among the plurality of first data to be compared and the plurality of second data to be compared, and this identifier is used as the first data identifier. For any first data identifier, the first data to be compared corresponding to the first data identifier is obtained from the plurality of first data to be compared as the first target data, and the second data to be compared corresponding to the first data identifier is obtained from the plurality of second data to be compared as the second target data; If the first transaction status of the first target data is not empty and the second transaction status of the second target data is not empty, then the second target data is updated based on the transaction sequence relationship between the first transaction status and the second transaction status.
2. The method as described in claim 1, characterized in that, Also includes: If the first transaction state in the first target data is not empty, the second transaction state in the second target data is not empty, and the first transaction state and the second transaction state are different, then based on the preset transaction state machine, the first state position corresponding to the first transaction state and the second state position corresponding to the second transaction state are determined respectively. If the first state position is after the second state position, then the second target data is updated using the first target data.
3. The method as described in claim 2, characterized in that, Also includes: If the first transaction status and the second transaction status are the same, then the transaction time point of the first target data and the transaction time point of the second target data are judged. If the transaction time of the first target data is later than the transaction time of the second target data, then the second target data is updated using the first target data.
4. The method as described in claim 1, characterized in that, Also includes: If the first transaction status in the first target data is empty and the second transaction status in the second target data is empty, then M first subsequent transaction data corresponding to the first target data and N second subsequent transaction data corresponding to the second target data are determined respectively; where M>=0 and N>=0; Each of the M first subsequent transaction data is determined to have a unique data identifier as a first subsequent identifier, and each of the N second subsequent transaction data is determined to have a unique data identifier as a second subsequent identifier. Based on a preset transaction state machine, the first subsequent state position corresponding to each of the M first subsequent transaction data is determined, and the second subsequent state position corresponding to each of the N second subsequent transaction data is determined. Based on the obtained M first subsequent identifiers and N second subsequent identifiers, as well as the M first subsequent state positions and N second subsequent state positions, the target subsequent transaction data chain is determined; Based on the target subsequent transaction data chain, the second transaction status in the second target data is updated.
5. The method as described in claim 4, characterized in that, The determination of the target subsequent transaction data chain based on the obtained M first subsequent identifiers and N second subsequent identifiers, as well as M first subsequent state positions and N second subsequent state positions, includes: If there is no common subsequent identifier among the M first subsequent identifiers and the N second subsequent identifiers, then the first subsequent transaction data corresponding to each of the M first subsequent state positions and the second subsequent transaction data corresponding to each of the N second subsequent state positions are taken as the target subsequent transaction data. The target subsequent transaction data is sorted according to the transaction time point to obtain the target subsequent transaction data chain.
6. The method as described in claim 5, characterized in that, Also includes: If there is a common subsequent identifier among the M first subsequent identifiers and the N second subsequent identifiers, the first and second subsequent identifiers with the same subsequent identifiers are grouped together to obtain at least one identifier matching group; and the first subsequent identifier in the identifier matching group is used as the first matching identifier, and the second subsequent identifier in the identifier matching group is used as the second matching identifier. For any identifier matching group, determine the first subsequent state position corresponding to the first matching identifier and the second subsequent state position corresponding to the second matching identifier; delete the subsequent transaction data corresponding to the later state position in the first and second subsequent state positions; The remaining P first subsequent state positions and the Q second subsequent state positions are used as the target subsequent transaction data; where 0 <= P <= M, 0 <= Q <= N; The target subsequent transaction data is sorted according to the transaction time point to obtain the target subsequent transaction data chain.
7. The method as described in claim 4, characterized in that, The step of updating the second transaction status in the second target data based on the target subsequent transaction data chain includes: For any two adjacent target subsequent transaction data in the target subsequent transaction data chain, a first positional relationship is determined based on the preset transaction state machine; Determine the second positional relationship between the two adjacent target subsequent transaction data in the target subsequent transaction data chain; If the first positional relationship and the second positional relationship are the same, then the second transaction state in the second target data is determined based on the transaction state corresponding to each target subsequent transaction data in the target subsequent transaction data chain.
8. The method as described in claim 1, characterized in that, The unique data identifier includes an application service unique identifier and a central service unique identifier; it also includes: From the plurality of first data to be compared and the plurality of second data to be compared, at least one pair of data to be compared is determined that has different application service unique identifiers but the same central service unique identifier; the pair of data to be compared includes first data to be compared and second data to be compared. For the at least one pair of data to be compared, if the transaction time of the first data to be compared in the pair is earlier than the transaction time of the second data to be compared in the pair, the first data to be compared in the pair is used to update the second data to be compared in the pair.
9. The method as described in claim 1, characterized in that, Also includes: For the first attribute identifier in the second target data, determine whether the first attribute value corresponding to the first attribute identifier is within a preset range. If not, add the second target data to the error file. For the first attribute identifier in the second target data, determine the second attribute identifier associated with the first attribute identifier, and determine whether the first attribute value corresponding to the first attribute identifier and the second attribute value corresponding to the second attribute identifier satisfy a preset relationship. If so, add the second target data to the exception file; the exception file is used for manual review.
10. A data fusion device, characterized in that, include: The acquisition module is used to acquire multiple first comparison data within a preset time period from a first data center, and to acquire multiple second comparison data within the preset time period from a second data center; The preset time period is determined based on the data center switching time point; The determining module is used to determine a unique identifier for the same data among the plurality of first data to be compared and the plurality of second data to be compared, and use it as the first data identifier; The update module is configured to, for any first data identifier, obtain the first data to be compared corresponding to the first data identifier from the plurality of first data to be compared as the first target data, and obtain the second data to be compared corresponding to the first data identifier from the plurality of second data to be compared as the second target data; If the first transaction status of the first target data is not empty and the second transaction status of the second target data is not empty, then the second target data is updated based on the transaction sequence relationship between the first transaction status and the second transaction status.
11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that, It stores a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the method according to any one of claims 1 to 9.