A data sharding management method and system under the unitized architecture of the financial industry

Abstract

The present invention discloses a data sharding management method and system under a unitized architecture of the financial industry, belonging to the field of computer technology; the method comprises: setting sharding nodes and mapping relationships of several units; obtaining business; determining the unit to be stored from the unit according to the business attributes, sharding nodes and mapping relationships of the business; and importing the business traffic into the unit to be stored. The present invention can reduce the complexity of data sharding node management in scenarios such as expansion and contraction under a unitized architecture, completely get rid of the traditional data source management method, and can dynamically load the data source according to the mapping relationship between the unit and the data source during the unit expansion and contraction process, thereby reducing the workload of data source change and improving work efficiency.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data sharding management method and system under a unitized architecture of the financial industry. Background Art

[0002] At present, the more common data sharding solution in business systems is to encapsulate the underlying data sharding nodes (data sources) and provide a unified entry for the business system. However, under the unitized architecture, this solution has obvious shortcomings, especially in the scenario of unit mutual backup or unit expansion and reduction. The business code needs a lot of changes to adapt to this scenario. This unstable scenario brings a lot of workload to business development.

[0003] Moreover, the traditional solution is to encapsulate the data source through coding. When scaling units, the code needs to be adjusted and the version needs to be re-released, which is extremely unfriendly to the system that has already been launched.

[0004] Currently, there is no mature solution in the industry for dynamic management of data shard nodes under a unitized architecture. Summary of the invention

[0005] The purpose of the present invention is to provide an efficient data sharding management method and system under the unitized architecture of the financial industry.

[0006] In order to solve the above technical problems, the present invention provides a data sharding management method under a unitized architecture of the financial industry, comprising the following steps:

[0007] Set the shard nodes and mapping relationships of several units;

[0008] Acquiring business;

[0009] Determine the unit to be stored from the unit according to the business attributes, shard nodes and mapping relationship of the business;

[0010] Import business traffic into the unit to be stored.

[0011] Preferably, according to the service attributes, shard nodes and mapping relationships of the service, determining the unit to be stored from the units specifically includes the following steps:

[0012] Determine the nodes to be stored from the shard nodes based on the business attributes of the business;

[0013] The unit to be stored is determined according to the node to be stored and the mapping relationship.

[0014] Preferably, according to the business attributes of the business, determining the node to be stored from the shard nodes specifically includes the following steps:

[0015] Determine the total number of shard nodes C based on the unique identifier of the business;

[0016] Extract the ID attribute value from the current request message of the service;

[0017] Take the modulus of the total number of shard nodes C according to the ID attribute value to obtain the modulus result;

[0018] According to the modulo result, the node to be stored is determined from the shard nodes.

[0019] Preferably, setting the shard nodes and mapping relationships of the unit specifically includes the following steps:

[0020] Set the shard nodes dn of the units U1 and U2; the shard nodes dn include dn1 to dn8;

[0021] Set the mapping relationship: U1 = {dn1, dn2, dn3, dn4}, U2 = {dn5, dn6, dn7, dn8}.

[0022] Preferably, the method further comprises the following steps:

[0023] The service in the unit starts to actively pull the mapping relationship between the unit and the data shard dn, and loads the shard node dn corresponding to each unit.

[0024] The present invention also provides a data sharding management system under a unitized architecture of the financial industry, comprising:

[0025] The setting module is used to set the shard nodes and mapping relationships of several units;

[0026] An acquisition module, used to acquire services;

[0027] A storage unit confirmation module is used to determine the unit to be stored from the unit according to the business attributes, shard nodes and mapping relationship of the business;

[0028] The import module is used to import business traffic into the unit to be stored.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The present invention can reduce the complexity of data sharding node management in scenarios such as expansion and contraction under a unitized architecture, and completely gets rid of the traditional data source management method. During the unit expansion and contraction process, the data source can be dynamically loaded according to the mapping relationship between the unit and the data source, thereby reducing the workload of data source change and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0032] Figure 1It is a flow chart of a data sharding management method under a unitized architecture of the financial industry according to the present invention. DETAILED DESCRIPTION

[0033] Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited to the specific implementation disclosed below.

[0034] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0035] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0036] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0037] like Figure 1 As shown, the present invention provides a data sharding management method under a unitized architecture of the financial industry, comprising the following steps:

[0038] Set the shard nodes and mapping relationships of several units;

[0039] Acquiring business;

[0040] Determine the unit to be stored from the unit according to the business attributes, shard nodes and mapping relationship of the business;

[0041] Import business traffic into the unit to be stored.

[0042] Preferably, according to the service attributes, shard nodes and mapping relationships of the service, determining the unit to be stored from the units specifically includes the following steps:

[0043] Determine the nodes to be stored from the shard nodes based on the business attributes of the business;

[0044] The unit to be stored is determined according to the node to be stored and the mapping relationship.

[0045] Preferably, according to the business attributes of the business, determining the node to be stored from the shard nodes specifically includes the following steps:

[0046] Determine the total number of shard nodes C based on the unique identifier of the business;

[0047] Extract the ID attribute value from the current request message of the service;

[0048] Take the modulus of the total number of shard nodes C according to the ID attribute value to obtain the modulus result;

[0049] According to the modulo result, the node to be stored is determined from the shard nodes.

[0050] Preferably, setting the shard nodes and mapping relationships of the unit specifically includes the following steps:

[0051] Set the shard nodes dn of the units U1 and U2; the shard nodes dn include dn1 to dn8;

[0052] Set the mapping relationship: U1 = {dn1, dn2, dn3, dn4}, U2 = {dn5, dn6, dn7, dn8}.

[0053] Preferably, the method further comprises the following steps:

[0054] The service in the unit starts to actively pull the mapping relationship between the unit and the data shard dn, and loads the shard node dn corresponding to each unit.

[0055] The present invention also provides a data sharding management system under a unitized architecture of the financial industry, comprising:

[0056] The setting module is used to set the shard nodes and mapping relationships of several units;

[0057] An acquisition module, used to acquire services;

[0058] A storage unit confirmation module is used to determine the unit to be stored from the unit according to the business attributes, shard nodes and mapping relationship of the business;

[0059] The import module is used to import business traffic into the unit to be stored.

[0060] In order to better illustrate the technical effect of the present invention, the present invention provides the following specific embodiments to illustrate the above technical process:

[0061] Embodiment 1: A data sharding management method under a unitized architecture of the financial industry, centrally configures the mapping relationship between each unit and the data sharding node, and provides an algorithm f(x) for calculating the unit UNIT and the data sharding node dn according to the business attributes. The business traffic inlet gateway uses the f(x) algorithm and the mapping relationship between the unit and the sharding node to calculate the specific unit, forwards the traffic to the specific unit according to the unit identifier, and uses the f(x) algorithm to calculate the specific sharding node dn.

[0062] The total data shard nodes (dn1...dn8) of the unit (U1, U2) have a mapping relationship of U1={dn1, dn2, dn3, dn4}U2={dn5, dn6, dn7, dn8}. The service in the unit starts to actively pull the mapping relationship U1, U2 between the unit and the data shard dn, and loads the shard node dn corresponding to each unit.

[0063] The gateway calculates the unit based on the service attributes: such as the service numerical attribute ID, determines the total number of shard nodes C based on the service unique identifier, extracts the unit division attribute ID from the current service request message, and modulos the total number of shard nodes C of units U1 and U2 according to the function f(x), that is, ID%C. Based on the modulo result and the mapping relationship between shard nodes and units, the unit UNIT is determined and the traffic is imported into the corresponding unit.

[0064] After receiving the traffic, the application in the unit obtains the business routing attribute ID and routes the specific data shard node dn through the function f(x); when the unit is expanded or reduced, it only needs to ensure that the number of shard nodes remains unchanged and adjust the mapping relationship between the unit UNIT and the shard node dn.

[0065] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules, modules or units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units, modules or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0066] The units may or may not be physically separated, and the components displayed as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0067] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0068] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present invention are executed. It should be noted that the above-mentioned computer-readable medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above.

[0069] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0070] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A data sharding management method under a unitized architecture in the financial industry, characterized in that: The following steps are involved: Set the shard nodes and mapping relationships of several units; Acquiring business; Determine the unit to be stored from the unit according to the business attributes, shard nodes and mapping relationship of the business; Import business traffic into the unit to be stored.

2. The data sharding management method under the unitized architecture of the financial industry according to claim 1 is characterized in that: According to the business attributes, shard nodes and mapping relationships of the business, the unit to be stored is determined from the units, specifically including the following steps: Determine the nodes to be stored from the shard nodes based on the business attributes of the business; The unit to be stored is determined according to the node to be stored and the mapping relationship.

3. The data sharding management method under the unitized architecture of the financial industry according to claim 2 is characterized in that: According to the business attributes of the business, determine the nodes to be stored from the shard nodes, specifically including the following steps: Determine the total number of shard nodes C based on the unique identifier of the business; Extract the ID attribute value from the current request message of the service; Take the modulus of the total number of shard nodes C according to the ID attribute value to obtain the modulus result; According to the modulo result, the node to be stored is determined from the shard nodes.

4. The data sharding management method under the unitized architecture of the financial industry according to claim 3 is characterized in that: Setting the shard nodes and mapping relationships of the unit includes the following steps: Set the shard nodes dn of the units U1 and U2; the shard nodes dn include dn1 to dn8; Set the mapping relationship: U1 = {dn1, dn2, dn3, dn4}, U2 = {dn5, dn6, dn7, dn8}.

5. The data sharding management method under the unitized architecture of the financial industry according to claim 4 is characterized in that: The following steps are also included: The service in the unit starts to actively pull the mapping relationship between the unit and the data shard dn, and loads the shard node dn corresponding to each unit.

6. A data sharding management system under a unitized architecture of the financial industry, used to implement the data sharding management method under a unitized architecture of the financial industry as described in any one of claims 1 to 5, characterized in that: include: The setting module is used to set the shard nodes and mapping relationships of several units; An acquisition module, used to acquire services; A storage unit confirmation module is used to determine the unit to be stored from the unit according to the business attributes, shard nodes and mapping relationship of the business; The import module is used to import business traffic into the unit to be stored.

Images