Sub-table execution control method and system and electronic equipment
By generating specific component dependency parameters and sharding strategies, the table sharding process is automatically executed, solving the problem of complex configuration in existing technologies and achieving the effect of simplifying learning costs and configuration complexity.
Patent Information
- Application Number
- CN202511098234.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-06
- Publication Date
- 2025-11-21
AI Technical Summary
Existing table partitioning components are complex to configure, especially in the case of composite shard keys, requiring backend maintenance personnel to manually rewrite SQL statements, resulting in high learning costs and configuration complexity.
Specific component dependency parameters are generated based on the attribute parameters of the data table. The table partitioning process is automatically executed using table partitioning and sharding strategies. Automatic table partitioning is achieved through table partitioning routing rules.
It simplifies the learning cost and configuration complexity of the table partitioning process, and realizes automatic table partitioning.
Smart Images
Figure CN120994663A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of data processing, and in particular to a table splitting execution control method and system and electronic equipment. BACKGROUND
[0002] With the development trend of application scenarios gradually becoming complex and cumbersome, the demand for table splitting processing of data tables gradually increases for backend applications. Since multiple data tables need to be split, the table splitting strategies between them are not the same, and specific configurations need to be made by backend maintenance personnel using related components to achieve this.
[0003] In the prior art, MyCat and Sharding-JDBC table splitting components are mainly used to implement complex table splitting business. Such table splitting components achieve the table splitting process by configuring specific strategies, and the overall configuration is relatively complex, which has a certain learning cost for backend maintenance personnel. In specific business, for the case of composite shard keys, the above organizations cannot specify specific primary shard keys, and backend maintenance personnel need to rewrite specific SQL statements to achieve table splitting, which has the problem of complex configuration. SUMMARY
[0004] Therefore, the purpose of the present application is to provide a table splitting execution control method, system and electronic equipment. The method can generate specific component dependency parameters based on data tables, and use the table splitting routing rules generated by the corresponding table splitting strategies and shard strategies to achieve automatic execution of the data table splitting process, thereby simplifying the learning cost and configuration complexity of the table splitting process.
[0005] In a first aspect, the present application provides a table splitting execution control method, which comprises:
[0006] determining the table splitting configuration parameters corresponding to the data table based on the attribute parameters of the data table, and determining the table splitting strategy of the data table using the table splitting configuration parameters;
[0007] determining the component dependency parameters corresponding to the data table according to the table splitting strategy, and constructing the shard strategy of the data table based on the component dependency parameters;
[0008] determining the table splitting name, table splitting field and shard rule in the data table that need to be split based on the shard strategy, and determining the table splitting routing rule corresponding to the data table using the table splitting name, table splitting field and shard rule;
[0009] determining the component service parameters of the data table using the table splitting routing rule, and controlling the data table to execute table splitting using the component service parameters.
[0010] Optionally, the data table corresponding table splitting configuration parameter is determined based on the attribute parameter of the data table, and the table splitting strategy of the data table is determined by using the table splitting configuration parameter, including:
[0011] The table splitting name parameter and the table splitting rule parameter of the data table are determined based on the attribute parameter of the data table, and the table splitting configuration parameter of the data table is determined by using the table splitting name parameter and the table splitting rule parameter;
[0012] The table name range parameter and the format parameter of the data table after table splitting are determined by using the table splitting configuration parameter, and the table splitting strategy of the data table is determined by using the table name range parameter and the format parameter.
[0013] Optionally, the component dependency parameter corresponding to the data table is determined according to the table splitting strategy, and the sharding strategy of the data table is constructed based on the component dependency parameter, including:
[0014] The database dependency package of the data table is determined based on the table splitting strategy, and the component assembly parameter corresponding to the database is constructed by using the database dependency package;
[0015] The component dependency parameter is determined according to the component assembly parameter, and the sharding data source corresponding to the data table is obtained by using the component dependency parameter;
[0016] The sharding strategy of the data table is determined by using the table splitting rule corresponding to the sharding data source.
[0017] Optionally, the table splitting name, the table splitting field and the sharding rule in the data table that need to be table split are determined based on the sharding strategy, and the table splitting routing rule corresponding to the data table is determined by using the table splitting name, the table splitting field and the sharding rule, including:
[0018] The table splitting name, the table splitting field and the primary sharding parameter in the data table that execute the table splitting processing are determined based on the sharding strategy;
[0019] The sharding execution strategy corresponding to the data table is determined by using the primary sharding parameter, and the sharding rule is determined by using the sharding execution strategy;
[0020] The table splitting routing rule corresponding to the data table is determined by using the table splitting name, the table splitting field and the sharding rule.
[0021] Optionally, the component service parameter of the data table is determined by using the table splitting routing rule, and the data table is controlled to execute table splitting by using the component service parameter, including:
[0022] The target processing data corresponding to the data table is determined according to the table splitting routing rule, and the target table splitting corresponding to the data table is determined by using the target processing data;
[0023] The data interaction strategy between the data table and the target table is obtained, and the component service parameter of the data table is determined by using the data interaction strategy;
[0024] The target processing data is transmitted to the corresponding target subtable by using the component service parameter control data table.
[0025] Optionally, the database dependency package of the data table is determined based on the subtable strategy, and the database dependency package comprises:
[0026] A ShardingSphere data service corresponding to the data table is constructed, and the ShardingSphere data service is used to determine a Sharding-JDBC data source corresponding to the data table.
[0027] The target processing data corresponding to the data table is obtained according to the subtable strategy, and a database dependency package corresponding to the target processing data is determined based on the Sharding-JDBC data source.
[0028] Optionally, the subtable routing rule corresponding to the data table is determined by using the subtable name, the subtable field and the sharding rule, and the subtable routing rule comprises:
[0029] The first parameter is determined by using the subtable name;
[0030] The second parameter is determined by using the name parameter corresponding to the subtable field;
[0031] The third parameter is determined by using the primary sharding annotation parameter corresponding to the subtable field;
[0032] The fourth parameter is determined by using the algorithm parameter corresponding to the sharding rule;
[0033] The fifth parameter is determined by using the sharding implementation parameter corresponding to the sharding rule;
[0034] The subtable routing rule is determined based on the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter.
[0035] Optionally, the algorithm parameter comprises a consistent hashing sharding key parameter, a composite primary sharding key parameter, a composite mixed sharding key parameter and a gene sharding key parameter;
[0036] The consistent hashing sharding key parameter is used to control the target processing data corresponding to the target field in the data table to perform a hash operation, and the obtained hash value is mapped to the corresponding shard;
[0037] The composite primary sharding key parameter is used to control the target field in the data table to be combined into a primary sharding key, and the primary sharding position is determined by using the primary sharding key;
[0038] The composite mixed sharding key parameter is used to control all sharding fields in the data table to be combined into a mixed sharding key, and the mixed sharding position is determined by using the mixed sharding key;
[0039] The gene sharding key parameter is used to control the target field in the data table to be sharded according to time and a marker value.
[0040] In a second aspect, the present application provides a table splitting execution control system, which comprises:
[0041] a table splitting strategy determination module configured to determine a table splitting configuration parameter corresponding to a data table based on an attribute parameter of the data table, and determine a table splitting strategy of the data table by using the table splitting configuration parameter;
[0042] a table splitting strategy determination module configured to determine a table splitting configuration parameter corresponding to a data table based on an attribute parameter of the data table, and determine a table splitting strategy of the data table by using the table splitting configuration parameter;
[0043] a table splitting strategy determination module configured to determine a table splitting configuration parameter corresponding to a data table based on an attribute parameter of the data table, and determine a table splitting strategy of the data table by using the table splitting configuration parameter;
[0044] a table splitting strategy determination module configured to determine a table splitting configuration parameter corresponding to a data table based on an attribute parameter of the data table, and determine a table splitting strategy of the data table by using the table splitting configuration parameter;
[0045] In a third aspect, the present application provides an electronic device, which comprises a processor and a memory. The memory stores computer executable instructions which can be executed by the processor. The processor executes the computer executable instructions to implement the steps of the table splitting execution control method provided in the first aspect.
[0046] In a fourth aspect, the present application provides a storage medium which stores computer executable instructions. When the computer executable instructions are called and executed by a processor, the computer executable instructions cause the processor to implement the steps of the table splitting execution control method provided in the first aspect.
[0047] The table splitting execution control method, system and electronic device provided by the present application can generate specific component dependency parameters based on a data table, and automatically execute the table splitting process of the data table by using the table splitting routing rules generated based on the table splitting strategy and the sharding strategy of the data table, thereby simplifying the learning cost and configuration complexity of the table splitting process.
[0048] Other features and advantages of the present application will be set forth in the descriptions that follow, and in part will be apparent from the description, or can be learned by practice of the application. The purposes and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
[0049] To make the above objectives, features and advantages of the present application more apparent, the following will specifically describe a preferred embodiment in combination with the accompanying drawings, and the detailed description is as follows. BRIEF DESCRIPTION OF DRAWINGS
[0050] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or the prior art description. Obviously, the drawings described below are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.
[0051] Figure 1 A flow chart of a sub-table execution control method provided by an embodiment of the present application;
[0052] Figure 2 A flow chart of step S101 in a sub-table execution control method provided by an embodiment of the present application;
[0053] Figure 3 A flow chart of step S102 in a sub-table execution control method provided by an embodiment of the present application;
[0054] Figure 4 A flow chart of step S103 in a sub-table execution control method provided by an embodiment of the present application;
[0055] Figure 5 A flow chart of step S104 in a sub-table execution control method provided by an embodiment of the present application;
[0056] Figure 6 A flow chart of determining a database dependent package of a data table based on a sub-table strategy in a sub-table execution control method provided by an embodiment of the present application;
[0057] Figure 7 A flow chart of step S403 in a sub-table execution control method provided by an embodiment of the present application;
[0058] Figure 8 A schematic diagram of a sub-table execution control system provided by an embodiment of the present application;
[0059] Figure 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application.
[0060] Icon:
[0061] 810-table strategy determination module; 820-fragmentation strategy determination module; 830-table routing rule acquisition module; 840-table execution module;
[0062] 101-processor; 102-memory; 103-bus; 104-communication interface. DETAILED DESCRIPTION
[0063] To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be described below in detail with embodiments. Obviously, the described embodiments are only some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection scope of the present application.
[0064] With the development trend of application scenarios gradually becoming complex and complicated, the demand for table splitting processing of data tables gradually increases for the backend application. Since table splitting needs to be performed on multiple data tables, the table splitting strategies between them are not the same, and specific configurations need to be made by the backend maintenance personnel using related components to achieve this.
[0065] In the prior art, MyCat and Sharding-JDBC and other table splitting components are mainly used to implement complex table splitting business. Such table splitting components achieve the table splitting process by configuring specific strategies, and the overall configuration is relatively complex, which has a certain learning cost for the backend maintenance personnel. In specific business, for the case of composite shard key, the above inter-organizations cannot specify a specific primary shard key, and the backend maintenance personnel needs to rewrite specific SQL statements to achieve table splitting, which has the problem of complex configuration.
[0066] Based on this, the present application provides a table splitting execution control method, system and electronic equipment. The method can generate specific component dependency parameters based on a data table, and use the table routing rules generated by the table splitting strategy and the fragmentation strategy corresponding to the data table to achieve automatic execution of the data table splitting process, thereby simplifying the learning cost and configuration complexity of the table splitting process.
[0067] To facilitate the understanding of the embodiments, first, a table splitting execution control method disclosed by the embodiments of the present application will be described in detail. As shown in Figure 1 , the method comprises:
[0068] Step S101, determining the table splitting configuration parameters corresponding to the data table based on the attribute parameters of the data table, and determining the table splitting strategy of the data table using the table splitting configuration parameters;
[0069] In step S102, the component dependency parameter corresponding to the data table is determined according to the table splitting strategy, and the table splitting strategy of the data table is constructed based on the component dependency parameter.
[0070] In step S103, the table splitting name, table splitting field and splitting rule of the data table that needs to be split are determined based on the splitting strategy, and the table splitting routing rule corresponding to the data table is determined by using the table splitting name, table splitting field and splitting rule.
[0071] In step S104, the component service parameter of the data table is determined by using the table splitting routing rule, and the table splitting of the data table is controlled by using the component service parameter.
[0072] Specifically, in the process of splitting the data table, the attribute parameter of the data table is used to determine the splitting process of the data table, and then the splitting process is determined based on the splitting. Therefore, the splitting configuration parameter of the data table is first determined according to the attribute parameter, and then the splitting strategy of the data table is determined by using the splitting configuration parameter; then the component dependency parameter of the data table is determined according to the splitting strategy, the component dependency parameter can be used for the construction of the component, and the splitting control process is realized by using the component as a carrier.
[0073] After the splitting strategy is obtained, the specific table splitting routing rule is determined in combination with the table splitting name, table splitting field and splitting rule of the data table that needs to be split. Specifically, after the component dependency relationship is introduced by using the component dependency parameter, the table information that needs to be split, the table splitting field and the splitting rule can be obtained by using the table splitting strategy and the splitting strategy in the component.
[0074] After the table splitting routing rule is obtained, the component service parameter of the data table is obtained by associating the data table with the component service, and the table splitting of the data table is controlled by using the component service parameter. In a specific scenario, by using the table splitting routing rule, the table name of the original table can be automatically rewritten when writing and querying the corresponding table according to the table splitting routing rule, so as to realize writing the business data in the data table into the split table or querying the corresponding data from the split table.
[0075] Optionally, based on the attribute parameter of the data table, the splitting configuration parameter corresponding to the data table is determined, and the splitting strategy of the data table is determined by using the splitting configuration parameter, as shown in Figure 2 , comprising:
[0076] In step S201, the table splitting name parameter and table splitting rule parameter of the data table are determined based on the attribute parameter of the data table, and the splitting configuration parameter of the data table is determined by using the table splitting name parameter and table splitting rule parameter.
[0077] In step S202, the table name range parameter and format parameter of the data table after splitting are determined by using the splitting configuration parameter, and the splitting strategy of the data table is determined by using the table name range parameter and format parameter.
[0078] The obtaining process of the sharding strategy mainly involves a sharding name and a sharding rule, and therefore, first, the sharding name parameter and the sharding rule parameter of the data table are obtained based on the attribute parameter of the data table, the corresponding sharding and rule are obtained through the sharding name parameter and the sharding rule parameter, and the sharding configuration parameter of the data table is determined. After the sharding configuration parameter is obtained, the table name range parameter and the format parameter of the data table after sharding are determined according to the parameter, and the sharding strategy of the data table is determined by using the above parameters.
[0079] Optionally, the component dependency parameter corresponding to the data table is determined according to the sharding strategy, and the sharding strategy of the data table is constructed based on the component dependency parameter, as shown in step S102. Figure 3
[0080] Step S301, the database dependency package of the data table is determined based on the sharding strategy, and the component assembly parameter corresponding to the database is constructed by using the database dependency package;
[0081] Step S302, the component dependency parameter is determined according to the component assembly parameter, and the sharding data source corresponding to the data table is obtained by using the component dependency parameter;
[0082] Step S303, the sharding strategy of the data table is determined by using the sharding rule corresponding to the sharding data source.
[0083] The obtaining process of the sharding strategy is implemented based on the sharding strategy, the database dependency package corresponding to the data table is obtained through the sharding strategy, for example, the Sharding-JDBC dependency package, and the component assembly parameter corresponding to the database is constructed. The component dependency parameter of the dependency package is determined according to the component assembly parameter, and is finally used for the component building process. After the component is built, the sharding data source corresponding to the data table is obtained by using the component dependency parameter, and the final sharding strategy is determined by obtaining the sharding rule corresponding to the sharding data source. Optionally, the database dependency package of the data table is determined based on the sharding strategy, as shown in step S102. Figure 6
[0084] Step S601, the ShardingSphere data service corresponding to the data table is constructed, and the Sharding-JDBC data source corresponding to the data table is determined by using the ShardingSphere data service;
[0085] Step S602, the target processing data corresponding to the data table is obtained according to the sharding strategy, and the database dependency package corresponding to the target processing data is determined based on the Sharding-JDBC data source.
[0086] In the implementation process, the Sharding-JDBC (JavaDataBase Connectivity) assembly can be implemented by using a Springboot Starter component. Table information that needs to be sharded is configured through an exposed configuration interface. Specifically, each table corresponding shard field, whether the shard field is a primary shard, and which shard algorithm is used for each shard field can be included.
[0087] In actual processing, a java project can be built through a maven framework. The Sharding-JDBC dependency package is introduced in the project. Then, KylinShardingAutoConfiguration is customized as an automatic assembly class. The class assembles the data source of Sharding-JDBC. The original data source in the related business system can be converted into Sharding-JDBC ShardingSphereDataSource in a wrapped manner. The table splitting configuration added in the business system is assembled into a table splitting rule source and added to the table splitting source. Subsequently, KylinShardingAutoConfiguration is added to the spring.factories file in the Springboot automatic assembly rule. After the related business system introduces the Springboot Starter component, the automatic assembly of Springboot can be automatically implemented.
[0088] Optionally, the table name, the shard field, and the shard rule of the data table that need to be sharded are determined based on the shard strategy, and the table routing rule corresponding to the data table is determined by using the table name, the shard field, and the shard rule in step S103, as shown in Figure 4 , which includes
[0089] In step S401, the table name, the shard field, and the primary shard parameter of the data table that needs to be sharded are determined based on the shard strategy.
[0090] In step S402, the shard execution strategy corresponding to the data table is determined by using the primary shard parameter, and the shard rule is determined by using the shard execution strategy.
[0091] In step S403, the table routing rule corresponding to the data table is determined by using the table name, the shard field, and the shard rule.
[0092] By introducing component dependency, the component corresponding subtable name, subtable field and master shard parameter can be used to construct subtable routing rules. Specifically, the subtable name, subtable field and master shard parameter for executing the subtable processing process are obtained through the sharding strategy; then the final sharding rule is determined by determining the data table corresponding sharding execution strategy through the master shard parameter, and then the subtable routing rule corresponding to the data table is determined by using the subtable name, subtable field and sharding rule.
[0093] Optionally, the step S403 of determining the subtable routing rule corresponding to the data table by using the subtable name, the subtable field and the sharding rule, as shown in Figure 7 , includes:
[0094] Step S701, determining a first parameter by using the subtable name;
[0095] Step S702, determining a second parameter by using the name parameter corresponding to the subtable field;
[0096] Step S703, determining a third parameter by using the master shard annotation parameter corresponding to the subtable field;
[0097] Step S704, determining a fourth parameter by using the algorithm parameter corresponding to the sharding rule;
[0098] Step S705, determining a fifth parameter by using the sharding implementation parameter corresponding to the sharding rule;
[0099] Step S706, determining the subtable routing rule based on the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter.
[0100] The subtable routing rule can be implemented by creating a new class, implementing the ShardingKeyParser interface, and rewriting the doParse method of the class to write custom sharding logic. Then, register this custom sharding algorithm in the configuration interface, and specify which sharding key it applies to. The class form is as follows:
[0101] ShardingParamRegister.registerShardingParam("kylin_document_info","dispute_id",false,ShardingParamTypeEnum.USER_DEFINE,newCustomShardingKeyParser()::doParse)
[0102] Wherein, the first parameter needs to be divided into table name, the second parameter is the shard field, the third field marks whether it is the primary shard, the fourth field marks the use of custom extended shard algorithm, and the fifth field is the extended ShardingKeyParser implementation class. Different shard algorithms are used for different shard strategies in different scenarios. Optionally, the algorithm parameters include: consistent hash shard key parameter, composite primary shard key parameter, composite mixed shard key parameter, and gene shard key parameter;
[0103] The consistent hash shard key parameter is used to control the target field in the data table to perform hash operation on the target processing data, and map the obtained hash value to the corresponding shard;
[0104] The composite primary shard key parameter is used to control the target field in the data table to combine the primary shard key, and determine the primary shard position by using the primary shard key;
[0105] The composite mixed shard key parameter is used to control all shard fields in the data table to combine the mixed shard key, and determine the mixed shard position by using the mixed shard key;
[0106] The gene shard key parameter is used to control the target field in the data table to perform shard processing according to time and mark value.
[0107] Specifically, the consistent hash shard key performs hash operation on the value of a certain field of the data, and maps the hash value to different shards to ensure uniform distribution of the data. Definition method: specify the shard field in the configuration interface, and select consistent hash as the shard algorithm.
[0108] The composite primary shard key uses the combination of multiple fields as the shard key, and these fields jointly determine the shard position of the data. Definition method: specify multiple shard fields in the configuration interface, and mark one or several as primary shard field, and select composite shard algorithm.
[0109] The composite mixed shard key is similar to the composite primary shard key, but allows the shard field to contain non-primary shard field, which plays an auxiliary role in shard. Definition method: specify all shard fields in the configuration interface, and mark the primary shard field, and select the composite mixed shard algorithm.
[0110] The gene shard key is based on certain characteristics or rules of data, such as according to certain rules of time, ID and other fields. Definition method: specify the shard field in the configuration interface, and select the gene shard algorithm, and configure the corresponding shard rule.
[0111] Optionally, the component service parameter of the data table is determined by using the table routing rule, and the component service parameter is used to control the data table to execute the table division step S104, as shown in Figure 5
[0112] In step S501, the target processing data corresponding to the data table is determined according to the sharding routing rule, and the target sharding corresponding to the data table is determined by using the target processing data.
[0113] In step S502, the data interaction strategy between the data table and the target sharding is obtained, and the component service parameter of the data table is determined by using the data interaction strategy.
[0114] In step S503, the component service parameter is used to control the data table to transmit the target processing data to the corresponding target sharding.
[0115] Specifically, after the Sharding-JDBC is assembled by using the Springboot Starter component, the component dependency is introduced, and the table information, sharding field and sharding rule required for sharding are maintained by using the sharding and sharding configuration functions in the component, so that a sharding routing rule is obtained. Then, the sharding routing rule is used, and when the corresponding table is written and queried by the related business system, the component will automatically rewrite the table name of the original table according to the sharding routing rule, so as to realize writing business data into the sharding table or querying corresponding data from the sharding table.
[0116] From the sharding execution control method mentioned in the above embodiment, it can be known that the method can generate specific component dependency parameters based on the data table, and the sharding routing rule generated by using the sharding strategy and the sharding strategy corresponding to the data table can realize automatic execution of the data table sharding process, thereby simplifying the learning cost and configuration complexity of the sharding process.
[0117] Corresponding to the sharding execution control method provided in the foregoing embodiment, an embodiment of the present application provides a sharding execution control system, as shown in Figure 8 The system comprises:
[0118] The sharding strategy determination module 810 is configured to determine the sharding configuration parameter corresponding to the data table based on the attribute parameter of the data table, and determine the sharding strategy of the data table by using the sharding configuration parameter.
[0119] The sharding strategy determination module 820 is configured to determine the component dependency parameter corresponding to the data table according to the sharding strategy, and construct the sharding strategy of the data table based on the component dependency parameter.
[0120] The sharding routing rule acquisition module 830 is configured to determine the sharding name, sharding field and sharding rule of the data table required for sharding based on the sharding strategy, and determine the sharding routing rule corresponding to the data table by using the sharding name, sharding field and sharding rule.
[0121] The sub-table execution module 840 is configured to determine a component service parameter of the data table by using the sub-table routing rule, and control the data table to perform sub-table by using the component service parameter.
[0122] From the sub-table execution control system mentioned in the above embodiment, it can be known that the system can generate specific component dependent parameters based on the data table, and automatically perform the sub-table process of the data table by using the sub-table routing rule generated by the corresponding sub-table strategy and the sharding strategy of the data table, so as to simplify the learning cost and configuration complexity of the sub-table process.
[0123] The sub-table execution control system provided in the embodiment has the same implementation principle and technical effects as the foregoing sub-table execution control method embodiment, and for brief description, the part not mentioned in the device embodiment can refer to the corresponding content in the foregoing sub-table execution control method embodiment.
[0124] The embodiment further provides an electronic device, a structure diagram of the electronic device is shown in Figure 9 The device includes a processor 101 and a memory 102; wherein the memory 102 is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the steps of the foregoing sub-table execution control method.
[0125] Figure 9 The electronic device shown in the figure further includes a bus 103 and a communication interface 104, and the processor 101, the communication interface 104 and the memory 102 are connected through the bus 103.
[0126] The memory 102 can include a high-speed random access memory (RAM, Random Access Memory), and can also include a non-volatile memory, for example, at least one disk memory. The bus 103 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 9 Only one bidirectional arrow is used to represent the bus in the figure, but it does not mean that there is only one bus or only one type of bus.
[0127] The communication interface 104 is configured to connect with at least one user terminal and other network units through a network interface, and send the encapsulated IPv4 packet or IPv4 packet to the user terminal through the network interface.
[0128] The processor 101 can be an integrated circuit chip with a processing capability of signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 101 or the instruction in the form of software. The processor 101 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiment of the present disclosure can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiment of the present disclosure can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the storage 102, and the processor 101 reads the information in the storage 102, and combines the hardware to complete the steps of the method of the above embodiment.
[0129] The embodiment of the present application further provides a storage medium, and the storage medium stores a computer program. When the computer program is run by a processor, the steps of the table separation execution control method in the above embodiment are executed.
[0130] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. The above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and other division manners can be used in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.
[0131] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.
[0132] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.
[0133] If the functions are realized in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and various program code storage media.
[0134] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present application, used to illustrate the technical solutions of the present application, and not to limit them, the protection scope of the present application is not limited thereto, although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: any skilled person in the art within the technical range disclosed by the present application, they can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and all should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims
1. A method of controlling a split table execution, characterized by, The method comprises: determining the data table corresponding to the table configuration parameter based on the attribute parameter of the data table, and determining the data table splitting strategy by using the table configuration parameter; determining the component dependency parameter corresponding to the data table according to the table splitting strategy, and constructing the data table splitting strategy based on the component dependency parameter; determining the table name, table field and shard rule of the data table that needs to be split based on the splitting strategy, and determining the data table corresponding to the table routing rule by using the table name, table field and shard rule; determining the component service parameter of the data table by using the table routing rule, and controlling the data table to perform table splitting by using the component service parameter.
2. The split table execution control method according to claim 1, characterized by, determining the data table corresponding to the table configuration parameter based on the attribute parameter of the data table, and determining the data table splitting strategy by using the table configuration parameter, comprising: determining the table name parameter and table rule parameter of the data table based on the attribute parameter of the data table, and determining the table configuration parameter of the data table by using the table name parameter and the table rule parameter; determining the table name range parameter and format parameter of the data table after splitting by using the table configuration parameter, and determining the table splitting strategy of the data table by using the table name range parameter and the format parameter.
3. The split table execution control method of claim 1, wherein, determining the component dependency parameter corresponding to the data table according to the table splitting strategy, and constructing the data table splitting strategy based on the component dependency parameter, comprising: determining the database dependency package of the data table based on the table splitting strategy, and constructing the component assembly parameter corresponding to the database by using the database dependency package; determining the component dependency parameter according to the component assembly parameter, obtaining the shard data source corresponding to the data table by using the component dependency parameter; determining the shard strategy of the data table by using the table rule corresponding to the shard data source.
4. The split table execution control method of claim 1, wherein, determining the table name, table field and shard rule of the data table that needs to be split based on the splitting strategy, and determining the data table corresponding to the table routing rule by using the table name, table field and shard rule, comprising: determining the table name, table field and primary shard parameter of the data table that performs table splitting processing based on the splitting strategy; determining the shard execution strategy corresponding to the data table by using the primary shard parameter, and determining the shard rule by using the shard execution strategy; determining the data table corresponding to the table routing rule by using the table name, table field and shard rule.
5. The method according to claim 1, wherein determining the component service parameter of the data table by using the table routing rule, and controlling the data table to perform table splitting by using the component service parameter, comprising: determining the target processing data corresponding to the data table according to the table routing rule, and determining the target table of the data table by using the target processing data; obtaining the data interaction strategy between the data table and the target table, and determining the component service parameter of the data table by using the data interaction strategy; The component service parameter is used for controlling the data table to transmit the target processing data to the corresponding target sub-table.
6. The split table execution control method of claim 3, wherein, The database dependency package of the data table is determined based on the sub-table strategy, including: The ShardingSphere data service corresponding to the data table is constructed, and the Sharding-JDBC data source corresponding to the data table is determined by using the ShardingSphere data service; The target processing data corresponding to the data table is obtained according to the sub-table strategy, and the database dependency package corresponding to the target processing data is determined based on the Sharding-JDBC data source.
7. The split table execution control method of claim 4, wherein, The sub-table routing rule corresponding to the data table is determined by using the sub-table name, the sub-table field and the sharding rule, including: The first parameter is determined by using the sub-table name; The second parameter is determined by using the name parameter corresponding to the sub-table field; The third parameter is determined by using the primary sharding annotation parameter corresponding to the sub-table field; The fourth parameter is determined by using the algorithm parameter corresponding to the sharding rule; The fifth parameter is determined by using the sharding implementation parameter corresponding to the sharding rule; The sub-table routing rule is determined based on the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter.
8. The split table execution control method according to claim 7, characterized by, The algorithm parameter includes a consistent hash sharding key parameter, a composite primary sharding key parameter, a composite mixed sharding key parameter and a gene sharding key parameter; The consistent hash sharding key parameter is used for controlling the target processing data corresponding to the target field in the data table to perform hash operation, and mapping the obtained hash value to the corresponding shard; The composite primary sharding key parameter is used for controlling the target field combination in the data table to be the primary sharding key, and determining the primary sharding position by using the primary sharding key; The composite mixed sharding key parameter is used for controlling all sharding fields in the data table to be combined into a mixed sharding key, and determining the mixed sharding position by using the mixed sharding key; The gene sharding key parameter is used for controlling the target field in the data table to be sharded according to time and marker value.
9. A split meter performance control system, characterized by, The system includes: A sub-table strategy determination module is configured to determine the sub-table configuration parameter corresponding to the data table based on the attribute parameter of the data table, and determine the sub-table strategy of the data table by using the sub-table configuration parameter; A sharding strategy determination module is configured to determine the component dependency parameter corresponding to the data table according to the sub-table strategy, and construct the sharding strategy of the data table based on the component dependency parameter; A sub-table routing rule acquisition module is configured to determine the sub-table name, the sub-table field and the sharding rule of the data table which needs to be sub-tabled based on the sharding strategy, and determine the sub-table routing rule corresponding to the data table by using the sub-table name, the sub-table field and the sharding rule; A sub-table execution module is configured to determine the component service parameter of the data table by using the sub-table routing rule, and control the data table to perform sub-table by using the component service parameter.
10. An electronic device, comprising: A computer program product including a processor and a memory storing computer executable instructions executable by the processor to implement the steps of the split table execution control method of any of claims 1 to 8.