Data processing method and device, equipment, medium and program product

Abstract

The embodiment of the invention provides a data processing method and device, equipment, a medium and a program product, and relates to the technical field of computers. The method comprises the following steps: receiving a first input of a user to a segmentation task interface; in response to the first input, a data segmentation task is determined, and the data segmentation task comprises a first data table and segmentation demand information used for segmenting the first data table; determining data segmentation state information of the data in the first data table according to the segmentation demand information; determining an SQL (Structured Query Language) statement corresponding to the data segmentation state information from preset SQL statements; and segmenting the first data table through the SQL statement to obtain at least two second data tables. Thus, through the preset SQL statement, a user can complete data segmentation of the first data table only by performing simple input operation on the segmentation task interface, and the problem that the splitting efficiency is low when the SQL statement is manually written for data table splitting can be solved.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a data processing method, device, equipment, medium and program product. Background Art

[0002] In database management, in order to improve the maintainability and query efficiency of the database, large-scale data tables in the database are usually split into several small-scale data tables according to preset classification rules such as time, region, and business type.

[0003] In the related art, the above-mentioned preset classification rules are determined by Structured Query Language (SQL) statements manually written by operators. However, manually writing SQL statements will reduce the efficiency of splitting data tables. Summary of the Invention

[0004] Embodiments of the present invention provide a data processing method, apparatus, device, medium, and program product, which can solve the problem of low splitting efficiency when manually writing SQL statements to split data tables.

[0005] In a first aspect, an embodiment of the present invention provides a data processing method, the data processing method comprising:

[0006] receiving a first input from a user on the segmentation task interface;

[0007] In response to a first input, determining a data segmentation task, the data segmentation task including a first data table and segmentation requirement information for segmenting the first data table;

[0008] Determine data segmentation status information of the data in the first data table according to the segmentation requirement information, where the data segmentation status information is used to represent the segmented records of the data in the first data table;

[0009] Determining an SQL statement corresponding to the data segmentation status information from a preset structured query language SQL statement, wherein the preset SQL statement is determined based on a plurality of reference segmentation strategies, each reference segmentation strategy corresponding to a reference segmentation domain rule and a reference data segmentation status information;

[0010] The first data table is segmented by using SQL statements to obtain at least two second data tables, where the second data tables include data of segmentation domains corresponding to segmentation requirement information.

[0011] In some possible implementations of the embodiments of the present application, the segmentation requirement information includes a segmentation field and a segmentation rule corresponding to the segmentation field;

[0012] Determining data segmentation status information of the data in the first data table according to the segmentation requirement information includes:

[0013] Construct data query conditions based on segmentation fields and segmentation rules;

[0014] According to the data query condition, the data in the first data table is queried to determine data segmentation status information of the data in the first data table, where the data segmentation status information includes at least one of the following: split domain data and unsplit domain data.

[0015] In some possible implementations of the embodiments of the present application, the split domain data includes at least one of the following: complete split domain data and incomplete split domain data; and the data processing method further includes:

[0016] Determining, according to the data structure, segmentation fields, and segmentation rules of the first data table, an associated field associated with the segmentation field;

[0017] In a case where the split domain data includes data corresponding to the associated field, determining that the split domain data is complete split domain data;

[0018] Alternatively, when the split domain data does not include data corresponding to the associated field, the split domain data is determined to be incomplete split domain data.

[0019] In some possible implementations of the embodiments of the present application, the split segmentation domain data includes at least one of the following: saturated segmentation domain data and unsaturated segmentation domain data; and the data processing method further includes:

[0020] In a case where the data storage capacity of the segmentation domain to which the split segmentation domain data belongs is greater than or equal to the reference data capacity, determining that the split segmentation domain data is saturated segmentation domain data;

[0021] Alternatively, when the data storage capacity of the segmented domain to which the split segmented domain data belongs is smaller than the reference data capacity, the split segmented domain data is determined to be non-saturated segmented domain data.

[0022] In some possible implementations of the embodiments of the present application, before executing the step of splitting the first data table using an SQL statement to obtain at least two second data tables, the data processing method further includes:

[0023] Obtaining a first storage data volume of a first database to which the first data table belongs in each of N first time windows, where the N first time windows start from the Tth first time window and extend to the T+Nth first time window;

[0024] determining, based on the first stored data volume in each of the N first time windows, a second stored data volume of the first database in a second time window, where the second time window occurs later than the N first time windows;

[0025] When the difference between the second stored data volume and the first stored data volume corresponding to the T+Nth first time window is less than or equal to the preset data volume difference, within the reference time window, the data of the first data table is split through SQL statements to obtain at least two second data tables, and the reference time window is determined based on the second time window.

[0026] In some possible implementations of the embodiments of the present application, the first data table is split using an SQL statement to obtain at least two second data tables, including:

[0027] When a preset split trigger condition is met, the first data table is split using an SQL statement to obtain at least two second data tables;

[0028] The preset segmentation trigger condition includes at least one of the following:

[0029] The data volume of the first data table is greater than or equal to a preset data volume threshold;

[0030] The current time is within the reference time window;

[0031] Receive the user's secondary confirmation instruction for the data segmentation task;

[0032] The evaluation value corresponding to the resource load of the database to which the first data table belongs is within a preset evaluation value range;

[0033] A service event associated with the first data table is detected.

[0034] In some possible implementations of the embodiments of the present application, before executing the SQL statement corresponding to the data segmentation status information determined from the preset structured query language SQL statement, the data processing method further includes:

[0035] According to the reference association relationship library, determine whether there is an SQL statement associated with the segmentation requirement information in the reference association relationship library;

[0036] Determining an SQL statement corresponding to the data segmentation status information from a preset structured query language SQL statement includes:

[0037] In the case that there is no SQL statement associated with the segmentation requirement information in the reference association relationship library, an SQL statement corresponding to the data segmentation status information is determined from preset structured query language SQL statements.

[0038] In some possible implementations of the embodiments of the present application, after executing the SQL statement to split the first data table to obtain at least two second data tables, the data processing method further includes:

[0039] Obtain the split status of the first data table split by the SQL statement;

[0040] When the split state includes a state indicating that the SQL statement successfully splits the first data table, establishing an association relationship between the SQL statement and the split requirement information;

[0041] Update the reference relationship database based on the relationship between SQL statements and segmentation requirement information;

[0042] The reference association relationship library is displayed in the display area of ​​the segmentation task interface.

[0043] In a second aspect, an embodiment of the present application provides a data processing device, the data processing device comprising:

[0044] A first receiving module is used to receive a first input from a user on the segmentation task interface;

[0045] A first determining module is configured to determine a data segmentation task in response to a first input, where the data segmentation task includes a first data table and segmentation requirement information for segmenting the first data table;

[0046] A second determining module is configured to determine data segmentation status information of the data in the first data table according to the segmentation requirement information, where the data segmentation status information is used to represent the segmented records of the data in the first data table;

[0047] a third determining module, configured to determine an SQL statement corresponding to the data segmentation status information from a preset structured query language SQL statement, wherein the preset SQL statement is determined based on a plurality of reference segmentation strategies, each reference segmentation strategy corresponding to a reference segmentation domain rule and a reference data segmentation status information;

[0048] The first segmentation module is used to segment the first data table through SQL statements to obtain at least two second data tables, where the second data tables include data of segmentation domains corresponding to segmentation requirement information.

[0049] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the data processing method as described in any one of the first aspects is implemented.

[0050] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, a data processing method as described in any one of the first aspects is implemented.

[0051] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, it implements the data processing method as any one of the first aspects.

[0052] The data processing method, apparatus, device, medium, and program product of the embodiments of the present invention, upon receiving a first input from a user on a segmentation task interface, determine a first data table and segmentation requirement information for segmenting the first data table based on the first input, and then accurately determine the data segmentation status information of the data in the first data table based on the segmentation requirement information. Based on the data segmentation status information, a corresponding SQL statement can be selected from preset SQL statements, thereby ensuring that the data segmentation operation can perform data segmentation in accordance with the user's segmentation requirement information. The preset SQL statements are pre-written based on a variety of reference segmentation strategies and can adapt to data segmentation tasks of different types and requirements. Thus, through the preset SQL statements, the user only needs to perform a simple input operation on the segmentation task interface to complete the data segmentation of the first data table, reducing the time and workload of manually writing SQL statements and executing data segmentation operations, and can solve the problem of low efficiency in manually writing SQL statements to split data tables. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 A schematic diagram showing a flow chart of a data processing method provided by some embodiments of the present application;

[0055] Figure 2 A flowchart illustrating a specific implementation of step 130 provided in some embodiments of the present application is shown;

[0056] Figure 3 A schematic flow chart showing a method for splitting a first data table in a data processing method provided in some embodiments of the present application is shown;

[0057] Figure 4 A schematic flow chart showing a method for updating a reference association relationship library in a data processing method provided in some embodiments of the present application is shown;

[0058] Figure 5 A schematic structural diagram of a data processing device provided in some embodiments of the present application is shown;

[0059] Figure 6A schematic structural diagram of an electronic device provided in some embodiments of the present application is shown. DETAILED DESCRIPTION

[0060] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the objects, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the present invention.

[0061] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0062] It should be noted that the acquisition, storage, use and processing of data in the embodiments of this application are in compliance with the relevant provisions of national laws and regulations.

[0063] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned. They should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0064] In order to solve the problems in the aforementioned related technologies, embodiments of the present invention provide a data processing method, apparatus, device, medium, and program product.

[0065] The following is combined with Figure 1 To the attached Figure 4 , the data processing method provided in the embodiment of the present application is described in detail through specific embodiments and their application scenarios.

[0066] Figure 1 FIG. 1 is a flow chart showing a data processing method according to an embodiment of the present invention. Figure 1As shown, the data processing method is applied to a data processing system, and may specifically include steps 110 to 150.

[0067] Step 110: receiving a first input from a user on the task segmentation interface;

[0068] Step 120: determining a data segmentation task in response to the first input, where the data segmentation task includes a first data table and segmentation requirement information for segmenting the first data table;

[0069] Step 130: determining data segmentation status information of the data in the first data table according to the segmentation requirement information, where the data segmentation status information is used to represent the segmented records of the data in the first data table;

[0070] Step 140: Determine an SQL statement corresponding to the data segmentation status information from a preset structured query language SQL statement, wherein the preset SQL statement is determined based on a plurality of reference segmentation strategies, each reference segmentation strategy corresponding to a reference segmentation domain rule and a reference data segmentation status information;

[0071] Step 150: Segment the first data table using SQL statements to obtain at least two second data tables, where the second data tables include data of segmentation domains corresponding to the segmentation requirement information.

[0072] Thus, upon receiving a first input from the user on the splitting task interface, the first data table and the splitting requirement information for splitting the first data table are determined based on the first input. Subsequently, based on the splitting requirement information, the data splitting status information of the data in the first data table can be accurately determined. Based on the data splitting status information, a corresponding SQL statement can be selected from the preset SQL statements, thereby ensuring that the data splitting operation can perform data splitting in accordance with the user's splitting requirement information. The preset SQL statements are pre-written based on a variety of reference splitting strategies and can adapt to data splitting tasks of different types and requirements. Thus, through the preset SQL statements, the user only needs to perform a simple input operation on the splitting task interface to complete the data splitting of the first data table, reducing the time and workload of manually writing SQL statements and executing data splitting operations, and solving the problem of low splitting efficiency when manually writing SQL statements to split data tables.

[0073] First, regarding step 110, the segmentation task interface in the embodiment of the present application is an interactive interface provided by the data processing system to the user specifically for initiating a data segmentation task. Specifically, it may include various operation options and input areas so that the user can input information related to the data segmentation task.

[0074] For example, a user can operate on a split task interface provided by the data processing system through an interactive device such as a mouse, keyboard, or touch screen, by clicking on interface elements, sliding pages, or entering specific instructions. The data processing system will capture and receive a first input. The first input is used to instruct the data processing system to initiate operations related to data splitting, and specifically may include the name and table structure information of the table to be split, split configuration parameters, and task scheduling information such as the scheduler, schedule start time, set schedule task start time, and schedule status.

[0075] Next, regarding step 120, the first data table in the embodiment of the present application refers to the data table to be segmented that stores the data to be segmented. The segmentation requirement information is the segmentation configuration parameters set by the user based on business requirements regarding how to segment the first data table. For example, data in the data table may be segmented into n data slices based on a certain attribute value range, or data in the data table may be segmented into m data slices based on the data timestamp.

[0076] Furthermore, regarding step 130, the data segmentation status information records the specific details of how the data in the first data table is segmented. Based on the data segmentation status information, it is possible to determine whether the data in the first data table has been divided into segmentation domains, how the data has been segmented, and the characteristics of each segmentation domain. For example, the data segmentation status information may indicate that the first data table is segmented into spring, summer, autumn, and winter according to seasons, and record the data records included in each segmentation domain.

[0077] In some embodiments of the present application, the segmentation requirement information includes a segmentation field and a segmentation rule corresponding to the segmentation field. Based on this, Figure 2 As shown, the above step 130 may specifically include step 1301 and step 1302.

[0078] Step 1301: Construct data query conditions based on the segmentation fields and segmentation rules.

[0079] A split field is a specific field used to divide data. For example, in a data table containing order information, the split field could be "Order Date," "Customer Region," or "Product Price." Using a split field, you can group the first data table based on the different values ​​or value ranges of the split field.

[0080] A segmentation rule is a rule for partitioning the data in the first data table, defined for the partition field. A segmentation rule can be based on a variety of conditions, such as time range, numeric range, or specific characters. For example, if the partition field is "Order Date," the segmentation rule might be to partition by month, with each month's data being divided into a separate partition field. If the partition field is "Product Price," the segmentation rule might be to partition by price range (e.g., 0-100 yuan, 101-200 yuan, etc.).

[0081] The data query condition refers to a condition constructed based on the split field and split rule for filtering data from the first data table. Specifically, it can be a logical expression used to limit the data range of the query. For example, if the split field is a date field and the split rule is to split by quarter, the data query condition can be constructed as a logical expression to filter data by quarter.

[0082] Step 1302: query the data in the first data table according to the data query condition to determine data segmentation status information of the data in the first data table. The data segmentation status information includes at least one of the following: split domain data and unsplit domain data.

[0083] Split domain data refers to data that meets the range defined by the data query criteria and has been divided into corresponding segment domains according to the segmentation rules. This data meets the range defined by the data query criteria. Unsplit domain data refers to data that does not meet the range defined by the data query criteria, that is, data that has not yet been divided into corresponding segment domains.

[0084] Thus, by constructing data query conditions based on the split fields and splitting rules, the data screening range can be precisely defined, improving the pertinence of data query operations and effectively avoiding indiscriminate scanning of the entire first data table, thereby improving data processing efficiency. Furthermore, dividing the data in the first data table into split field data and unsplit field data helps accurately determine the SQL statement that matches the data distribution in the first data table.

[0085] In some embodiments of the present application, for the determined split domain data, it is necessary to further check whether the data in each split domain is complete. The split domain data includes at least one of the following: complete split domain data and incomplete split domain data. Based on this, the above data processing method may further include:

[0086] Determining, according to the data structure, segmentation fields, and segmentation rules of the first data table, an associated field associated with the segmentation field;

[0087] When the split domain data includes data corresponding to the associated fields, the split domain data is determined to be complete split domain data; or, when the split domain data does not include data corresponding to the associated fields, the split domain data is determined to be incomplete split domain data.

[0088] The data structure of the first data table refers to the definitions of each field in the first data table and the relationship between the fields, wherein the definitions of each field may include field name, data type, etc. The segmentation rule specifies how to divide the data based on the segmentation field. The data corresponding to the associated field is related to the segmentation field in terms of business logic or data relationship. For example, in a sales data table, if the segmentation field is "sales date", the data corresponding to the associated field may be the data corresponding to the "order number" or the data corresponding to the "customer number". By determining the data corresponding to the associated field, the integrity of the data can be evaluated more comprehensively.

[0089] Complete split field data refers to data in the first data table that has been split, and the data in each split field is intact. Incomplete split field data refers to data in the first data table that has been split, but the data in some split fields is incomplete, such as missing data corresponding to associated fields, or missing or damaged records in split fields.

[0090] Exemplarily, for the determined split domain data, the data in each split domain is further examined in depth.

[0091] Based on the data results and integrity requirements of the first data table, the data corresponding to the split field and the associated fields of the split field are checked to determine whether they all have values ​​and whether there are any missing or damaged records. If all fields in all records are complete, then the split domain data is complete. For example, in an e-commerce order database segmented by quarter, key fields such as order number, customer information, and product information in the order table within each quarter's split domain are checked. If all fields are complete and not missing, then the quarterly split domain data is complete. Conversely, if there are data records with missing or damaged data, such as data with null values ​​in certain fields or data with record formats that do not meet the requirements, these data records are considered incomplete split domain data.

[0092] Therefore, the inspection and classification of the integrity of the split domain data helps the data processing system to better manage and utilize the data. For incomplete data, further data repair or supplementation can be performed to improve the quality and availability of the data.

[0093] In some other embodiments of the present application, the split segmentation domain data includes at least one of the following: saturated segmentation domain data, non-saturated segmentation domain data. Based on this, the above data processing method may further include:

[0094] When the data storage capacity of the segmented domain to which the split segmented domain data belongs is greater than or equal to the reference data capacity, the split segmented domain data is determined to be saturated segmented domain data; or, when the data storage capacity of the segmented domain to which the split segmented domain data belongs is less than the reference data capacity, the split segmented domain data is determined to be non-saturated segmented domain data.

[0095] The reference data capacity can be an empirical value set by the user based on experience, or it can be determined based on the adjustment coefficient corresponding to the segmentation domain. The reference data capacity represents the maximum capacity that the field value corresponding to the segmentation field in the segmentation domain can reach under the current segmentation rules. Saturated segmentation domain data means that the data in the first data table has been segmented according to certain rules, and the data in each segmentation domain has reached the preset maximum capacity, namely the reference data capacity. Unsaturated segmentation domain data means that the data in the first data table has been segmented according to certain rules, but the data in some segmentation domains has not reached the maximum preset capacity, namely the reference data capacity.

[0096] Exemplarily, the reference data capacity of the reference segmentation domain is 100,000. If the data storage capacity of the reference segmentation domain is 100,000, it means that the data corresponding to the reference segmentation domain is saturated segmentation domain data. If the data storage capacity of the reference segmentation domain is 80,000, it means that the data corresponding to the reference segmentation domain is non-saturated segmentation domain data.

[0097] Therefore, by using the reference data capacity as the judgment basis, it is possible to distinguish which split domains in the split domain data have data greater than or equal to the reference data capacity and which split domains have data less than the reference data capacity. This can provide conditions for subsequent targeted segmentation processing of data in different states, and help the data processing system adopt different segmentation strategies according to different data segmentation status information, thereby improving the accuracy and efficiency of the segmentation of the first data table.

[0098] Next, in step 140, the data processing system maintains a preset SQL statement library. The SQL statements in this preset SQL statement library are pre-written according to different reference segmentation strategies. After obtaining data segmentation status information, the data processing system matches the obtained data segmentation status information with the reference data segmentation status information corresponding to each SQL statement in the preset SQL statement library. When a matching reference data segmentation status information is found, the SQL statement corresponding to the reference data segmentation status information is determined to be the SQL statement corresponding to the current data segmentation status information. In this way, the SQL statement suitable for the data segmentation task of the first data table can be quickly and accurately found from the preset SQL statement library, avoiding the complexity of manually writing SQL statements and improving the efficiency of data segmentation for the first data table.

[0099] Among them, SQL statements include two categories. One is SQL instructions, which are used to perform segmentation operations on a single segmentation domain. One instruction corresponds to a single segmentation task. The other is SQL scripts, which arrange multiple SQL statements in a specific order and can execute segmentation tasks on multiple segmentation domains in batches.

[0100] Exemplarily, the SQL statement corresponding to the data segmentation status information includes at least one of the following: an SQL statement corresponding to the split domain data, an SQL statement corresponding to the unsplit domain data, an SQL statement corresponding to the complete split domain data, an SQL statement corresponding to the incomplete split domain data, an SQL statement corresponding to the saturated domain data, and an SQL statement corresponding to the unsaturated domain data.

[0101] Then, in step 150 , the second data table includes data of the segmentation domain corresponding to the segmentation requirement information, that is, the data in each second data table meets the above segmentation requirement information.

[0102] In some embodiments of the present application, before the above step 150, as Figure 3 As shown, the above data processing method may further include steps 210 to 230.

[0103] Step 210 , obtaining a first storage data volume of a first database to which the first data table belongs in each of N first time windows, where the N first time windows start from the Tth first time window and continue to the T+Nth first time window.

[0104] Wherein, N and T are both positive integers greater than or equal to 1. The first time window is a pre-set time period with a fixed time span, used to count the first amount of stored data in the database within the time period. For example, a month is divided into 30 first time windows of one day.

[0105] N first time windows represent N consecutive first time windows starting from the Tth first time window. For example, if T=1, N=5, and the first time window is one day, then it means starting from the 1st day and continuing for 5 consecutive days.

[0106] In this way, by obtaining the first storage data volume of the first database in N first time windows, we can clearly understand the changes in the data volume of the first database over time, which provides basic data for the subsequent prediction of the second storage data volume in the second time window and helps to reasonably arrange data segmentation operations.

[0107] Step 220 : Determine a second amount of stored data in the first database in a second time window according to the first amount of stored data in each of the N first time windows, where the second time window occurs later than the N first time windows.

[0108] The second time window is a time period that occurs later than the N first time windows and is used to predict the first storage data volume of the database within the time period. The second storage data volume is the predicted first storage data volume of the first database within the second time window.

[0109] For example, a time series analysis method can be used to establish a database performance indicator observation model, and the database performance indicator observation model can be used to monitor and predict database segmentation domain data, thereby avoiding database performance bottlenecks and failure risks in advance.

[0110] Among them, time series analysis is a statistical method that predicts future trends based on historical data. In the database performance indicator observation model, this principle can be used to predict future performance by analyzing the database's past performance indicator data, and then monitor and manage the database segmentation domain data. In order to ensure the accuracy of the algorithm, the performance indicator of the target database under observation for the past two weeks can be selected as the initial value X t The performance indicators here can be the database data size, response time, throughput, etc. In this example, the database data size is used as an example.

[0111]

[0112] Table 1

[0113] Since the time series for database performance monitoring is fixed, a model parameter α value of 0.5 can be selected. α is the smoothing coefficient in exponential smoothing, which represents the weighting of historical and current data in the forecast. As you can understand, the closer α is to 1, the greater the weight given to current data; the closer α is to 0, the greater the weight given to historical data. Selecting 0.5 gives equal weight to historical and current data.

[0114] The average of the first three periods of data in Table 1 can be used as the initial smoothing value F0. Taking the data volume of the database in the past 15 days as an example, the data volume of the first three periods is 23GB, 22GB, and 24GB respectively, so the initial smoothing value F0 = 23GB.

[0115] The exponential smoothing method can be expressed by the following formula (1):

[0116] F t+1 =αX t +(1-α)F t (1)

[0117] Among them, F t+1 is the smoothing period of period t+1, i.e. the predicted value; X t is the actual value of period t; F t is the smoothed value of period t, and α is the smoothing coefficient.

[0118] For example, the calculation of F1 can be expressed as F1 = 0.5 * 23 + (1-0.5) * 23 = 23GB. Based on this, the calculation of F2 and F 3… F 15 The predicted values ​​are: F2=23.0, F3=22.5, F4=23.3, F5=23.6, F6=22.3, F7=22.2, F8=21.1, F9=22.0, F 10 =21.0, F 11 =22.5, F 12 =21.3, F 13 =20.6, F 14 =22.3, F 15 =23.2.

[0119] Furthermore, by calculating the smoothed values ​​of the first 15 periods, we can get the data size F of the 16th period. 16 According to the principle of exponential smoothing, the forecast value of the 16th period is the smoothed value of the 15th period, 23.3GB.

[0120] By establishing a database performance indicator observation model and utilizing time series analysis to monitor and predict database segmentation domain data, we can proactively identify changing database performance trends and take timely measures to prevent database failures caused by performance bottlenecks. This approach improves database reliability and stability, ensuring the normal operation of business systems. Furthermore, splitting data tables based on prediction results optimizes the database's storage structure and improves data query and processing efficiency.

[0121] Step 230: When the difference between the second stored data volume and the first stored data volume corresponding to the T+Nth first time window is less than or equal to the preset data volume difference, within the reference time window, the data of the first data table is split using an SQL statement to obtain at least two second data tables, and the reference time window is determined based on the second time window.

[0122] The preset data volume difference is a pre-set threshold used to determine whether the difference between the second stored data volume and the first stored data volume corresponding to the T+Nth first time window is within an acceptable range. In one example, a subset of the second time window can be determined as the reference time window. For example, if the second time window is "the 1st to the 5th of each month," the reference time window can be limited to "the 1st to the 2nd of each month."

[0123] For example, the predicted second stored data volume is subtracted from the first stored data volume corresponding to the T+Nth first time window to obtain a difference. The calculated difference is then compared with a preset data volume difference. If the difference is less than or equal to the preset data volume difference, the growth or change in the database data volume is within an acceptable range, and the data splitting operation can be performed. The data in the first data table is then split by executing a pre-written SQL statement to obtain at least two second data tables.

[0124] Referring to the example of step 220 above, after step 230 above, when the difference between the predicted data volume of the 16th period and the data volume of the 15th period is less than or equal to the preset data volume difference, the first data table is split.

[0125] Therefore, by comparing the difference between the second stored data volume and the first stored data volume, and performing the data splitting operation when the difference is less than or equal to the preset data volume difference, it can be ensured that the data splitting operation is performed when the data volume of the database is relatively stable, thereby reducing the impact of data splitting on database performance.

[0126] In some embodiments of the present application, the above step 150 may specifically include: when a preset split trigger condition is met, splitting the first data table using an SQL statement to obtain at least two second data tables;

[0127] The preset segmentation trigger condition includes at least one of the following:

[0128] The data volume of the first data table is greater than or equal to a preset data volume threshold;

[0129] The current time is within the reference time window;

[0130] Receive the user's secondary confirmation instruction for the data segmentation task;

[0131] The evaluation value corresponding to the resource load of the database to which the first data table belongs is within a preset evaluation value range;

[0132] A service event associated with the first data table is detected.

[0133] The preset data volume threshold is a pre-set data volume standard. When the data volume of the first data table is greater than or equal to the preset data volume threshold, a data split operation is triggered. The setting of this preset data volume threshold can be determined based on factors such as database performance, storage capacity, and business needs, and is not limited here. By setting a data volume threshold and performing real-time monitoring, data table data can be split promptly when the data volume reaches a certain scale, avoiding database performance degradation due to excessive data volume and improving data query and processing efficiency.

[0134] The reference time window refers to a pre-set time range within which the first data table is allowed to perform data segmentation operations. For example, the reference time window can be determined with reference to steps 210 to 230 above.

[0135] A secondary confirmation instruction means that before performing a data splitting operation, the data processing system generates a random numeric code and sends it to a reserved number for the approving user. After confirming that the data splitting operation for the first data table can be performed, the approving user synchronizes the code with the operating user. The operating user must enter the received verification code in the verification area of ​​the splitting task interface. The server communicating with the data processing system automatically compares the verification code entered by the user with the verification code sent by the platform and verifies whether the verification code is within the validity period. If the two are consistent and the verification code is still valid, the data processing system completes a secondary verification, indicating that the user has received a secondary confirmation instruction for the data splitting task. This secondary confirmation mechanism effectively blocks the execution of illegal operations, prevents data splitting caused by erroneous operations, improves the security and reliability of data splitting, and ensures that data splitting is performed with the user's explicit authorization. For example, if a person accidentally presses the data splitting button or an external party attempts to intrude on the system to perform data splitting, this mechanism can prevent such abnormal operations through verification of the verification code, effectively ensuring data security.

[0136] The evaluation value corresponding to the resource load refers to a value obtained after a comprehensive evaluation of the resource usage of the database. The resource load can include the central processing unit (CPU) usage, memory usage, disk load, etc. The preset evaluation value range refers to a pre-set evaluation value interval. When the resource load evaluation value of the database is within this interval, it is considered that the resource load of the database is in an appropriate state and data segmentation operations can be performed. By monitoring the resource load of the database and performing data segmentation under the appropriate resource load state, it is possible to avoid excessive burden on the database system due to data segmentation operations, thereby ensuring the normal operation and stable performance of the database.

[0137] Business events associated with a data table refer to business operations or events related to the first data table. When these events occur, data splitting operations on the first data table may be triggered, for example, the completion of monthly financial settlement, the start of a new business cycle, etc.

[0138] In some embodiments of the present application, before step 150, the data processing method may further include: determining, based on the reference association relationship library, whether there is an SQL statement associated with the segmentation requirement information in the reference association relationship library.

[0139] The reference relationship database is a database that stores the correspondence between segmentation requirements and SQL statements. With this database, you can quickly find SQL statements that match specific segmentation requirements, improving the efficiency of data segmentation operations.

[0140] Based on this, the above step 150 may specifically include, when there is no SQL statement associated with the segmentation requirement information in the reference association library, determining an SQL statement corresponding to the data segmentation status information from preset structured query language SQL statements.

[0141] Therefore, by performing a matching query in the reference association library, the SQL statement associated with the segmentation requirement information can be quickly located, improving the accuracy and consistency of the data segmentation operation. Furthermore, when the reference association library cannot provide a matching SQL statement, it can select from preset SQL statements, allowing the data processing system to find the appropriate SQL statement for data segmentation in various situations, ensuring the smooth execution of the data segmentation task.

[0142] In some embodiments of the present application, after the above step 150, as shown in FIG. Figure 4 As shown, the above data processing may further include steps 310 to 340.

[0143] Step 310: Obtain the split status of the first data table split by the SQL statement.

[0144] The split status refers to the result status presented after the SQL statement performs a split operation on the first data table, which may include a success status or a failure status.

[0145] Step 320 : When the split status includes a status indicating that the SQL statement successfully splits the first data table, an association relationship between the SQL statement and the split requirement information is established.

[0146] Step 330: Update the reference association relationship database according to the association relationship between the SQL statement and the segmentation requirement information.

[0147] Exemplarily, key elements such as the name of the segmentation field, the description of the segmentation rule, etc. are extracted from the segmentation requirement information, and after associating this information with the successfully executed SQL statement, it is stored in the reference association relationship library.

[0148] Step 340: Display the reference association relationship library in the display area of ​​the segmentation task interface.

[0149] For example, data visualization tools can be used to display relevant information about the segmentation operation in the form of charts, reports, etc., providing intuitive information to operations and maintenance personnel, helping them to better plan and execute future data segmentation tasks. For example, by analyzing the data distribution under different segmentation rules in historical data, operations and maintenance personnel can select more appropriate segmentation rules to improve the efficiency and accuracy of data segmentation.

[0150] Therefore, after establishing the association between the SQL statement and the segmentation requirement information, it is helpful to quickly find the corresponding SQL statement based on the segmentation requirement information to execute the data segmentation operation of the data table, thereby effectively improving the efficiency of data segmentation.

[0151] In some embodiments of the present application, each SQL statement in the reference association library may also be associated with a segmentation duration and segmentation resources. Segmentation resources include the various resources occupied by the database when executing the SQL statement to segment data, such as CPU, memory, and disk I / O. By recording the segmentation resources associated with each SQL statement, it is possible to gain a deeper understanding of the resource requirements of different segmentation tasks. This allows for the subsequent data segmentation operation to be performed based on this information, allowing for the proper resource planning in advance and avoiding performance issues caused by inappropriate resource allocation. For example, SQL statements with high segmentation resource requirements can be scheduled for execution during periods of ample system resources, or more computing resources can be allocated to ensure efficient completion of the segmentation task. By analyzing the segmentation resource usage of different SQL statements, it is possible to identify areas with low resource utilization efficiency and optimize them. For example, if a SQL statement is found to be consuming excessive disk I / O resources when executing a segmentation task, but the actual segmentation effect is unsatisfactory, the SQL statement can be optimized to improve overall resource utilization.

[0152] The split duration records the time it takes for the SQL statement to complete the data split operation. When scheduling tasks, you can use the estimated split duration to rationally arrange the order and timing of tasks to avoid task backlogs or excessive wait times. For example, in a scenario requiring multiple data split tasks, you can schedule tasks with shorter split durations first to improve overall processing efficiency.

[0153] Furthermore, the SQL statement's associated split duration and split resources can be displayed in the split task interface's display area. This allows users, when faced with multiple split scenarios, to quickly understand the differences in time and resource consumption for each. For example, if users need to obtain split results quickly, they can prioritize solutions with shorter split durations. If their system resources are limited, they can refer to the split resource information and select solutions with fewer resource constraints.

[0154] Based on the data processing method provided in the above embodiments, the present application also provides a specific implementation of a data processing device. Please refer to the following embodiments.

[0155] See first Figure 5 , the data processing device 400 provided in the embodiment of the present application includes:

[0156] A first receiving module 410 is configured to receive a first input from a user on the task segmentation interface;

[0157] A first determining module 420 is configured to determine a data segmentation task in response to a first input, where the data segmentation task includes a first data table and segmentation requirement information for segmenting the first data table;

[0158] A second determining module 430 is configured to determine data segmentation status information of the data in the first data table according to the segmentation requirement information, where the data segmentation status information is used to represent the segmented records of the data in the first data table;

[0159] A third determining module 440 is configured to determine an SQL statement corresponding to the data segmentation status information from a preset structured query language SQL statement, wherein the preset SQL statement is determined based on a plurality of reference segmentation strategies, each reference segmentation strategy corresponding to a reference segmentation domain rule and a reference data segmentation status information;

[0160] The first segmentation module 450 is configured to segment the first data table using SQL statements to obtain at least two second data tables, where the second data tables include data in segmentation domains corresponding to the segmentation requirement information.

[0161] Thus, upon receiving a first input from the user on the splitting task interface, the first data table and the splitting requirement information for splitting the first data table are determined based on the first input. Subsequently, based on the splitting requirement information, the data splitting status information of the data in the first data table can be accurately determined. Based on the data splitting status information, a corresponding SQL statement can be selected from the preset SQL statements, thereby ensuring that the data splitting operation can perform data splitting in accordance with the user's splitting requirement information. The preset SQL statements are pre-written based on a variety of reference splitting strategies and can adapt to data splitting tasks of different types and requirements. Thus, through the preset SQL statements, the user only needs to perform a simple input operation on the splitting task interface to complete the data splitting of the first data table, reducing the time and workload of manually writing SQL statements and executing data splitting operations, and solving the problem of low splitting efficiency when manually writing SQL statements to split data tables.

[0162] In some embodiments of the present application, the second determining module 430 is specifically configured to:

[0163] In the case where the segmentation requirement information includes a segmentation field and a segmentation rule corresponding to the segmentation field, constructing a data query condition according to the segmentation field and the segmentation rule;

[0164] According to the data query condition, the data in the first data table is queried to determine data segmentation status information of the data in the first data table, where the data segmentation status information includes at least one of the following: split domain data and unsplit domain data.

[0165] In some embodiments of the present application, the data processing device may further include:

[0166] a fourth determining module configured to determine, when the split domain data includes at least one of the following: complete split domain data and incomplete split domain data, an associated field associated with the split field according to the data structure, the split field, and the split rule of the first data table;

[0167] The fifth determination module is used to determine that the split domain data is complete split domain data when the split domain data includes data corresponding to the associated field; or to determine that the split domain data is incomplete split domain data when the split domain data does not include data corresponding to the associated field.

[0168] In some embodiments of the present application, the data processing device may further include:

[0169] The sixth determination module is used to determine that the split segmented domain data is saturated segmented domain data when the split segmented domain data includes at least one of the following: saturated segmented domain data and unsaturated segmented domain, and when the data storage capacity of the segmented domain to which the split segmented domain data belongs is greater than or equal to the reference data capacity; or to determine that the split segmented domain data is unsaturated segmented domain data when the data storage capacity of the segmented domain to which the split segmented domain data belongs is less than the reference data capacity.

[0170] In some embodiments of the present application, the data processing device may further include:

[0171] A first acquisition module is configured to acquire, before splitting the first data table using an SQL statement to obtain at least two second data tables, a first amount of stored data in each of N first time windows of a first database to which the first data table belongs, where the N first time windows start from the Tth first time window and extend to the T+Nth first time window;

[0172] an eighth determining module, configured to determine, based on the first stored data amount in each of the N first time windows, a second stored data amount of the first database in a second time window, the second time window occurring later than the N first time windows;

[0173] The second segmentation module is used to segment the data of the first data table through SQL statements within the reference time window to obtain at least two second data tables when the difference between the second stored data volume and the first stored data volume corresponding to the T+Nth first time window is less than or equal to the preset data volume difference, and the reference time window is determined based on the second time window.

[0174] In some embodiments of the present application, the first segmentation module 450 may be specifically used to:

[0175] When a preset split trigger condition is met, the first data table is split using an SQL statement to obtain at least two second data tables;

[0176] The preset segmentation trigger condition includes at least one of the following:

[0177] The data volume of the first data table is greater than or equal to a preset data volume threshold;

[0178] The current time is within the reference time window;

[0179] Receive the user's secondary confirmation instruction for the data segmentation task;

[0180] The evaluation value corresponding to the resource load of the database to which the first data table belongs is within a preset evaluation value range;

[0181] A service event associated with the first data table is detected.

[0182] In some embodiments of the present application, the data processing device may further include:

[0183] a ninth determining module, configured to determine, based on a reference association relationship database, whether there is an SQL statement associated with the segmentation requirement information before determining the SQL statement corresponding to the data segmentation status information from the preset structured query language SQL statement;

[0184] Based on this, the third determining module 440 can be used to determine an SQL statement corresponding to the data segmentation status information from a preset structured query language SQL statement when there is no SQL statement associated with the segmentation requirement information in the reference association relationship library.

[0185] In some embodiments of the present application, the data processing device may further include:

[0186] A second acquisition module is configured to, after the first data table is split using an SQL statement to obtain at least two second data tables, acquire a split status of the first data table split by the SQL statement;

[0187] An establishing module, configured to establish an association relationship between the SQL statement and the segmentation requirement information when the segmentation state includes a state indicating that the SQL statement successfully segments the first data table;

[0188] An update module is used to update the reference association relationship library according to the association relationship between the SQL statement and the segmentation requirement information;

[0189] The display module is used to display the reference association relationship library in the display area of ​​the segmentation task interface.

[0190] Each module of the data processing device 400 provided in the embodiment of the present application can realize Figures 1 to 4 The functions of each step of the provided data processing method and its ability to achieve corresponding technical effects are described briefly and will not be elaborated here.

[0191] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in some embodiments of the present application is shown.

[0192] The electronic device may include a processor 501 and a memory 502 storing computer program instructions.

[0193] Specifically, the processor 501 may include a central processing unit (CPU) or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0194] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In one example, the memory 502 may include a removable or non-removable (or fixed) medium, or the memory 502 may be a non-volatile solid-state memory. The memory 502 may be inside or outside the integrated gateway disaster recovery device.

[0195] In one example, the memory 502 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0196] The processor 501 reads and executes the computer program instructions stored in the memory 502 to implement Figures 1 to 4 The data processing method in the illustrated embodiment.

[0197] In one example, the electronic device may further include a communication interface 503 and a bus 504. Figure 6 As shown, the processor 501 , the memory 502 , and the communication interface 503 are connected via a bus 504 and communicate with each other.

[0198] The communication interface 503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiment of the present invention.

[0199] Bus 504 includes hardware, software or both, and couples the components of the online data traffic metering device to each other. For example, and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 504 may include one or more buses. Although the embodiments of the present invention describe and illustrate specific buses, the present invention contemplates any suitable bus or interconnect.

[0200] The electronic device can execute the data processing method in the embodiment of the present application, thereby realizing the combination Figures 1 to 2 Describe the data processing methods.

[0201] In addition, in combination with the data processing method in the above embodiment, the embodiment of the present application may provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the data processing methods in the above embodiment is implemented. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as portable disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, etc.

[0202] In addition, in conjunction with the data processing methods in the above embodiments, embodiments of the present application may be implemented by providing a computer program product. The program product is stored in a storage medium and may specifically include a computer program or instructions. When the computer program or instructions are executed by a processor, any of the data processing methods in the above embodiments is implemented. The program product is executed by at least one processor to implement the various processes of the above data processing method embodiments, and can achieve the same technical effects. To avoid repetition, it will not be described here.

[0203] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0204] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium, or transmitted on a transmission medium or communication link via a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memories (ROMs), flash memories, erasable read-only memories (EROMs), floppy disks, compact disc read-only memories (CD-ROMs), optical discs, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segments can be downloaded via computer networks such as the Internet and intranets.

[0205] It should also be noted that the exemplary embodiments described herein describe methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the steps described above. In other words, the steps may be performed in the order described in the embodiments, or in a different order, or several steps may be performed simultaneously.

[0206] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0207] The above is only a specific embodiment of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A data processing method, characterized in that: include: receiving a first input from a user on the segmentation task interface; In response to the first input, determining a data segmentation task, the data segmentation task including a first data table and segmentation requirement information for segmenting the first data table; Determining data segmentation status information of the data in the first data table according to the segmentation requirement information, wherein the data segmentation status information is used to represent the segmented records of the data in the first data table; Determining an SQL statement corresponding to the data segmentation status information from a preset structured query language SQL statement, wherein the preset SQL statement is determined based on a plurality of reference segmentation strategies, each reference segmentation strategy corresponding to a reference segmentation domain rule and a reference data segmentation status information; The first data table is segmented by the SQL statement to obtain at least two second data tables, wherein the second data tables include data of segmentation domains corresponding to the segmentation requirement information.

2. The method according to claim 1, characterized in that The segmentation requirement information includes a segmentation field and a segmentation rule corresponding to the segmentation field; Determining data segmentation status information of the data in the first data table according to the segmentation requirement information includes: Constructing data query conditions according to the segmentation fields and the segmentation rules; According to the data query condition, the data in the first data table is queried to determine data segmentation status information of the data in the first data table, wherein the data segmentation status information includes at least one of the following: split domain data and unsplit domain data.

3. The method according to claim 2, characterized in that The split domain data includes at least one of the following: complete split domain data and incomplete split domain data; the method further includes: Determining, according to the data structure of the first data table, the segmentation field, and the segmentation rule, an associated field associated with the segmentation field; In a case where the split domain data includes data corresponding to the associated field, determining that the split domain data is complete split domain data; Alternatively, when the split domain data does not include data corresponding to the associated field, it is determined that the split domain data is incomplete split domain data.

4. The method according to claim 2, characterized in that The split segmentation domain data includes at least one of the following: saturated segmentation domain data and unsaturated segmentation domain data; the method further includes: In a case where the data storage capacity of the segmented domain to which the split segmented domain data belongs is greater than or equal to the reference data capacity, determining that the split segmented domain data is saturated segmented domain data; Alternatively, when the data storage capacity of the segmented domain to which the split segmented domain data belongs is smaller than the reference data capacity, the split segmented domain data is determined to be non-saturated segmented domain data.

5. The method according to any one of claims 1 to 4, characterized in that Before dividing the first data table by the SQL statement to obtain at least two second data tables, the method further includes: Obtaining a first storage data volume of a first database to which the first data table belongs in each of N first time windows, where the N first time windows start from the Tth first time window and extend to the T+Nth first time window; determining, based on a first amount of stored data in each of the N first time windows, a second amount of stored data in the first database in a second time window, where the second time window occurs later than the N first time windows; When the difference between the second stored data volume and the first stored data volume corresponding to the T+Nth first time window is less than or equal to the preset data volume difference, within the reference time window, the data of the first data table is split using the SQL statement to obtain at least two second data tables, and the reference time window is determined based on the second time window.

6. The method according to any one of claims 1 to 4, characterized in that The first data table is split by the SQL statement to obtain at least two second data tables, including: When a preset split trigger condition is met, the first data table is split using the SQL statement to obtain at least two second data tables; The preset segmentation trigger condition includes at least one of the following: The data volume of the first data table is greater than or equal to a preset data volume threshold; The current time is within the reference time window; Receiving a second confirmation instruction from the user for the data segmentation task; The evaluation value corresponding to the resource load of the database to which the first data table belongs is within a preset evaluation value range; A service event associated with the first data table is detected.

7. The method according to any one of claims 1 to 4, characterized in that Before determining the SQL statement corresponding to the data segmentation status information from the preset structured query language SQL statement, the method further includes: Determining, based on a reference association relationship database, whether there is an SQL statement associated with the segmentation requirement information in the reference association relationship database; The determining of the SQL statement corresponding to the data segmentation status information from a preset structured query language SQL statement includes: In the case that there is no SQL statement associated with the segmentation requirement information in the reference association relationship library, an SQL statement corresponding to the data segmentation status information is determined from preset structured query language SQL statements.

8. The method according to claim 7, characterized in that After the first data table is split using the SQL statement to obtain at least two second data tables, the method further includes: Obtaining a split status of the first data table split by the SQL statement; When the split status includes a status indicating that the SQL statement successfully splits the first data table, establishing an association relationship between the SQL statement and the split requirement information; updating the reference association relationship library according to the association relationship between the SQL statement and the segmentation requirement information; The reference association relationship library is displayed in a display area of ​​the segmentation task interface.

9. A data processing device, characterized in that: The device comprises: A first receiving module is used to receive a first input from a user on the segmentation task interface; a first determining module, configured to determine a data segmentation task in response to the first input, the data segmentation task including a first data table and segmentation requirement information for segmenting the first data table; a second determining module, configured to determine data segmentation status information of the data in the first data table according to the segmentation requirement information, wherein the data segmentation status information is used to represent a record in which the data in the first data table is segmented; a third determining module, configured to determine an SQL statement corresponding to the data segmentation status information from a preset structured query language SQL statement, wherein the preset SQL statement is determined based on a plurality of reference segmentation strategies, each reference segmentation strategy corresponding to a reference segmentation domain rule and a reference data segmentation status information; The first segmentation module is configured to segment the first data table using the SQL statement to obtain at least two second data tables, wherein the second data tables include data of segmentation domains corresponding to the segmentation requirement information.

10. An electronic device, characterized in that: The device includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the data processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer storage medium stores computer program instructions, which, when executed by a processor, implement the data processing method according to any one of claims 1 to 8.

12. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to perform the data processing method according to any one of claims 1 to 8.

Images