Database partitioning method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of database partitioning, and discloses a database partitioning method and device, computer equipment and a storage medium. Batch data of the day are written into a new partition of a database, and after the batch data of the day are written into the new partition of the database, the new partition of the database is partitioned; asynchronously playing back data in the new partition and the latest historical partition of the database to a temporary table; after asynchronous playback of the temporary table is completed, deleting the new partition, and expanding the partition boundary of the historical newest partition to the time of the day to obtain a target historical newest partition; after performing partition exchange on the temporary table and the target historical latest partition, creating a next-day batch partition of the database; therefore, the query performance of the database in the batch business scene can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of database partitioning, and in particular to a database partitioning method, device, computer equipment and computer-readable storage medium. Background Art

[0002] At present, in today's information age, the demand for database systems in enterprise-level applications is growing, especially in key industries such as finance, telecommunications, and e-commerce. These industries face the challenges of storing and managing massive amounts of historical data, while also meeting the stringent requirements of high-concurrency queries and daily large-volume data writing. Massive historical data is like a huge information treasure house, containing rich business insights and value, but its storage and query efficiency has become a bottleneck restricting the development of enterprises. High-concurrency queries are coming like a tide, and users, systems, and business processes are racing against time to obtain data. Once the query efficiency is low, it will directly affect the user experience, the timeliness of business decisions, and the smoothness of enterprise operations. The daily large-volume data writing task is like a huge rock pressing down on the top, causing a huge impact on the I / O resources of the database system, not only slowing down the writing speed, but also triggering a chain reaction of query performance, causing the entire database system to fall into a performance quagmire.

[0003] In order to cope with this complex workload, database partitioning technology came into being and became a key means to improve the performance and scalability of database systems. Database partitioning divides a large data table into multiple smaller partitions, each partition storing a portion of the data, thereby significantly reducing the amount of data that needs to be scanned during query. When a user initiates a query request, the database system only needs to retrieve data in the relevant partitions instead of traversing the entire large table. This undoubtedly greatly improves query efficiency and makes data retrieval as easy and fast as catching fish in a small pond. At the same time, partitioning technology can also disperse the I / O pressure of write operations, distributing the write load originally concentrated on one storage device to the storage devices corresponding to multiple partitions, avoiding congestion of I / O resources and making data writing smoother, just like distributing traffic to multiple roads to alleviate traffic congestion.

[0004] However, in the database system of an enterprise, static partitioning is the most common application form. This partitioning method is usually defined by the database administrator in advance during the database design phase based on the estimated data distribution and query mode. Although static partitioning optimizes query efficiency to a certain extent, it also has many limitations. First, data distribution and query mode are not static. As time goes by and the business develops, the original partitioning rules may no longer apply, resulting in uneven data distribution, too much data in some partitions, and too little data in other partitions. This data skew phenomenon will seriously affect query performance and the rational use of storage resources. Secondly, static partitions are often concentrated on a storage device with database deployment, which means that all partitions share the I / O resources of the same storage device. In the case of large-scale write scenarios, the huge I / O resource consumption will inevitably affect the query operation. The write operation and the query operation compete for I / O resources, just like two forces colliding violently in the same narrow channel, resulting in longer query response time, slower write speed, and greatly reduced overall database performance.

[0005] Based on this, how to provide a database partitioning method, device, computer equipment and computer-readable storage medium that can effectively improve the query performance of the database in batch business scenarios is a problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide a database partitioning method, apparatus, computer equipment and computer-readable storage medium, aiming to solve the problem of how to effectively improve the query performance of the database in a batch business scenario.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] In a first aspect, the present invention provides a database partitioning method, which comprises:

[0009] Writing the batch data of the day into a newly created partition of the database, and after the writing of the batch data of the day into the database is completed, asynchronously playing back the data in the newly created partition and the latest historical partition of the database to a temporary table;

[0010] After the asynchronous playback of the temporary table is completed, the newly created partition is deleted, and the partition boundary of the latest historical partition is extended to the time of the day to obtain the target latest historical partition;

[0011] After performing partition exchange on the temporary table and the target historical latest partition, a next-day batch partition of the database is created.

[0012] In a second aspect, the present invention provides a database partitioning device, comprising:

[0013] A write and playback module, used to write the batch data of the day into a newly created partition of the database, and after the batch data of the day is written into the database, asynchronously playback the data in the newly created partition and the latest historical partition of the database to a temporary table;

[0014] A deletion and expansion module is used to delete the newly created partition after the asynchronous playback of the temporary table is completed, and to expand the partition boundary of the latest historical partition to the time of the day to obtain the target latest historical partition;

[0015] The exchange and creation module is used to create the next day's batch partition of the database after performing partition exchange on the temporary table and the target historical latest partition.

[0016] In a third aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the database partitioning method as described above when executing the computer program.

[0017] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program implements the database partitioning method as described above when executed by a processor.

[0018] Compared with the prior art, the present invention provides a database partitioning method, apparatus, computer device and computer-readable storage medium, wherein the day's batch data is written into a newly created partition of a database, and after the day's batch data is written into the database, the data in the newly created partition and the latest historical partition of the database are asynchronously played back to a temporary table; after the asynchronous playback of the temporary table is completed, the newly created partition is deleted, and the partition boundary of the latest historical partition is extended to the day time to obtain the target latest historical partition; after performing partition exchange on the temporary table and the target latest historical partition, the next day's batch partition of the database is created; thereby, the present invention can effectively improve the query performance of the database in the batch business scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of an application environment of a database partitioning method provided by an embodiment of the present invention.

[0021] Figure 2 A schematic diagram of a flow chart of a database partitioning method provided by an embodiment of the present invention.

[0022] Figure 3 A schematic diagram of a program module of a database partitioning device provided by an embodiment of the present invention.

[0023] Figure 4 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention.

[0024] Figure 5 Another structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0027] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0028] As used in the present specification and the appended claims, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0029] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0030] References to "one embodiment" or "some embodiments" etc. described in the present specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0031] It should be understood that the order of execution of the steps in the following embodiments does not imply a precedence of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0032] In order to illustrate the technical solution of the present invention, specific embodiments are provided below for illustration.

[0033] An embodiment of the present invention provides a database partitioning method, which can be applied in Figure 1 In the application environment shown, the client and the server communicate through the network. The client includes but is not limited to PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, personal digital assistants (PDAs) and other computer devices. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0034] See also Figure 2 An embodiment of the present invention provides a database partitioning method, wherein the method comprises the following steps:

[0035] S100, writing the batch data of the day into a newly created partition of the database, and after the batch data of the day is written into the database, asynchronously playing back the data in the newly created partition and the latest historical partition of the database to a temporary table;

[0036] S200, after the asynchronous playback of the temporary table is completed, the newly created partition is deleted, and the partition boundary of the latest historical partition is extended to the time of the day to obtain the target latest historical partition;

[0037] S300: After performing partition exchange on the temporary table and the target historical latest partition, create a next-day batch partition for the database.

[0038] During specific implementation, this embodiment achieves the technical effect of effectively improving database query performance in the batch business scenario through carefully designed steps. First, in step S100, the batch data of the day is written into the newly created partition. This operation separates the new data from the historical data, avoids the interference of the new data writing on the existing query operation, and reduces the amount of data that needs to be scanned during the query, thereby improving the query efficiency. At the same time, the data of the newly created partition and the latest historical partition are asynchronously played back to the temporary table, so that the data integration work is carried out in the background, and the query operation will not be blocked, which further ensures the stability of the query performance. Secondly, in step S200, the newly created partition is deleted and the partition boundary of the latest historical partition is expanded. This process simplifies the data structure, reduces the number of partitions, and reduces the complexity of database management. At the same time, the target latest historical partition (that is, the latest historical partition corresponding to the database when the batch is run the next day) can contain all the data of the day, providing a complete and continuous data basis for subsequent query operations, which helps to optimize the query path and improve the query speed. Finally, in step S300, after executing the partition exchange, the next day's batch partition is created (i.e., the newly created partition corresponding to the database when the batch is run the next day). This operation realizes the rapid update of data and the dynamic maintenance of the partition, ensuring that the database can reflect the latest business data in a timely manner, and also prepares for the writing of the next (next day) batch data, avoiding the impact on the database query performance when the data is written. Overall, this implementation method realizes the effective improvement of database query performance in the batch business scenario by reasonably planning the partition strategy and optimizing the data processing process, and provides strong support for high-concurrency queries and large-volume data writing.

[0039] It should be noted that in the patent scope to be protected and disclosed by the present invention, "same day" and "next day" refer to specific time periods, which can be limited according to actual needs, such as 12 hours, 24 hours, etc.

[0040] Furthermore, in one embodiment, in the database partitioning method, the newly created partition exclusively occupies a first storage device, the temporary table exclusively occupies a second storage device, and the latest historical partition exclusively occupies a third storage device.

[0041] In specific implementation, this embodiment allows the newly created partition to exclusively occupy the first storage device, the temporary table to exclusively occupy the second storage device, and the latest historical partition to exclusively occupy the third storage device. The database partitioning method of this embodiment realizes the complete isolation of different data sets at the physical storage level. This isolation method can significantly reduce I / O conflicts during data operations, because the data read and write operations of each partition will only be performed on its corresponding storage device, and will not interfere with the data operations of other partitions. For example, when writing the batch data of the day, the first storage device can run at full speed without being affected by other queries or data integration operations; similarly, when asynchronously replaying data to the temporary table, the second storage device can efficiently handle data integration tasks without dragging down the performance of the third storage device when querying historical data. In addition, this strategy of exclusive storage devices also helps to more accurately monitor and manage the performance of each partition, and can perform targeted performance optimization and resource allocation according to the load conditions and data characteristics of each storage device, such as configuring a higher capacity hard disk for the third storage device that stores a large amount of historical data, and equipping the second storage device that needs to be read and written frequently with a solid-state hard disk with higher I / O performance. In general, this technical approach of the present embodiment greatly improves the data processing efficiency and stability of the database system, and provides a solid and reliable storage foundation for high-concurrency queries and large-volume data writing.

[0042] Further, in one embodiment, the database partitioning method, wherein the step S100, writing the batch data of the day into the newly created partition of the database, and after the batch data of the day is written into the database, asynchronously playing back the data in the newly created partition and the latest historical partition of the database to a temporary table, specifically comprises the steps of:

[0043] Determine the newly created partition of the database, and write the batch data of the day into the newly created partition;

[0044] After the batch data of the day is written into the database, the data in the newly created partition and the latest historical partition of the database are asynchronously played back to the temporary table;

[0045] During the asynchronous playback process, the progress and status of the data playback in the temporary table are monitored in real time.

[0046] In specific implementation, this embodiment realizes an efficient and reliable data processing process by accurately controlling the data flow and monitoring the data status in real time. First, the newly created partition is clearly determined and is specifically used to write the batch data of the day. This targeted data writing strategy avoids data mixing, ensures the orderliness and traceability of the data, and lays a solid foundation for subsequent data analysis and query. Secondly, after the batch data of the day is written, the data of the newly created partition and the latest historical partition are asynchronously replayed to the temporary table. This process makes full use of the advantages of asynchronous operations, so that data integration work can be carried out efficiently in the background, and other operations of the database will not be blocked, ensuring the smooth operation of the database system. At the same time, asynchronous playback can also reduce data redundancy. By integrating new and old data at one time, data duplication storage is avoided and storage space is saved. Finally, the progress and status of data playback in the temporary table are monitored in real time during the asynchronous playback process. This monitoring mechanism provides visualization and controllability for data processing, so that database administrators can grasp the details of data playback in a timely manner, such as the speed of data playback, the amount of data played back, whether there are errors or exceptions, etc. Once a problem is found, measures can be taken quickly to deal with it, such as pausing playback, fixing errors, optimizing playback strategies, etc., so as to ensure the accuracy and integrity of data playback, avoid data loss or damage, and improve the reliability of data processing. Overall, these steps of this embodiment work together to enable the database to efficiently and stably process massive amounts of data in batch business scenarios, meet the needs of high-concurrency queries and large-volume data writing, and provide strong support for the company's data management and business operations.

[0047] Further, in one embodiment, the database partitioning method, wherein the step S200, after the asynchronous playback of the temporary table is completed, deletes the newly created partition, and expands the partition boundary of the latest historical partition to the time of the day to obtain the target latest historical partition, specifically includes the steps of:

[0048] After the asynchronous playback of the temporary table is completed, verifying the data in the temporary table;

[0049] When the data in the temporary table is successfully verified, the newly created partition is deleted, and the partition boundary of the latest historical partition is extended to the time of the day to obtain the target latest historical partition.

[0050] During specific implementation, this embodiment ensures the accuracy of data and the efficient operation of the database through strict data verification and timely partition adjustment. After the asynchronous playback of the temporary table is completed, the data in the temporary table is verified. This step is crucial. It can detect whether the data is wrong or lost during the playback process to ensure the integrity and consistency of the data. Only when the data verification is successful will the subsequent operations of deleting the newly created partition and expanding the partition boundaries of the latest historical partition be executed. This conditional execution process greatly reduces the risk of data errors. Deleting the newly created partition releases storage space, simplifies the database structure, and reduces the complexity of subsequent management and maintenance; at the same time, the partition boundaries of the latest historical partition are expanded to the time of the day, and the target latest historical partition is obtained. This operation enables the database to reflect the latest business data in a timely manner, providing a complete and continuous data foundation for subsequent queries and analysis, helping to optimize the query path and improve query efficiency. Overall, these steps of this embodiment jointly realize the efficient integration and accurate update of data, ensure the stability and reliability of the database in the batch business scenario, and provide strong support for the data management and business decision-making of the enterprise.

[0051] Furthermore, in one embodiment, the database partitioning method, wherein the step S300, after performing partition exchange on the temporary table and the target historical latest partition, creates the next day batch partition of the database, specifically comprises the steps of:

[0052] After performing partition exchange on the data and indexes in the temporary table and the target historical latest partition, verifying the exchange result;

[0053] When the exchange result is successful, the next day's batch partition of the database is created.

[0054] During specific implementation, this embodiment achieves efficient updating and continuous and stable operation of the database by ensuring the accuracy of data exchange and creating new batch partitions in a timely manner. After performing partition exchange on the temporary table and the latest partition of the target history, the exchange result is immediately verified. This step can promptly discover and handle problems that may arise during the exchange process, such as data loss, index errors, etc., to ensure that the integrity and consistency of the data are guaranteed. Only when the verification result shows that the exchange is successful will the next day's batch partition be created. This conditional execution process avoids the creation of new partitions when data exchange fails, thereby preventing potential data errors from affecting subsequent batch operations. Creating the next day's batch partition prepares for the writing of the next batch data, ensures that the database can continuously process new business data, and maintains the efficient operation of the database system and the continuity of the business. Overall, these steps of this embodiment jointly ensure the data accuracy and system stability of the database in the batch business scenario, and provide strong support for the data management and business operations of the enterprise.

[0055] Furthermore, in one embodiment, the database partitioning method, wherein when the exchange result is successful, creating the next day batch partition of the database, further comprises:

[0056] When the exchange result is successful, the second storage device and the third storage device are exchanged for use.

[0057] In specific implementation, in this embodiment, when the partition exchange result of the data and index in the temporary table and the target historical latest partition is a successful exchange, the second storage device and the third storage device are exchanged for use. This strategy can achieve efficient recycling and load balancing of storage resources. After the partition exchange is successful, the second storage device that originally carried the temporary table data has completed its task, and the third storage device originally carried the data of the historical latest partition. By exchanging the two storage devices, the second storage device can be reused to avoid idle resources, and the third storage device can be released to provide storage space for subsequent data processing tasks. This way of dynamically adjusting the use of storage devices helps to balance the load of different storage devices, avoid the performance degradation or failure of some storage devices due to continuous high load, thereby improving the stability and performance of the entire database system. In addition, it also provides greater flexibility for the expansion and maintenance of the database. According to actual business needs and data growth, the use strategy of the storage device can be flexibly adjusted to optimize the allocation of storage resources and ensure that the database can efficiently support the data management and business operations of the enterprise.

[0058] Furthermore, in order to improve the availability and scalability of the database partitioning method of the present invention, in some embodiments, a method similar to a "consistent hash ring" is introduced to optimize the organization of partition-storage device mapping:

[0059] By hashing the storage device ID, you can confirm the location of the storage device on the hash ring, such as Disk1, Disk2, Disk3, etc.; when binding the new partition PartitionN to the disk, use the same hash algorithm to calculate the partition ID to confirm the location of the partition on the hash ring, and use this location as the origin to take the next storage node Disk2 clockwise on the hash ring for binding. When the storage device Disk2 bound to the new partition fails, just continue to take the next available storage node Disk3 clockwise along the hash ring. At the same time, in view of the need to identify the health status of the storage device, a background resident process can be used to determine the health status of the storage device and synchronously update the catalog information of the storage node status.

[0060] Furthermore, when using the "consistent hash ring" to organize the partition-storage device mapping relationship, given the problem of hash collisions that easily lead to identical values, this may cause many partitions to be assigned to the same storage node in actual use, resulting in data skew. In addition, the multiple storage devices on the server in the actual operating environment may have different performance (capacity). Simply selecting available storage nodes to bind partitions may result in insufficient available space on small-capacity storage devices, while large-capacity storage devices are still largely idle. In order to balance data skew and storage space utilization issues, in some embodiments, the "consistent hash ring with virtual nodes" may be cited to further optimize the above-mentioned organization method:

[0061] By setting the standard capacity of the virtual storage node (1T), when cataloging a new storage device, it can be split into multiple virtual storage nodes according to the actual available storage capacity of the storage device. Calculate the hash value of the newly added partition to confirm its position on the hash ring, take the next virtual storage node clockwise, and use its corresponding actual storage node to bind it to the partition. By introducing a consistent hash ring for virtual nodes, the capacity differences of storage devices of different capacities on the hash ring are eliminated, and large-capacity storage devices naturally have higher weights in actual allocation. In addition, by appropriately setting the standard capacity size of the virtual storage node (such as 100M), a large number of blanks on the hash ring can be filled, eliminating the problem of hash collisions with the same value that is prone to occur when there are fewer nodes, better leveraging the advantages of the hash algorithm, and better allocating storage nodes.

[0062] In addition, in some embodiments, in addition to monitoring the health status of the storage device, the background process can also collect the capacity water level and write it into the storage device catalog information. After the newly added partition finds an available storage node on the hash ring, it determines the capacity water level of the storage node. If it reaches the preset threshold (90%), it continues to look for the next available storage node.

[0063] It can be seen from the above method embodiments that the database partitioning method provided by the present invention includes: writing the batch data of the day into the newly created partition of the database, and after the writing of the batch data of the day in the database is completed, asynchronously replaying the data in the newly created partition and the latest historical partition of the database to a temporary table; after the asynchronous playback of the temporary table is completed, deleting the newly created partition, and expanding the partition boundary of the latest historical partition to the time of the day to obtain the target latest historical partition; after performing partition exchange on the temporary table and the target latest historical partition, creating the next day's batch partition of the database. In this way, the query performance of the database in the batch business scenario can be effectively improved by the method of the present invention.

[0064] It should be understood that, although the present application provides method operation steps as described in the embodiments or flowcharts, more or less operation steps may be included based on conventional or non-creative labor, and these operation steps are not necessarily performed in sequence according to the embodiment or flowchart. The order of steps listed in the embodiment or flowchart is only one way of executing the order of many steps, and does not represent the only execution order. It should be noted that there is not necessarily a certain order between the above steps. A person of ordinary skill in the art can understand from the description of the embodiment of the present invention that in different embodiments, the above steps may have different execution orders, that is, they may be executed in parallel, or they may be executed in exchange, etc. Moreover, at least a part of the steps in the embodiment or flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but may be executed in turn, alternately or synchronously with other steps or at least a part of the sub-steps or stages of other steps.

[0065] Based on the above method embodiment, please refer to Figure 3 Another embodiment of the present invention further provides a database partitioning device, wherein the device comprises:

[0066] The writing and playback module 11 is used to write the batch data of the day into the newly created partition of the database, and after the batch data of the day is written into the database, asynchronously playback the data in the newly created partition and the latest historical partition of the database to a temporary table;

[0067] The deletion and expansion module 12 is used to delete the newly created partition after the asynchronous playback of the temporary table is completed, and expand the partition boundary of the latest historical partition to the time of the day to obtain the target latest historical partition;

[0068] The exchange and creation module 13 is used to create the next day's batch partition of the database after performing partition exchange on the temporary table and the target historical latest partition.

[0069] Furthermore, in one embodiment, in the database partitioning apparatus, the newly created partition exclusively occupies a first storage device, the temporary table exclusively occupies a second storage device, and the latest historical partition exclusively occupies a third storage device.

[0070] Furthermore, in one embodiment, in the database partitioning device, the writing and playback module 11 is specifically used for:

[0071] Determine the newly created partition of the database, and write the batch data of the day into the newly created partition;

[0072] After the batch data of the day is written into the database, the data in the newly created partition and the latest historical partition of the database are asynchronously played back to the temporary table;

[0073] During the asynchronous playback process, the progress and status of the data playback in the temporary table are monitored in real time.

[0074] Furthermore, in one embodiment, in the database partitioning device, the deletion and expansion module 12 is specifically used for:

[0075] After the asynchronous playback of the temporary table is completed, verifying the data in the temporary table;

[0076] When the data in the temporary table is successfully verified, the newly created partition is deleted, and the partition boundary of the latest historical partition is extended to the time of the day to obtain the target latest historical partition.

[0077] Furthermore, in one embodiment, in the database partitioning device, the exchange and creation module 13 is specifically used for:

[0078] After performing partition exchange on the data and indexes in the temporary table and the target historical latest partition, verifying the exchange result;

[0079] When the exchange result is successful, the next day's batch partition of the database is created.

[0080] Furthermore, in one embodiment, the database partitioning device, wherein when the exchange result is successful, creates the next day batch partition of the database, further comprising:

[0081] When the exchange result is successful, the second storage device and the third storage device are exchanged for use.

[0082] It should be noted that in the embodiment of the device of the present invention, the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the aforementioned method embodiment part and will not be repeated here.

[0083] Based on the above method embodiment, another embodiment of the present invention further provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the functions or steps of the database partitioning method server side in any of the above method embodiments are implemented.

[0084] Based on the above method embodiment, another embodiment of the present invention further provides a computer device, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the functions or steps of the client side of the database partitioning method in any of the above method embodiments are implemented.

[0085] Those skilled in the art will understand that Figure 4 and Figure 5The structural schematic diagram shown in the figure is only a schematic diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0086] The processor may be a CPU, or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0087] Among them, the memory includes a readable storage medium, an internal memory, etc., wherein the internal memory can be the memory of a computer device, and the internal memory provides an environment for the operation of an operating system and computer-readable instructions in the readable storage medium. The readable storage medium can be a hard disk of a computer device, and in other embodiments, it can also be an external storage device of a computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on a computer device. Further, the memory can also include both an internal storage unit of a computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program, etc. The memory can also be used to temporarily store data that has been output or is to be output.

[0088] Based on the above method embodiments, another embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the database partitioning method in any of the above method embodiments is implemented. The computer-readable storage medium may be non-volatile or volatile.

[0089] It should be noted that the above-mentioned functions or steps that can be implemented by the computer-readable storage medium or computer device, and the technical effects brought about by the functions / steps, can be found in the relevant descriptions in the aforementioned method embodiments. To avoid repetition, they will not be described one by one here.

[0090] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM). The disclosed memory components or memories of the operating environments described herein are intended to comprise one or more of these and / or any other suitable types of memory.

[0091] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, in the embodiment of the device of the present invention, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0092] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0093] In the embodiments provided by the present invention, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0094] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0095] It should be noted that if software tools or components other than those of the Company appear in the embodiments of the present application, they are only used for illustration and do not represent actual use. The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A database partitioning method, characterized in that: include: Writing the batch data of the day into a newly created partition of the database, and after the writing of the batch data of the day into the database is completed, asynchronously replaying the data in the newly created partition and the latest historical partition of the database to a temporary table; After the asynchronous playback of the temporary table is completed, the newly created partition is deleted, and the partition boundary of the latest historical partition is extended to the time of the day to obtain the target latest historical partition; After performing partition exchange on the temporary table and the target historical latest partition, a next-day batch partition of the database is created.

2. The database partitioning method according to claim 1, characterized in that: The step of writing the batch data of the day into the newly created partition of the database, and after the batch data of the day is written into the database, asynchronously playing back the data in the newly created partition and the latest historical partition of the database to a temporary table includes: Determine the newly created partition of the database, and write the batch data of the day into the newly created partition; After the batch data of the day is written into the database, the data in the newly created partition and the latest historical partition of the database are asynchronously played back to the temporary table; During the asynchronous playback process, the progress and status of the data playback in the temporary table are monitored in real time.

3. The database partitioning method according to claim 1, characterized in that: After the asynchronous playback of the temporary table is completed, the newly created partition is deleted, and the partition boundary of the latest historical partition is extended to the time of the day to obtain the target latest historical partition, including: After the asynchronous playback of the temporary table is completed, verifying the data in the temporary table; When the data in the temporary table is successfully verified, the newly created partition is deleted, and the partition boundary of the latest historical partition is extended to the time of the day to obtain the target latest historical partition.

4. The database partitioning method according to claim 1, 2 or 3, characterized in that: The newly created partition exclusively occupies the first storage device, the temporary table exclusively occupies the second storage device, and the latest historical partition exclusively occupies the third storage device.

5. The database partitioning method according to claim 4, characterized in that: After performing partition exchange on the temporary table and the target historical latest partition, creating a next-day batch partition of the database includes: After performing partition exchange on the data and indexes in the temporary table and the target historical latest partition, verifying the exchange result; When the exchange result is successful, the next day's batch partition of the database is created.

6. The database partitioning method according to claim 5, characterized in that: When the exchange result is successful, creating the next day batch partition of the database, further comprising: When the exchange result is successful, the second storage device and the third storage device are exchanged for use.

7. A database partitioning device, characterized in that: include: A write and playback module, used to write the batch data of the day into a newly created partition of the database, and after the batch data of the day is written into the database, asynchronously playback the data in the newly created partition and the latest historical partition of the database to a temporary table; A deletion and expansion module is used to delete the newly created partition after the asynchronous playback of the temporary table is completed, and to expand the partition boundary of the latest historical partition to the time of the day to obtain the target latest historical partition; The exchange and creation module is used to create the next day's batch partition of the database after performing partition exchange on the temporary table and the target historical latest partition.

8. The database partitioning device according to claim 7, characterized in that: The newly created partition exclusively occupies the first storage device, the temporary table exclusively occupies the second storage device, and the latest historical partition exclusively occupies the third storage device.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the database partitioning method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the database partitioning method according to any one of claims 1 to 6 is implemented.

Images