Database table read-write method and device, computer equipment and storage medium

Abstract

The invention relates to a database table read-write method and device, computer equipment, a storage medium and a computer program product, and is applied to the technical field of big data. The method comprises the following steps: determining to-be-transmitted financial data, and obtaining data partition information corresponding to each piece of to-be-transmitted financial data from a first database table; based on the data partition information, performing concurrent reading on the to-be-transmitted financial data corresponding to different data partitions in the first database table; and concurrently writing the concurrently read financial data to be transmitted into a second database table according to the data partition information. By adopting the method, concurrent writing and concurrent reading of the data of different data partitions can be realized based on the data partition information of the financial data to be transmitted, and due to the fact that reading and writing of the data of different partitions do not influence each other, the mode can be beneficial to improving the reading and writing efficiency of a financial database table.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a database table reading and writing method, apparatus, computer equipment, storage medium, and computer program product. Background Art

[0002] With the development of big data technology, a technology has emerged that transfers processed financial data from big data to other databases for storage. This technology can use big data processing tools to read the financial data that needs to be transferred to other database tables from the database tables of the processed database, and use the tool to write the read financial data into other database tables for storage.

[0003] Traditionally, database table reading and writing typically involves concurrently reading data from the original database storing the financial data to be transferred, and then concurrently writing the read financial data to other database tables. However, this concurrent reading and writing approach still results in low efficiency for financial data reading and writing. Summary of the Invention

[0004] Based on this, it is necessary to provide a database table reading and writing method, device, computer equipment, computer-readable storage medium and computer program product that can improve the efficiency of financial data reading and writing in order to address the above technical problems.

[0005] In a first aspect, the present application provides a method for reading and writing a database table, the method comprising:

[0006] Determining financial data to be transmitted, and obtaining data partition information corresponding to each of the financial data to be transmitted from the first database table;

[0007] Based on the data partition information, concurrently reading the to-be-transmitted financial data corresponding to different data partitions in the first database table;

[0008] The concurrently read financial data to be transmitted is concurrently written into the second database table according to the data partition information.

[0009] In one embodiment, the concurrent reading of the financial data to be transmitted corresponding to different data partitions in the first database table based on the data partition information includes: based on the data partition information, classifying the financial data to be transmitted corresponding to the same data partition information to obtain multiple financial data sets to be transmitted corresponding to different data partitions; obtaining the concurrent reading data volume set for each of the financial data sets to be transmitted; for each of the financial data sets to be transmitted, according to the concurrent reading data volume, determining the current financial data to be transmitted corresponding to each of the financial data sets to be transmitted from the financial data to be transmitted contained in each of the financial data sets to be transmitted; and concurrently reading the current financial data to be transmitted corresponding to each of the financial data sets to be transmitted.

[0010] In one embodiment, for each of the financial data sets to be transmitted, according to the concurrently read data volume, the current financial data to be transmitted corresponding to each of the financial data sets to be transmitted is determined from the financial data to be transmitted contained in each of the financial data sets to be transmitted, including: obtaining the current financial data set to be transmitted, and the current concurrently read data volume corresponding to the current financial data set to be transmitted; determining candidate transmission data from the current financial data set to be transmitted, and obtaining a data ranking of each of the candidate transmission data; based on the data ranking and the current concurrently read data volume, determining the current financial data to be transmitted corresponding to the current financial data set to be transmitted from the candidate transmission data.

[0011] In one embodiment, determining candidate transmission data from the current set of financial data to be transmitted includes: taking the financial data to be transmitted that does not carry a transmitted identifier among the financial data to be transmitted contained in the current set of financial data to be transmitted as the candidate transmission data; the transmitted identifier is used to indicate that the financial data to be transmitted has been written into the second database table; and after concurrently reading the current financial data to be transmitted corresponding to each of the financial data sets to be transmitted, further includes: adding the transmitted identifier to the current financial data to be transmitted corresponding to each of the financial data sets to be transmitted.

[0012] In one embodiment, after concurrently writing the concurrently read financial data to be transmitted into the second database table according to the data partition information, the step further includes: returning to execute the step of using the financial data to be transmitted included in the current set of financial data to be transmitted, which does not carry the transmitted identifier, as the candidate transmission data, until all the financial data to be transmitted included in each set of financial data to be transmitted carries the transmitted identifier.

[0013] In one embodiment, the concurrently reading financial data to be transmitted and writing them concurrently into a second database table according to the data partition information includes: obtaining multiple data partitions contained in the second database table; determining, from the multiple data partitions contained in the second database table according to the data partition information, the partitions to be written corresponding to each of the financial data to be transmitted; and concurrently writing each of the financial data to be transmitted into the partitions to be written corresponding to each of the financial data to be transmitted.

[0014] In a second aspect, the present application further provides a database table reading and writing device, the device comprising:

[0015] a data partition acquisition module, configured to determine the financial data to be transmitted and obtain data partition information corresponding to each of the financial data to be transmitted from the first database table;

[0016] a concurrent data reading module, configured to concurrently read the financial data to be transmitted corresponding to different data partitions in the first database table based on the data partition information;

[0017] The data concurrent writing module is used to concurrently write the concurrently read financial data to be transmitted into the second database table according to the data partition information.

[0018] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0019] Determining financial data to be transmitted, and obtaining data partition information corresponding to each of the financial data to be transmitted from the first database table;

[0020] Based on the data partition information, concurrently reading the to-be-transmitted financial data corresponding to different data partitions in the first database table;

[0021] The concurrently read financial data to be transmitted is concurrently written into the second database table according to the data partition information.

[0022] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0023] Determining financial data to be transmitted, and obtaining data partition information corresponding to each of the financial data to be transmitted from the first database table;

[0024] Based on the data partition information, concurrently reading the to-be-transmitted financial data corresponding to different data partitions in the first database table;

[0025] The concurrently read financial data to be transmitted is concurrently written into the second database table according to the data partition information.

[0026] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0027] Determining financial data to be transmitted, and obtaining data partition information corresponding to each of the financial data to be transmitted from the first database table;

[0028] Based on the data partition information, concurrently reading the to-be-transmitted financial data corresponding to different data partitions in the first database table;

[0029] The concurrently read financial data to be transmitted is concurrently written into the second database table according to the data partition information.

[0030] The above-mentioned database table reading and writing method, apparatus, computer equipment, storage medium and computer program product determine the financial data to be transmitted and obtain the data partition information corresponding to each financial data to be transmitted from the first database table; based on the data partition information, concurrently read the financial data to be transmitted corresponding to different data partitions in the first database table; and concurrently write the concurrently read financial data to be transmitted to the second database table according to the data partition information. The present application determines the data partition information of each financial data to be transmitted that needs to be transmitted to the second database table from the first data table, thereby concurrently reading the financial data to be transmitted in different data partitions in the first database table based on the above data partition information, and when writing to the second database table, the financial data to be transmitted can also be concurrently written according to the above data partition information. The present application can realize concurrent writing and concurrent reading of data in different data partitions based on the data partition information of the financial data to be transmitted. Since the reading and writing of data in different partitions do not affect each other, this method can be beneficial to improving the reading and writing efficiency of the financial database table. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG1 is an application environment diagram of a database table reading and writing method according to an embodiment;

[0032] FIG2 is a schematic diagram of a flow chart of a method for reading and writing a database table in one embodiment;

[0033] FIG3 is a schematic diagram of a process for concurrently reading financial data to be transmitted in one embodiment;

[0034] FIG4 is a schematic diagram of a process for determining the current financial data to be transmitted in one embodiment;

[0035] FIG5 is a schematic diagram of a process for concurrently writing financial data to be transmitted in one embodiment;

[0036] FIG6 is a block diagram of a database table reading and writing device according to an embodiment;

[0037] FIG7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0039] The database table reading and writing method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. Among them, the data reading and writing platform 102 communicates with the first server 101 and the second server 103 through the network. The first server 101 can store a first database table, and the second server 103 can store a second database table. The first server 101 and the second server 102 can be financial servers. Specifically, the data reading and writing platform 102 can first establish a connection with the first server 101, and after determining the financial data to be transmitted, obtain the data partition information of each financial data to be transmitted in the first database 101 from the first database table carried by the first database 101, and then based on the above data partition information, the financial data to be transmitted in different data partitions in the first database table can be concurrently read. Afterwards, the data reading and writing platform 102 can disconnect from the first database 101, thereby connecting to the second database 103, and concurrently write the financial data to be transmitted read from the concurrent reading to the second database table in the second database 103 according to the data partition information. The first server 101 , the data reading and writing platform 102 and the second database 103 may be implemented by using independent servers or a server cluster consisting of multiple servers.

[0040] In one embodiment, as shown in FIG2 , a method for reading and writing a database table is provided. The method is described by taking the application of the method to the data reading and writing platform 102 in FIG1 as an example, and includes the following steps:

[0041] Step S201: determine the financial data to be transmitted, and obtain data partition information corresponding to each financial data to be transmitted from the first database table.

[0042] The financial data to be transmitted refers to financial data that needs to be transferred and written to the second database 103. For example, it can be financial data processed in the first database 101. The first database table is a database table stored in the first database 101. The data partition information refers to the partition information of the data partitions storing the financial data to be transmitted. In this embodiment, the first database table can contain the data partition information of each data stored in the first database 101. After the data reading and writing platform 102 determines the financial data to be transmitted, it can further obtain the data partition information corresponding to each piece of financial data to be transmitted from the first database table.

[0043] Step S202 : Based on the data partition information, concurrently read the financial data to be transmitted corresponding to different data partitions in the first database table.

[0044] Since the reading and writing of data in different partitions do not affect each other, when concurrently reading data, the financial data to be transmitted corresponding to different data partitions in the first database table can be concurrently read, thereby improving the efficiency of reading the financial data to be transmitted from the first database table.

[0045] Step S203: The concurrently read financial data to be transmitted is concurrently written into the second database table according to the data partition information.

[0046] The second database table is a database table in the second database 103. Similar to step S202, since the reading and writing of data in different partitions do not affect each other, when concurrently writing financial data, the financial data to be transmitted in different data partitions can be concurrently written to the second database table in the second database 103, thereby improving the efficiency of writing the financial data to be transmitted into the second database table.

[0047] In the above-mentioned database table reading and writing method, by determining the financial data to be transmitted and obtaining the data partition information corresponding to each financial data to be transmitted from the first database table; based on the data partition information, concurrently reading the financial data to be transmitted corresponding to different data partitions in the first database table; and concurrently writing the concurrently read financial data to be transmitted to the second database table according to the data partition information. The present application determines the data partition information of each financial data to be transmitted that needs to be transmitted to the second database table from the first data table, thereby concurrently reading the financial data to be transmitted in different data partitions in the first database table based on the above-mentioned data partition information, and when writing to the second database table, the financial data to be transmitted can also be concurrently written according to the above-mentioned data partition information. The present application can realize concurrent writing and concurrent reading of data in different data partitions based on the data partition information of the financial data to be transmitted. Since the reading and writing of data in different partitions do not affect each other, this method can be beneficial to improving the reading and writing efficiency of the financial database table.

[0048] In one embodiment, as shown in FIG3 , step S202 may further include:

[0049] Step S301 : Based on the data partition information, the financial data to be transmitted corresponding to the same data partition information are classified and processed to obtain a plurality of financial data sets to be transmitted corresponding to different data partitions.

[0050] The financial data set to be transmitted refers to a data set formed by multiple financial data to be transmitted. In this embodiment, the data reading and writing platform 102 can first classify the financial data to be transmitted corresponding to the same data partition based on the data partition information, thereby forming a financial data set to be transmitted corresponding to the financial data to be transmitted in different data partitions.

[0051] For example, the financial data to be transmitted may include data 1, data 2, data 3, data 4 and data 5, where the data partition corresponding to data 1 and data 3 is partition A, the data partition corresponding to data 2 and data 4 is partition B, and the data partition corresponding to data 5 is partition C. Then the data reading and writing platform 102 can divide the above-mentioned financial data to be transmitted into set A, set B and set C, where set A contains data 1 and data 3, set B contains data 2 and data 4, and set C contains data 5.

[0052] Step S302: obtaining the concurrent read data volume set for each financial data set to be transmitted;

[0053] Step S303 : for each financial data set to be transmitted, according to the concurrently read data volume, determine the current financial data to be transmitted corresponding to each financial data set to be transmitted from the financial data to be transmitted contained in each financial data set to be transmitted.

[0054] The concurrent read data volume refers to the amount of data that can be concurrently read from each set of financial data to be transmitted, while the current set of financial data to be transmitted refers to the financial data to be transmitted that is being read and transmitted. In this embodiment, each set of financial data to be transmitted is set with a maximum data volume for a single concurrent read transmission, which is the concurrent read data volume. After the data reading and writing platform 102 determines the concurrent read data volume corresponding to each set of financial data to be transmitted, it can also filter the current set of financial data to be transmitted from the financial data to be transmitted contained in each set of financial data to be transmitted based on the concurrent read data volume. For example, for a set A containing 1000 financial data to be transmitted, if the concurrent read data volume corresponding to set A is 300, then 300 data items can be filtered out of the 1000 financial data to be transmitted as the current set of financial data to be transmitted.

[0055] Step S304 : concurrently reading the current financial data to be transmitted corresponding to each of the financial data sets to be transmitted.

[0056] After determining the current financial data to be transmitted corresponding to each set of financial data to be transmitted, the data reading and writing platform 102 may also concurrently read the current financial data to be transmitted of each set of financial data to be transmitted.

[0057] In this embodiment, the data reading and writing platform 102 can also pre-set the concurrent reading data volume for the financial data sets to be transmitted in different data partitions, thereby determining the financial data to be transmitted that is read at each data partition based on the concurrent reading data volume, thereby concurrently reading the financial data to be transmitted at multiple data partitions, which can further improve the efficiency of financial data reading.

[0058] Furthermore, as shown in FIG4 , step S303 may further include:

[0059] Step S401: obtaining the current financial data set to be transmitted and the current concurrent read data volume corresponding to the current financial data set to be transmitted.

[0060] The current financial data set to be transmitted refers to any one of the multiple financial data sets to be transmitted in different data partitions, and the current concurrent read data volume refers to the concurrent read data volume corresponding to the current financial data set to be transmitted. The data reading and writing platform 102 can combine any one of the financial data sets to be transmitted as the current financial data set to be transmitted, and use the concurrent read data volume corresponding to the current financial data set to be transmitted as the current concurrent read data volume.

[0061] Step S402 : determining candidate transmission data from the current set of financial data to be transmitted, and obtaining the data ranking of each candidate transmission data.

[0062] Candidate transmission data refers to the financial data to be transmitted that may be read from the current set of financial data to be transmitted and transferred and written to the second database table. For example, it may be data within the current set of financial data to be transmitted that has not yet been written to the second database table. Because the transmission of the financial data to be transmitted in the current set of financial data to be transmitted is affected by the current concurrent read data volume, if the data is large, multiple reads, transfers, and writes are often required. If this read is the first read, the candidate transmission data may be all the financial data to be transmitted in the current set of financial data to be transmitted. If this read is not the first read, the candidate transmission data may be the financial data to be transmitted that has not yet been read or written.

[0063] Data sorting refers to the data order of candidate transmission data in the current set of financial data to be transmitted. The data reading and writing platform 102 can determine the financial data to be transmitted that has not been read or written in the current set of financial data to be transmitted as candidate transmission data, and can determine the data order of each candidate transmission data in the current set of financial data to be transmitted.

[0064] Step S403 : Based on the data sorting and the current concurrently read data volume, the current financial data to be transmitted corresponding to the current financial data set to be transmitted is determined from the candidate transmission data.

[0065] Finally, the data reading and writing platform 102 can determine the current financial data to be transmitted corresponding to the current set of financial data to be transmitted from the candidate transmission data based on the data sorting and the current concurrent read data volume. For example, the candidate transmission data with the same number of candidate transmission data as the current concurrent read data volume before sorting can be used as the current financial data to be transmitted. For example, if the current concurrent read data volume is 300, the data reading and writing platform 102 can sort the candidate transmission data that ranks in the top 300 as the current financial data to be transmitted. In this way, the financial data to be transmitted corresponding to each set of financial data to be transmitted can be determined.

[0066] In this embodiment, the data reading and writing platform 102 can determine the financial data to be transmitted corresponding to each financial data set to be transmitted based on the concurrent read data volume of the financial data set to be transmitted and the data sorting of the candidate transmission data in the financial data set to be transmitted. In this way, the accuracy of obtaining the financial data to be transmitted can be further improved.

[0067] In one embodiment, step S402 may further include: taking the financial data to be transmitted that does not carry the transmitted identifier among the financial data to be transmitted contained in the current financial data set to be transmitted as candidate transmission data; the transmitted identifier is used to indicate that the financial data to be transmitted has been written into the second database table; after step S304, it may also include: adding the transmitted identifier to the current financial data to be transmitted corresponding to each financial data set to be transmitted.

[0068] The "transmitted" flag can be used to indicate that certain financial data to be transmitted has already been transmitted and written to the second database table. Therefore, if certain financial data to be transmitted already carries the "transmitted" flag, it has already been written to the second database table and will not be written again as the current financial data to be transmitted, thereby preventing duplicate writing of the data to be transmitted. To this end, the data reading and writing platform 102 can identify the financial data to be transmitted that does not carry the "transmitted" flag within the current set of financial data to be transmitted as candidate data to be transmitted and written to the second database table. Furthermore, after completing the concurrent read of the current set of financial data to be transmitted, to ensure that the current set of financial data to be transmitted is not read again during the next read, a "transmitted" flag can be added to the current set of financial data to be transmitted, indicating that the current set of financial data to be transmitted has been read and written to the second database table.

[0069] In this embodiment, a transmitted flag can be set to indicate whether each financial data to be transmitted has been read and written into the second database table. In this way, it can be ensured that the financial data to be transmitted will not be read repeatedly and written into the second database table, further improving the accuracy of financial data reading and writing.

[0070] In addition, after step S203, the method may further include: returning to the step of using the financial data to be transmitted that does not carry the transmitted identifier in the financial data to be transmitted included in the current financial data set to be transmitted as candidate transmission data, until all the financial data to be transmitted included in each financial data set to be transmitted carries the transmitted identifier.

[0071] In this embodiment, after completing the concurrent reading of the current financial data to be transmitted corresponding to each set of financial data to be transmitted and adding a transmitted identifier, the data reading and writing platform 102 may also concurrently write each of the read financial data to be transmitted into the second database table according to the data partition information. Thereafter, the data reading and writing platform 102 may return to the step of using the financial data to be transmitted included in the current set of financial data to be transmitted that does not carry a transmitted identifier as candidate transmission data. That is, for each set of financial data to be transmitted, the financial data to be transmitted that has not yet been written into the second database table is used as candidate transmission data, and the current financial data to be transmitted corresponding to each set of financial data to be transmitted is again screened out for parallel reading until the financial data to be transmitted in all the sets of financial data to be transmitted are all added with a transmitted identifier, indicating that all the financial data to be transmitted have been written into the second database table.

[0072] In this embodiment, after completing writing the current financial data to be transmitted into the second database table, the data reading and writing platform 102 returns to execute the step of using the financial data to be transmitted that does not carry the transmitted identifier as candidate transmission data until all the financial data to be transmitted are added with the above-mentioned transmitted identifier. In this way, the financial data to be transmitted can be read and written in multiple times, and it can be ensured that all the financial data to be transmitted are written into the second database table, further improving the integrity of the reading and writing of the financial data to be transmitted.

[0073] In one embodiment, as shown in FIG5 , step S203 may further include:

[0074] Step S501: Acquire multiple data partitions contained in a second database table.

[0075] In this embodiment, the second database table corresponding to the second server 103 may also be composed of multiple data partitions. When writing the financial data to be transmitted, the data reading and writing platform 102 may first determine the multiple data partitions included in the second database table.

[0076] Step S502: Determine, based on the data partition information, the partitions to be written corresponding to the respective financial data to be transmitted from the multiple data partitions included in the second database table;

[0077] Step S503 : concurrently writing each financial data to be transmitted into the to-be-written partitions corresponding to each financial data to be transmitted.

[0078] The partition to be written refers to the data partition where the financial data to be transmitted needs to be written. Since each financial data to be transmitted corresponds to corresponding data partition information, when writing the financial data to be transmitted, it can also be written according to the above data partition information. For example, the data partition information corresponding to data 1 is partition A, then when writing to the second database table, data 1 also needs to be written to the data partition A corresponding to the second database table, then the data partition A is the partition to be written, then when concurrently writing the financial data to be transmitted, the above-mentioned financial data to be transmitted, that is, data 1, can be concurrently written to the data partition A corresponding to the second database table. If the data partition information corresponding to data 2 is partition B, then when writing to the second database table, data 3 also needs to be written to the data partition B corresponding to the second database table, then the data partition B is the partition to be written, then when concurrently writing the financial data to be transmitted, the above-mentioned financial data to be transmitted, that is, data 2, can be concurrently written to the data partition B corresponding to the second database table.

[0079] In this embodiment, each financial data to be transmitted can also be concurrently written into the corresponding partition to be written in the second database table according to the data partition information. In this way, the accuracy of financial data writing can be further improved while ensuring the efficiency of financial data writing.

[0080] In one embodiment, a data transmission optimization method based on TBDS Hadoop is also provided, which can utilize the partition information in the library table to improve the efficiency of concurrent reading and writing. Specifically, the following steps may be included:

[0081] Step 1: Connect to the big data database table.

[0082] Step 2: Execute SQL to read data from the big data database table. In this step, the partition information in the big database table is used to read data from different partitions concurrently.

[0083] Step 3: Release the database connection.

[0084] Step 4: Connect to other databases (such as MySQL database).

[0085] Step 5: Identify the partition information in the MySQL database table and divide the data read in step 2 into different partitions in MySQL. Write data to different partitions concurrently based on the MySQL partition information.

[0086] Step 6: For the same partition data in MySQL, submit it in batches multiple times instead of submitting all the data at once.

[0087] Step 7: Release the database connection.

[0088] In this embodiment, the partition information of the table can be used to improve the read and write concurrency speed. Since the reading and writing of data in different partitions do not affect each other, this feature can be used to improve the read and write speed and perform concurrent reading and writing.

[0089] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0090] Based on the same inventive concept, embodiments of the present application further provide a database table reading and writing device for implementing the aforementioned database table reading and writing method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more database table reading and writing device embodiments provided below can be found in the above-mentioned limitations of the database table reading and writing method, and will not be repeated here.

[0091] In one embodiment, as shown in FIG6 , a database table reading and writing device is provided, comprising: a data partition acquisition module 601 , a data concurrent reading module 602 , and a data concurrent writing module 603 , wherein:

[0092] The data partition acquisition module 601 is used to determine the financial data to be transmitted and obtain the data partition information corresponding to each financial data to be transmitted from the first database table;

[0093] A concurrent data reading module 602 is configured to concurrently read the financial data to be transmitted corresponding to different data partitions in the first database table based on the data partition information;

[0094] The data concurrent writing module 603 is used to concurrently write the concurrently read financial data to be transmitted into the second database table according to the data partition information.

[0095] In one embodiment, the data concurrent reading module 602 is further used to classify the financial data to be transmitted corresponding to the same data partition information based on the data partition information to obtain multiple financial data sets to be transmitted corresponding to different data partitions; obtain the concurrent reading data volume set for each financial data set to be transmitted; for each financial data set to be transmitted, according to the concurrent reading data volume, determine the current financial data to be transmitted corresponding to each financial data set to be transmitted from the financial data to be transmitted contained in each financial data set to be transmitted; and concurrently read the current financial data to be transmitted corresponding to each financial data set to be transmitted.

[0096] In one embodiment, the data concurrent reading module 602 is further used to obtain the current set of financial data to be transmitted and the current concurrent reading data volume corresponding to the current set of financial data to be transmitted; determine the candidate transmission data from the current set of financial data to be transmitted, and obtain the data ranking of each candidate transmission data; based on the data ranking and the current concurrent reading data volume, determine the current financial data to be transmitted corresponding to the current set of financial data to be transmitted from the candidate transmission data.

[0097] In one embodiment, the concurrent data reading module 602 is further used to select the financial data to be transmitted that does not carry a transmitted identifier among the financial data to be transmitted included in the current financial data set to be transmitted as candidate transmission data; the transmitted identifier is used to indicate that the financial data to be transmitted has been written into the second database table; and is also used to add a transmitted identifier to the current financial data to be transmitted corresponding to each financial data set to be transmitted.

[0098] In one embodiment, the concurrent data reading module 602 is further used to return to the step of using the financial data to be transmitted included in the current set of financial data to be transmitted, which does not carry the transmitted identifier, as candidate transmission data, until all the financial data to be transmitted included in each set of financial data to be transmitted carry the transmitted identifier.

[0099] In one embodiment, the concurrent data writing module 603 is further used to obtain multiple data partitions contained in the second database table; determine the partitions to be written corresponding to each financial data to be transmitted from the multiple data partitions contained in the second database table based on the data partition information; and concurrently write each financial data to be transmitted into the partitions to be written corresponding to each financial data to be transmitted.

[0100] Each module in the above-mentioned database table reading and writing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor of the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0101] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in Figure 7. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store financial data to be transmitted. The network interface of the computer device is used to communicate with an external terminal via a network connection. When executed by the processor, the computer program implements a method for reading and writing a database table.

[0102] Those skilled in the art will understand that the structure shown in FIG7 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0103] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0104] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0105] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0106] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0107] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0108] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0109] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for reading and writing a database table, characterized in that: The method comprises: Determine the financial data to be transmitted, and obtain data partition information corresponding to each of the financial data to be transmitted from the first database table; Based on the data partition information, concurrently read the financial data to be transmitted corresponding to different data partitions in the first database table; The concurrently read financial data to be transmitted is concurrently written into the second database table according to the data partition information.

2. The method according to claim 1, characterized in that The concurrently reading the to-be-transmitted financial data corresponding to different data partitions in the first database table based on the data partition information includes: Based on the data partition information, the financial data to be transmitted corresponding to the same data partition information are classified and processed to obtain multiple sets of financial data to be transmitted corresponding to different data partitions; Obtaining the concurrent read data volume set for each of the financial data sets to be transmitted; For each of the financial data sets to be transmitted, according to the concurrently read data volume, determine the current financial data to be transmitted corresponding to each of the financial data sets to be transmitted from the financial data to be transmitted contained in each of the financial data sets to be transmitted; Concurrently read the current financial data to be transmitted corresponding to each of the financial data sets to be transmitted.

3. The method according to claim 2, characterized in that The determining, for each of the financial data sets to be transmitted, according to the concurrently read data volume, from the financial data to be transmitted contained in each of the financial data sets to be transmitted, the current financial data to be transmitted corresponding to each of the financial data sets to be transmitted, includes: Acquire the current financial data set to be transmitted and the current concurrent read data volume corresponding to the current financial data set to be transmitted; Determine candidate transmission data from the current set of financial data to be transmitted, and obtain a data ranking of each candidate transmission data; Based on the data sorting and the current concurrently read data volume, the current financial data to be transmitted corresponding to the current financial data set to be transmitted is determined from the candidate transmission data.

4. The method according to claim 3, characterized in that The step of determining candidate transmission data from the current set of financial data to be transmitted includes: taking the financial data to be transmitted that does not carry a transmitted identifier among the financial data to be transmitted included in the current set of financial data to be transmitted as the candidate transmission data; the transmitted identifier is used to indicate that the financial data to be transmitted has been written into the second database table; After concurrently reading the current financial data to be transmitted corresponding to each of the financial data sets to be transmitted, the method further includes: The transmitted identifier is added to the current financial data to be transmitted corresponding to each of the financial data sets to be transmitted.

5. The method according to claim 4, characterized in that After concurrently writing the concurrently read financial data to be transmitted into the second database table according to the data partition information, the method further includes: Return to the step of using the financial data to be transmitted included in the current set of financial data to be transmitted, which does not carry the transmitted identifier, as the candidate transmission data until all the financial data to be transmitted included in the sets of financial data to be transmitted carry the transmitted identifier.

6. The method according to claim 1, characterized in that The concurrently writing the concurrently read financial data to be transmitted into the second database table according to the data partition information includes: Acquire multiple data partitions contained in the second database table; According to the data partition information, determining the partitions to be written corresponding to the financial data to be transmitted from the multiple data partitions included in the second database table; The financial data to be transmitted are concurrently written into the partitions to be written corresponding to the financial data to be transmitted.

7. A database table reading and writing device, characterized in that: The device comprises: A data partition acquisition module, used to determine the financial data to be transmitted, and obtain data partition information corresponding to each of the financial data to be transmitted from the first database table; A data concurrent reading module, configured to concurrently read the financial data to be transmitted corresponding to different data partitions in the first database table based on the data partition information; The data concurrent writing module is used to concurrently write the concurrently read financial data to be transmitted into the second database table according to the data partition information.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images