Data writing method, device, system, equipment, medium and program product

Abstract

The present disclosure provides a data writing method, device, system, equipment, medium and program product, which can be applied to the fields of database technology and computer technology. The method includes: monitoring the target node corresponding to the target data table in the registration center; in the case of monitoring that the batch write configuration information has changed, adjusting the first batch write threshold corresponding to the nth batch write operation according to the changed batch write configuration information, and obtaining the second batch write threshold corresponding to the n+1th batch write operation; in the case that the nth batch write operation has been executed, comparing the batch write parameter information of the current data to be written stored in the memory of the execution end with the second batch write threshold, and obtaining a first comparison result; in the case that the first comparison result meets the first preset condition, by executing the n+1th batch write operation, the current data to be written stored in the memory is written into the target data table.

Description

Technical Field

[0001] The present disclosure relates to the fields of database technology and computer technology, and more specifically, to a data writing method, device, system, equipment, medium and program product. Background Art

[0002] ClickHosue (hereinafter referred to as CK) is a column-based, storage-computing integrated open source database with extremely fast query and analysis performance. It has outstanding advantages in log analysis, user behavior analysis, data monitoring, real-time data warehouse, etc., and is widely favored.

[0003] However, due to the limitations of ClickHouse's underlying design, each time a write operation is performed, a corresponding small file will be generated in the ClickHouse storage layer. Once the write frequency is too high, there will be too many small files, causing the CK background data merge thread to be overloaded and unable to handle it, resulting in errors. Although some other open source components provide tools for batch writing to CK, there is no way to dynamically adjust the threshold for batch writing without shutting down the server.

[0004] Therefore, how to dynamically adjust the threshold of batch writing without stopping the system is a technical problem that needs to be solved in the prior art. Summary of the invention

[0005] In view of this, the present disclosure provides a data writing method, apparatus, system, device, medium and program product.

[0006] One aspect of the present disclosure provides a data writing method, comprising:

[0007] Monitoring a target node corresponding to a target data table in a registration center, wherein the target data table is stored in a target database, and the target node stores batch write configuration information corresponding to the target data table;

[0008] When a change is detected in the batch write configuration information, the first batch write threshold corresponding to the nth batch write operation is adjusted according to the changed batch write configuration information to obtain a second batch write threshold corresponding to the (n+1)th batch write operation, where n is an integer greater than or equal to 1;

[0009] When the nth batch write operation has been executed, comparing the batch write parameter information of the current data to be written stored in the memory of the execution end with the second batch write threshold to obtain a first comparison result; and

[0010] In the case where the first comparison result satisfies the first preset condition, the current data to be written stored in the memory is written into the target data table by executing the (n+1)th batch write operation.

[0011] Another aspect of the present disclosure provides a data writing method, comprising:

[0012] In response to detecting a configuration modification operation for a target data table, displaying a configuration interface, wherein the target data table is stored in a target database;

[0013] When the modified batch write configuration information of the target data table is modified in the configuration interface, the modified batch write configuration information is pushed to the target node corresponding to the target data table in the registration center, so that when the first comparison result satisfies the first preset condition, the execution end writes the current to-be-written data stored in the memory into the target data table by executing the n+1th batch write operation.

[0014] Among them, the above-mentioned first comparison result is obtained by comparing the batch write parameter information of the current data to be written stored in the memory of the above-mentioned execution end with the second batch write threshold when the above-mentioned nth batch write operation has been executed; the above-mentioned second batch write threshold is obtained by adjusting the first batch write threshold corresponding to the above-mentioned nth batch write operation according to the above-mentioned changed batch write configuration information when the above-mentioned execution end monitors that the batch write configuration information stored in the above-mentioned target node has changed; n is an integer greater than or equal to 1.

[0015] Another aspect of the present disclosure provides a data writing device, including: a monitoring module, an adjustment module, a first comparison module and a first writing module. The monitoring module is used to monitor the target node corresponding to the target data table in the registration center, wherein the target data table is stored in the target database, and the target node stores the batch writing configuration information corresponding to the target data table. The adjustment module is used to adjust the first batch writing threshold corresponding to the nth batch writing operation according to the changed batch writing configuration information when the batch writing configuration information is detected to change, and obtain the second batch writing threshold corresponding to the n+1th batch writing operation, wherein n is an integer greater than or equal to 1. The first comparison module is used to compare the batch writing parameter information of the current data to be written stored in the memory of the execution end with the second batch writing threshold when the nth batch writing operation has been executed, and obtain a first comparison result. The first writing module is used to write the current data to be written stored in the memory into the target data table by executing the n+1th batch writing operation when the first comparison result meets the first preset condition.

[0016] Another aspect of the present disclosure provides a data writing device, including: a first display module and a first push module. The first display module is used to display a configuration interface in response to detecting a configuration modification operation for a target data table, wherein the target data table is stored in a target database. The first push module is used to push the changed batch write configuration information of the target data table to a target node corresponding to the target data table in a registration center after the changed batch write configuration information of the target data table is modified in the configuration interface, so that when the first comparison result satisfies the first preset condition, the execution end writes the current data to be written stored in the memory into the target data table by executing the n+1th batch write operation.

[0017] Among them, the above-mentioned first comparison result is obtained by comparing the batch write parameter information of the current data to be written stored in the memory of the above-mentioned execution end with the second batch write threshold when the above-mentioned nth batch write operation has been executed; the above-mentioned second batch write threshold is obtained by adjusting the first batch write threshold corresponding to the above-mentioned nth batch write operation according to the above-mentioned changed batch write configuration information when the above-mentioned execution end monitors that the batch write configuration information stored in the above-mentioned target node has changed; n is an integer greater than or equal to 1.

[0018] Another aspect of the present disclosure provides a data writing system, including: a configuration end and an execution end. The configuration end is used to: in response to detecting a configuration modification operation for a target data table, display a configuration interface, wherein the target data table is stored in a target database; when the modified batch write configuration information of the target data table is modified in the configuration interface, push the modified batch write configuration information to a target node corresponding to the target data table in a registration center. The execution end is used to: monitor the above-mentioned target node; when it is monitored that the batch write configuration information stored in the above-mentioned target node has changed, adjust the first batch write threshold corresponding to the nth batch write operation according to the above-mentioned changed batch write configuration information, and obtain the second batch write threshold corresponding to the n+1th batch write operation, wherein n is an integer greater than or equal to 1; when the above-mentioned nth batch write operation has been executed, compare the batch write parameter information of the current data to be written stored in the memory of the above-mentioned execution end with the above-mentioned second batch write threshold to obtain a first comparison result; when the above-mentioned first comparison result meets the first preset condition, by executing the above-mentioned n+1th batch write operation, the current data to be written stored in the above-mentioned memory is written into the above-mentioned target data table.

[0019] Another aspect of the present disclosure provides an electronic device, including: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0020] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0021] Another aspect of the present disclosure provides a computer program product, which includes computer executable instructions, and when the instructions are executed, are used to implement the method as described above.

[0022] According to the data writing method, device, system, equipment, medium and program product provided by the present disclosure, when the execution end performs batch writing on the target data table in the target database, the execution end monitors the target node corresponding to the target data table in the registration center to monitor whether the batch writing configuration information corresponding to the target data table stored in the target node has changed, so that when the batch writing configuration information has changed, the execution end can detect it in time, and dynamically adjust the first batch writing threshold corresponding to the current nth batch writing operation according to the changed batch writing configuration information in time, and obtain the second batch writing threshold that can be used to determine whether to execute the n+1th batch writing operation, so as to perform the next batch writing operation according to the latest batch writing configuration information. A judgment is made as to whether a write operation is to be executed, and, in the case where the nth batch write operation has been executed, by comparing the batch write parameter information of the current data to be written stored in the memory with the second batch write threshold, so as to judge whether the n+1th batch write operation can be executed at the current moment according to the first comparison result, so that when the first comparison result satisfies the first preset condition, the n+1th batch write operation can be executed to write the current data to be written into the target data table. Since the execution end monitors in real time whether the batch write configuration information stored in the target node has changed, in the case of a change, the batch write threshold can be changed based on the changed batch write configuration information without shutting down the system, thereby realizing dynamic adjustment of the batch write threshold. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0024] Figure 1 An exemplary system architecture of a data writing method according to an embodiment of the present disclosure is schematically shown;

[0025] Figure 2A flowchart of a data writing method according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 3 A flowchart of a data writing method according to another embodiment of the present disclosure is schematically shown;

[0027] Figure 4 The structure diagram of the data writing system according to the embodiment of the present disclosure is schematically shown;

[0028] Figure 5 A structural block diagram of a data writing device applied to an execution end according to an embodiment of the present disclosure is schematically shown;

[0029] Figure 6 A structural block diagram schematically shows a data writing device applied to a configuration terminal according to an embodiment of the present disclosure; and

[0030] Figure 7 A block diagram of an electronic device suitable for implementing a data writing method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0031] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0032] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0033] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0034] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0035] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0036] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0037] Figure 1 The exemplary system architecture 100 of the data writing method according to the embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0038] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first server 101, a network 102, and a second server 103. The network 102 is used to provide a medium for a communication link between the first server 101 and the second server 103. The network 102 may include various connection types, such as wired and / or wireless communication links, etc.

[0039] The registration center may be constructed based on the first server 101, and the execution end may be constructed based on the second server 103. The first server 101 interacts with the second server 103 via the network 102 to receive or send messages and the like.

[0040] The first server 101 and the second server 103 may be servers that provide various services, such as a background management server that provides support to the registration center and the execution end (only as an example). The background management server may analyze and process the received user request and other data.

[0041] For example, the target node corresponding to the target data table in the registration center can be monitored by the second server 103, wherein the target node stores batch write configuration information corresponding to the target data table, so that when it is monitored that the batch write configuration information has changed, the first batch write threshold corresponding to the nth batch write operation is adjusted according to the changed batch write configuration information to obtain the second batch write threshold corresponding to the n+1th batch write operation, and when the nth batch write operation has been executed, the batch write parameter information of the current data to be written stored in the memory of the execution end is compared with the second batch write threshold to obtain a first comparison result; when the first comparison result meets the first preset condition, the current data to be written stored in the memory is written into the target data table by executing the n+1th batch write operation.

[0042] It should be noted that the data writing method provided in the embodiment of the present disclosure can generally be executed by the second server 103. Accordingly, the data writing device provided in the embodiment of the present disclosure can generally be arranged in the second server 103. The data writing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the second server 103 and can communicate with the first server 101. Accordingly, the data writing device provided in the embodiment of the present disclosure can also be arranged in a server or server cluster that is different from the second server 103 and can communicate with the first server 101. Alternatively, the data writing method provided in the embodiment of the present disclosure can also be executed by the first server 101, or can also be executed by other servers different from the first server 101. Accordingly, the data writing device provided in the embodiment of the present disclosure can also be arranged in the first server 101, or can be arranged in other servers different from the first server 101.

[0043] It should be understood that Figure 1 The number of networks and servers in the embodiment is only for illustration. Any number of networks and servers may be provided as required.

[0044] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 The data writing method of the embodiment of the present disclosure is described in detail.

[0045] Figure 2 The flowchart of the data writing method according to the embodiment of the present disclosure is schematically shown.

[0046] like Figure 2 As shown, the method 200 includes operations S210 to S240.

[0047] In operation S210, a target node corresponding to a target data table in a registration center is monitored.

[0048] The target data table is stored in the target database, and the target node stores batch write configuration information corresponding to the target data table.

[0049] According to an embodiment of the present disclosure, a registration center may include multiple nodes, each of which may store batch write configuration information corresponding to a data table, and there may be at least one data table, which is stored in a corresponding database. The database may include a ClickHosue database, and the database may also include other types of databases with the same performance or characteristics as the ClickHosue database. The target data table may include at least one data table, and the target data table is a data table to be batch written by the execution end.

[0050] Among them, the registration center ZooKeeper is a distributed, open source distributed application coordination service.

[0051] According to an embodiment of the present disclosure, when the execution end performs batch writes to the target data table in the target database, the execution end will monitor the target node corresponding to the target data table in the registration center to monitor whether the batch write configuration information corresponding to the target data table stored in the target node has changed, so that the execution end can promptly reflect the changes in the batch write configuration information to adjust the corresponding batch write threshold.

[0052] The batch write configuration information may include the time interval between the start time of the last batch write operation and the start time of the next batch write operation, and the number of data items and data bytes that can be written each time a batch write operation is executed.

[0053] In operation S220, when a change in the batch write configuration information is detected, the first batch write threshold corresponding to the nth batch write operation is adjusted according to the changed batch write configuration information to obtain a second batch write threshold corresponding to the (n+1)th batch write operation.

[0054] Wherein, n is an integer greater than or equal to 1.

[0055] According to an embodiment of the present disclosure, when the execution end has executed the nth batch write operation and has not executed the n+1th batch write operation, the execution end monitors that the batch write configuration information corresponding to the target data table stored in the target node has changed. Then, according to the changed batch write configuration information, the execution end adjusts the first batch write threshold currently corresponding to the nth batch write operation to obtain the second batch write threshold corresponding to the n+1th batch write operation.

[0056] According to an embodiment of the present disclosure, the nth batch write operation is performed based on the first batch write threshold. When the nth batch write operation has been performed, if the batch write configuration information corresponding to the target data table changes, the n+1th batch write operation needs to be performed based on the second batch write operation.

[0057] According to an embodiment of the present disclosure, the batch write threshold may include a time interval threshold, a data number threshold, and a data byte number threshold. At least one of the time interval threshold, the data number threshold, and the data byte number threshold in the first batch write threshold and the second batch write threshold is different.

[0058] In operation S230, when the nth batch write operation has been performed, the batch write parameter information of the current data to be written stored in the memory of the execution end is compared with the second batch write threshold to obtain a first comparison result.

[0059] According to an embodiment of the present disclosure, when the nth batch write operation has been executed, the execution end may compare the batch write parameter information of the current data to be written stored in the memory with the second batch write threshold to obtain a first comparison result.

[0060] According to an embodiment of the present disclosure, the current data to be written is all the data currently stored in the memory of the execution end. The batch write parameter information may include the time interval between the start time of the nth batch write operation and the current time, the number of data items and the number of data bytes of the current data to be written.

[0061] According to an embodiment of the present disclosure, by comparing the batch write parameter information of the data to be written currently with the second batch write threshold, it is determined whether the (n+1)th batch write operation can be performed at the current moment.

[0062] In operation S240, when the first comparison result satisfies the first preset condition, the current data to be written stored in the memory is written into the target data table by performing the (n+1)th batch write operation.

[0063] According to an embodiment of the present disclosure, when the first comparison result satisfies the first preset condition, it can indicate that the n+1th batch write operation can be performed at the current moment, that is, the current data to be written stored in the memory can be written into the target data table by executing the n+1th batch write operation.

[0064] According to an embodiment of the present disclosure, when the execution end performs batch writing on the target data table in the target database, the execution end monitors the target node corresponding to the target data table in the registration center to monitor whether the batch write configuration information corresponding to the target data table stored in the target node has changed, so that when the batch write configuration information has changed, the execution end can detect it in time, and dynamically adjust the first batch write threshold corresponding to the current n-th batch write operation according to the changed batch write configuration information in time, and obtain the second batch write threshold that can be used to determine whether to execute the n+1-th batch write operation, so as to determine whether to execute the next batch write operation according to the latest batch write configuration information. , and, in the case where the nth batch write operation has been executed, by comparing the batch write parameter information of the current data to be written stored in the memory with the second batch write threshold, it is determined whether the n+1th batch write operation can be executed at the current moment according to the first comparison result, so that when the first comparison result meets the first preset condition, the n+1th batch write operation can be executed to write the current data to be written into the target data table. Since the execution end monitors in real time whether the batch write configuration information stored in the target node has changed, in the case of a change, the batch write threshold can be changed based on the changed batch write configuration information without shutting down the system, thereby realizing dynamic adjustment of the batch write threshold.

[0065] According to an embodiment of the present disclosure, the batch write parameter information includes the time interval between the start time of the nth batch write operation and the current time, the current number of data items and the current number of data bytes of the data to be written stored in the memory; the second batch write threshold includes the time interval threshold, the number of data items threshold and the number of data bytes threshold; the first preset condition includes at least one of the following: the time interval is greater than or equal to the time interval threshold; the current number of data items is greater than or equal to the number of data items threshold; the current number of data bytes is greater than or equal to the number of data bytes threshold.

[0066] According to an embodiment of the present disclosure, the first preset condition may include at least one of a time interval greater than or equal to a time interval threshold, a current number of data items greater than or equal to a data item threshold, and a current number of data bytes greater than or equal to a data byte threshold. This is equivalent to determining that the n+1th batch write operation can be performed at the current moment when the first comparison result satisfies at least one of the time interval greater than or equal to the time interval threshold, the current number of data items greater than or equal to the data item threshold, and the current number of data bytes greater than or equal to the data byte threshold.

[0067] For example, when the first comparison result is that the time interval is greater than or equal to the time interval threshold, the current number of data items is less than the data item threshold, and the current number of data bytes is less than the data byte threshold, the first comparison result satisfies the first preset condition, that is, the n+1th batch write operation can be performed at the current moment. When the first comparison result is that the time interval is less than the time interval threshold, the current number of data items is greater than or equal to the data item threshold, and the current number of data bytes is greater than or equal to the data byte threshold, the first comparison result also satisfies the first preset condition, that is, the n+1th batch write operation can be performed at the current moment.

[0068] According to an embodiment of the present disclosure, in order to effectively avoid the problem of multiple nodes writing to the same table resulting in excessive request peaks at the same time, the time interval is determined based on the start time of the last batch write operation rather than using a natural time interval.

[0069] For example, if there are 100 nodes that need to write to table A, if a natural time interval is used, writing every 2 minutes, then at the 0th minute, these 100 nodes will write once, and at the 2nd minute, these 100 nodes will also write once, that is, these 100 nodes will write at the same time; but the time interval is calculated from the start time of the last write, so that among the 100 nodes, some may write in the 1st minute, and some may write in the 2nd minute, which is equivalent to spreading the request peak to avoid excessive request peak.

[0070] According to the embodiments of the present disclosure, the time interval threshold is set to avoid excessive delay in batch writing of data, the data number threshold is set to avoid too frequent writing, and the data byte number threshold is set to avoid excessive memory usage on the execution end. By comparing the batch write parameter information of the current data to be written with the second batch write threshold, the best time to execute the n+1th batch write operation is determined to minimize the problems of excessive data write delay, too frequent writing and excessive memory usage.

[0071] According to an embodiment of the present disclosure, the current data to be written stored in the memory is obtained through the following operations, including: from the starting moment of executing the nth batch write operation, when the execution end receives the data to be written for the mth time, the data to be written received for the mth time is saved in the memory to obtain the current data to be written stored in the memory, where m is an integer greater than or equal to 1.

[0072] According to an embodiment of the present disclosure, starting from the start moment when the execution end executes the nth batch write operation, each time the execution end receives data to be written, it saves the received data to be written into the memory to obtain the current data to be written stored in the memory.

[0073] According to an embodiment of the present disclosure, during the period from the start time of executing the nth batch write operation to the start time of executing the n+1th batch write operation, the execution end will continuously receive the data to be written, and from the start time of executing the nth batch write operation, the number of times the execution end receives the data to be written will increase from 1, that is, m will increase from 1. In the case where the execution end receives the data to be written for the mth time, the execution end will save the data to be written received for the mth time to the memory to obtain the current data to be written, that is, the current data to be written may include all the data to be written received by the execution end m times.

[0074] The data to be written received by the execution end each time may include at least one piece of data, and the number of bytes of each piece of data is also uncertain.

[0075] According to an embodiment of the present disclosure, at the start time of executing the nth batch write operation, the execution end will write all the data to be written previously stored in the memory into the target data table, that is, the memory of the execution end will clear all the data to be written previously stored, and restart to store the data to be written as the data to be written into the target data table when executing the n+1th batch write operation. After receiving the data to be written each time, the execution end will save the received data to be written into the memory, so as to write the current data to be written stored in the memory into the target data table when the first preset condition is met.

[0076] According to an embodiment of the present disclosure, when it is monitored that the batch write configuration information has not changed, the above-mentioned data writing method also includes: when the nth batch write operation has been executed, the batch write parameter information of the current data to be written stored in the memory is compared with the first batch write threshold to obtain a second comparison result; when the second comparison result meets the second preset condition, the current data to be written stored in the memory is written into the target data table by executing the n+1th batch write operation.

[0077] According to an embodiment of the present disclosure, the execution end monitors in real time whether the batch write configuration information corresponding to the target data table stored in the target node has changed. If the execution end detects that the batch write configuration information has not changed, when the execution end has executed the nth batch write operation, the batch write parameter information of the current data to be written stored in the memory can be compared with the first batch write threshold, that is, when the batch write configuration information has changed, the batch write parameters of the current data to be written are compared with the adjusted batch write threshold; when the batch write configuration information has not changed, the batch write parameters of the current data to be written continue to be compared with the previous batch write threshold.

[0078] According to an embodiment of the present disclosure, the second preset condition may also include at least one of the following: the time interval is greater than or equal to the time interval threshold; the current number of data items is greater than or equal to the data item threshold; the current number of data bytes is greater than or equal to the data byte threshold. However, the three thresholds in the first preset condition and the second preset condition are not exactly the same.

[0079] According to an embodiment of the present disclosure, when the second comparison result satisfies the second preset condition, it can be shown that the n+1th batch write operation can be performed at the current moment, that is, the current data to be written stored in the memory can be written into the target data table by executing the n+1th batch write operation.

[0080] According to the embodiments of the present disclosure, since the data to be written is written into the target data table in a batch writing manner, the throughput of the target database can be improved.

[0081] According to an embodiment of the present disclosure, when the execution end has executed the nth batch write operation, if the execution end monitors that the batch write configuration information corresponding to the target data table stored in the target node has not changed, it continues to compare the current data to be written with the original batch write threshold to determine whether the n+1th batch write operation can be performed at the current moment.

[0082] According to an embodiment of the present disclosure, the above-mentioned data writing method also includes: when the first comparison result does not meet the first preset condition, starting from the start time of executing the nth batch write operation, the data to be written received by the execution end for the m+1th time continues to be saved in the memory; the batch write parameter information of the current data to be written stored in the memory is compared with the second batch write threshold to obtain a third comparison result; when the third comparison result meets the first preset condition, the n+1th batch write operation is executed.

[0083] According to an embodiment of the present disclosure, starting from the start moment of executing the nth batch write operation, when the execution end receives the data to be written for the mth time, the data to be written received for the mth time can be first saved in the memory to obtain the current data to be written, and then the batch write parameter information of the current data to be written can be compared with the second batch write threshold to obtain a first comparison result.

[0084] According to an embodiment of the present disclosure, when the first comparison result does not meet the first preset condition, the data to be written received by the execution end for the m+1th time will continue to be saved in the memory. If it is monitored that the batch write configuration information corresponding to the target data table stored in the target node has not changed again, the batch write parameter information of the current data to be written stored in the memory is compared with the second batch write threshold to obtain a third comparison result.

[0085] According to an embodiment of the present disclosure, when the third comparison result satisfies the first preset condition, the (n+1)th batch write operation may be performed to write the current to-be-written data stored in the memory into the target data table.

[0086] According to an embodiment of the present disclosure, when the third comparison result does not meet the first preset condition, the data to be written received by the execution end for the m+2th time will continue to be saved in the memory. If it is monitored that the batch write configuration information corresponding to the target data table stored in the target node has not changed again, the batch write parameter information of the current data to be written stored in the memory is compared with the second batch write threshold to obtain a fourth comparison result.

[0087] According to an embodiment of the present disclosure, when the fourth comparison result satisfies the first preset condition, the n+1th batch write operation can be performed to write the current data to be written stored in the memory into the target data table. If the fourth comparison result still does not meet the first preset condition, the data to be written received by the execution end for the m+3th, m+4th, ... times will continue to be saved in the memory until the comparison result of the batch write parameter information of the current data to be written stored in the memory and the second batch write threshold value meets the first preset condition.

[0088] According to an embodiment of the present disclosure, after receiving data to be written each time, the execution end will first save the received data to be written into the memory, and then compare the batch write parameter information of the current data to be written stored in the memory with the batch write threshold.

[0089] According to an embodiment of the present disclosure, when the third comparison result does not meet the first preset condition, the data to be written received by the execution end for the m+2th time will continue to be saved in the memory. If it is monitored that the batch write configuration information corresponding to the target data table stored in the target node has changed again, the execution end can adjust the second batch write threshold according to the batch write configuration information after the change again to obtain the third batch write threshold.

[0090] According to an embodiment of the present disclosure, the batch write parameter information of the current data to be written stored in the memory is compared with the third batch write threshold to obtain a fifth comparison result. If the fifth comparison result meets the third preset condition, the n+1th batch write operation is performed.

[0091] The third preset condition may include at least one of the following: the time interval is greater than or equal to the time interval threshold; the current number of data items is greater than or equal to the data item threshold; the current number of data bytes is greater than or equal to the data byte threshold. However, the third preset condition is not completely the same as the three thresholds in the first preset condition.

[0092] According to an embodiment of the present disclosure, when the first comparison result does not satisfy the first preset condition, that is, the next batch write operation cannot be performed at present, the data to be written received for the m+1th time continues to be saved in the memory, and the batch write parameter information of the current data to be written stored in the memory is compared with the second batch write threshold, so that when the third comparison result satisfies the first preset condition, the n+1th batch write operation can be performed; if the third comparison result does not satisfy the first preset condition, the data to be written received for the m+2th, m+3th, etc. times continues to be saved in the memory until the current data to be written stored in the memory satisfies the first preset condition, and the n+1th batch write operation is performed.

[0093] According to an embodiment of the present disclosure, from the start time of executing the nth batch write operation, when the execution end receives the data to be written for the mth time, the data to be written received for the mth time is saved in the memory, and if it is monitored that the batch write configuration information has not changed, the batch write parameter information of the current data to be written stored in the memory is compared with the first batch write threshold to obtain a second comparison result; when the second comparison result meets the second preset condition, the current data to be written stored in the memory is written into the target data table by executing the n+1th batch write operation.

[0094] According to an embodiment of the present disclosure, when the second comparison result does not meet the second preset condition, the data to be written received by the execution end for the m+1th, m+2th, etc. times will continue to be saved in the memory, and if it is monitored that the batch write configuration information has not changed, the batch write parameter information of the current data to be written stored in the memory will be compared with the first batch write threshold until the comparison result meets the second preset condition, and the n+1th batch write operation will be performed.

[0095] According to an embodiment of the present disclosure, when n takes a value of 1, the batch write threshold corresponding to the first batch write operation is determined by the following operations, including: pulling the initial batch write configuration information corresponding to the target data table from the target node; and determining the batch write threshold corresponding to the first batch write operation based on the initial batch write configuration information.

[0096] According to an embodiment of the present disclosure, since the registration center includes a target node corresponding to the target data table, and the target node stores batch write configuration information corresponding to the target data table, when the execution end just starts to perform batch writes on the target data table, that is, when the execution end is about to perform the first batch write operation, the execution end can pull the initial batch write configuration information corresponding to the target data table from the target node, so as to determine the batch write threshold corresponding to the first batch write operation based on the initial batch write configuration information, so that it can be judged whether the first batch write operation can be performed at the current moment based on the batch write threshold.

[0097] According to an embodiment of the present disclosure, at the beginning, the target node stores initial batch write configuration information corresponding to the target data table, but during the batch writing process on the execution end, the batch write configuration information corresponding to the target data table stored in the target node may change, that is, the execution end may subsequently determine whether to perform the next batch write operation based on the changed batch write configuration information.

[0098] According to an embodiment of the present disclosure, initial batch write configuration information corresponding to the target data table is initially stored in the target node of the registration center. When the execution end starts to write data to the target data table, the execution end can directly pull the initial batch write configuration information corresponding to the target data table from the target node to determine the batch write threshold corresponding to the first batch write operation; in the event that the batch write configuration information corresponding to the target data table stored in the subsequent target node changes, the target node can actively push the changed batch write configuration information to the execution end, so that the execution end can adjust the batch write threshold in time according to the changed batch write configuration information to obtain the batch write threshold for the next comparison.

[0099] Figure 3 The flowchart of a data writing method according to another embodiment of the present disclosure is schematically shown.

[0100] like Figure 3 As shown, the method 300 includes operation S310 and operation S320.

[0101] In operation S310 , in response to detecting a configuration modification operation on a target data table, a configuration interface is displayed.

[0102] The target data table is stored in the target database.

[0103] According to an embodiment of the present disclosure, the batch write configuration information of the target data table can be configured and modified on the configuration interface of the configuration end. When the configuration end detects a configuration modification operation for the target data table, the configuration interface can be displayed so that the user can modify the batch write configuration information of the target data table in the configuration interface.

[0104] For example, at least one of the following can be modified in the configuration interface: the time interval between the start time of the last batch write operation and the start time of the next batch write operation, the number of data items and the number of data bytes that can be written each time a batch write operation is executed.

[0105] In operation S320, when the changed batch write configuration information of the target data table is modified in the configuration interface, the changed batch write configuration information is pushed to the target node corresponding to the target data table in the registration center, so that when the first comparison result satisfies the first preset condition, the execution end writes the current data to be written stored in the memory into the target data table by executing the n+1th batch write operation.

[0106] Among them, the first comparison result is obtained by comparing the batch write parameter information of the current data to be written stored in the memory of the execution end with the second batch write threshold when the nth batch write operation has been executed; the second batch write threshold is obtained by adjusting the first batch write threshold corresponding to the nth batch write operation according to the changed batch write configuration information when the execution end monitors that the batch write configuration information stored in the target node has changed; n is an integer greater than or equal to 1.

[0107] According to an embodiment of the present disclosure, when the changed batch write configuration information of the target data table is modified in the configuration interface, the configuration end can, with the help of the server end, push the changed batch write configuration information of the target data table to the execution end, so that the execution end can adjust the first batch write threshold corresponding to the nth batch write operation according to the changed batch write configuration information, and obtain the second batch write threshold, so as to compare the batch write parameter information of the current data to be written stored in the memory of the execution end with the second batch write threshold, so that the n+1th batch write operation can be performed when the first comparison result satisfies the first preset condition.

[0108] According to an embodiment of the present disclosure, in the configuration interface of the configuration end, based on the interface configurability, the batch write configuration information of the target data table can be dynamically updated, and the changed batch write configuration information can be pushed to the target node. Since the execution end monitors the target node in real time, when it is detected that the batch write configuration information corresponding to the target data table stored in the target node has changed, the batch write threshold can be changed in time without downtime.

[0109] According to an embodiment of the present disclosure, before the execution end executes the first batch write operation, the above-mentioned data writing method also includes: in response to detecting a configuration operation for the batch write configuration information corresponding to the data table, displaying a configuration interface; after the initial batch write configuration information of each data table is configured in the configuration interface, the initial batch write configuration information of each data table is pushed to the node corresponding to the data table in the registration center, so that the execution end pulls the initial batch write configuration information corresponding to the target data table from the target node, and determines the batch write threshold corresponding to the first batch write operation based on the initial batch write configuration information.

[0110] According to an embodiment of the present disclosure, initially, the configuration terminal may display a configuration interface when a configuration operation for configuration information of multiple data tables is detected, so that the user may configure batch writing of configuration information for each data table in the configuration interface.

[0111] According to an embodiment of the present disclosure, when the initial batch write configuration information of each data table is configured in the configuration interface, the configuration end can, with the help of the server, push the initial batch write configuration information of each data table to the node corresponding to each data table in the registration center, so that when the execution end wants to start batch writing to the target data table, the initial batch write configuration information corresponding to the target data table is pulled from the target node corresponding to the target data table in the registration center, and the batch write threshold corresponding to the first batch write operation is determined based on the initial batch write configuration information.

[0112] According to an embodiment of the present disclosure, in the initial stage, the batch write configuration information of each data table can be configured in the configuration interface, and the initial batch write configuration information of each data table can be pushed to the corresponding node in the registration center, so that when the subsequent execution end wants to perform batch writes on a data table, the initial batch write configuration information corresponding to the data table can be directly pulled from the corresponding node to determine the batch write threshold corresponding to the first batch write operation.

[0113] Figure 4 The structural diagram of the data writing system according to the embodiment of the present disclosure is schematically shown.

[0114] like Figure 4 As shown, the data writing system 400 for implementing the data writing method may include a configuration end 410 , a service end 420 , a registration center 430 and an execution end 440 .

[0115] According to an embodiment of the present disclosure, the configuration terminal 410 can be used to: display a configuration interface in response to detecting a configuration modification operation for a target data table, wherein the target data table is stored in a target database; and push the changed batch write configuration information to a target node corresponding to the target data table in the registration center 430 after the changed batch write configuration information of the target data table is modified in the configuration interface. The configuration terminal 410 can push the changed batch write configuration information to the target node of the registration center 430 with the help of the service terminal 420, and the configuration terminal 410 can be used to perform the above operations S310 and S320.

[0116] According to an embodiment of the present disclosure, the execution end 440 can be used to: monitor the target node; when it is detected that the batch write configuration information stored in the target node has changed, adjust the first batch write threshold corresponding to the nth batch write operation according to the changed batch write configuration information, and obtain the second batch write threshold corresponding to the n+1th batch write operation, wherein n is an integer greater than or equal to 1; when the nth batch write operation has been executed, compare the batch write parameter information of the current data to be written stored in the memory of the execution end 410 with the second batch write threshold to obtain a first comparison result; when the first comparison result satisfies the first preset condition, write the current data to be written stored in the memory into the target data table by executing the n+1th batch write operation.

[0117] In which, when the batch write configuration information stored in the target node changes, the registration center 430 will actively push the changed batch write configuration information to the execution end 440; the execution end 440 can be used to execute the above operations S210 to S240.

[0118] According to an embodiment of the present disclosure, the configuration end 410, the service end 420 and the registration center 430 can be regarded as a tool 450 for implementing a data writing method. The execution end 440 can use the tool 450 to dynamically adjust the corresponding batch write threshold to determine whether the next batch write operation can be performed at the current moment based on real-time batch write configuration information.

[0119] According to an embodiment of the present disclosure, the tool 450 may be provided with various services by a server, and the execution end 440 may be provided with various services by another server.

[0120] Based on the above data writing method, the present disclosure also provides a data writing device applied to the execution end. Figure 5 The device is described in detail.

[0121] Figure 5 The structural block diagram of a data writing device applied to an execution end according to an embodiment of the present disclosure is schematically shown.

[0122] like Figure 5 As shown, the data writing device 500 of this embodiment includes a monitoring module 510 , an adjustment module 520 , a first comparison module 530 and a first writing module 540 .

[0123] The monitoring module 510 is used to monitor the target node corresponding to the target data table in the registration center, wherein the target data table is stored in the target database, and the target node stores the batch write configuration information corresponding to the target data table. In one embodiment, the monitoring module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0124] The adjustment module 520 is used to adjust the first batch write threshold corresponding to the nth batch write operation according to the changed batch write configuration information when the batch write configuration information is detected to obtain the second batch write threshold corresponding to the (n+1)th batch write operation, where n is an integer greater than or equal to 1. In one embodiment, the adjustment module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0125] The first comparison module 530 is used to compare the batch write parameter information of the current data to be written stored in the memory of the execution end with the second batch write threshold value to obtain a first comparison result when the nth batch write operation has been performed. In one embodiment, the first comparison module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0126] The first writing module 540 is used to write the current to-be-written data stored in the memory into the target data table by performing the n+1th batch writing operation when the first comparison result satisfies the first preset condition. In one embodiment, the first writing module 540 can be used to perform the operation S240 described above, which will not be described in detail here.

[0127] According to an embodiment of the present disclosure, the first comparison module 520 includes a storage unit.

[0128] A saving unit is used to save the data to be written received for the mth time into the memory from the start time of executing the nth batch write operation when the execution end receives the data to be written for the mth time, so as to obtain the current data to be written stored in the memory, wherein m is an integer greater than or equal to 1.

[0129] According to an embodiment of the present disclosure, the data writing device 500 further includes a second comparing module and a second writing module.

[0130] The second comparison module is used to compare the batch write parameter information of the current data to be written stored in the memory with the first batch write threshold to obtain a second comparison result when it is monitored that the batch write configuration information has not changed and the nth batch write operation has been performed.

[0131] The second writing module is used to write the current to-be-written data stored in the memory into the target data table by executing the (n+1)th batch writing operation when the second comparison result satisfies the second preset condition.

[0132] According to an embodiment of the present disclosure, the data writing module 500 further includes a saving module, a third comparing module and an executing module.

[0133] The saving module is used to continue to save the to-be-written data received by the execution end for the m+1th time into the memory from the start time of executing the nth batch writing operation when the first comparison result does not meet the first preset condition.

[0134] The third comparison module is used to compare the batch write parameter information of the data to be written currently stored in the memory with the second batch write threshold value to obtain a third comparison result.

[0135] The execution module is used to execute the (n+1)th batch write operation when the third comparison result meets the first preset condition.

[0136] According to an embodiment of the present disclosure, the data writing device 500 further includes a pulling module and a determining module.

[0137] The pull module is used to pull the initial batch write configuration information corresponding to the target data table from the target node.

[0138] The determination module is used to determine the batch write threshold corresponding to the first batch write operation according to the initial batch write configuration information.

[0139] Based on the above another data writing method, the present disclosure also provides a data writing device applied to the configuration end. Figure 6 The device is described in detail.

[0140] Figure 6 The structural block diagram of a data writing device applied to a configuration terminal according to another embodiment of the present disclosure is schematically shown.

[0141] like Figure 6 As shown, the data writing device 600 of this embodiment includes a first display module 610 and a first pushing module 620 .

[0142] The first display module 610 is used to display a configuration interface in response to detecting a configuration modification operation for a target data table, wherein the target data table is stored in a target database. In one embodiment, the first display module 610 can be used to perform the operation S310 described above, which will not be described in detail here.

[0143] The first push module 620 is used to push the changed batch write configuration information to the target node corresponding to the target data table in the registration center when the changed batch write configuration information of the target data table is modified in the configuration interface, so that when the first comparison result satisfies the first preset condition, the execution end writes the current to-be-written data stored in the memory into the target data table by executing the n+1th batch write operation. In one embodiment, the first push module 620 can be used to execute the operation S320 described above, which will not be described in detail here.

[0144] According to an embodiment of the present disclosure, the data writing device 600 further includes a second display module and a second push module.

[0145] The second display module is used to display a configuration interface in response to detecting a configuration operation for batch writing configuration information corresponding to the data table.

[0146] The second push module is used to push the initial batch write configuration information of each data table to the node corresponding to the data table in the registration center after the initial batch write configuration information of each data table is configured in the configuration interface, so that the execution end pulls the initial batch write configuration information corresponding to the target data table from the target node, and determines the batch write threshold corresponding to the first batch write operation based on the initial batch write configuration information.

[0147] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0148] For example, any of the monitoring module 510, the adjustment module 520, the first comparison module 530 and the first writing module 540 can be combined in one module / unit / sub-unit for implementation, or any of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the monitoring module 510, the adjustment module 520, the first comparison module 530 and the first writing module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware and firmware or by a suitable combination of any of them. Alternatively, at least one of the monitoring module 510 , the adjusting module 520 , the first comparing module 530 and the first writing module 540 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0149] For example, any multiple of the first display module 610 and the first push module 620 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the first display module 610 and the first push module 620 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the first display module 610 and the first push module 620 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0150] It should be noted that the data writing device part in the embodiment of the present disclosure corresponds to the data writing method part in the embodiment of the present disclosure. The description of the data writing device part specifically refers to the data writing method part, which will not be repeated here.

[0151] Figure 7 A block diagram of an electronic device suitable for implementing the data writing method described above according to an embodiment of the present disclosure is schematically shown. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0152] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0153] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0154] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0155] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0156] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0157] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0158] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703 .

[0159] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the data writing method provided by the embodiment of the present disclosure.

[0160] When the computer program is executed by the processor 701, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0161] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0162] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0163] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0164] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A data writing method, comprising: Monitoring a target node corresponding to a target data table in a registration center, wherein the target data table is stored in a target database, and the target node stores batch write configuration information corresponding to the target data table; When a change is detected in the batch write configuration information, adjusting the first batch write threshold corresponding to the nth batch write operation according to the changed batch write configuration information to obtain a second batch write threshold corresponding to the (n+1)th batch write operation, where n is an integer greater than or equal to 1; When the nth batch write operation has been executed, comparing the batch write parameter information of the current data to be written stored in the memory of the execution end with the second batch write threshold to obtain a first comparison result; and When the first comparison result satisfies the first preset condition, the current data to be written stored in the memory is written into the target data table by executing the (n+1)th batch write operation.

2. The method according to claim 1, wherein: The batch write parameter information includes the time interval between the start time of the nth batch write operation and the current time, the current number of data items and the current number of data bytes of the data to be written stored in the memory; the second batch write threshold includes the time interval threshold, the number of data items threshold and the number of data bytes threshold; the first preset condition includes at least one of the following: The time interval is greater than or equal to the time interval threshold; The current number of data items is greater than or equal to the data item number threshold; The current number of data bytes is greater than or equal to the data byte number threshold.

3. The method according to claim 1, wherein: The current data to be written stored in the memory is obtained by the following operations, including: From the start time of executing the nth batch write operation, when the execution end receives the data to be written for the mth time, the data to be written received for the mth time is saved in the memory to obtain the current data to be written stored in the memory, where m is an integer greater than or equal to 1.

4. The method according to claim 1, wherein: In the case where it is detected that the batch write configuration information has not changed, the method further includes: In the case where the nth batch write operation has been performed, comparing the batch write parameter information of the current data to be written stored in the memory with the first batch write threshold to obtain the second comparison result; When the second comparison result satisfies the second preset condition, the current data to be written stored in the memory is written into the target data table by executing the (n+1)th batch write operation.

5. The method according to claim 3, further comprising: If the first comparison result does not satisfy the first preset condition, starting from the start time of executing the nth batch write operation, the to-be-written data received by the execution end for the m+1th time is continuously saved in the memory; Compare the batch write parameter information of the current data to be written stored in the memory with the second batch write threshold to obtain a third comparison result; When the third comparison result satisfies the first preset condition, the (n+1)th batch write operation is performed.

6. The method according to any one of claims 1 to 5, wherein When the value of n is 1, the batch write threshold corresponding to the first batch write operation is determined by the following operations, including: Pulling initial batch write configuration information corresponding to the target data table from the target node; and A batch write threshold corresponding to the first batch write operation is determined according to the initial batch write configuration information.

7. A data writing method, comprising: In response to detecting a configuration modification operation for a target data table, displaying a configuration interface, wherein the target data table is stored in a target database; When the modified batch write configuration information of the target data table is modified in the configuration interface, the modified batch write configuration information is pushed to the target node corresponding to the target data table in the registration center, so that when the first comparison result satisfies the first preset condition, the execution end writes the current to-be-written data stored in the memory into the target data table by executing the n+1th batch write operation, Among them, the first comparison result is obtained by comparing the batch write parameter information of the current data to be written stored in the memory of the execution end with the second batch write threshold when the nth batch write operation has been executed; the second batch write threshold is obtained by adjusting the first batch write threshold corresponding to the nth batch write operation according to the changed batch write configuration information when the execution end monitors that the batch write configuration information stored in the target node has changed; n is an integer greater than or equal to 1.

8. The method according to claim 7, wherein: Before the execution end executes the first batch write operation, the method further includes: In response to detecting a configuration operation for batch writing configuration information corresponding to the data table, displaying the configuration interface; and When the initial batch write configuration information of each of the data tables is configured in the configuration interface, the initial batch write configuration information of each of the data tables is pushed to the node corresponding to the data table in the registration center, so that the execution end pulls the initial batch write configuration information corresponding to the target data table from the target node, and determines the batch write threshold corresponding to the first batch write operation based on the initial batch write configuration information.

9. A data writing device, comprising: A monitoring module, used for monitoring a target node corresponding to a target data table in a registration center, wherein the target data table is stored in a target database, and the target node stores batch write configuration information corresponding to the target data table; an adjustment module, configured to adjust a first batch write threshold corresponding to the nth batch write operation according to the changed batch write configuration information when a change is detected in the batch write configuration information, so as to obtain a second batch write threshold corresponding to the (n+1)th batch write operation, wherein n is an integer greater than or equal to 1; A first comparison module is used to compare the batch write parameter information of the current data to be written stored in the memory of the execution end with the second batch write threshold value to obtain a first comparison result when the nth batch write operation has been executed; and The first writing module is used to write the current to-be-written data stored in the memory into the target data table by executing the (n+1)th batch writing operation when the first comparison result satisfies a first preset condition.

10. A data writing device, comprising: A first display module, configured to display a configuration interface in response to detecting a configuration modification operation on a target data table, wherein the target data table is stored in a target database; The first push module is used for pushing the changed batch write configuration information of the target data table to the target node corresponding to the target data table in the registration center when the changed batch write configuration information of the target data table is modified in the configuration interface, so that the execution end writes the current to-be-written data stored in the memory into the target data table by executing the n+1th batch write operation when the first comparison result satisfies the first preset condition, Among them, the first comparison result is obtained by comparing the batch write parameter information of the current data to be written stored in the memory of the execution end with the second batch write threshold when the nth batch write operation has been executed; the second batch write threshold is obtained by adjusting the first batch write threshold corresponding to the nth batch write operation according to the changed batch write configuration information when the execution end monitors that the batch write configuration information stored in the target node has changed; n is an integer greater than or equal to 1.

11. A data writing system, comprising: Configuration terminal, used for: In response to detecting a configuration modification operation for a target data table, displaying a configuration interface, wherein the target data table is stored in a target database; When the modified batch write configuration information of the target data table is modified in the configuration interface, the modified batch write configuration information is pushed to the target node corresponding to the target data table in the registration center; The execution end is used to: Monitoring the target node; When it is detected that the batch write configuration information stored in the target node has changed, adjusting the first batch write threshold corresponding to the nth batch write operation according to the changed batch write configuration information to obtain a second batch write threshold corresponding to the (n+1)th batch write operation, where n is an integer greater than or equal to 1; When the nth batch write operation has been executed, comparing the batch write parameter information of the current data to be written stored in the memory of the execution end with the second batch write threshold to obtain a first comparison result; and When the first comparison result satisfies the first preset condition, the current data to be written stored in the memory is written into the target data table by executing the (n+1)th batch write operation.

12. An electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 6 or any one of claims 7 to 8.

13. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method of any one of claims 1 to 6 or any one of claims 7 to 8.

14. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 6 or any one of claims 7 to 8.

Images