Methods, apparatus and devices for controlling write operations

Abstract

This application discloses a method, apparatus, and device for controlling write operations, relating to the field of big data technology. The specific implementation scheme is as follows: By setting a first threshold and a second threshold, the current number of SSTable files is compared with the first and second thresholds respectively. If the current number is greater than or equal to the first threshold but less than the second threshold, it indicates that continuing to write will risk the number of SSTable files exceeding the second threshold. Therefore, all data write operations are rejected within a preset time period within a unit of time. After the preset time period ends, all data write operations are allowed within the remaining time period. This approach controls the rejection of further write operations by controlling the preset time period, only performing SSTable file merging to reduce the number of underlying SSTable files, thereby improving control flexibility and achieving better control performance.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to the field of big data technology. Background Technology

[0002] LSM tree is a disk-based data structure that includes two types of nodes: MemoryTable nodes and SSTable (hereinafter referred to as SST) nodes. MemoryTables reside in memory. When data is written, it is first written to multiple MemoryTables. When the number of MemoryTables reaches a threshold, the data in the MemoryTables is written to multiple SSTables. When the number of SSTables reaches the threshold, a compaction process is triggered, merging multiple SSTables into a single SSTable. When the write speed is too high, the compaction speed cannot keep up with the write speed, and the number of underlying SSTables will continue to increase. Since reading data requires querying each SSTable, the more SSTables there are, the greater the latency for reading data.

[0003] To control the number of underlying SSTables, existing technologies set a threshold to limit the number of SSTables. When the number of SSTables reaches the threshold, no more writes are allowed; only SSTable merging is performed. Writes can only resume once the number of merged underlying SSTables is less than the threshold. However, with this method, writes are completely unaffected before the number of underlying SSTables reaches the threshold; once the threshold is reached, write traffic drops drastically. Therefore, the existing method lacks flexibility and is ineffective in controlling the number of underlying SSTables. Summary of the Invention

[0004] This application provides a method, apparatus, and device for controlling write operations, which improves the flexibility of control and results in better control performance when controlling the number of underlying SSTables.

[0005] In a first aspect, embodiments of this application provide a control method for a data writing operation, which may include:

[0006] Get the current number of SSTable files.

[0007] If the current quantity is greater than or equal to the first threshold and less than the second threshold, then all data write operations are rejected within the first preset time period of the unit time. After the preset time period ends, all data write operations are allowed within the time period other than the preset time period.

[0008] Therefore, in this embodiment of the application, when controlling the number of underlying SSTables, if the current number is greater than or equal to the first threshold and less than the second threshold, it indicates that if writing continues, the number of SSTable files may exceed the second threshold. In this case, all data write operations are rejected within the first preset time period of the unit time. That is, within the first preset time period, only SSTable file merging is performed to reduce the number of underlying SSTable files. After the preset time period ends, all data write operations are allowed in the time period other than the preset time period. Thus, by controlling the rejection of further write operations and only performing SSTable file merging to reduce the number of underlying SSTable files through the first preset time period, the flexibility of control is improved, resulting in a better control effect.

[0009] In one possible implementation, before rejecting all data write operations within a pre-preset time period within a unit of time, the method further includes:

[0010] Obtain the rejection rate of all data write operations within the unit time period; and determine a preset time period for rejecting all data write operations within the unit time period based on the rejection rate.

[0011] In this way, after calculating the preset time period, all data write operations can be rejected within the first preset time period of the unit time. After the preset time period ends, all data write operations are allowed within the time period other than the preset time period. Thus, by controlling the rejection of further write operations through the first preset time period, only the merging of SSTable files is performed to reduce the number of underlying SSTable files, thereby improving the flexibility of control and making the control effect better.

[0012] In one possible implementation, obtaining the rejection ratio of all data write operations rejected within the unit time period includes:

[0013] Calculate the first difference between the second threshold and the first threshold, and the second difference between the current quantity and the first threshold; and based on the first difference and the second difference, obtain the rejection ratio of all data write operations rejected per unit time.

[0014] By calculating the rejection rate of all data write operations per unit time based on the first and second differences, the preset time period is determined based on the rejection rate, thereby improving the accuracy of the preset time period calculation.

[0015] In one possible implementation, obtaining the rejection ratio of all data write operations rejected per unit time based on the first difference and the second difference includes:

[0016] Calculate the ratio of the second difference to the first difference; and based on the ratio of the second difference to the first difference, obtain the rejection ratio of all data write operations rejected within the unit time period, which can also improve the accuracy of the calculation for the preset time period.

[0017] In one possible implementation, obtaining the rejection ratio of all data write operations rejected per unit time based on the ratio of the second difference to the first difference includes:

[0018] according to Obtain the rejection rate of all data write operations rejected within the specified unit of time.

[0019] Where P represents the rejection ratio of all data write operations, n is an integer greater than 0, v2 represents the second difference, and v1 represents the first difference.

[0020] In one possible implementation, the method further includes:

[0021] If the current quantity is less than the first threshold, then all data write operations are allowed to be performed until the current quantity is greater than or equal to the first threshold.

[0022] Therefore, when the current number of SSTable files is less than the first threshold, it indicates that the current number of SSTable files is relatively small. Continuing to perform write operations will not lead to an excessive number of SSTable files. In this case, all data write operations can continue to be performed, meaning all data write operations are allowed. It should be noted that this does not mean that all data write operations are allowed indefinitely, but rather that data write operations will stop when the current number of SSTable files is greater than or equal to the first threshold.

[0023] In one possible implementation, the method further includes:

[0024] If the current number is greater than or equal to the second threshold, then no write operation is allowed for any data until the number of SSTable files is less than the second threshold.

[0025] Therefore, when the current number of SSTable files is greater than or equal to the first threshold, it indicates that the current number of SSTable files is too large, and no write operations are allowed. Since reading data requires querying each SSTable, the larger the current number of SSTable files, the greater the latency of reading data from the SSTable files. At this point, all data write operations can be rejected, that is, all data write operations are not allowed until the number of SSTable files is less than the second threshold. This can avoid large latency in reading expired data due to the large number of SSTable files, thereby reducing the data reading latency.

[0026] Secondly, embodiments of this application also provide a control device for a data writing operation, which may include:

[0027] The retrieval module is used to retrieve the current number of SSTable files;

[0028] The control module is configured to, if the current quantity is greater than or equal to a first threshold and less than a second threshold, reject all data write operations within a preset time period within a unit of time, and allow all data write operations within a time period other than the preset time period after the preset time period ends.

[0029] In one possible implementation, the acquisition module is further configured to acquire the rejection ratio of all data write operations rejected within the unit time period.

[0030] The control module is also used to determine a preset time period for rejecting all data write operations within the unit time period based on the rejection ratio.

[0031] In one possible implementation, the acquisition module is specifically used to calculate the first difference between the second threshold and the first threshold, and the second difference between the current quantity and the first threshold; and based on the first difference and the second difference, to obtain the rejection ratio of rejecting all data write operations within the unit time period.

[0032] In one possible implementation, the acquisition module is specifically used to calculate the ratio of the second difference to the first difference; and based on the ratio of the second difference to the first difference, to obtain the rejection ratio of rejecting all data write operations within the unit time period.

[0033] In one possible implementation, the acquisition module is specifically used to... Obtain the rejection rate of all data write operations rejected within the given unit of time.

[0034] Where P represents the rejection ratio of all data write operations, n is an integer greater than 0, v2 represents the second difference, and v1 represents the first difference.

[0035] In one possible implementation, the control module is further configured to allow all data write operations to be performed if the current quantity is less than the first threshold, until the current quantity is greater than or equal to the first threshold.

[0036] In one possible implementation, the control module is further configured to prevent all data from being written if the current quantity is greater than or equal to the second threshold, until the number of SSTable files is less than the second threshold.

[0037] Thirdly, embodiments of this application also provide an electronic device, which may include:

[0038] At least one processor; and

[0039] A memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the control method for the data write operation described in any of the first aspects above.

[0041] Fourthly, embodiments of this application also provide a non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to control the computer to perform the data writing operation described in any of the first aspects above.

[0042] Fifthly, embodiments of this application also provide a method for controlling a data write operation, which may include:

[0043] Get the current number of SSTable files.

[0044] If the current quantity is greater than or equal to the first threshold and less than the second threshold, then all data write operations are rejected within the first preset time period, and then all data write operations are allowed within the time period other than the preset time period.

[0045] An embodiment of the above application has the following advantages or beneficial effects: When controlling the number of underlying SSTables, by setting a first threshold and a second threshold, when the current number of SSTable files is obtained, the current number of SSTable files is compared with the first threshold and the second threshold respectively. If the current number is greater than or equal to the first threshold and less than the second threshold, it indicates that if writing continues, the number of SSTable files may exceed the second threshold. Therefore, all data write operations are rejected within the first preset time period of the unit time. That is, within the first preset time period, only the merging of SSTable files is performed to reduce the number of underlying SSTable files. After the preset time period ends, all data write operations are allowed in the time period other than the preset time period. Thus, by controlling the rejection of continued write operations and only performing the merging of SSTable files to reduce the number of underlying SSTable files through the first preset time period, the control flexibility is improved and the control effect is better.

[0046] Other effects of the above-mentioned alternative methods will be described below in conjunction with specific embodiments. Attached Figure Description

[0047] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this application. Wherein:

[0048] Figure 1 This is a scenario diagram illustrating a control method for implementing the write operation in the embodiments of this application;

[0049] Figure 2 This is a flowchart illustrating the control method for data writing operations according to the first embodiment of this application;

[0050] Figure 3 This is a flowchart illustrating the control method for data writing operations according to the second embodiment of this application;

[0051] Figure 4 This is a schematic diagram illustrating the relationship between the current number of SSTable files and the rejection rate according to the second embodiment of this application;

[0052] Figure 5 This is a schematic diagram of the control device for data writing operations according to the third embodiment of this application;

[0053] Figure 6 This is a block diagram of an electronic device for controlling a data writing operation according to an embodiment of this application. Detailed Implementation

[0054] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0055] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0056] Figure 1 This is a scenario diagram illustrating a control method for implementing the write operation in the embodiments of this application. For example, please refer to... Figure 1 As shown, this write operation control method can be applied to LSM tree data structures. When data is written, it is first written to multiple memory tables. When the number of memory tables reaches a threshold, the data in the memory tables is written to multiple sorted string tables (SSTables). When the number of SSTables reaches a threshold, the multiple SSTables are merged into one SSTable. When the write speed is too high, the merging speed cannot keep up with the write speed, resulting in an ever-increasing number of underlying SSTables.

[0057] To control the number of underlying SSTables, existing technologies set a threshold to limit the number of SSTables. When the number of SSTables reaches the threshold, no more writes are allowed; only SSTable merging is performed. Writes can only resume once the number of merged underlying SSTables is less than the threshold. However, with this method, writes are completely unaffected before the number of underlying SSTables reaches the threshold; once the threshold is reached, write traffic drops drastically. Therefore, the existing method lacks flexibility and is ineffective in controlling the number of underlying SSTables.

[0058] To improve control flexibility and achieve better control performance, this application provides a write operation control method. By setting a first threshold and a second threshold, when the current number of SSTable files is obtained, the current number of SSTable files is compared with the first threshold and the second threshold respectively. If the current number is greater than or equal to the first threshold and less than the second threshold, it indicates that continuing to write will risk the number of SSTable files exceeding the second threshold. Therefore, all data write operations are rejected within a preset time period within a unit of time. That is, within the preset time period, only SSTable file merging is performed to reduce the number of underlying SSTable files. After the preset time period ends, all data write operations are allowed in the remaining time periods. Thus, by controlling and rejecting further write operations within the preset time period and only performing SSTable file merging to reduce the number of underlying SSTable files, the control flexibility is improved, resulting in better control performance.

[0059] It should be noted that, in this embodiment, all data write operations are rejected within a pre-set time period within a unit of time. That is, within this pre-set time period, only SSTable file merging is performed to reduce the number of underlying SSTable files. However, all data write operations are allowed during the time period outside the pre-set time period. That is, during the time period outside the pre-set time period, all data write operations are performed, and SSTable file merging is also performed to reduce the number of underlying SSTable files.

[0060] Understandably, all data write operations rejected within the pre-set time period will be allowed to execute when they re-initiate the data write operation request. Therefore, for users performing data write operations, these data write operations only take a long time to complete, but will not cause business interruption.

[0061] The technical solutions of this application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0062] Example 1

[0063] Figure 2 This is a flowchart illustrating a control method for a data writing operation according to the first embodiment of this application. This control method can be executed by software and / or hardware devices. For example, the hardware device can be a control device for the data writing operation, which can be installed in an electronic device. For an example, please refer to [link to example]. Figure 2As shown, the control method for this data write operation may include:

[0064] S201. Get the current number of SSTable files.

[0065] For example, in this embodiment of the application, when obtaining the current number of SSTable files, the number of SSTable files can be detected in real time to obtain the current number of SSTable files; alternatively, the number of SSTable files can be detected non-real-time to obtain the current number of SSTable files. The specific settings can be configured according to actual needs. It is understood that in order to obtain the current number of SSTable files in a timely manner, the number of SSTable files can be detected in real time.

[0066] Once the current number of SSTable files is obtained, it can be compared with a pre-set first threshold and a second threshold to determine the relationship between the current number of SSTable files and the first and second thresholds, i.e., the following S202 is executed:

[0067] S202. Determine whether the current number of SSTable files is greater than or equal to the first threshold and less than the second threshold.

[0068] Wherein, the first threshold is less than the second threshold. For example, the first threshold can be 320 and the second threshold can be 400. Of course, the first threshold can also be 300 and the second threshold can also be 420. The specific values ​​can be set according to actual needs. Here, this application embodiment does not further limit the size of the first and second thresholds.

[0069] When determining whether the current number of SSTable files is greater than or equal to the first threshold and less than the second threshold, it can be seen that the determination result can be divided into three different cases. In one possible case, as shown in S203, the current number of SSTable files is greater than or equal to the first threshold and less than the second threshold. In another possible case, as shown in S204, the current number of SSTable files is less than the first threshold. In yet another possible case, as shown in S205, the current number of SSTable files is greater than or equal to the second threshold. The following will explain these three possible cases in detail with reference to S203-S205.

[0070] S203. If the current quantity is greater than or equal to the first threshold and less than the second threshold, then all data write operations are rejected within the first preset time period of the unit time. After the preset time period ends, all data write operations are allowed within the time period other than the preset time period.

[0071] It is understandable that when determining the unit of time, either one second or every 2 seconds can be used as the unit of time, depending on the actual needs. For example, in this embodiment of the application, one second can be used as the unit of time.

[0072] It should be noted that when the current number of SSTable files is greater than or equal to the first threshold and less than the second threshold, all data write operations need to be rejected for the first preset time period within a unit of time. Therefore, the preset time period needs to be determined first. For example, the preset time period can be determined based on the current number of SSTable files, the first threshold, and the second threshold. If, after determination, the preset time period is 1 / 10 of a second, and the unit of time is one second, then when the current number of SSTable files is greater than or equal to the first threshold and less than the second threshold, all data write operations are rejected for the first 1 / 10 of a second. This allows only SSTable file merging to be performed during this first 1 / 10 of a second, reducing the number of underlying SSTable files. After the first 1 / 10 of a second ends, all data write operations are allowed for the remaining 9 / 10 of a second. Thus, by controlling and rejecting further write operations during the first 1 / 10 of a second and only performing SSTable file merging to reduce the number of underlying SSTable files, the control flexibility is improved, resulting in better control performance.

[0073] It should be noted that, in this embodiment, all data write operations are rejected for the first 1 / 10 of a second; that is, only SSTable file merging is performed during this first 1 / 10 of a second to reduce the number of underlying SSTable files. During the remaining 9 / 10 of a second, all data write operations are allowed; that is, all data write operations are performed during this remaining 9 / 10 of a second, and SSTable file merging is also performed to reduce the number of underlying SSTable files.

[0074] Understandably, all data write operations rejected within that 1 / 10-second period will be allowed to execute when they re-initiate the data write operation request. Therefore, for users performing data write operations, these data write operations only take a long time to complete, but will not cause business interruption.

[0075] S204. If the current quantity is less than the first threshold, then all data write operations are allowed until the current quantity is greater than or equal to the first threshold.

[0076] Understandably, when the current number of SSTable files is less than the first threshold, it means the current number of SSTable files is relatively small, and continuing to perform write operations will not cause the number of SSTable files to become excessive. In this case, all data write operations can continue to be performed, meaning all data write operations are allowed. It should be noted that this does not mean that all data write operations are allowed indefinitely, but rather that data write operations will stop when the current number of SSTable files is greater than or equal to the first threshold.

[0077] Taking a first threshold of 320 as an example, when the current number of SSTable files is less than 320, it means that the current number of SSTable files is relatively small, and continuing to perform write operations will not cause the number of SSTable files to exceed 400. At this time, all data write operations can be allowed to continue. This does not mean that all data write operations are allowed without limit, but rather that data write operations will stop when the current number of SSTable files is greater than or equal to 320.

[0078] S205. If the current quantity is greater than or equal to the second threshold, then no write operation is allowed for any data until the number of SSTable files is less than the second threshold.

[0079] Understandably, when the current number of SSTable files is greater than or equal to the first threshold, it indicates that the current number of SSTable files is too large, and no write operations are allowed. Since reading data requires querying each SSTable, the larger the current number of SSTable files, the greater the latency of reading data from the SSTable files. At this point, all data write operations can be rejected, meaning that no data write operations are allowed until the number of SSTable files is less than the second threshold. This can avoid large latency in reading expired data due to the large number of SSTable files, thereby reducing the data reading latency.

[0080] It is understandable that the period until the number of SSTable files is less than the second threshold can include two different scenarios. One scenario is that the number of SSTable files is less than both the second and first thresholds. In this case, all data write operations are allowed, as described in S204 above, and will not be repeated here. The other scenario is that the number of SSTable files is less than the second threshold but greater than or equal to the first threshold. In this case, all data write operations can be rejected for the first preset time period within a unit of time. After the preset time period ends, all data write operations are allowed for the remaining time periods, as described in S203 above, and will not be repeated here.

[0081] For example, taking a second threshold of 400 as an example, when the current number of SSTable files is greater than or equal to 400, it means that the current number of SSTable files is already too large. Continuing to perform write operations will not cause the number of SSTable files to exceed 400. Since reading data requires querying each SSTable, the latency of reading data from this excessive number of SSTable files will be greater. At this time, all data write operations can be rejected, that is, all data write operations are not allowed to be performed until the current number of SSTable files is less than 400. This can avoid the large latency of reading expired data due to the large number of SSTable files, thereby reducing the data reading latency.

[0082] Example 2

[0083] Understandably, based on the above Figure 2 In the illustrated embodiment, if the current quantity is greater than or equal to a first threshold and less than a second threshold when executing S203, all data write operations are rejected within the first preset time period of the unit time. Before allowing all data write operations in the time period other than the preset time period after the preset time period ends, the preset time period needs to be obtained first. Only then can all data write operations be rejected within the first preset time period of the unit time, and all data write operations be allowed in the time period other than the preset time period after the preset time period ends. For an example, please refer to [link to example]. Figure 3 As shown, Figure 3 This is a flowchart illustrating a control method for a data writing operation according to a second embodiment of this application. The method may further include:

[0084] S301. Calculate the first difference between the second threshold and the first threshold, and the second difference between the current quantity and the first threshold.

[0085] For example, taking a first threshold of 320 and a second threshold of 400 as an example, the first difference between the second threshold and the first threshold can be calculated to be 80. If the current number of SSTable files is 320, the second difference between the current number and the first threshold can be calculated to be 0.

[0086] After calculating the first difference between the second threshold and the first threshold, and the second difference between the current quantity and the first threshold, the rejection ratio of all data write operations rejected per unit time can be obtained based on the first difference and the second difference. Then, based on the rejection ratio, a preset time period for rejecting all data write operations per unit time can be determined, i.e., the following steps S302-S303 are executed:

[0087] S302. Based on the first difference and the second difference, obtain the rejection rate of all data write operations rejected per unit time.

[0088] Optionally, to obtain the rejection ratio of all data write operations rejected per unit time based on the first difference and the second difference, the ratio of the second difference to the first difference can be calculated first; and the rejection ratio of all data write operations rejected per unit time can be obtained based on the ratio of the second difference to the first difference.

[0089] For example, when obtaining the rejection rate of all data write operations rejected per unit time based on the ratio of the second difference to the first difference, it can be based on... Get the rejection rate of all data write operations rejected within a unit of time.

[0090] Where P represents the rejection ratio of all data write operations, n is an integer greater than 0, v2 represents the second difference, and v1 represents the first difference.

[0091] Based on the description in S301 above, when the first difference between the second threshold and the first threshold is calculated to be 80, and the second difference between the current quantity and the first threshold is 0, it can be determined according to... By calculating the rejection rate of all data write operations within a unit of time, we can obtain that the rejection rate of all data write operations within a unit of time is 1 / 10.

[0092] To more clearly illustrate the relationship between the current number of SSTable files and the rejection rate, see the example below. Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the relationship between the current number of SSTable files and the rejection rate according to the second embodiment of this application, combined with... Figure 4As can be seen, when the current number of SSTable files is greater than or equal to the first threshold of 320 and less than the second threshold of 400, the more SSTable files there are, the greater the corresponding rejection ratio, meaning that the preset time period for rejecting writes will be longer within a unit of time. When the current number of SSTable files is greater than the second threshold of 400, the corresponding rejection ratio is 1, meaning that data write operations are rejected throughout the entire unit of time. In this case, it is not necessary to calculate the rejection ratio first. Instead, when the current number of SSTable files is greater than the second threshold of 400, all data write operations are not allowed until the number of SSTable files is less than the second threshold of 400. For details, please refer to the description in S205 above, which will not be repeated here.

[0093] After obtaining the rejection rate of all data write operations within a unit of time through S302, the following S303 can be executed:

[0094] S303. Based on the rejection ratio, determine the preset time period for rejecting all data write operations per unit time.

[0095] When calculating the preset time period for rejecting all data write operations per unit time based on the rejection comparison, the product of the unit time and the rejection ratio can be calculated to obtain the preset time period for rejecting all data write operations per unit time.

[0096] For example, referring to the description in S302 above, after calculating the rejection ratio of 1 / 10, if the unit time is one second, then a preset time period of 1 / 10 second is obtained within that unit time to reject all data write operations. Thus, when the current number of SSTable files is greater than or equal to 320 and less than 400, all data write operations can be rejected for the first 1 / 10 second of the first second, allowing only SSTable file merging to reduce the current number of underlying SSTable files. After the first 1 / 10 second ends, all data write operations are allowed for the remaining 9 / 10 seconds. Therefore, by controlling the rejection of further write operations during the first 1 / 10 second and only performing SSTable file merging to reduce the number of underlying SSTable files, the control flexibility is improved, resulting in better control performance.

[0097] Example 3

[0098] Figure 5 This is a schematic diagram of the structure of a control device 50 for a data writing operation according to a third embodiment of this application. The control device 50 for the data writing operation may include:

[0099] Module 501 is used to obtain the current number of SSTable files.

[0100] The control module 502 is configured to, if the current quantity is greater than or equal to a first threshold and less than a second threshold, reject all data write operations within a preset time period within a unit time period, and allow all data write operations within a time period other than the preset time period after the preset time period ends.

[0101] Optionally, the acquisition module 501 is further configured to acquire the rejection ratio of all data write operations rejected within the unit time period.

[0102] The control module 502 is further configured to determine, based on the rejection ratio, a preset time period for rejecting all data write operations within the unit time period.

[0103] Optionally, the acquisition module 501 is specifically used to calculate the first difference between the second threshold and the first threshold, and the second difference between the current quantity and the first threshold; and to obtain the rejection ratio of rejecting all data write operations within the unit time based on the first difference and the second difference.

[0104] Optionally, the acquisition module 501 is specifically used to calculate the ratio of the second difference to the first difference; and based on the ratio of the second difference to the first difference, to obtain the rejection ratio of rejecting all data write operations within the unit time.

[0105] Optionally, the acquisition module 501 is specifically used to obtain based on Obtain the rejection rate of all data write operations rejected within the specified unit of time.

[0106] Where P represents the rejection ratio of all data write operations, n is an integer greater than 0, v2 represents the second difference, and v1 represents the first difference.

[0107] Optionally, the control module 502 is further configured to allow all data write operations to be performed if the current quantity is less than the first threshold, until the current quantity is greater than or equal to the first threshold.

[0108] Optionally, the control module 502 is further configured to prevent all data from being written if the current quantity is greater than or equal to the second threshold, until the number of SSTable files is less than the second threshold.

[0109] The data writing operation control device 50 provided in this application embodiment can execute the technical solution of the data writing operation control method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the data writing operation control method. Please refer to the implementation principle and beneficial effects of the data writing operation control method. It will not be repeated here.

[0110] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0111] like Figure 6 As shown, Figure 6 This is a block diagram of an electronic device for controlling a data write operation according to embodiments of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0112] like Figure 6 As shown, the electronic device includes one or more processors 601, a memory 602, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 6 Take the 601 processor as an example.

[0113] The memory 602 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor to cause the at least one processor to perform the control method for data write operations provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the control method for data write operations provided in this application.

[0114] Memory 602, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the data write operation control method in the embodiments of this application (e.g., attached...). Figure 5 (The acquisition module 501 and control module 502 are shown). The processor 601 executes various functional applications and data processing of the server by running non-transient software programs, instructions, and modules stored in the memory 602, that is, it implements the control method for data writing operations in the above method embodiments.

[0115] Memory 602 may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function. The data storage area may store data created by the use of the electronic device according to the control method of the data write operation. Furthermore, memory 602 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 602 may optionally include memory remotely located relative to processor 601, and these remote memories can be connected to the electronic device of the control method of the data write operation via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0116] The electronic device for controlling the data writing operation may further include an input device 603 and an output device 604. The processor 601, memory 602, input device 603, and output device 604 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0117] Input device 603 can receive input digital or character information, as well as key signal inputs related to user settings and function control of electronic devices that generate control methods for data writing operations, such as touch screens, keypads, mice, trackpads, touchpads, joysticks, one or more mouse buttons, trackballs, joysticks, etc. Output device 604 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The display device may include, but is not limited to, liquid crystal displays (LCDs), light-emitting diode (LED) displays, and plasma displays. In some embodiments, the display device may be a touch screen.

[0118] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0119] These computational programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0120] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0121] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0122] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0123] According to the technical solution of this application embodiment, when controlling the number of underlying SSTables, a first threshold and a second threshold are set. When the current number of SSTable files is obtained, the current number of SSTable files is compared with the first threshold and the second threshold respectively. If the current number is greater than or equal to the first threshold and less than the second threshold, it indicates that if writing continues, the number of SSTable files may exceed the second threshold. Therefore, all data write operations are rejected within the first preset time period of the unit time. That is, within the first preset time period, only SSTable file merging is performed to reduce the number of underlying SSTable files. After the preset time period ends, all data write operations are allowed in the time period other than the preset time period. Thus, by controlling the rejection of continued write operations and only performing SSTable file merging to reduce the number of underlying SSTable files through the first preset time period, the control flexibility is improved and the control effect is better.

[0124] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0125] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A control method for data write operations, characterized in that, include: Get the current number of elements in the sorted string table SSTable file; If the current quantity is greater than or equal to the first threshold and less than the second threshold, then all data write operations are rejected within the first preset time period of the unit time. After the preset time period ends, all data write operations are allowed within the time period other than the preset time period. Before rejecting all data write operations within a preset time period within a unit of time, the method further includes: Calculate the first difference between the second threshold and the first threshold, and the second difference between the current quantity and the first threshold, respectively; Based on the ratio of the second difference to the first difference, the rejection ratio of all data write operations rejected within the unit time period is obtained; Based on the rejection ratio, a preset time period is determined for rejecting all data write operations within the unit time period.

2. The method according to claim 1, characterized in that, The step of obtaining the rejection ratio of all data write operations rejected per unit time based on the ratio of the second difference to the first difference includes: according to Obtain the rejection rate of all data write operations rejected within the given unit of time. in, This indicates the rejection rate for all data write operations. It is an integer greater than 0. Indicates the second difference. This represents the first difference.

3. The method according to any one of claims 1-2, characterized in that, The method further includes: If the current quantity is less than the first threshold, then all data write operations are allowed to be performed until the current quantity is greater than or equal to the first threshold.

4. The method according to any one of claims 1-2, characterized in that, The method further includes: If the current number is greater than or equal to the second threshold, then no write operation is allowed for any data until the number of SSTable files is less than the second threshold.

5. A control device for data writing operations, characterized in that, include: The retrieval module is used to retrieve the current number of elements in the sorted string table SSTable file. The control module is configured to, if the current quantity is greater than or equal to a first threshold and less than a second threshold, reject all data write operations within a preset time period within a unit of time, and allow all data write operations within a time period other than the preset time period after the preset time period ends. The acquisition module is further configured to calculate the first difference between the second threshold and the first threshold, and the second difference between the current quantity and the first threshold; and to obtain the rejection ratio of rejecting all data write operations within the unit time period based on the ratio of the second difference to the first difference. The control module is also used to determine a preset time period for rejecting all data write operations within the unit time period based on the rejection ratio.

6. The apparatus according to claim 5, characterized in that, The acquisition module is specifically used to obtain based on Obtain the rejection rate of all data write operations rejected within the given unit of time. in, This indicates the rejection rate for all data write operations. It is an integer greater than 0. Indicates the second difference. This represents the first difference.

7. The apparatus according to any one of claims 5-6, characterized in that, The control module is further configured to allow all data write operations to be performed if the current quantity is less than the first threshold, until the current quantity is greater than or equal to the first threshold.

8. The apparatus according to any one of claims 5-6, characterized in that, The control module is further configured to prevent all data from being written if the current quantity is greater than or equal to the second threshold, until the number of SSTable files is less than the second threshold.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the control method for the data write operation according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to control the computer to perform the data writing operation as described in any one of claims 1-7.

Images