Sequence value caching method and related device

Abstract

The present description provides a sequence value caching method and a related device. A distributed cluster maintains a target data table and an internal table. Memories of various nodes of the distributed cluster cache a plurality of consecutive sequence values that are obtained by the nodes on the basis of a preset sequence parameter from the internal table, starting from a starting sequence value in the internal table. The plurality of sequence values are used for filling a target field in the target data table. The method comprises: if a sequence parameter is modified, reclaiming unused caches comprised in memories of various nodes; and determining, from a plurality of reclaimed unused caches, a comprised target cache having a sequence value closest to the starting sequence value, and updating the starting sequence value to a starting sequence value among several sequence values comprised in the target cache so that, on the basis of the modified sequence parameter, the various nodes acquire, from an internal table, a plurality of consecutive sequence values starting from the updated starting sequence value, and load the plurality of sequence values as caches into the memories of the nodes.

Description

Sequence value cache processing method and related device TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of database, and in particular, to a sequence value cache processing method and related device. BACKGROUND

[0002] In the database technology, for a certain field (usually a primary key field) in a data table, a unique sequence value can be automatically generated for the primary key field in a new data record inserted into the data table using SEQUENCE, so as to uniquely identify each row of data record by using the sequence value, and ensure the uniqueness of each row of data record, so as to facilitate subsequent data query, update and deletion operations.

[0003] In a distributed scenario, in order to improve the generation efficiency of sequence values, each node in a distributed cluster can first obtain a plurality of sequence values starting from a starting sequence value (i.e. the starting sequence value of a plurality of sequence values maintained in an internal table of the distributed cluster) in a continuous manner (including increment or decrement) based on the SEQUENCE parameter, and then load the plurality of sequence values as cache into the local memory, and subsequently each node can directly read the sequence values in order from the respective cache, so as to fill the primary key field based on the read sequence values.

[0004] It should be noted that after any node obtains a plurality of sequence values from the internal table, the starting sequence value in the internal table needs to be updated to the next sequence value after the last sequence value in the plurality of sequence values in the specified direction, so that other nodes can subsequently obtain a plurality of sequence values starting from the updated starting sequence value in a continuous manner from the internal table, avoiding obtaining repeated sequence values by multiple nodes.

[0005] However, once the SEQUENCE parameter is modified, it will cause the cache of each node and the SEQUENCE parameter to be inconsistent, and further cause problems such as sequence value holes or sequence value repetition, which seriously affects the database performance, and even causes correctness problems. Based on this, the local cache in each node needs to be emptied, and then each node needs to obtain a plurality of sequence values starting from the current starting sequence value in a continuous manner from the internal table based on the modified SEQUENCE parameter, thereby causing a serious waste of a large number of unused sequence values in the cache. SUMMARY

[0006] Therefore, one or more embodiments of the present specification provide a sequence value cache processing method and related device.

[0007] In a first aspect, the present specification provides a sequence value cache processing method, applied to a scheduling node in a distributed cluster, wherein the distributed cluster maintains a target data table and an internal table; the internal table is used to maintain a plurality of sequence values continuous from a starting sequence value; the plurality of sequence values are used to sequentially fill a target field in the target data table; and the memory of each node included in the distributed cluster caches a plurality of sequence values continuous from the starting sequence value, which are obtained by the node from the internal table based on a preset sequence parameter.

[0008] The method comprises:

[0009] In response to the sequence parameter being modified, the memory of each node includes unused caches; wherein the unused caches include a plurality of sequence values that have not been used to fill the target field;

[0010] A target cache is determined from the plurality of recovered unused caches, which includes sequence values closest to the starting sequence value;

[0011] The starting sequence value in the internal table is updated to the starting sequence value in the plurality of sequence values included in the target cache, so that each node obtains a plurality of sequence values continuous from the updated starting sequence value from the internal table based on the modified sequence parameter, loads the plurality of sequence values as caches into the memory of the node, and reads the sequence values from the caches in sequence to fill the target field based on the read sequence values.

[0012] In a second aspect, the present specification provides a sequence value cache processing device, applied to a scheduling node in a distributed cluster, wherein the distributed cluster maintains a target data table and an internal table; the internal table is used to maintain a plurality of sequence values continuous from a starting sequence value; the plurality of sequence values are used to sequentially fill a target field in the target data table; and the memory of each node included in the distributed cluster caches a plurality of sequence values continuous from the starting sequence value, which are obtained by the node from the internal table based on a preset sequence parameter.

[0013] The device comprises:

[0014] A recovery unit is configured to, in response to the sequence parameter being modified, recover the unused caches included in the memory of each node; wherein the unused caches include a plurality of sequence values that have not been used to fill the target field;

[0015] A determination unit is configured to determine a target cache from the plurality of recovered unused caches, which includes sequence values closest to the starting sequence value;

[0016] An updating unit is configured to update a starting sequence value in the internal table to a starting sequence value among a plurality of sequence values contained in the target cache, so that each node obtains, based on the modified sequence parameter, a plurality of sequence values from the internal table starting from the updated starting sequence value, loads the plurality of sequence values into the memory of the node as a cache, reads the sequence values in sequence from the cache, and fills the target field based on the read sequence values.

[0017] Correspondingly, the specification also provides a computing device, comprising a memory and a processor; the memory stores a computer program / instruction executable by the processor; when the processor executes the computer program / instruction, the sequence value cache processing method of the first aspect is executed.

[0018] Correspondingly, the specification also provides a computer readable storage medium, which stores a computer program / instruction, and when the processor executes the computer program / instruction, the sequence value cache processing method of the first aspect is executed.

[0019] Correspondingly, the specification also provides a computer program product, which comprises a computer program / instruction, and when the processor executes the computer program / instruction, the sequence value cache processing method of the first aspect is executed.

[0020] In summary, the present application can be applied to a scheduling node in a distributed cluster, which maintains a target data table and an internal table. The internal table is used to maintain a plurality of sequence values that are continuous from a starting sequence value. The memory of each node in the distributed cluster caches a plurality of sequence values that are continuous from the starting sequence value and are obtained by the node from the internal table based on a preset sequence parameter. Accordingly, each node can read the sequence values in sequence from the cache to fill a target field (e.g., a primary key field) in the target data table based on the read sequence values. In an illustrative embodiment, the scheduling node can reclaim unused caches contained in the memory of each node in response to the sequence parameter being modified, wherein the unused caches contain a plurality of sequence values that have not been used to fill the target field. Further, the scheduling node can determine a target cache from the reclaimed unused caches that contains sequence values closest to the starting sequence value. Further, the scheduling node can update the starting sequence value in the internal table to a starting sequence value in the sequence values contained in the target cache, so that each node obtains a plurality of sequence values that are continuous from the updated starting sequence value from the internal table based on the modified sequence parameter, loads the plurality of sequence values into the memory of the node as a cache, and reads the sequence values in sequence from the cache to fill the target field based on the read sequence values. In this way, when the sequence parameter is modified, the scheduling node first reclaims the unused caches in the memory of each node, and determines a target cache from the reclaimed unused caches that contains sequence values closest to the starting sequence value. It should be understood that the starting sequence value of the current internal table is necessarily after the ending sequence value contained in the target cache, but the sequence values in the target cache have not been used. Based on this, the starting sequence value of the current internal table can be updated to the starting sequence value in the target cache, i.e., the starting sequence value is rolled back to the starting sequence value in the target cache. In this way, each node can obtain a plurality of sequence values that are continuous from the updated starting sequence value, and so on, thereby reducing the waste of sequence values as much as possible and improving the performance of the database on the premise that each node does not obtain duplicate sequence values. BRIEF DESCRIPTION OF DRAWINGS

[0021] FIG. 1 is a schematic diagram of a system architecture of a distributed cluster according to an example embodiment;

[0022] FIG. 2 is a schematic diagram of a sequence value cache processing method according to an example embodiment;

[0023] FIG. 3 is a schematic diagram of cache reclaiming according to an example embodiment;

[0024] FIG. 4 is another schematic diagram of cache reclaiming according to an example embodiment;

[0025] FIG. 5 is a timing diagram of a work thread obtaining a lock according to an example embodiment;

[0026] FIG. 6 is a timing diagram of a DDL thread acquiring a lock according to an example embodiment;

[0027] FIG. 7 is a timing diagram of another work thread acquiring a lock according to an example embodiment;

[0028] FIG. 8 is a timing diagram of yet another work thread acquiring a lock according to an example embodiment;

[0029] FIG. 9 is a structural diagram of a sequence value cache processing apparatus according to an example embodiment;

[0030] FIG. 10 is a structural diagram of a computing device according to an example embodiment. DETAILED DESCRIPTION

[0031] The example embodiments will be described in detail herein with reference to the attached drawings. In the following description, like reference numerals refer to like elements, unless the context clearly dictates otherwise. The following description of example embodiments is not representative of all possible embodiments consistent with one or more aspects of the present specification. Rather, it is merely an example of apparatuses and methods consistent with some aspects of one or more embodiments of the present specification as detailed in the appended claims.

[0032] It should be noted that the steps of the methods in other embodiments need not necessarily be performed in the order shown and described in the present specification. In some other embodiments, the steps of the methods can include more or fewer steps than those described in the present specification. Furthermore, a single step described in the present specification can be split into multiple steps in other embodiments; and multiple steps described in the present specification can be combined into a single step in other embodiments.

[0033] It should be noted that the term "plurality" as used in the present specification means two or more.

[0034] Furthermore, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present specification are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation portals for the user to choose authorization or refusal.

[0035] First, some terms in the present specification are explained to facilitate understanding by those skilled in the art.

[0036] (1) SEQUENCE, a database object in a database for generating a unique numerical sequence, which is mainly used to automatically generate a unique sequence value for a certain field (usually a primary key field) in a data table. Whenever a new data record needs to be inserted into the data table, SEQUENCE can automatically generate a unique sequence value for the primary key field in the data record to uniquely identify each row of data record with the sequence value, ensuring the uniqueness of each row of data record, so as to facilitate subsequent data query, update and deletion operations, etc.

[0037] For example, the above-mentioned data table can be a commodity transaction order table, which can contain multiple fields (or data columns) such as commodity ID, payment time, and amount, and each row of data record can be a commodity transaction record. For example, the commodity ID field in the table can be the primary key field, and SEQUENCE can automatically generate 1, 2, 3, 4, …, etc. unique sequence values for the commodity ID field in each row of data record in the table as commodity ID values, thereby realizing the unique identification of each row of data record with commodity ID values.

[0038] In generating the sequence value, specifically, the disk of the database can be accessed to obtain the sequence value from the internal table stored in the disk, which is not limited in the present specification.

[0039] Further, in order to improve the generation efficiency of the sequence value, the SEQUENCE CACHE parameter can be used, so that a plurality of sequence values can be obtained from the internal table first, and then the plurality of sequence values are loaded into the local memory as a cache, and subsequently the sequence values can be read in order from the cache to fill the primary key field with the read sequence values, and when the sequence values in the cache are all used up, a plurality of sequence values are obtained from the internal table and cached, and so on.

[0040] In addition, in the scenario of a distributed database constructed based on a distributed cluster, each node in the plurality of nodes included in the distributed cluster can obtain a plurality of sequence values from the internal table of the disk and load the plurality of sequence values into the memory of the node as a cache, and subsequently each node can read the sequence values in order from the respective cache, etc.

[0041] It should be noted that when the plurality of sequence values are obtained from the internal table, the plurality of sequence values that are continuous from the starting sequence value in the internal table can be obtained based on a preset SEQUENCE parameter. In an illustrated embodiment, the plurality of sequence values that are continuous from the starting sequence value can be a plurality of sequence values that are increasing from the starting sequence value, or can also be a plurality of sequence values that are decreasing from the starting sequence value, which is not limited in the present specification. For example, the parameter can include a step size for sequentially reading sequence values from the cache, and a specification (i.e., a cache size) of the cache. The cache size can refer to a number of sequence values that can be used to fill the primary key field and are sequentially read from all sequence values contained in the cache according to the set step size. Accordingly, each node can determine a target number of sequence values to be obtained from the internal table according to the step size and the cache size set in the parameter, and then obtain the target number of sequence values that are continuous from the starting sequence value in the internal table.

[0042] In addition, after any node obtains the plurality of sequence values, the starting sequence value of the internal table needs to be updated to a next sequence value after an end sequence value in the obtained plurality of sequence values, so that other nodes can subsequently obtain continuous plurality of sequence values from the starting sequence value in the current internal table, avoiding that the plurality of nodes obtain repeated sequence values.

[0043] In the following, an example of an increasing sequence is taken to illustrate the sequence value obtaining process in a distributed scenario.

[0044] Example 1

[0045] Suppose that the distributed cluster includes node 1, node 2 and node 3, the starting sequence value in the current internal table is 1, the cache size set in the SEQUENCE parameter is 10, and the step size for reading sequence values from the cache is 1. Again, the cache size of 10 means that the number of sequence values that can be read from the cache for filling the primary key field is 10 according to the set step size, not that the total number of sequence values contained in the cache is 10.

[0046] First, node 1 obtains 10 increasing sequence values from 1 to 10 from the internal table based on the starting sequence value 1 of the current internal table, loads the 10 sequence values into the memory of node 1 as a cache (or can be that the sequence value interval [1, 10] is obtained and loaded into the memory of node 1 as a cache), and updates the starting sequence value of the internal table to 11. Subsequently, node 1 can sequentially read sequence values 1, 2, 3, …, 10 from the cache according to the step size 1, and fill the primary key field with the read sequence values.

[0047] Secondly, node 2 gets 10 incremental sequence values 11, 12, 13, …, 20 from the internal table based on the current start sequence value 11 of the internal table, loads the 10 sequence values into the memory of node 2 as a cache, and updates the start sequence value of the internal table to 21. Subsequently, node 2 can read the sequence values 11, 12, 13, …, 20 from the cache in sequence with a step length of 1, and fill the primary key field with the read sequence values.

[0048] Then, node 3 gets 10 incremental sequence values 21, 22, 23, …, 30 from the internal table based on the current start sequence value 21 of the internal table, loads the 10 sequence values into the memory of node 3 as a cache, and updates the start sequence value of the internal table to 31. Subsequently, node 3 can read the sequence values 21, 22, 23, …, 30 from the cache in sequence with a step length of 1, and fill the primary key field with the read sequence values.

[0049] Example Two:

[0050] Suppose that the distributed cluster contains node 1, node 2 and node 3, the start sequence value in the current internal table is 1, the cache size set in the SEQUENCE parameter is 10, and the step length for reading sequence values from the cache is 2.

[0051] Firstly, node 1 gets 20 incremental sequence values 1-20 from the internal table based on the current start sequence value 1, loads the 20 sequence values into the memory of node 1 as a cache, and updates the start sequence value of the internal table to 21. Subsequently, node 1 can read the sequence values 1, 3, 5, …, 19 from the cache in sequence with a step length of 2, and fill the primary key field with the read sequence values.

[0052] Secondly, node 2 gets 20 incremental sequence values 21, 22, 23, …, 40 from the internal table based on the current start sequence value 21, loads the 20 sequence values into the memory of node 2 as a cache, and updates the start sequence value of the internal table to 41. Subsequently, node 2 can read the sequence values 21, 23, 25, …, 39 from the cache in sequence with a step length of 2, and fill the primary key field with the read sequence values.

[0053] Then, node 3 gets 20 incremental sequence values 41, 42, 43, …, 60 from the internal table based on the current start sequence value 41, loads the 20 sequence values into the memory of node 3 as a cache, and updates the start sequence value of the internal table to 61. Subsequently, node 3 can read the sequence values 41, 43, 45, …, 59 from the cache in sequence with a step length of 2, and fill the primary key field with the read sequence values.

[0054] However, once the SEQUENCE parameter is modified, the caches of the respective nodes and the SEQUENCE parameter will be inconsistent. As described in the second example above, when the step X = 2 and the cache size N = 10, the number of sequence values obtained from the internal table and loaded into the memory cache of the respective nodes is N X X = 20, which is an integer multiple of the step 2. However, once the step is modified to Y = 3, the 20 sequence values in the original cache are obviously not an integer multiple of 3, and if the sequence values are read from the remaining sequence values to fill the target field according to the step 3, various problems may occur, such as sequence value holes, or multiple nodes generating duplicate sequence values, etc., thereby causing the sequence values to be unable to uniquely identify each row of data records, resulting in errors in subsequent data query, update and deletion operations, and seriously affecting the performance of the database.

[0055] In order to avoid the above problems, the local caches of the respective nodes need to be emptied directly after the SEQUENCE parameter is modified. Taking the second example above as an example, assuming that when the node 1 uses the sequence values 17 in its cache, the node 2 uses the sequence values 25 in its cache, and the node 3 uses the sequence values 43 in its cache, the parameter is modified, then the unused caches of the node 1 containing the sequence values 18-20, the unused caches of the node 2 containing the sequence values 26-40, and the unused caches of the node 3 containing the sequence values 44-60 need to be emptied. Subsequently, the node 1 reacquires the 30 incremental sequence values 61-90 (taking the cache size 10 and the step 3 as an example) based on the starting sequence value 61 of the current internal table, and so on.

[0056] In this way, although it can be avoided that the respective nodes generate duplicate sequence values, it will cause a serious waste of unused sequence values in the cache, resulting in a serious sequence value hole, and reducing the performance of the entire distributed database system.

[0057] Based on this, the present specification provides a technical solution that, after the sequence parameter is modified, the unused caches in the memory of the respective nodes are recovered first, the target cache containing the sequence values closest to the starting sequence value in the current internal table is determined, and the starting sequence value is rolled back to the starting sequence value in the target cache, thereby minimizing the waste of sequence values on the premise of avoiding the respective nodes obtaining duplicate sequence values.

[0058] In implementation, the present application can be applied to a scheduling node in a distributed cluster, where a target data table and an internal table are maintained. The internal table is used to maintain a plurality of sequence values continuous from a starting sequence value. The memory of each node included in the distributed cluster caches a plurality of sequence values continuous from the starting sequence value, which are obtained by the node from the internal table based on a preset sequence parameter. Accordingly, each node can read the sequence values in sequence from the cache to fill a target field (e.g., a primary key field) in the target data table based on the read sequence values. In an illustrated embodiment, the scheduling node can reclaim unused caches included in the memory of each node in response to the sequence parameter being modified, where the unused caches include a plurality of sequence values that have not been used to fill the target field. Further, the scheduling node can determine a target cache from the reclaimed unused caches, which includes sequence values closest to the starting sequence value. Further, the scheduling node can update the starting sequence value in the internal table to a starting sequence value among the sequence values included in the target cache, so that each node obtains a plurality of sequence values continuous from the updated starting sequence value from the internal table based on the modified sequence parameter, loads the plurality of sequence values into the memory as a cache, and reads the sequence values in sequence from the cache to fill the target field based on the read sequence values.

[0059] In the above technical solution, when the sequence parameter is modified, the scheduling node first reclaims the unused caches in the memory of each node, and determines a target cache from the reclaimed unused caches, which includes sequence values closest to the starting sequence value. It should be understood that the starting sequence value of the current internal table is necessarily after the ending sequence value included in the target cache, but the sequence values in the target cache have not been used. Based on this, the starting sequence value of the current internal table can be updated to the starting sequence value in the target cache, i.e., the starting sequence value is rolled back to the starting sequence value in the target cache. In this way, each node can obtain a plurality of sequence values continuous from the updated starting sequence value, and so on, so as to minimize the waste of sequence values and improve the database performance on the premise that each node does not obtain duplicate sequence values.

[0060] Referring to FIG. 1, FIG. 1 is a schematic diagram of a system architecture of a distributed cluster according to an example embodiment. One or more embodiments provided in the present specification can be implemented in the system architecture shown in FIG. 1 or a similar system architecture. As shown in FIG. 1, the distributed cluster can include a plurality of nodes, such as node 100a, node 100b, node 100c, and node 100d, etc. Among them, node 100a can be a scheduling node in the distributed cluster, and node 100a, node 100b, node 100c, and node 100d, etc. can establish a communication connection in any possible manner, which is not limited in the present specification.

[0061] As shown in FIG. 1, the distributed cluster maintains a target data table in the distributed database, which can contain a target field and other multiple fields. The target field can be a primary key field in the target data table, and the target field can be automatically generated with unique and continuous sequence values by using SEQUENCE. For example, the target data table can be a commodity transaction order table containing commodity ID, payment time, amount, and other multiple fields, in which the commodity ID field can be the target field. For example, the target data table can also be a merchant information table containing merchant ID, registration time, praise rate, and fan number, in which the merchant ID field can be the target field, and the like, which are not limited in the present specification.

[0062] In an illustrated embodiment, the distributed cluster also maintains an internal table in the distributed database, which is used to maintain multiple sequence values starting from a starting sequence value. The internal table can be stored in a disk. The starting sequence value in the internal table is the starting sequence value of the multiple sequence values maintained by the internal table.

[0063] In an illustrated embodiment, each of the node 100b, the node 100c, and the node 100d can obtain the multiple sequence values starting from the starting sequence value from the internal table based on the sequence parameter (i.e., the SEQUENCE parameter), and load the obtained multiple sequence values as a cache into the memory of each node. Further, each node can read the sequence values in sequence from the cache to fill the target field in the target data table based on the read sequence values. Further, when the sequence values in the cache of any node are all used up, the node can obtain multiple sequence values from the internal table and cache again, and so on.

[0064] It should be understood that, in order to avoid multiple nodes obtaining repeated sequence values from the internal table at the same time, the node 100b, the node 100c, and the node 100d cannot access the internal table at the same time, and the specific obtaining process can refer to the description of the above example one and example two, which will not be described here.

[0065] In an illustrated embodiment, in response to the sequence parameter being modified, the node 100a as the scheduling node can first reclaim the unused buffers contained in the memory of each node. The unused buffers contain a number of sequence values that have not been used to fill the target field, i.e., a number of unused sequence values. It should be noted that in the case where the step size is greater than 1 (e.g., the step size is equal to 2, 3, or 5, etc.), as described in the above-described Example 2, only part of the sequence values contained in the buffer can be read and used to fill the target field. Based on this, the number of unused sequence values contained in the unused buffer referred to in the embodiments of the present specification refers to a number of sequence values after the currently used sequence values, and does not include the sequence values that have not been read due to the step size being greater than 1 and that cannot be used to fill the target field before this.

[0066] Further, the node 100a can determine a target buffer from the reclaimed unused buffers, the target buffer containing a sequence value closest to the start sequence value in the current internal table. It can be understood that in the case of an increasing sequence, the so-called target buffer containing a sequence value closest to the start sequence value in the current internal table, i.e., the target buffer containing the maximum sequence value, is determined from the reclaimed unused buffers; in the case of a decreasing sequence, the so-called target buffer containing a sequence value closest to the start sequence value in the current internal table, i.e., the target buffer containing the minimum sequence value, is determined from the reclaimed unused buffers.

[0067] Further, the node 100a can determine the start sequence value in the number of sequence values contained in the target buffer. It can be understood that in the case of an increasing sequence, the start sequence value in the target buffer, i.e., the minimum sequence value in the target buffer; in the case of a decreasing sequence, the start sequence value in the target buffer, i.e., the maximum sequence value in the target buffer.

[0068] Further, the node 100a can update the start sequence value in the internal table to the start sequence value in the target buffer. Correspondingly, each node can acquire a number of sequence values starting from the updated start sequence value from the internal table based on the modified sequence parameter, load the number of sequence values as a buffer into the memory of the node, and read the sequence values from the buffer in sequence to fill the target field based on the read sequence values, etc.

[0069] For example, still taking the scenario of the increasing sequence as an example, assuming that the node 100b obtains the sequence values 1-20 from the internal table and caches, after the sequence parameter is modified, the unused cache in the memory of the node 100b contains the unused sequence values 18-20; assuming that the node 100c obtains the sequence values 21-40 from the internal table and caches, after the sequence parameter is modified, the unused cache in the memory of the node 100c contains the unused sequence values 26-40; assuming that the node 100d obtains the sequence values 41-60 from the internal table and caches, after the sequence parameter is modified, the unused cache in the memory of the node 100d contains the unused sequence values 44-60; at this time, the starting sequence value in the internal table is 61.

[0070] Further, the node 100a recycles the unused caches in the memories of the node 100b, the node 100c and the node 100d, and determines that the target cache containing the sequence values closest to the current starting sequence value in the plurality of unused caches is the unused cache (containing the sequence values 44-60) in the memory of the node 100d. Further, the node 100a can update the starting sequence value in the current internal table to 44. Subsequently, the node 100b can obtain the plurality of sequence values starting from 44 from the internal table, reducing the waste of sequence values.

[0071] In an illustrated embodiment, the node 100a as the scheduling node can also perform the above-mentioned steps related to the generation of sequence values, including obtaining the plurality of sequence values from the internal table and loading the sequence values as caches into the memory, reading the sequence values from the caches to fill the target fields, etc., which are not specifically limited in the specification.

[0072] In an illustrated embodiment, the above-mentioned node 100a, the node 100b, the node 100c and the node 100d, etc. can be desktop computers, servers or server clusters composed of multiple servers having the above-mentioned functions, etc., which are not specifically limited in the specification.

[0073] It should be understood that the system architecture of the distributed cluster shown in FIG. 1 is only an exemplary illustration, and in some possible embodiments, the distributed cluster can further include more or fewer devices than those shown in FIG. 1, which are not specifically limited in the specification.

[0074] Referring to FIG. 2, FIG. 2 is a flowchart of a sequence value caching processing method according to an example embodiment. The method can be applied to the distributed cluster shown in FIG. 1, and specifically can be applied to the scheduling node in the distributed cluster shown in FIG. 1. As described above, the distributed cluster maintains a target data table and an internal table in the distributed database; the target data table contains a target field, and the internal table is used to maintain a plurality of sequence values that are continuous from a starting sequence value; the memory of each node in the distributed cluster caches a plurality of sequence values that are continuous from the starting sequence value and are obtained by the node from the internal table based on preset sequence parameters; and the plurality of sequence values cached in the memory of each node are used to sequentially fill the target field. As shown in FIG. 2, the method can specifically include the following steps S201-S203.

[0075] In step S201, in response to the sequence parameters being modified, unused caches contained in the memory of each node are reclaimed; wherein the unused caches contain a plurality of sequence values that have not been used to fill the target field.

[0076] In an example embodiment, during the process of performing sequence value generation by each node (including obtaining a plurality of sequence values from the internal table and caching, and sequentially reading sequence values from the cache to fill the target field, etc.), the scheduling node can modify the sequence parameters according to the user's request, such as modifying the step size and the cache size, etc. Further, in response to the sequence parameters being modified, the scheduling node can reclaim the unused caches contained in the memory of each node, which can be specifically referred to the description of the corresponding embodiment of FIG. 1 above, and will not be described here.

[0077] In step S202, a target cache is determined from the reclaimed plurality of unused caches, which contains sequence values closest to the starting sequence value in the internal table.

[0078] Further, the scheduling node can determine a target cache from the reclaimed plurality of unused caches, which contains sequence values closest to the starting sequence value in the internal table, which can be specifically referred to the description of the corresponding embodiment of FIG. 1 above, and will not be described here.

[0079] In step S203, the starting sequence value in the internal table is updated to a starting sequence value in a plurality of sequence values contained in the target cache, so that each node obtains a plurality of sequence values that are continuous from the updated starting sequence value from the internal table based on the modified sequence parameters, loads the plurality of sequence values as a cache into the memory of the node, and sequentially reads sequence values from the cache to fill the target field based on the read sequence values.

[0080] Further, the scheduling node can update the start sequence value in the internal table to the start sequence value in the target cache, so as to realize rollback of the start sequence value in the internal table, which is equivalent to writing the unused cache back to the internal table, reduces waste of the sequence values in the unused cache, and relieves the sequence value hole. For details, refer to the description in the embodiment corresponding to FIG. 1, and details are not described herein again.

[0081] In an illustrated embodiment, referring to FIG. 3, which is a schematic diagram of cache recovery provided in an exemplary embodiment. As shown in FIG. 3, the caches in the memories of node 1, node 2 and node 3 each contain used part of the cache and unused part of the cache. As shown in FIG. 3, the unused cache of node 3 is the target cache containing the sequence value closest to the start sequence value in the current internal table, and the start sequence value of the current internal table should be the next sequence value after the end sequence value of the target cache, and the sequence values in the target cache have not been used, so the start sequence value in the internal table can be updated to the start sequence value in the target cache.

[0082] In an illustrated embodiment, considering that the cache recovery can fail, for example, node 3 is down or network communication fails, the scheduling node can fail to successfully recover the unused cache in node 3 shown in FIG. 3. In this way, the scheduling node recovers the unused caches of node 1 and node 2, and at this time, the scheduling node determines the target cache as the unused cache of node 2, and directly updating the start sequence value of the internal table to the start sequence value in the unused cache of node 2 will cause the part of the cache that has been used in node 3 to be incorrectly written back to the internal table. Subsequent nodes will obtain the repeated sequence values in the used cache of node 3 from the internal table, and further cause the sequence values to be unable to uniquely identify each row of data records, so that subsequent data query, update and deletion and the like are incorrect, and seriously affect the database performance.

[0083] Therefore, before updating the start sequence value in the internal table, the scheduling node needs to verify the target cache currently determined.

[0084] In an illustrated embodiment, the scheduling node can determine whether the sequence values in the internal table and the sequence values contained in the target cache form continuous sequence values. Specifically, the start sequence value in the internal table can be obtained, and it is determined whether the start sequence value is the same as the next sequence value after the end sequence value in the sequence values contained in the target cache. If the sequence values form continuous sequence values, the scheduling node can update the start sequence value in the internal table.

[0085] In an illustrated embodiment, if the start sequence value is continuous with the end sequence value contained in the target cache, it can be determined that the unused caches in the current nodes are all successfully reclaimed, and then the start sequence value in the internal table can be updated to the start sequence value contained in the target cache. For example, in the case that the scheduling node successfully reclaims the unused caches in node 1, node 2 and node 3, the reference sequence of the current internal table is 61, and the target cache determined is the unused cache in node 3, which contains sequence values 44-60, and the maximum sequence value is 60, which is continuous with the start sequence value. Therefore, it can be determined that the unused caches in the current nodes are all successfully reclaimed, and then the start sequence value 61 can be updated to the start sequence value 44 contained in the unused cache in node 3.

[0086] In an illustrated embodiment, if the start sequence value is not continuous with the end sequence value contained in the target cache, it indicates that the unused cache in at least one of the current nodes fails to be reclaimed. Therefore, all the unused caches on the nodes can be directly cleared, and the start sequence value in the current internal table does not need to be updated. For example, in the case that the scheduling node successfully reclaims the unused caches in node 1 and node 2, but fails to reclaim the unused cache in node 3, the reference sequence of the current internal table is 61, and the target cache determined is the unused cache in node 2, which contains sequence values 26-40, and the end sequence value is 40, which is not continuous with the start sequence value. Therefore, it can be determined that the unused caches in the current nodes are not all successfully reclaimed. Since node 3 has used sequence values 41-43, if the start sequence value 61 is directly updated to the start sequence value 26 in the unused cache in node 2 at this time, the sequence values 41-44 used in node 3 will be repeatedly acquired by the subsequent nodes.

[0087] It should be understood that the present application mainly aims to reduce the waste of sequence values as much as possible under the premise of avoiding the repetition of sequence values. However, in the case that the unused cache fails to be reclaimed, in order to ensure that the already used cache is not written back to the internal table by mistake, the sequence values in the unused cache can only be wasted to avoid the repetition of sequence values.

[0088] In addition, in order to further improve the efficiency of sequence value generation, each node in the present application can also perform cache prefetching. For example, in the case that the sequence values in the cache have not been used up, but the unused sequence values are insufficient, the node can access the internal table in advance to obtain and cache multiple sequence values continuous with the current start sequence value from the internal table in advance, instead of waiting until the sequence values in the cache are used up, and then obtaining multiple sequence values continuous with the current start sequence value from the internal table.

[0089] In an illustrated embodiment, after reading the sequence values in the cache in order, each node can further determine whether the number of unused sequence values contained in the cache is less than a preset threshold.

[0090] In an illustrated embodiment, if the number of unused sequence values contained in the cache is less than the preset threshold, the node can obtain, based on the sequence parameter, a plurality of sequence values that are continuous from the current starting sequence value from the internal table in advance, and load the plurality of sequence values obtained in advance as a prefetch cache into the memory of the node. Subsequently, in response to the sequence values in the original cache of the node being used up (i.e., the sequence values are all read and used to fill the target field), the sequence values can be read from the prefetch cache in order to fill the target field; specifically, the prefetch cache can be used as a new cache, and the sequence values can be read from the new cache in order to fill the target field.

[0091] It should be noted that the specific value of the preset threshold is not particularly limited in the present specification, and in an illustrated embodiment, a suitable preset threshold can be set according to actual needs. For example, when the cache size is set to 10 and the step size is set to 2 in the sequence parameter, the preset threshold can be 5; when the cache size is set to 10 and the step size is set to 1 in the sequence parameter, the preset threshold can be 4 or 6, etc., which is not particularly limited in the present specification.

[0092] As described above, in the scenario supporting cache prefetching, there can be two caches, i.e., the cache being used and the prefetch cache, in some nodes of the distributed cluster, and the two caches are discontinuous, i.e., the sequence values contained in the two caches are discontinuous. Based on this, the unused caches recovered from each node by the scheduling node can include: the unused part of the cache that has started to be used, and the complete unused prefetch cache.

[0093] Correspondingly, when determining the target cache from the plurality of recovered unused caches, the scheduling node can include: first screening the plurality of unused partial caches other than the prefetch cache from the plurality of recovered unused caches; and then determining the target cache containing the sequence value closest to the starting sequence value from the plurality of unused partial caches.

[0094] Correspondingly, when checking the target cache determined at present, the scheduling node can include: determining several prefetch caches containing sequence values after the target cache from the recycled unused caches; then, it can be determined whether the sequence values in the internal table, the sequence values contained in the target cache, and the sequence values contained in the several prefetch caches constitute continuous sequence values. Specifically, the scheduling node can obtain the starting sequence value in the internal table and the total number of sequence values contained in the several prefetch caches; further, in the scenario of increasing sequence, the scheduling node can calculate the subtraction result of the starting sequence value and the total number, and determine whether the subtraction result is the same as the next sequence value after the ending sequence value in the target cache; or in the scenario of decreasing sequence, the scheduling node can calculate the addition result of the starting sequence value and the total number, and determine whether the addition result is the same as the next sequence value after the ending sequence value in the target cache. If it can constitute continuous sequence values, the scheduling node can update the starting sequence value in the internal table.

[0095] In an illustrated embodiment, if the subtraction result / addition result is continuous with the ending sequence value in the target cache, it can be determined that the unused caches of the current nodes are all recycled successfully, and then the starting sequence value in the internal table can be updated to the starting sequence value in the target cache.

[0096] In an illustrated embodiment, if the subtraction result / addition result is not continuous with the ending sequence value in the target cache, it indicates that the recycling of the unused cache in at least one of the current nodes fails, so all the unused caches on the nodes can be directly cleared, and the starting sequence value of the current internal table does not need to be updated.

[0097] In an illustrated embodiment, referring to FIG. 4, FIG. 4 is a schematic diagram of another cache recycling provided in an exemplary embodiment. As shown in FIG. 4, the caches in the memories of node 1, node 2 and node 3 each contain used part of the cache, unused part of the cache and complete unused prefetch cache. As shown in FIG. 4, the unused part of the cache in node 3 is the target cache containing sequence values closest to the starting sequence value in the current internal table, the starting sequence value in the current internal table should be the next sequence value after the ending sequence value in the prefetch cache of node 3, and the sequence values in the target cache and the prefetch caches in node 1 and node 3 have not been used, so the starting sequence value in the internal table can be updated to the starting sequence value in the target cache.

[0098] For example, in the case that the scheduling node successfully recovers the unused caches in node 1, node 2 and node 3, the starting sequence value of the current internal table is 121, the unused partial cache of node 3 contains sequence values 64-80, the prefetch cache of node 1 contains sequence values 81-100, and the prefetch cache of node 3 contains sequence values 101-120. The scheduling node calculates the difference between the starting sequence value 121 and the two prefetch caches in node 1 and node 3, and the difference is 81, which is continuous with the ending sequence value 80 in the unused partial cache of node 3, so it can be determined that the unused caches of the current nodes are all successfully recovered, and then the starting sequence value 121 can be updated to the starting sequence value 64 in the unused partial cache of node 3.

[0099] Next, the specific threads and lock logic involved in the execution of the above method by each node in the application will be described.

[0100] 1. Lock logic of the working thread

[0101] First, in order to avoid conflicts among the nodes in the distributed cluster when accessing the internal table, an internal table lock needs to be set, and each node needs to preempt the internal table lock before accessing the internal table. Only the node that successfully preempts the internal table lock can access the internal table to obtain multiple sequence values, while the remaining nodes can only wait. Further, after the node successfully obtains multiple sequence values from the internal table, the internal table lock can be released, so that each node can again preempt the internal table lock.

[0102] Secondly, multiple parallel working threads can be enabled in each node, and each working thread can be used to execute the method steps related to the generation of the above sequence values (including obtaining multiple sequence values from the internal table and caching, and reading sequence values from the cache to fill the target field, etc.), so a cache lock is also needed in each node to coordinate the concurrent access among multiple working threads.

[0103] In an illustrative embodiment, referring to FIG. 5, which is a timing diagram of a working thread obtaining a lock according to an exemplary embodiment. As shown in FIG. 5, before reading sequence values from the cache, multiple working threads in each node need to first preempt the cache lock in the node. In response to any working thread of the multiple working threads successfully preempts the cache lock, the working thread can read sequence values from the cache of the node in order to fill the target field based on the read sequence values. Further, the working thread can determine whether the sequence values in the current cache have been used up.

[0104] In an illustrative embodiment, if the sequence values in the current cache have not been used up, the cache lock can be released.

[0105] In an illustrated embodiment, the internal table lock can be further preempted if the sequence values in the current cache have all been used up. In response to the internal table lock being successfully preempted, the worker thread can obtain, from the internal table, a plurality of sequence values that are continuous starting from the current starting sequence value based on the sequence parameter. Further, the worker thread can update the starting sequence value in the internal table to a next sequence value after an end sequence value in the plurality of sequence values after obtaining the plurality of sequence values from the internal table, and release the internal table lock after the update is completed. Further, the worker thread can load the plurality of sequence values obtained as a cache into the memory of the node, and release the cache lock after the loading is completed.

[0106] Correspondingly, in a scenario where cache pre-fetching is supported, the worker thread can further determine whether the number of unused sequence values contained in the cache is less than a preset threshold after reading the sequence values from the cache, and preempt the internal table lock to obtain a plurality of sequence values that are continuous starting from the current starting sequence value from the internal table in advance and load the plurality of sequence values obtained as a pre-fetch cache into the memory of the node, etc. if the number of unused sequence values is less than the preset threshold.

[0107] 2. Lock logic of a DDL (Data Definition Language) thread

[0108] A DDL (Data Definition Language) thread is run in a scheduling node of a distributed cluster, and can be used to modify a sequence parameter (for example, a SEQUENCE parameter) and perform method steps related to the sequence value cache processing described above (including recycling unused caches of various nodes and updating starting sequence values of internal tables, etc.). Based on this, in order to successfully recycle the unused caches of various nodes, the DDL thread needs to lock the internal table first to prevent worker threads of various nodes from continuing to obtain sequence values from the internal table and cache. Then, the DDL thread also needs to lock the cache of each node to prevent the worker thread from continuing to read sequence values from the cache.

[0109] In an illustrated embodiment, referring to FIG. 6, FIG. 6 is a timing diagram of a DDL thread acquiring locks according to an example embodiment. As shown in FIG. 6, the DDL thread in the scheduling node first preempts the internal table lock, and in response to the internal table lock being successfully preempted, further preempts the cache lock in each of the plurality of nodes. Then, in response to the cache lock in each of the plurality of nodes being successfully preempted, the unused cache contained in the memory of each of the plurality of nodes can be reclaimed, and after the reclaiming is completed, the cache lock in each of the plurality of nodes is released. In an illustrated embodiment, the DDL thread can release the cache lock in each of the plurality of nodes after reclaiming the unused cache in each of the plurality of nodes, which is not limited in the present specification. Further, the DDL thread can determine a target cache from the reclaimed unused caches that contains a sequence value closest to the starting sequence value in the current internal table, and update the starting sequence value in the internal table to the starting sequence value in the target cache, which can be referred to the description in the above embodiments, and will not be repeated here. Further, after the starting sequence value is updated, the DDL thread can release the internal table lock.

[0110] As described above, the working thread acquires the cache lock first and then acquires the internal table lock, while the DDL thread acquires the internal table lock first and then acquires the cache lock, and the order of acquiring the locks is completely opposite. If the DDL thread has successfully preempted the internal table lock and the working thread has successfully preempted the cache lock, the two will wait for each other on the lock of the other, thereby causing a deadlock.

[0111] To solve the above problem of deadlock, the lock logic of the working thread is adjusted in the present application.

[0112] In an illustrated embodiment, referring to FIG. 7, FIG. 7 is another timing diagram of a working thread acquiring locks according to an example embodiment. As shown in FIG. 7, after any working thread successfully preempts the cache lock and reads the sequence values from the cache to fill the target field, if it is determined that the sequence values in the cache have been used up or are insufficient, and thus the internal table lock needs to be further preempted, the working thread can first temporarily release the cache lock and further preempt the internal table lock. In response to the internal table lock being successfully preempted, the working thread can re-preempt the cache lock, and at the same time, can acquire a plurality of sequence values from the internal table. Similarly, after the plurality of sequence values are acquired, the working thread can update the starting sequence value in the internal table to the next sequence value after the end sequence value in the plurality of sequence values, and after the updating is completed, release the internal table lock. Further, the working thread can load the plurality of sequence values acquired as the cache into the memory of the node, and after the loading is completed, release the cache lock re-preempted.

[0113] Thus, if the sequence parameter is modified, the DDL thread can immediately preempt the cache lock in the process that the worker thread temporarily releases the cache lock, solving the problem of deadlock. It should be understood that, as shown in FIG. 6, since the DDL thread has successfully preempted the internal table lock before this time, after the worker thread releases the cache lock, the worker thread cannot further successfully preempt the internal table lock, and loses the cache lock, and can only wait until the DDL thread releases the cache lock and the internal table lock in turn as shown in FIG. 7, and then preempts the cache lock and the internal table lock.

[0114] Further, considering the case that the unused sequence values in the cache of each node are just insufficient, any worker thread in each node will temporarily release the cache lock to further preempt the internal table lock after preempts the cache lock as shown in FIG. 7. During the process that the cache lock is released, the remaining multiple worker threads in each node will try to preempt the cache lock, and another worker thread that successfully preempts the cache lock will still temporarily release the cache lock to further preempt the internal table lock due to the insufficient unused sequence values in the cache, and this process is repeated, resulting in that all the worker threads on each node are competing for the internal table lock, which seriously affects the performance of the internal table, and causes a large number of worker threads to wait for a long time, and further affects the performance of the entire distributed cluster.

[0115] In order to solve the problem that all the worker threads compete for the internal table lock due to the temporary release of the cache lock, the present application further presets a target lock, for example, a fetch lock, before the cache lock based on the lock logic shown in FIG. 7. In an illustrated embodiment, the present application can set a fetch lock on each node, or set a globally unique fetch lock, which is not specifically limited in the present specification.

[0116] In an illustrated embodiment, please refer to FIG. 8, which is a timing diagram of another lock acquisition process of a worker thread provided in an exemplary embodiment. As shown in FIG. 8, the multiple worker threads in each node need to first preempt the fetch lock before preemting the cache lock of the node, and in response to any worker thread of the multiple worker threads successfully preemting the fetch lock, the worker thread can further execute the lock logic as described in FIG. 7, which is not described herein again. Finally, as shown in FIG. 8, when the worker thread releases the cache lock that is successfully preemted again, the worker thread can further release the fetch lock.

[0117] As described above, by introducing the new fetch lock, all the worker threads in the node can first wait on the fetch lock, avoiding the case that all the worker threads on multiple nodes compete for the internal table lock, and ensuring the performance of the internal table and the entire distributed cluster.

[0118] Corresponding to the method flow implementation described above, the embodiments of the present specification also provide a sequence value cache processing apparatus. Please refer to FIG. 9, which is a structural schematic diagram of a sequence value cache processing apparatus provided by an exemplary embodiment. The apparatus 90 can be applied to the distributed cluster shown in FIG. 1, and specifically can be applied to the scheduling node in the distributed cluster shown in FIG. 1. As described above, the target data table in the distributed database and the internal table are maintained in the distributed cluster; the target data table contains target fields, and the internal table is used to maintain a plurality of sequence values that are continuous from a starting sequence value; the memory of each node in the distributed cluster caches a plurality of sequence values that are continuous from the starting sequence value and are obtained by the node from the internal table based on preset sequence parameters; and the plurality of sequence values cached in the memory of each node are used to sequentially fill the target fields. As shown in FIG. 9, the apparatus 90 includes:

[0119] a recycling unit 901 configured to recycle a plurality of unused caches contained in the memory of each node in response to the sequence parameters being modified; wherein the plurality of unused caches contain a plurality of sequence values that have not been used to fill the target fields;

[0120] a determining unit 902 configured to determine a target cache from the plurality of recycled unused caches, the target cache containing sequence values closest to the starting sequence value;

[0121] an updating unit 903 configured to update the starting sequence value in the internal table to a starting sequence value in the plurality of sequence values contained in the target cache, so that each node obtains a plurality of sequence values that are continuous from the updated starting sequence value from the internal table based on the modified sequence parameters, loads the plurality of sequence values as caches into the memory of the node, and sequentially reads sequence values from the caches to fill the target fields based on the read sequence values.

[0122] In an illustrated embodiment, the updating unit 903 is specifically configured to:

[0123] determine whether the plurality of sequence values in the internal table and the plurality of sequence values contained in the target cache constitute continuous sequence values;

[0124] if the plurality of sequence values constitute continuous sequence values, update the starting sequence value to the starting sequence value in the plurality of sequence values contained in the target cache

[0125] In an illustrated embodiment, after each node obtains the plurality of sequence values, the starting sequence value of the internal table is updated to a next sequence value after an end sequence value in the plurality of sequence values.

[0126] In an illustrated embodiment, after each node reads the sequence values in the cache in sequence, it determines whether the number of sequence values contained in the cache that are not used to fill the target field is less than a preset threshold value;

[0127] If so, it acquires a plurality of sequence values from the internal table that are continuous from the start sequence value based on the sequence parameter, and loads the acquired plurality of sequence values as a prefetch cache into the memory of the node, and after all the sequence values in the cache are read, reads the sequence values in the prefetch cache in sequence as a new cache.

[0128] In an illustrated embodiment, the unused caches include unused partial caches contained in the caches that have already started to be used, and complete unused prefetch caches;

[0129] The determination unit 902 is specifically configured to:

[0130] select a plurality of unused partial caches from the plurality of recovered unused caches, except for the prefetch caches;

[0131] determine a target cache from the plurality of unused partial caches that contains sequence values closest to the start sequence value.

[0132] In an illustrated embodiment, the update unit 903 is specifically configured to:

[0133] determine a plurality of prefetch caches from the plurality of recovered unused caches that contain sequence values after the target cache;

[0134] determine whether the plurality of sequence values in the internal table, the sequence values contained in the target cache, and the sequence values contained in the plurality of prefetch caches constitute continuous sequence values;

[0135] If so, update the start sequence value to the start sequence value in the sequence values contained in the target cache.

[0136] In an illustrated embodiment, a plurality of work threads are enabled in parallel on each node, and each work thread is configured to:

[0137] preempt the cache lock of the node, and in response to successful preemption of the cache lock, read the sequence values in the cache of the node in sequence to fill the target field based on the read sequence values; and,

[0138] determine whether the number of sequence values contained in the cache that are not used to fill the target field is less than a preset threshold value, and if so, release the cache lock and further preempt the internal table lock;

[0139] In response to the successful preemption of the internal table lock, based on the sequence parameter, a plurality of consecutive sequence values ​​starting from the initial sequence value are obtained from the internal table; and the cache lock is re-preempted.

[0140] After obtaining the plurality of sequence values, the starting sequence value in the internal table is updated to the next sequence value after the ending sequence value among the plurality of sequence values, and the internal table lock is released after the update is completed;

[0141] The acquired sequence values ​​are loaded into the memory of the node as a prefetch cache, and the cache lock is released after loading is complete.

[0142] In one illustrated implementation, each worker thread on each node preempts a preset target lock before preempting the cache lock on that node, and in response to the successful preemption of the target lock, further preempts the cache lock; and each worker thread further releases the target lock after releasing the cache lock.

[0143] In one illustrated embodiment, the recycling unit 901 is specifically used for:

[0144] Preempt the internal table lock, and in response to the successful preemption of the internal table lock, further preempt the cache locks of each of the multiple nodes;

[0145] In response to successful cache lock preemption on each node, unused cache in the memory of each node is reclaimed, and the cache lock on each node is released after reclamation is complete.

[0146] In one illustrated embodiment, the update unit 903 is specifically used for:

[0147] Update the starting sequence value in the internal table to the starting sequence value among several sequence values ​​contained in the target cache, and release the internal table lock after the update is completed.

[0148] In one illustrated embodiment, the sequence parameters include: a step size for sequentially reading sequence values ​​from the cache, and a specification of the cache; wherein the cache specification includes the number of sequence values ​​that can be sequentially read from all sequence values ​​contained in the cache according to the step size and used to fill the target field;

[0149] Each node obtains multiple consecutive sequence values ​​starting from the initial sequence value from the internal table based on the sequence parameters, including: each node determining a target number of sequence values ​​to obtain from the internal table according to the step size and the cache specification; and obtaining the target number of consecutive sequence values ​​starting from the initial sequence value from the internal table.

[0150] In an illustrated embodiment, the plurality of consecutive sequence values from the starting sequence value includes a plurality of sequence values that are incremented from the starting sequence value, or a plurality of sequence values that are decremented from the starting sequence value.

[0151] The functions and effects of the units of the apparatus 90 are described in detail in the above embodiments, and will not be repeated here. It should be understood that the apparatus 90 can be implemented by software, or by hardware or a combination of software and hardware. For example, as a logical apparatus, the corresponding computer program instructions are read into the memory by the processor (CPU) of the device to form. From the hardware level, in addition to the CPU and the memory, the device where the apparatus is located usually also includes other hardware such as a chip for wireless signal transmission and reception, and / or other hardware such as a board for realizing network communication function.

[0152] The apparatus embodiments described above are only schematic, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical modules, that is, they can be located in one place, or distributed on multiple network modules. Part or all of the units or modules can be selected according to actual needs to achieve the purpose of the scheme of the present specification. Those skilled in the art can understand and implement it without creative labor.

[0153] The apparatus, units and modules described in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver device, a game console, a tablet computer, a wearable device, a vehicle-mounted computer or a combination of any of these devices.

[0154] Corresponding to the method embodiments, the embodiments of the present specification also provide a computing device. Please refer to FIG. 10, which is a structural schematic diagram of a computing device provided in an exemplary embodiment. The computing device shown in FIG. 10 can be a scheduling node in the distributed cluster shown in FIG. 1. As described above, the distributed cluster maintains a target data table and an internal table in a distributed database; the target data table contains a target field, and the internal table is used to maintain a plurality of sequence values that are continuous from a starting sequence value; the memory of each node included in the distributed cluster caches a plurality of sequence values that are continuous from the starting sequence value and are obtained by the node from the internal table based on preset sequence parameters; and the plurality of sequence values cached in the memory of each node are used to sequentially fill the target field. As shown in FIG. 10, the computing device includes a processor 1001 and a memory 1002, and can further include an input device 1004 (such as a keyboard, etc.) and an output device 1005 (such as a display, etc.). The processor 1001, the memory 1002, the input device 1004 and the output device 1005 can be connected through a bus or other means. As shown in FIG. 10, the memory 1002 includes a computer readable storage medium 1003 that stores computer programs capable of being run by the processor 1001. The processor 1001 can be a CPU, a microprocessor, or an integrated circuit for controlling the execution of the above method embodiments. When the processor 1001 runs the stored computer programs, it can execute each step of the sequence value cache processing method in the embodiments of the present specification, including: in response to the sequence parameters being modified, reclaiming unused caches contained in the memory of each node; wherein the unused caches contain a plurality of sequence values that have not been used to fill the target field; determining a target cache from the reclaimed plurality of unused caches that contains sequence values closest to the starting sequence value; updating the starting sequence value in the internal table to a starting sequence value in the plurality of sequence values contained in the target cache, so that each node obtains a plurality of sequence values that are continuous from the updated starting sequence value from the internal table based on the modified sequence parameters, loads the plurality of sequence values as caches into the memory of the node, and sequentially reads sequence values from the caches to fill the target field based on the read sequence values, and the like.

[0155] For detailed description of each step of the sequence value cache processing method, please refer to the previous content, which will not be repeated here.

[0156] Corresponding to the method embodiments, the embodiments of the present specification also provide a computer readable storage medium, which stores computer programs that, when run by a processor, execute each step of the sequence value cache processing method in the embodiments of the present specification. For specific description, please refer to the description of the above embodiments, which will not be repeated here.

[0157] The foregoing merely illustrates the principles of the application. Various modifications and alterations to the methods and devices described herein will become apparent to those skilled in the art from the foregoing description. It will be understood that the above description is intended to be illustrative and not restrictive. Any number of changes or modifications can be made by those skilled in the art to the methods and devices described herein with the principles of the application in mind and without departing from the scope of the application as claimed below.

[0158] In a typical configuration, a terminal device includes one or more CPUs, input / output interfaces, network interfaces, and memory.

[0159] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) having a common memory space employable to store the execution of various applications and / or data files; and non-volatile memory, such as flash memory or the like, which can serve as a persistent memory. The memory is an example of computer-readable media.

[0160] Computer-readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. The computer-readable media includes, but is not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

[0161] It is also important to note that the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0162] It is also important to note that the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0163] Those skilled in the art will appreciate that embodiments of the present description can be devised for a method, a system, or a computer program product. Accordingly, embodiments of the present description can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, and the like) embodying computer-readable program code thereon for use by or in connection with an instruction execution system.

Claims

1. A method of processing a sequence value cache, characterized by, The application is applied to a scheduling node in a distributed cluster, the distributed cluster maintains a target data table and an internal table; the internal table is used for maintaining a plurality of sequence values which are continuous starting from a starting sequence value; the plurality of sequence values are used for filling a target field in the target data table in sequence; The memory of each node in the distributed cluster caches a plurality of sequence values which are continuous starting from the starting sequence value and are obtained by the node from the internal table based on preset sequence parameters; The method comprises: In response to the sequence parameters being modified, recycling unused caches contained in the memory of each node; wherein the unused caches contain a plurality of sequence values which have not been used for filling the target field; Determining a target cache from the recycled plurality of unused caches, the target cache contains sequence values closest to the starting sequence value; Updating the starting sequence value in the internal table to a starting sequence value in a plurality of sequence values contained in the target cache, so that each node obtains a plurality of sequence values which are continuous starting from the updated starting sequence value from the internal table based on the modified sequence parameters, loads the plurality of sequence values as a cache into the memory of the node, and reads sequence values from the cache in sequence to fill the target field based on the read sequence values.

2. The method of claim 1, wherein, The updating of the starting sequence value in the internal table to the starting sequence value in the plurality of sequence values contained in the target cache comprises: Determining whether the plurality of sequence values in the internal table and the plurality of sequence values contained in the target cache constitute continuous sequence values; If the plurality of sequence values constitute continuous sequence values, updating the starting sequence value to the starting sequence value in the plurality of sequence values contained in the target cache.

3. The method of claim 1, wherein, The method further comprises: After each node obtains the plurality of sequence values, updating the starting sequence value of the internal table to a next sequence value after an end sequence value in the plurality of sequence values.

4. The method of claim 3, wherein, The method further comprises: After each node reads sequence values from the cache in sequence, determining whether the number of sequence values contained in the cache which have not been used for filling the target field is less than a preset threshold value; If the number is less than the preset threshold value, obtaining a plurality of sequence values which are continuous starting from the starting sequence value from the internal table based on the sequence parameters, and loading the obtained plurality of sequence values as a prefetch cache into the memory of the node; in response to all sequence values in the cache being read, using the prefetch cache as a new cache, and reading sequence values from the new cache in sequence.

5. The method of claim 4, wherein, The unused caches comprise unused partial caches contained in caches which have already started to be used, and a complete unused prefetch cache; The determining of the target cache from the recycled plurality of unused caches, the target cache contains sequence values closest to the starting sequence value, comprises: Screening a plurality of unused partial caches from the recycled plurality of unused caches, except for the prefetch cache; Determining the target cache from the plurality of unused partial caches, the target cache contains sequence values closest to the starting sequence value.

6. The method of claim 5, wherein, The updating of the starting sequence value in the internal table to the starting sequence value in the plurality of sequence values contained in the target cache comprises: determining, from the plurality of unused caches recovered, a number of prefetch caches containing sequence values subsequent to the target cache; determining whether the plurality of sequence values in the internal table, the sequence values contained in the target cache, and the sequence values contained in the number of prefetch caches constitute continuous sequence values; if the sequence values constitute continuous sequence values, updating the start sequence value to a start sequence value among the sequence values contained in the target cache.

7. The method of claim 4, wherein, each node has a plurality of parallel work threads enabled thereon, each work thread being configured to: preempt a cache lock of the node, in response to successful preemption of the cache lock, sequentially read sequence values from a cache of the node, and fill the target field based on the read sequence values; and, determining whether a number of sequence values contained in the cache and not used to fill the target field is less than a preset threshold, and if so, releasing the cache lock and further preempting an internal table lock; in response to successful preemption of the internal table lock, obtaining, from the internal table, a plurality of sequence values continuous from the start sequence value based on the sequence parameter; and, re-preempting the cache lock; after obtaining the plurality of sequence values, updating the start sequence value in the internal table to a next sequence value subsequent to an end sequence value among the plurality of sequence values, and releasing the internal table lock after the updating is completed; loading the obtained plurality of sequence values as prefetch caches into a memory of the node, and releasing the cache lock after the loading is completed.

8. The method of claim 7, wherein, each work thread on each node preempts a preset target lock before preempts the cache lock of the node, in response to successful preemption of the target lock, further preempts the cache lock; and each work thread further releases the target lock after releasing the cache lock.

9. The method of claim 7, wherein, the recovering of the unused caches contained in the memory of each node includes: preempting the internal table lock, in response to successful preemption of the internal table lock, further preempts a cache lock of each node among the plurality of nodes; in response to successful preemption of the cache lock of each node, recovering the unused caches contained in the memory of each node, and releasing the cache lock of each node after the recovering is completed.

10. The method of claim 9, wherein, the updating of the start sequence value in the internal table to a start sequence value among the sequence values contained in the target cache includes: updating the start sequence value in the internal table to a start sequence value among the sequence values contained in the target cache, and releasing the internal table lock after the updating is completed.

11. The method of claim 1, wherein, the sequence parameter includes a step length for sequentially reading sequence values from the cache, and a specification of the cache; wherein the specification of the cache includes a number of sequence values that can be used to fill the target field and are sequentially read from all sequence values contained in the cache according to the step length; the obtaining, from the internal table, of the plurality of sequence values continuous from the start sequence value based on the sequence parameter by each node includes: Each node determines a target number of sequence values to fetch from the internal table, according to the step size and the size of the cache; and fetches the target number of sequence values from the internal table, which are consecutive to the starting sequence value.

12. The method according to any one of claims 1 to 11, characterized in that, The sequence values consecutive to the starting sequence value include: sequence values that are incremented from the starting sequence value; or sequence values that are decremented from the starting sequence value.

13. A sequence value cache processing apparatus characterized by comprising: The scheduling node is applied to a distributed cluster, the distributed cluster maintains a target data table and an internal table; the internal table is used to maintain a plurality of sequence values that are consecutive to a starting sequence value; the plurality of sequence values are used to sequentially fill a target field in the target data table; The memory of each node in the distributed cluster caches a plurality of sequence values that are consecutive to the starting sequence value, which are fetched by the node from the internal table based on a preset sequence parameter; The apparatus comprises: A recycling unit is configured to recycle a plurality of unused caches in the memory of each node, in response to the sequence parameter being modified; wherein the plurality of unused caches contain a plurality of sequence values that have not been used to fill the target field; A determining unit is configured to determine a target cache from the plurality of recycled unused caches, which contains a plurality of sequence values closest to the starting sequence value; An updating unit is configured to update the starting sequence value in the internal table to a starting sequence value in the plurality of sequence values contained in the target cache, so that each node fetches a plurality of sequence values that are consecutive to the updated starting sequence value from the internal table based on the modified sequence parameter, loads the plurality of sequence values as a cache into the memory of the node, and sequentially reads sequence values from the cache to fill the target field based on the read sequence values.

14. A computing device, comprising: Comprise: a memory and a processor; the memory stores computer programs / instructions executable by the processor; the processor executes the computer programs / instructions to perform the method of any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, a computer program / instruction stored thereon, which is executed by a processor to implement the method of any one of claims 1-12.

16. A computer program product, characterised in that, The computer program product comprises computer programs / instructions, which are executed by a processor to implement the method of any one of claims 1-12.

Images