Sequence value cache processing method and related equipment

Abstract

The invention provides a sequence value cache processing method and related equipment. The distributed cluster maintains a target data table and an internal table; a plurality of continuous sequence values starting from a reference sequence value recorded in an internal table and obtained from the internal table based on a preset sequence parameter of the node are cached in a memory of each node of the distributed cluster; the plurality of sequence values are used for filling a target field in the target data table. The method comprises the steps that if sequence parameters are modified, unused caches contained in memories of all nodes are recycled; determining a target cache of which the sequence value is closest to the reference sequence value from the recovered unused caches, and updating the reference sequence value to be a starting sequence value in a plurality of sequence values contained in the target cache, so that each node is based on the modified sequence parameter. And acquiring a plurality of continuous sequence values starting from the updated reference sequence value from the internal table, and loading the plurality of sequence values as caches into a memory of the node.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of database technology, and in particular, to a sequence value cache processing method and related equipment. Background Art

[0002] In database technology, for a field in a data table (usually the primary key field), whenever a new data record needs to be inserted into the data table, a SEQUENCE can be used to automatically generate a unique sequence value for the primary key field in the data record. The sequence value can be used to uniquely identify each row of data records, ensuring the uniqueness of each row of data records, so as to facilitate subsequent data query, update, and delete operations.

[0003] In a distributed scenario, in order to improve the efficiency of sequence value generation, each node in the distributed cluster can first obtain multiple sequence values ​​that are continuous (including increasing or decreasing) starting from the baseline sequence value recorded in the internal table maintained by the distributed cluster based on the SEQUENCE parameter, and then load the multiple sequence values ​​into the local memory as a cache. Subsequently, each node can directly read the sequence values ​​in sequence from its own cache to fill the primary key field based on the read sequence values.

[0004] It should be noted that after any node obtains multiple sequence values ​​from the internal table, it is necessary to update the baseline sequence value in the internal table to the next sequence value after the last sequence value in the specified direction among the multiple sequence values ​​obtained, so that other nodes can then obtain multiple consecutive sequence values ​​starting from the updated baseline sequence value from the internal table, avoiding multiple nodes from obtaining duplicate sequence values.

[0005] However, once the SEQUENCE parameter is modified, the cache of each node will be inconsistent with the SEQUENCE parameter, resulting in problems such as sequence value holes or sequence value duplication, which seriously affects database performance and even causes correctness problems. Based on this, it is necessary to clear all local caches in each node, and then each node re-obtains multiple consecutive sequence values ​​starting from the current baseline sequence value from the internal table based on the modified SEQUENCE parameter, resulting in a serious waste of a large number of unused sequence values ​​in the cache. Summary of the invention

[0006] In view of this, one or more embodiments of the present specification provide a sequence value cache processing method and related devices.

[0007] In a first aspect, the present specification provides a sequence value cache processing method, which is applied to a scheduling node in a distributed cluster, wherein the distributed cluster maintains a target data table and an internal table; the internal table records a reference sequence value; the memory of each node included in the distributed cluster caches a plurality of sequence values ​​starting from the reference sequence value obtained from the internal table by the node based on a preset sequence parameter; the plurality of sequence values ​​are used to sequentially fill a target field in the target data table;

[0008] The method comprises:

[0009] In response to the sequence parameter being modified, reclaiming unused cache contained in the memory of each node; wherein the unused cache contains a number of sequence values ​​that have not been used to fill the target field;

[0010] Determine a target cache containing a sequence value closest to the reference sequence value from among the recovered multiple unused caches;

[0011] The base sequence value in the internal table is updated to a starting sequence value among several sequence values ​​contained in the target cache, so that each node obtains a plurality of continuous sequence values ​​starting from the updated base sequence value from the internal table based on the modified sequence parameters, loads the plurality of sequence values ​​into the memory of the node as a cache, and reads the sequence values ​​in sequence from the cache to fill the target field based on the read sequence values.

[0012] In a second aspect, this specification provides a sequence value cache processing device,

[0013] Applied to a scheduling node in a distributed cluster, the distributed cluster maintains a target data table and an internal table; the internal table records a reference sequence value; the memory of each node included in the distributed cluster caches a plurality of sequence values ​​starting from the reference sequence value obtained from the internal table based on a preset sequence parameter by the node; the plurality of sequence values ​​are used to sequentially fill in a target field in the target data table;

[0014] The device comprises:

[0015] A recycling unit, configured to recycle unused cache contained in the memory of each node in response to the sequence parameter being modified; wherein the unused cache contains a number of sequence values ​​that have not been used to fill the target field;

[0016] A determination unit, configured to determine, from among the recovered multiple unused caches, a target cache containing a sequence value closest to the reference sequence value;

[0017] An updating unit is used to update the reference sequence value in the internal table to a starting sequence value among the plurality of sequence values ​​contained in the target cache, so that each node obtains a plurality of sequence values ​​starting from the updated reference sequence value from the internal table based on the modified sequence parameter, loads the plurality of sequence values ​​into the memory of the node as a cache, and reads the sequence values ​​from the cache in sequence to fill the target field based on the read sequence values.

[0018] Accordingly, the present specification also provides a computing device, comprising: a memory and a processor; the memory stores a computer program / instruction that can be executed by the processor; when the processor executes the computer program / instruction, it executes the sequence value cache processing method described in the first aspect above.

[0019] Accordingly, the present specification also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the sequence value cache processing method described in the first aspect above is executed.

[0020] Accordingly, the present specification also provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the sequence value cache processing method described in the first aspect above is executed.

[0021] In summary, the present application can be applied to a scheduling node in a distributed cluster, in which a target data table and an internal table are maintained; a benchmark sequence value is recorded in the internal table. The memory of each node contained in the distributed cluster caches multiple sequence values ​​starting from the benchmark sequence value obtained by the node from the internal table based on a preset sequence parameter. Accordingly, each node can read the sequence value from the cache in sequence to fill the target field (such as the primary key field) in the target data table based on the read sequence value. In an illustrated embodiment, the scheduling node can recycle the unused cache contained in the memory of each node in response to the sequence parameter being modified; wherein the unused cache contains several sequence values ​​that have not yet been used to fill the target field. Furthermore, the scheduling node can determine the target cache containing the sequence value closest to the above-mentioned benchmark sequence value from the recovered multiple unused caches. Further, the scheduling node can update the reference sequence value in the internal table to the starting sequence value of the several sequence values ​​contained in the target cache, so that each node obtains multiple sequence values ​​starting from the updated reference sequence value from the internal table based on the modified sequence parameter, and loads the multiple sequence values ​​into the memory of the node as a cache, and reads the sequence values ​​from the cache in order to fill the above-mentioned target field based on the read sequence value. In this way, when the sequence parameter is modified, the scheduling node first recycles the unused cache in the memory of each node, and determines the target cache whose sequence value contained in the recycled multiple unused caches is closest to the reference sequence value. It should be understood that the reference sequence value of the current internal table must be after the end sequence value contained in the target cache, but the sequence values ​​in the target cache have not been used yet. Based on this, the reference sequence value of the current internal table can be updated to the starting sequence value in the target cache, that is, the reference sequence value is rolled back to the starting sequence value in the target cache. In this way, each node can obtain multiple consecutive sequence values ​​starting from the updated baseline sequence value, and so on, thereby minimizing the waste of sequence values ​​and improving database performance while avoiding each node from obtaining duplicate sequence values. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a schematic diagram of a system architecture of a distributed cluster provided by an exemplary embodiment;

[0023] Figure 2 It is a flowchart of a sequence value cache processing method provided by an exemplary embodiment;

[0024] Figure 3 is a schematic diagram of cache recycling provided by an exemplary embodiment;

[0025] Figure 4 is a schematic diagram of another cache recycling provided by an exemplary embodiment;

[0026] Figure 5 It is a timing diagram of a working thread acquiring a lock provided by an exemplary embodiment;

[0027] Figure 6 It is a timing diagram of a DDL thread acquiring a lock provided by an exemplary embodiment;

[0028] Figure 7 is another timing diagram of a worker thread acquiring a lock provided by an exemplary embodiment;

[0029] Figure 8 It is a timing diagram of another working thread acquiring a lock provided by an exemplary embodiment;

[0030] Fig. 9 It is a structural schematic diagram of a sequence value cache processing device provided by an exemplary embodiment;

[0031] Fig.10 It is a structural diagram of a computing device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0032] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0033] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0034] It should be noted that the “plurality” mentioned in this application refers to two or more than two.

[0035] In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0036] First, some terms in this specification are explained to facilitate understanding by those skilled in the art.

[0037] (1) SEQUENCE is a database object used to generate a unique numeric sequence in a database. Its main purpose is to automatically generate a unique sequence value for a field (usually the primary key field) in a data table. Whenever a new data record needs to be inserted into the data table, SEQUENCE can automatically generate a unique sequence value for the primary key field in the data record, so as to use the sequence value to uniquely identify each row of data records, ensure the uniqueness of each row of data records, and facilitate subsequent data query, update, and delete operations.

[0038] Exemplarily, the above data table can be a commodity transaction order table, which can contain multiple fields (or data columns) such as commodity ID, payment time, amount, etc., and each row of data record can be a commodity transaction record. Exemplarily, the commodity ID field in the table can be the primary key field, and SEQUENCE can automatically generate 1, 2, 3, 4, etc. unique sequence values ​​as commodity ID values ​​for the commodity ID field in each row of data record in the table, thereby realizing the use of commodity ID values ​​to uniquely identify each row of data record.

[0039] When the sequence value is generated, the disk of the database may be accessed to obtain the sequence value from an internal table of the database stored on the disk, which is not specifically limited in this specification.

[0040] Furthermore, in order to improve the efficiency of generating sequence values, the SEQUENCE CACHE parameter can be used. In this way, multiple sequence values ​​can be obtained from the internal table first, and then loaded into the local memory as a cache. Subsequently, the sequence values ​​can be directly read from the cache in sequence to fill the primary key field with the read sequence values. When all the sequence values ​​in the cache are used up, multiple sequence values ​​are obtained from the internal table and cached, and so on.

[0041] In addition, in the scenario of a distributed database built based on a distributed cluster, each of the multiple nodes contained in the distributed cluster can obtain multiple sequence values ​​from the internal table of the disk respectively, and load the multiple sequence values ​​into the memory of the node as a cache. Subsequently, each node can directly read the sequence values ​​in sequence from its own cache, and so on.

[0042] It should be noted that when obtaining multiple sequence values ​​from the internal table, it can be based on the preset SEQUENCE parameter to obtain a specified number of multiple sequence values ​​starting from the benchmark sequence value recorded in the internal table. In an illustrated embodiment, the multiple sequence values ​​starting from the benchmark sequence value can specifically be multiple sequence values ​​that increase from the benchmark sequence value, or can also be multiple sequence values ​​that decrease from the benchmark sequence value. This specification does not make specific limitations on this. Exemplarily, the parameter may include: the step length for sequentially reading sequence values ​​from the cache, and the specification of the cache (i.e., the cache size); wherein the cache size may refer to the number of sequence values ​​that can be used to fill the primary key field and are sequentially read from all sequence values ​​contained in the cache according to the above-set step length. Accordingly, each node can determine the target number of sequence values ​​to be obtained from the internal table based on the step length and cache size set in the parameters; then, each node can obtain the target number of multiple sequence values ​​starting from the benchmark sequence value from the internal table.

[0043] In addition, after any node obtains multiple sequence values, it is necessary to update the baseline sequence value of the internal table to the next sequence value after the end sequence value among the multiple sequence values ​​obtained, so that other nodes can then obtain multiple consecutive sequence values ​​starting from the baseline sequence value in the current internal table, avoiding multiple nodes from obtaining duplicate sequence values.

[0044] The following will take an increasing sequence as an example to illustrate the process of obtaining sequence values ​​in a distributed scenario.

[0045] Example 1:

[0046] Assume that the distributed cluster contains nodes 1, 2, and 3, the current base sequence value recorded in the internal table is 1, the cache size set in the SEQUENCE parameter is 10, and the step length for reading sequence values ​​from the cache is 1. Once again, the cache size of 10 means that according to the set step length, the number of sequence values ​​that can be read from the cache to fill the primary key field is 10, not that the total number of sequence values ​​contained in the cache is 10.

[0047] First, based on the current internal table's baseline sequence value 1, node 1 obtains 10 increasing sequence values ​​1-10 from the internal table, loads the 10 sequence values ​​into node 1's memory as cache (or obtains the sequence value interval [1,10] and loads the sequence value interval [1,10] into node 1's memory as cache), and updates the internal table's baseline sequence value to 11. Subsequently, node 1 can read sequence values ​​1, 2, 3, ..., 10 in order from the cache with a step size of 1, and use the read sequence values ​​to fill the primary key field.

[0048] Secondly, based on the current internal table's base sequence value 11, node 2 obtains 10 increasing sequence values ​​11, 12, 13, ..., 20 from the internal table, loads the 10 sequence values ​​into the memory of node 2 as cache, and updates the base sequence value of the internal table to 21. Subsequently, node 2 can read sequence values ​​11, 12, 13, ..., 20 in order from the cache with a step size of 1, and use the read sequence values ​​to fill the primary key field.

[0049] Then, based on the current internal table's base sequence value 21, node 3 obtains 10 increasing sequence values ​​21, 22, 23, ..., 30 from the internal table, loads the 10 sequence values ​​into the memory of node 3 as a cache, and updates the base sequence value of the internal table to 31. Subsequently, node 3 can read sequence values ​​21, 22, 23, ..., 30 in order from the cache with a step size of 1, and use the read sequence values ​​to fill the primary key field.

[0050] Example 2:

[0051] Assume that the distributed cluster contains nodes 1, 2, and 3, the benchmark sequence value recorded in the current internal table is 1, the cache size set in the SEQUENCE parameter is 10, and the step size for reading the sequence value from the cache is 2.

[0052] First, based on the current base sequence value 1, node 1 obtains 20 increasing sequence values ​​1-20, loads the 20 sequence values ​​into the memory of node 1 as a cache, and updates the base sequence value of the internal table to 21. Subsequently, node 1 can read sequence values ​​1, 3, 5, ..., 19 in order from the cache with a step size of 2, and use the read sequence values ​​to fill the primary key field.

[0053] Secondly, based on the current base sequence value 21, node 2 obtains 20 increasing sequence values ​​21, 22, 23, ..., 40, loads the 20 sequence values ​​into the memory of node 2 as cache, and updates the base sequence value of the internal table to 41. Subsequently, node 2 can read sequence values ​​21, 23, 25, ..., 39 in order from the cache with a step size of 2, and use the read sequence values ​​to fill the primary key field.

[0054] Then, based on the current base sequence value 41, node 3 obtains 20 increasing sequence values ​​41, 42, 43, ..., 60, loads the 20 sequence values ​​into the memory of node 3 as a cache, and updates the base sequence value of the internal table to 61. Subsequently, node 3 can read sequence values ​​41, 43, 45, ..., 59 in order from the cache with a step size of 2, and use the read sequence values ​​to fill the primary key field.

[0055] However, once the SEQUENCE parameter is modified, the cache of each node will be inconsistent with the SEQUENCE parameter. As described in the above example 2, when the step size X = 2 and the cache size N = 10, the number of sequence values ​​obtained from the internal table and loaded into the memory cache by each node is N×X = 20, which is an integer multiple of the step size 2. However, once the step size is changed to Y = 3, the 20 sequence values ​​originally in the cache are obviously not an integer multiple of 3. If the sequence values ​​are continued to be read from the remaining sequence values ​​to fill the target field according to the step size 3, various problems may arise, such as the generation of sequence value holes, or multiple nodes generating duplicate sequence values, etc., which will make it impossible to use the sequence value to uniquely identify each row of data records, causing errors in subsequent data query, update and delete operations, seriously affecting database performance.

[0056] In order to avoid the above problems, it is necessary to directly clear the local cache of each node after the SEQUENCE parameter is modified. Taking the above example 2 as an example, assuming that when node 1 uses the sequence value 17 in its cache, node 2 uses the sequence value 25 in its cache, and node 3 uses the sequence value 43 in its cache, the parameter is modified. Then, it is necessary to clear all the unused local caches of node 1 containing sequence values ​​18-20, the unused local caches of node 2 containing sequence values ​​26-40, and the unused local caches of node 3 containing sequence values ​​44-60. Subsequently, node 1 re-obtains the 30 incremental sequence values ​​61-90 (taking the cache size of 10 and the step size of 3 as an example) based on the current internal table's baseline sequence value 61, and so on.

[0057] In this way, although it can avoid that each node generates duplicate sequence values, it will cause serious waste of unused sequence values ​​in the cache, resulting in serious holes in the sequence values ​​and reducing the performance of the entire distributed database system.

[0058] Based on this, this specification provides a technical solution. After the sequence parameters are modified, the unused cache in the memory of each node is first recovered, the target cache whose sequence value is closest to the benchmark sequence value in the current internal table is determined, and the benchmark sequence value is rolled back to the starting sequence value in the target cache, thereby minimizing the waste of sequence values ​​while avoiding each node from obtaining duplicate sequence values.

[0059] In implementation, the present application can be applied to a scheduling node in a distributed cluster, in which a target data table and an internal table are maintained; a benchmark sequence value is recorded in the internal table. The memory of each node contained in the distributed cluster caches multiple sequence values ​​starting from the benchmark sequence value obtained by the node from the internal table based on preset sequence parameters. Accordingly, each node can read the sequence value from the cache in sequence to fill the target field (such as the primary key field) in the target data table based on the read sequence value. In an illustrated embodiment, the scheduling node can recycle the unused cache contained in the memory of each node in response to the sequence parameter being modified; wherein the unused cache contains several sequence values ​​that have not yet been used to fill the target field. Furthermore, the scheduling node can determine the target cache containing the sequence value closest to the above-mentioned benchmark sequence value from the recovered multiple unused caches. Furthermore, the scheduling node can update the baseline sequence value in the internal table to a starting sequence value among several sequence values ​​contained in the target cache, so that each node obtains multiple consecutive sequence values ​​starting from the updated baseline sequence value from the internal table based on the modified sequence parameters, loads the multiple sequence values ​​into the memory of the node as a cache, and reads the sequence values ​​from the cache in sequence to fill the above-mentioned target field based on the read sequence values.

[0060] In the above technical solution, when the sequence parameters are modified, the scheduling node first recycles the unused cache in the memory of each node, and determines the target cache whose sequence value contained in the recovered multiple unused caches is closest to the benchmark sequence value. It should be understood that the benchmark sequence value of the current internal table must be after the end sequence value contained in the target cache, but the sequence values ​​in the target cache have not been used yet. Based on this, the benchmark sequence value of the current internal table can be updated to the starting sequence value in the target cache, that is, the benchmark sequence value is rolled back to the starting sequence value in the target cache. In this way, each node can obtain multiple consecutive sequence values ​​starting from the updated benchmark sequence value, and so on, thereby minimizing the waste of sequence values ​​and improving database performance while avoiding each node from obtaining duplicate sequence values.

[0061] See also Figure 1 , Figure 1 This is a schematic diagram of a distributed cluster system architecture provided by an exemplary embodiment. One or more embodiments provided in this specification may be Figure 1 The system architecture shown in the figure or a similar system architecture is specifically implemented. Figure 1As shown, the distributed cluster may include multiple nodes, such as node 100a, node 100b, node 100c, and node 100d, etc. Among them, node 100a may be a scheduling node in the distributed cluster, and nodes 100a, node 100b, node 100c, and node 100d, etc. may establish communication connections in any possible manner, and this specification does not specifically limit this.

[0062] like Figure 1 As shown, the distributed cluster maintains a target data table in a distributed database, and the target data table may include a target field and multiple other fields, wherein the target field may specifically be a primary key field in the target data table, and SEQUENCE may be used to automatically generate a unique and continuous sequence value for the target field. Exemplarily, the target data table may be a commodity transaction order table, including multiple fields such as commodity ID, payment time, amount, etc., wherein the commodity ID field may be the target field. Exemplarily, the target data table may also be a merchant information table, which may include multiple fields such as merchant ID, registration time, praise rate, number of fans, etc., wherein the merchant ID field may be the target field, and so on, and this specification does not make specific limitations on this.

[0063] In an illustrated embodiment, the distributed cluster further maintains an internal table in the distributed database, and the internal table records the reference sequence value. The internal table can be stored in a disk.

[0064] In an illustrated embodiment, each of the nodes 100b, 100c, and 100d can obtain multiple sequence values ​​starting from a reference sequence value from an internal table based on a sequence parameter (i.e., a SEQUENCE parameter), and load the obtained multiple sequence values ​​into their respective memories as a cache. Further, each node can read sequence values ​​from the cache in order to fill the target field in the target data table based on the read sequence values. Further, when all the sequence values ​​in the cache of any node are used up, the node can obtain multiple sequence values ​​from the internal table and cache them, and so on.

[0065] It should be understood that in order to prevent multiple nodes from simultaneously obtaining duplicate sequence values ​​from the internal table, node 100b, node 100c and node 100d cannot access the internal table simultaneously. The specific acquisition process can refer to the description of Example 1 and Example 2 above, which will not be repeated here.

[0066] In an illustrated embodiment, the node 100a as a scheduling node can first reclaim the unused cache contained in the memory of each node in response to the sequence parameter being modified. Among them, the unused cache contains several sequence values ​​that have not been used to fill the target field, that is, several unused sequence values. It should be noted that, when the step size is greater than 1 (for example, the step size is equal to 2, 3 or 5, etc.), such as the description in the above example 2, only some of the sequence values ​​contained in the cache can be read and used to fill the target field. Based on this, the unused sequence values ​​contained in the unused cache referred to in the embodiments of this specification refer to several sequence values ​​after the currently used sequence value, and do not include sequence values ​​that were not read before because the step size was greater than 1 and could not be used to fill the target field.

[0067] Further, the node 100a can determine the target cache whose sequence value is closest to the benchmark sequence value in the current internal table from the recovered multiple unused caches. It can be understood that in the scenario of increasing sequence, the so-called determining the target cache whose sequence value is closest to the benchmark sequence value in the current internal table from the recovered multiple unused caches is, that is, determining the target cache whose sequence value is the largest from the recovered multiple unused caches; in the scenario of decreasing sequence, the so-called determining the target cache whose sequence value is closest to the benchmark sequence value in the current internal table from the recovered multiple unused caches is, that is, determining the target cache whose sequence value is the smallest from the recovered multiple unused caches.

[0068] Further, the node 100a may determine a starting sequence value among several sequence values ​​contained in the target cache. It is understandable that, in an increasing sequence scenario, the starting sequence value in the target cache is the minimum sequence value in the target cache; in a decreasing sequence scenario, the starting sequence value in the target cache is the maximum sequence value in the target cache.

[0069] Furthermore, the node 100a may update the reference sequence value in the internal table to the starting sequence value in the target cache. Accordingly, each node may obtain multiple sequence values ​​starting from the updated reference sequence value from the internal table based on the modified sequence parameter, load the multiple sequence values ​​into the memory of the node as a cache, and read the sequence values ​​from the cache in order to fill the target field based on the read sequence values, and so on.

[0070] Exemplarily, still taking the scenario of increasing sequence as an example, assuming that node 100b obtains sequence values ​​1-20 from the internal table and caches them, after the sequence parameters are modified, the unused cache in the memory of node 100b contains unused sequence values ​​18-20; assuming that node 100c obtains sequence values ​​21-40 from the internal table and caches them, after the sequence parameters are modified, the unused cache in the memory of node 100c contains unused sequence values ​​26-40; assuming that node 100d obtains sequence values ​​41-60 from the internal table and caches them, after the sequence parameters are modified, the unused cache in the memory of node 100d contains unused sequence values ​​44-60; at this time, the baseline sequence value in the internal table is 61.

[0071] Further, node 100a reclaims the unused caches in the memory of node 100b, node 100c, and node 100d, and determines that the target cache whose sequence value contained in the multiple unused caches is closest to the current reference sequence value is the unused cache in the memory of node 100d (including sequence values ​​44-60). Further, node 100a can update the reference sequence value in the current internal table to 44. Subsequently, node 100b can obtain multiple sequence values ​​that increase from 44 from the internal table, reducing the waste of sequence values.

[0072] In an illustrated embodiment, the node 100a as a scheduling node may also perform the above steps related to sequence value generation, including obtaining multiple sequence values ​​from an internal table and loading the sequence values ​​into a memory as a cache, and reading the sequence values ​​from the cache to fill the target field, etc. This specification does not specifically limit this.

[0073] In one illustrated embodiment, the above-mentioned nodes 100a, 100b, 100c and 100d etc. may be desktop computers, servers or a server cluster composed of multiple servers with the above-mentioned functions, etc., and this specification does not make any specific limitation on this.

[0074] It should be understood that Figure 1 The system architecture of the distributed cluster shown is only an example. In some possible implementations, the distributed cluster may also include Figure 1 There may be more or fewer devices shown, and this specification does not make any specific limitations on this.

[0075] See also Figure 2 , Figure 2 is a flowchart of a sequence value cache processing method provided by an exemplary embodiment. The method can be applied to Figure 1 The distributed cluster shown can be applied to Figure 1The scheduling node in the distributed cluster shown. As described above, the distributed cluster maintains the target data table and internal table in the distributed database; the target data table contains the target field, and the internal table records the benchmark sequence value; the memory of each node contained in the distributed cluster caches multiple sequence values ​​starting from the benchmark sequence value obtained from the internal table based on the preset sequence parameters; the multiple sequence values ​​cached in the memory of each node are used to fill the target field in sequence. Figure 2 As shown, the method may specifically include the following steps S201 to S203.

[0076] Step S201 , in response to a sequence parameter being modified, reclaiming unused cache contained in the memory of each node; wherein the unused cache contains a number of sequence values ​​that have not been used to fill a target field.

[0077] In an illustrated embodiment, during the process of executing sequence value generation by each node (including obtaining multiple sequence values ​​from an internal table and caching them, and sequentially reading sequence values ​​from the cache to fill the target field, etc.), the scheduling node can modify the sequence parameters according to the user's request, such as modifying the step size and cache size, etc. Further, in response to the sequence parameters being modified, the scheduling node can reclaim the unused cache contained in the memory of each node, and the details can be referred to above. Figure 1 The description in the corresponding embodiment will not be repeated here.

[0078] Step S202: determine, from the recovered multiple unused caches, a target cache containing a sequence value closest to the reference sequence value in the internal table.

[0079] Furthermore, the scheduling node can determine the target cache whose sequence value is closest to the reference sequence value in the internal table from the recovered multiple unused caches. For details, please refer to the above Figure 1 The description in the corresponding embodiment will not be repeated here.

[0080] Step S203, updates the baseline sequence value in the internal table to a starting sequence value among several sequence values ​​contained in the target cache, so that each node obtains multiple consecutive sequence values ​​starting from the updated baseline sequence value from the internal table based on the modified sequence parameters, loads the multiple sequence values ​​into the memory of the node as a cache, and reads the sequence values ​​from the cache in sequence to fill the target field based on the read sequence values.

[0081] Furthermore, the scheduling node can update the reference sequence value in the internal table to the starting sequence value in the target cache to implement the rollback of the reference sequence value in the internal table, which is equivalent to writing the unused cache back to the internal table, reducing the waste of sequence values ​​in the unused cache and alleviating sequence value holes. For details, please refer to the above Figure 1 The description in the corresponding embodiment will not be repeated here.

[0082] In one illustrated embodiment, see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of a cache recycling method provided by an exemplary embodiment. Figure 3 As shown in FIG, the caches in the memory of each of Node 1, Node 2, and Node 3 contain some used caches and some unused caches. Figure 3 As shown, the unused cache of node 3 is the target cache containing a sequence value that is closest to the baseline sequence value in the current internal table. The baseline sequence value of the current internal table should be the next sequence value after the end sequence value of the target cache, and several sequence values ​​in the target cache have not been used. Therefore, the baseline sequence value in the internal table can be updated to the starting sequence value in the target cache.

[0083] In an illustrated embodiment, considering that cache reclaim may fail, for example, node 3 may crash or network communication may fail, the scheduling node may not be able to successfully reclaim Figure 3 The unused cache in node 3 shown in the figure. In this way, the multiple unused caches recovered by the scheduling node only include the unused caches in node 1 and node 2. At this time, the target cache determined by the scheduling node is the unused cache in node 2. If the baseline sequence value of the internal table is directly updated to the starting sequence value in the unused cache of node 2, part of the cache that has been used in node 3 will be written back to the internal table by mistake. Subsequent nodes will obtain sequence values ​​from the internal table that are repeated with those in the used cache of node 3, which will make it impossible to use sequence values ​​to uniquely identify each row of data records, causing errors in subsequent data query, update, and deletion operations, seriously affecting database performance.

[0084] In view of this, the scheduling node needs to verify the currently determined target cache before updating the benchmark sequence value in the internal table.

[0085] In an illustrated implementation, the scheduling node may obtain a reference sequence value in an internal table, and determine whether the reference sequence value is continuous with a terminal sequence value among a plurality of sequence values ​​contained in a target cache.

[0086] In an illustrated embodiment, if the reference sequence value is continuous with the end sequence value in the target cache, it can be determined that all the unused caches of the current nodes have been successfully recovered, and the reference sequence value in the internal table can be updated to the starting sequence value in the target cache. For example, when the scheduling node successfully recovers the unused caches in nodes 1, 2, and 3, the reference sequence of the current internal table is 61, and the determined target cache is the unused cache in node 3, which contains sequence values ​​44-60, of which the maximum sequence value is 60, which is continuous with the reference sequence value, then it can be determined that all the unused caches of the current nodes have been successfully recovered, and the reference sequence value 61 can be updated to the starting sequence value 44 contained in the unused cache of node 3.

[0087] In an illustrated embodiment, if the baseline sequence value is not continuous with the end sequence value contained in the target cache, it indicates that the unused cache in at least one node has failed to be recovered. In this way, all unused caches on each node can be directly cleared without updating the baseline sequence value of the current internal table. For example, when the scheduling node successfully recovers the unused cache in nodes 1 and 2, but the unused cache in node 3 fails to be recovered, the baseline sequence of the current internal table is 61, and the determined target cache is the unused cache in node 2, which contains sequence values ​​26-40, of which the end sequence value is 40, which is not continuous with the baseline sequence value. It can be determined that the unused cache of each node has not been successfully recovered. Since node 3 has already used sequence values ​​41-43, if the baseline sequence value 61 is directly updated to the starting sequence value 26 in the unused cache of node 2 at this time, it will cause subsequent nodes to repeatedly obtain the sequence values ​​41-44 that have been used in node 3.

[0088] It should be understood that this application is mainly to reduce the waste of sequence values ​​as much as possible under the premise of avoiding duplication of sequence values. However, in the case of failure of unused cache recovery, in order to ensure that the used cache will not be written back to the internal table by mistake and avoid duplication of sequence values, the sequence values ​​in the unused cache can only be wasted.

[0089] In addition, in order to further improve the efficiency of sequence value generation, each node in the present application can also perform cache pre-fetching. For example, when the sequence values ​​in the cache have not been fully used, but the unused sequence values ​​are insufficient, the node can access the internal table in advance to obtain and cache multiple consecutive sequence values ​​starting from the current baseline sequence value from the internal table in advance.

[0090] In an illustrated implementation, after reading the sequence values ​​from the cache in order, each node may further determine whether the number of unused sequence values ​​contained in the cache is less than a preset threshold.

[0091] In an illustrated embodiment, if the number of unused sequence values ​​contained in the cache is less than a preset threshold, the node can obtain multiple sequence values ​​starting from the current reference sequence value from the internal table in advance based on the sequence parameters, and load the multiple sequence values ​​obtained in advance into the memory of the node as a pre-fetch cache. Subsequently, in response to the sequence values ​​in the original cache of the node being used up (i.e., the sequence values ​​are all read and used to fill the target field), the sequence values ​​can then be read in order from the pre-fetch cache to fill the target field.

[0092] It should be noted that this specification does not specifically limit the specific value of the above-mentioned preset threshold. In an embodiment shown in FIG. 1 , a suitable preset threshold can be set according to actual needs. For example, when the cache size is set to 10 and the step size is 2 in the sequence parameters, the preset threshold can be 5; when the cache size is set to 10 and the step size is 1 in the sequence parameters, the preset threshold can be 4 or 6, and so on. This specification does not specifically limit this.

[0093] As described above, in a scenario where cache prefetching is supported, some nodes in a distributed cluster may have two cache segments, namely, the cache in use and the prefetch cache, and the two cache segments are discontinuous, that is, the sequence values ​​contained in the two segments are discontinuous. Based on this, the unused cache recovered by the scheduling node from each node may include: the unused partial cache contained in the cache that has been started to be used, and the complete unused prefetch cache.

[0094] Correspondingly, when the scheduling node determines the target cache from the recovered multiple unused caches, it can include: first filtering out multiple unused partial caches except the prefetch cache from the recovered multiple unused caches; then, determining the target cache whose sequence value is closest to the benchmark sequence value from the multiple unused partial caches.

[0095] Accordingly, when the scheduling node verifies the currently determined target cache, it may include: determining several pre-fetch caches containing sequence values ​​after the target cache from multiple recovered unused caches; then, obtaining the benchmark sequence value in the internal table, and obtaining the total number of sequence values ​​contained in the several pre-fetch caches; further, in the scenario of an increasing sequence, the scheduling node may calculate the subtraction result of the benchmark sequence value and the total number, and determine whether the subtraction result is continuous with the end sequence value in the target cache; or, in the scenario of a decreasing sequence, the scheduling node may calculate the addition result of the benchmark sequence value and the total number, and determine whether the addition result is continuous with the end sequence value in the target cache.

[0096] In one illustrated embodiment, if the subtraction result / addition result is continuous with the end sequence value in the target cache, it can be determined that all unused caches of the current nodes have been successfully recovered, and the reference sequence value in the internal table can be updated to the starting sequence value in the target cache.

[0097] In one illustrated embodiment, if the subtraction result / addition result is discontinuous with the end sequence value in the target cache, it indicates that the unused cache recovery in at least one current node has failed. In this way, all unused caches on each node can be directly cleared without updating the baseline sequence value of the current internal table.

[0098] In one illustrated embodiment, see Figure 4 , Figure 4 FIG. 1 is a schematic diagram of another cache recycling method provided by an exemplary embodiment. Figure 4 As shown, the caches in the memory of Node 1, Node 2, and Node 3 each contain a used partial cache, an unused partial cache, and a complete unused pre-fetch cache. Figure 4 As shown, the unused partial cache in node 3 is the target cache containing a sequence value closest to the reference sequence value in the current internal table. The reference sequence value of the current internal table should be the next sequence value after the end sequence value in the prefetch cache of node 3, and several sequence values ​​in the target cache and the prefetch caches in nodes 1 and 3 have not been used. Therefore, the reference sequence value in the internal table can be updated to the starting sequence value in the target cache.

[0099] For example, when the scheduling node successfully reclaims the unused cache in nodes 1, 2, and 3, the current internal table's base sequence value is 121, the unused partial cache of node 3 contains sequence values ​​64-80, the prefetch cache of node 1 contains sequence values ​​81-100, and the prefetch cache of node 3 contains sequence values ​​101-120. The scheduling node calculates the base sequence value 121 minus the two prefetch caches in nodes 1 and 3, and the difference is 81, which is continuous with the end sequence value 80 in the unused partial cache of node 3. Therefore, it can be determined that the unused cache of each node is successfully reclaimed, and the base sequence value 121 can be updated to the starting sequence value 64 in the unused partial cache of node 3.

[0100] Next, the specific threads and lock logic involved in each node in the present application when executing the above method flow will be described.

[0101] 1. Lock logic of working thread

[0102] First, in order to avoid conflicts between nodes in the distributed cluster when accessing the internal table, an internal table lock needs to be set. Each node needs to seize the internal table lock before accessing the internal table. Only the node that successfully seizes the lock can access the internal table to obtain multiple sequence values, while the other nodes can only wait. Furthermore, after the node successfully obtains multiple sequence values ​​from the internal table, the internal table lock can be released, so that each node can seize the internal table lock again.

[0103] Secondly, multiple parallel worker threads can be enabled in each node, and each worker thread can be used to execute the method steps related to the above-mentioned sequence value generation (including obtaining multiple sequence values ​​from the internal table and caching them, and reading sequence values ​​from the cache to fill the target field, etc.). Therefore, a cache lock is also required in each node to coordinate concurrent access between multiple worker threads.

[0104] In one illustrated embodiment, see Figure 5 , Figure 5 FIG. 1 is a timing diagram of a working thread acquiring a lock provided by an exemplary embodiment. Figure 5 As shown, before reading sequence values ​​from the cache, multiple worker threads in each node need to seize the cache lock in the node first. In response to any worker thread in the multiple worker threads successfully seizing the cache lock, the worker thread can read the sequence value from the cache of the node in order to fill the target field based on the read sequence value. Further, the worker thread can determine whether all the sequence values ​​in the current cache have been used up.

[0105] In an illustrated implementation, if all sequence values ​​in the current cache have not been used up, the cache lock may be released.

[0106] In an illustrated embodiment, if all sequence values ​​in the current cache have been used up, the internal table lock can be further seized. In response to the successful seizure of the internal table lock, the worker thread can obtain multiple sequence values ​​starting from the current baseline sequence value from the internal table based on the sequence parameters. Further, after obtaining multiple sequence values ​​from the internal table, the worker thread can update the baseline sequence value in the internal table to the next sequence value after the terminal sequence value in the multiple sequence values, and release the internal table lock after the update is completed. Further, the worker thread can load the obtained multiple sequence values ​​into the memory of the node as a cache, and release the cache lock after the loading is completed.

[0107] Correspondingly, in a scenario that supports cache prefetching, after reading the sequence value from the cache, the above-mentioned worker thread can further determine whether the number of unused sequence values ​​contained in the cache is less than a preset threshold. If so, it can further seize the internal table lock to obtain multiple consecutive sequence values ​​starting from the current baseline sequence value from the internal table in advance, and load the obtained multiple sequence values ​​as a prefetch cache into the memory of the node, and so on.

[0108] 2. Lock logic of DDL (Data Definition Language) threads

[0109] A DDL thread runs in the scheduling node of the distributed cluster. The DDL thread can be used to modify sequence parameters (such as SEQUENCE parameters) and execute method steps related to the above-mentioned sequence value cache processing (including recycling unused caches of each node and updating the baseline sequence value of the internal table, etc.). Based on this, in order to smoothly recycle the unused cache of each node, the DDL thread needs to lock the internal table first to prevent the working threads of each node from continuing to obtain sequence values ​​from the internal table and cache. Then, the DDL thread also needs to lock the cache of each node to prevent the working thread from continuing to read the sequence value from the cache.

[0110] In one illustrated embodiment, see Figure 6 , Figure 6 FIG. 1 is a timing diagram of a DDL thread acquiring a lock provided by an exemplary embodiment. Figure 6 As shown, the DDL thread in the scheduling node first seizes the internal table lock, and in response to the successful seizure of the internal table lock, further seizes the cache lock of each node in the multiple nodes. Then, in response to the successful seizure of the cache lock of each node, the unused cache contained in the memory of each node can be recovered, and after the recovery is completed, the cache lock of each node is released. In an illustrated embodiment, the DDL thread can release the cache lock of a node every time it completes the recovery of the unused cache in the node, and this specification does not make specific limitations on this. Further, the DDL thread can determine the target cache whose sequence value is closest to the benchmark sequence value in the current internal table from the recovered multiple unused caches, and update the benchmark sequence value in the internal table to the starting sequence value in the target cache. For details, please refer to the description in the above embodiment, which will not be repeated here. Further, after the benchmark sequence value is updated, the DDL thread can release the internal table lock.

[0111] As mentioned above, the worker thread first obtains the cache lock and then the internal table lock, while the DDL thread first obtains the internal table lock and then the cache lock. The order in which the two acquire locks is completely opposite. If the DDL thread has successfully seized the internal table lock and the worker thread has successfully seized the cache lock, the two will wait for each other's locks, resulting in a deadlock.

[0112] In order to solve the above deadlock problem, the present application adjusts the lock logic of the working thread.

[0113] In one illustrated embodiment, see Figure 7 , Figure 7 FIG. 1 is another timing diagram of a worker thread acquiring a lock provided by an exemplary embodiment. Figure 7 As shown, after any worker thread successfully seizes the cache lock and reads the sequence value from the cache to fill the target field, if it is determined that the sequence values ​​in the cache have been fully used or the sequence values ​​are insufficient, and thus it is necessary to further seize the internal table lock, the worker thread can temporarily release the cache lock and further seize the internal table lock. In response to the successful seizure of the internal table lock, the worker thread can re-seize the cache lock and obtain multiple sequence values ​​from the internal table at the same time. Similarly, after obtaining the multiple sequence values, the worker thread can update the baseline sequence value in the internal table to the next sequence value after the end sequence value in the multiple sequence values, and release the internal table lock after the update is completed. Further, the worker thread can load the multiple sequence values ​​obtained as a cache into the memory of the node, and after the loading is completed, release the re-seized cache lock.

[0114] In this way, if the sequence parameters are modified, the DDL thread can immediately seize the cache lock while the worker thread temporarily releases the cache lock, thus solving the deadlock problem. Figure 6 As shown in the figure, since the DDL thread has successfully seized the internal table lock before, the worker thread cannot further successfully seize the internal table lock after releasing the cache lock, and also loses the cache lock. It can only wait for the DDL thread to Figure 7 As shown in the figure, the cache lock and the internal table lock are released in sequence, and then the cache lock and the internal table lock are seized.

[0115] Furthermore, considering that the unused sequence values ​​in the cache of each node are just insufficient, any worker thread in each node will, after seizing the cache lock, Figure 7As shown in the figure, the cache lock is temporarily released to further seize the internal table lock. While the cache lock is being released, the remaining multiple worker threads in each node will rush to seize the cache lock, and another worker thread that has successfully seized the cache lock will still temporarily release the cache lock to further seize the internal table lock due to insufficient unused sequence values ​​in the cache. This will happen repeatedly, causing all worker threads on each node to compete for the internal table lock, seriously affecting the performance of the internal table, and causing a large number of worker threads to wait for a long time, thereby affecting the performance of the entire distributed cluster.

[0116] In order to solve the above problem that all worker threads compete for internal table locks due to temporary release of cache locks, this application is based on Figure 7 The lock logic shown also presets a target lock, such as a fetch lock, before the cache lock. In an embodiment shown, the present application may set a fetch lock on each node, or may set a globally unique fetch lock, which is not specifically limited in this specification.

[0117] In one illustrated embodiment, see Figure 8 , Figure 8 FIG. 1 is another timing diagram of a worker thread acquiring a lock provided by an exemplary embodiment. Figure 8 As shown, before the multiple worker threads in each node seize the cache lock of the node, they need to seize the fetch lock first. In response to any worker thread in the multiple worker threads successfully seizing the fetch lock, the worker thread can further execute the following steps: Figure 7 The lock logic mentioned above will not be described in detail here. Figure 8 As shown, after the worker thread releases the cache lock that has been successfully re-captured, the fetch lock can be further released.

[0118] As mentioned above, by introducing the new fetch lock, all worker threads in the node can wait on the fetch lock first, avoiding the situation where all worker threads on multiple nodes compete for the internal table lock, thereby ensuring the performance of the internal table and the entire distributed cluster.

[0119] Corresponding to the implementation of the above method flow, the embodiment of this specification also provides a sequence value cache processing device. Fig. 9 , Fig. 9 is a schematic diagram of a sequence value cache processing device provided by an exemplary embodiment. The device 90 can be applied to Figure 1 The distributed cluster shown can be applied to Figure 1The scheduling node in the distributed cluster shown. As described above, the distributed cluster maintains the target data table and internal table in the distributed database; the target data table contains the target field, and the internal table records the benchmark sequence value; the memory of each node contained in the distributed cluster caches multiple sequence values ​​starting from the benchmark sequence value obtained from the internal table based on the preset sequence parameters; the multiple sequence values ​​cached in the memory of each node are used to fill the target field in sequence. Fig. 9 As shown, the device 90 comprises:

[0120] A recycling unit 901 is used to recycle unused cache contained in the memory of each node in response to the sequence parameter being modified; wherein the unused cache contains a number of sequence values ​​that have not been used to fill the target field;

[0121] A determining unit 902 is configured to determine a target cache containing a sequence value closest to the reference sequence value from among the recovered multiple unused caches;

[0122] An updating unit 903 is used to update the baseline sequence value in the internal table to a starting sequence value among several sequence values ​​contained in the target cache, so that each node obtains a plurality of continuous sequence values ​​starting from the updated baseline sequence value from the internal table based on the modified sequence parameters, loads the plurality of sequence values ​​into the memory of the node as a cache, and reads the sequence values ​​in sequence from the cache to fill the target field based on the read sequence values.

[0123] In an illustrated embodiment, the updating unit 903 is specifically configured to:

[0124] Obtaining a reference sequence value in the internal table, and determining whether the reference sequence value is continuous with a terminal sequence value among a plurality of sequence values ​​contained in the target cache;

[0125] If they are continuous, the reference sequence value is updated to the starting sequence value among the sequence values ​​contained in the target cache.

[0126] In an illustrated embodiment, after acquiring the multiple sequence values, each node updates the reference sequence value of the internal table to the next sequence value after the terminal sequence value among the multiple sequence values.

[0127] In an illustrated embodiment, after reading the sequence values ​​from the cache in order, each node determines whether the number of sequence values ​​contained in the cache that are not used to fill the target field is less than a preset threshold;

[0128] If it is less than, then based on the sequence parameters, multiple sequence values ​​starting from the benchmark sequence value are obtained from the internal table in advance, and the multiple sequence values ​​obtained in advance are loaded into the memory of the node as a pre-fetch cache. In response to all the sequence values ​​in the cache being read, the sequence values ​​are read in sequence from the pre-fetch cache.

[0129] In an illustrated embodiment, the unused cache includes an unused partial cache included in a cache that has started to be used, and a complete unused pre-fetch cache;

[0130] The determining unit 902 is specifically configured to:

[0131] Filter out multiple unused partial caches except the pre-fetch cache from the recycled multiple unused caches;

[0132] A target cache whose sequence value is closest to the reference sequence value is determined from the multiple unused partial caches.

[0133] In an illustrated embodiment, the updating unit 903 is specifically configured to:

[0134] Determine, from the multiple unused caches that are recovered, a number of pre-fetch caches that contain sequence values ​​after the target cache;

[0135] Obtaining a reference sequence value in the internal table and a total number of sequence values ​​contained in the plurality of pre-fetch buffers;

[0136] Calculate a subtraction result of the reference sequence value and the total number, and determine whether the subtraction result and a terminal sequence value among the plurality of sequence values ​​contained in the target cache are continuous; or calculate an addition result of the reference sequence value and the total number, and determine whether the addition result and a terminal sequence value among the plurality of sequence values ​​contained in the target cache are continuous;

[0137] If they are continuous, the reference sequence value is updated to a starting sequence value among the sequence values ​​contained in the target cache.

[0138] In an illustrated embodiment, multiple parallel working threads are enabled on each node, and each working thread is used to:

[0139] Seizing the cache lock of the node, and in response to the cache lock being successfully seized, sequentially reading the sequence value from the cache of the node to fill the target field based on the read sequence value; and

[0140] Determine whether the number of sequence values ​​contained in the cache that are not used to fill the target field is less than a preset threshold, and if so, release the cache lock and further seize the internal table lock;

[0141] In response to the internal table lock being seized successfully, obtaining a plurality of sequence values ​​that are continuous from the reference sequence value from the internal table based on the sequence parameter; and re-seizing the cache lock;

[0142] After acquiring the multiple sequence values, updating the reference sequence value in the internal table to a next sequence value after the terminal sequence value in the multiple sequence values, and releasing the internal table lock after the update is completed;

[0143] The multiple sequence values ​​obtained are loaded into the memory of the node as a pre-fetch cache, and after the loading is completed, the cache lock is released.

[0144] In one illustrated embodiment, each worker thread on each node first seizes a preset target lock before seizing the cache lock of the node, and in response to successful seizure of the target lock, further seizes the cache lock; and each worker thread further releases the target lock after releasing the cache lock.

[0145] In an illustrated embodiment, the recovery unit 901 is specifically used for:

[0146] Seizing the internal table lock, and in response to the internal table lock being successfully seized, further seizing the cache lock of each node in the multiple nodes;

[0147] In response to the cache lock of each node being successfully seized, the unused cache contained in the memory of each node is recycled, and after the recycling is completed, the cache lock of each node is released.

[0148] In an illustrated embodiment, the updating unit 903 is specifically configured to:

[0149] The reference sequence value in the internal table is updated to a starting sequence value among the plurality of sequence values ​​contained in the target cache, and after the updating is completed, the internal table lock is released.

[0150] In an illustrated embodiment, the sequence parameters include: a step length for sequentially reading sequence values ​​from the cache, and a specification of the cache; wherein the specification of the cache includes the number of sequence values ​​that can be used to fill the target field and are sequentially read from all sequence values ​​contained in the cache according to the step length;

[0151] Based on the sequence parameters, each node obtains multiple sequence values ​​starting from the benchmark sequence value from the internal table, including: each node determines the target number of sequence values ​​to be obtained from the internal table according to the step size and the specification of the cache; and, obtains the target number of multiple sequence values ​​starting from the benchmark sequence value from the internal table.

[0152] In an illustrated embodiment, the plurality of sequence values ​​continuous from the reference sequence value include: a plurality of sequence values ​​increasing from the reference sequence value; or a plurality of sequence values ​​decreasing from the reference sequence value.

[0153] The implementation process of the functions and effects of each unit in the above-mentioned device 90 is specifically described in the above-mentioned embodiment, and will not be repeated here. It should be understood that the above-mentioned device 90 can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor (CPU) of the device where it is located reading the corresponding computer program instructions into the memory and running them. From the hardware level, in addition to the CPU and the memory, the device where the above-mentioned device is located usually also includes other hardware such as chips for wireless signal transmission and reception, and / or other hardware such as boards and cards for realizing network communication functions.

[0154] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the units or modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. Those of ordinary skill in the art may understand and implement it without creative work.

[0155] The devices, units, and modules described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device, a vehicle-mounted computer, or a combination of any of these devices.

[0156] Corresponding to the above method embodiment, the embodiment of this specification also provides a computing device. Fig.10 , Fig.10 It is a structural diagram of a computing device provided by an exemplary embodiment. Fig.10 The computing device shown may be the above Figure 1The scheduling node in the distributed cluster shown. As described above, the distributed cluster maintains the target data table and internal table in the distributed database; the target data table contains the target field, and the internal table records the benchmark sequence value; the memory of each node contained in the distributed cluster caches multiple sequence values ​​starting from the benchmark sequence value obtained from the internal table based on the preset sequence parameters; the multiple sequence values ​​cached in the memory of each node are used to fill the target field in sequence. Fig.10 As shown, the computing device includes a processor 1001 and a memory 1002, and may further include an input device 1004 (such as a keyboard, etc.) and an output device 1005 (such as a display, etc.). The processor 1001, the memory 1002, the input device 1004, and the output device 1005 may be connected via a bus or other means. Fig.10 As shown, the memory 1002 includes a computer-readable storage medium 1003, which stores a computer program that can be run by the processor 1001. The processor 1001 can be a CPU, a microprocessor, or an integrated circuit for controlling the execution of the above method embodiment. When the processor 1001 runs the stored computer program, it can execute the various steps of the sequence value cache processing method in the embodiment of this specification, including: in response to the modification of the sequence parameter, reclaiming the unused cache contained in the memory of each node; wherein the unused cache contains several sequence values ​​that have not been used to fill the target field; determining the target cache containing the sequence value closest to the reference sequence value from the recovered multiple unused caches; updating the reference sequence value in the internal table to the starting sequence value of the several sequence values ​​contained in the target cache, so that each node obtains multiple sequence values ​​starting from the updated reference sequence value from the internal table based on the modified sequence parameter, loads the multiple sequence values ​​as cache into the memory of the node, and reads the sequence values ​​in sequence from the cache to fill the target field based on the read sequence value, and so on.

[0157] For a detailed description of each step of the above sequence value cache processing method, please refer to the previous content and will not be repeated here.

[0158] Corresponding to the above method embodiment, the embodiment of this specification also provides a computer-readable storage medium, on which a computer program is stored, and when these computer programs are executed by a processor, the various steps of the sequence value cache processing method in the embodiment of this specification are executed. Please refer to the description of the above embodiment for details, which will not be repeated here.

[0159] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

[0160] In a typical configuration, a terminal device includes one or more CPUs, input / output interfaces, network interfaces, and memory.

[0161] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0162] Computer readable media include permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer readable instructions, data structures, program modules or other data.

[0163] Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0164] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0165] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

Claims

1. A sequence value cache processing method, characterized in that: Applied to a scheduling node in a distributed cluster, the distributed cluster maintains a target data table and an internal table; the internal table records a benchmark sequence value; the memory of each node included in the distributed cluster caches a plurality of sequence values ​​starting from the benchmark sequence value obtained from the internal table based on a preset sequence parameter by the node; The multiple sequence values ​​are used to fill the target fields in the target data table in sequence; The method comprises: In response to the sequence parameter being modified, reclaiming unused cache contained in the memory of each node; wherein the unused cache contains a number of sequence values ​​that have not been used to fill the target field; Determine a target cache containing a sequence value closest to the reference sequence value from among the recovered multiple unused caches; The base sequence value in the internal table is updated to a starting sequence value among several sequence values ​​contained in the target cache, so that each node obtains a plurality of continuous sequence values ​​starting from the updated base sequence value from the internal table based on the modified sequence parameters, loads the plurality of sequence values ​​into the memory of the node as a cache, and reads the sequence values ​​in sequence from the cache to fill the target field based on the read sequence values.

2. The method according to claim 1, characterized in that The updating of the reference sequence value in the internal table to a starting sequence value among a plurality of sequence values ​​contained in the target cache comprises: Obtaining a reference sequence value in the internal table, and determining whether the reference sequence value is continuous with a terminal sequence value among a plurality of sequence values ​​contained in the target cache; If they are continuous, the reference sequence value is updated to a starting sequence value among the sequence values ​​contained in the target cache.

3. The method according to claim 1, characterized in that The method further comprises: After acquiring the multiple sequence values, each node updates the reference sequence value of the internal table to the next sequence value after the terminal sequence value among the multiple sequence values.

4. The method according to claim 3, characterized in that The method further comprises: After reading the sequence values ​​from the cache in order, each node determines whether the number of sequence values ​​contained in the cache that are not used to fill the target field is less than a preset threshold; If it is less than, then based on the sequence parameters, multiple sequence values ​​starting from the benchmark sequence value are obtained from the internal table in advance, and the multiple sequence values ​​obtained in advance are loaded into the memory of the node as a pre-fetch cache. In response to all the sequence values ​​in the cache being read, the sequence values ​​are read in sequence from the pre-fetch cache.

5. The method according to claim 4, characterized in that The unused cache includes an unused partial cache included in the cache that has started to be used, and a complete unused pre-fetch cache; The step of determining, from among the recovered multiple unused caches, a target cache containing a sequence value closest to the reference sequence value, comprises: Filter out multiple unused partial caches except the pre-fetch cache from the recycled multiple unused caches; A target cache whose sequence value is closest to the reference sequence value is determined from the multiple unused partial caches.

6. The method according to claim 5, characterized in that The updating of the reference sequence value in the internal table to a starting sequence value among a plurality of sequence values ​​contained in the target cache comprises: Determine, from the multiple unused caches that are recovered, a number of pre-fetch caches that contain sequence values ​​after the target cache; Obtaining a reference sequence value in the internal table and a total number of sequence values ​​contained in the plurality of pre-fetch buffers; Calculate a subtraction result of the reference sequence value and the total number, and determine whether the subtraction result and a terminal sequence value among the plurality of sequence values ​​contained in the target cache are continuous; or calculate an addition result of the reference sequence value and the total number, and determine whether the addition result and a terminal sequence value among the plurality of sequence values ​​contained in the target cache are continuous; If they are continuous, the reference sequence value is updated to a starting sequence value among the sequence values ​​contained in the target cache.

7. The method according to claim 4, characterized in that Multiple parallel worker threads are enabled on each node, and each worker thread is used to: Seizing the cache lock of the node, and in response to the cache lock being successfully seized, sequentially reading the sequence value from the cache of the node to fill the target field based on the read sequence value; as well as, Determine whether the number of sequence values ​​contained in the cache that are not used to fill the target field is less than a preset threshold, and if so, release the cache lock and further seize the internal table lock; In response to the internal table lock being successfully seized, a plurality of sequence values ​​that are continuous starting from the reference sequence value are acquired from the internal table based on the sequence parameter; and, re-seizing the cache lock; After acquiring the multiple sequence values, updating the reference sequence value in the internal table to a next sequence value after the terminal sequence value in the multiple sequence values, and releasing the internal table lock after the update is completed; The multiple sequence values ​​obtained are loaded into the memory of the node as a pre-fetch cache, and after the loading is completed, the cache lock is released.

8. The method according to claim 7, characterized in that Each worker thread on each node first seizes a preset target lock before seizing the cache lock of the node, and further seizes the cache lock in response to successful seizure of the target lock; and each worker thread further releases the target lock after releasing the cache lock.

9. The method according to claim 7, characterized in that: The reclaiming of unused cache contained in the memory of each node includes: Seizing the internal table lock, and in response to the internal table lock being successfully seized, further seizing the cache lock of each node in the multiple nodes; In response to the cache lock of each node being successfully seized, the unused cache contained in the memory of each node is recycled, and after the recycling is completed, the cache lock of each node is released.

10. The method according to claim 9, characterized in that The updating of the reference sequence value in the internal table to a starting sequence value among a plurality of sequence values ​​contained in the target cache comprises: The reference sequence value in the internal table is updated to a starting sequence value among the plurality of sequence values ​​contained in the target cache, and after the update is completed, the internal table lock is released.

11. The method according to claim 1, characterized in that The sequence parameters include: a step length for sequentially reading sequence values ​​from the cache, and a specification of the cache; wherein the specification of the cache includes the number of sequence values ​​that can be used to fill the target field and are sequentially read from all sequence values ​​contained in the cache according to the step length; Each node obtains a plurality of sequence values ​​starting from the reference sequence value from the internal table based on the sequence parameter, including: Each node determines a target number of sequence values ​​to be obtained from the internal table according to the step size and the specification of the cache; and obtains from the internal table a plurality of sequence values ​​of the target number that are continuous starting from the reference sequence value.

12. The method according to any one of claims 1 to 11, characterized in that: The plurality of sequence values ​​that are continuous from the reference sequence value include: a plurality of sequence values ​​that are incremented from the reference sequence value; or a plurality of sequence values ​​that are decremented from the reference sequence value.

13. A sequence value cache processing device, characterized in that: Applied to a scheduling node in a distributed cluster, the distributed cluster maintains a target data table and an internal table; the internal table records a benchmark sequence value; the memory of each node included in the distributed cluster caches a plurality of sequence values ​​starting from the benchmark sequence value obtained from the internal table based on a preset sequence parameter by the node; The multiple sequence values ​​are used to fill the target fields in the target data table in sequence; The device comprises: A recycling unit, configured to recycle unused cache contained in the memory of each node in response to the sequence parameter being modified; wherein the unused cache contains a number of sequence values ​​that have not been used to fill the target field; A determination unit, configured to determine, from among the recovered multiple unused caches, a target cache containing a sequence value closest to the reference sequence value; An updating unit is used to update the reference sequence value in the internal table to a starting sequence value among the plurality of sequence values ​​contained in the target cache, so that each node obtains a plurality of sequence values ​​starting from the updated reference sequence value from the internal table based on the modified sequence parameter, loads the plurality of sequence values ​​into the memory of the node as a cache, and reads the sequence values ​​from the cache in sequence to fill the target field based on the read sequence values.

14. A computing device, characterized in that include: Memory and processor; The memory has stored thereon computer programs / instructions executable by the processor; When the processor runs the computer program / instructions, it performs the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: A computer program / instruction is stored thereon, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

16. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images