Method for implementing page access, server and system
By setting up multiple preset storage spaces in each server of a distributed database cluster, storing page home information, and migrating this information when the cluster adds or deletes the server, the business access freezing problem caused by the reallocation of page home information in the prior art is solved, and the user's business availability and experience improvement during capacity expansion or reduction is achieved.
Patent Information
- Application Number
- PCT/CN2024/101452
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-09-12
- Filing Date
- 2024-06-25
- Publication Date
- 2025-03-20
AI Technical Summary
When existing distributed database clusters expand or shrink, they need to reassign page home information, resulting in business access being frozen and affecting user experience.
By setting multiple preset storage spaces in each server, the page home information is stored, and when the cluster adds or deletes the server, the page home information is migrated and the status of the preset storage space is marked to achieve load balancing.
During the expansion or reduction of distributed database clusters, maintain the availability of user services, avoid business access being frozen, and improve user experience.
Smart Images

Figure CN2024101452_20032025_PF_FP_ABST
Abstract
Description
Method, server and system for implementing page access
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 12, 2023, with application number 202311177481.9 and application name “A method, server and system for implementing page access”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present application relates to the field of distributed database technology, and in particular to a method, server, and system for implementing page access. Background Art
[0003] A distributed database cluster based on a shared disk architecture consists of multiple servers that host client access. Multiple servers access the same shared disk device to achieve data sharing, and the data within the database is typically stored in a page format. Direct server access to the shared disk device often incurs significant signaling overhead. To mitigate this overhead, a page is typically assigned to a server, loaded into that server's page buffer pool, and the page's ownership information is also stored on a specific server. When a server in the database cluster wishes to access the page, it can obtain the page's ownership information through network communication between servers. Based on this information, the server to which the page belongs is then found and the page is retrieved from that server.
[0004] Distributed database scalability refers to the ability to adjust database capacity by expanding or shrinking the distributed database cluster, ensuring database performance while minimizing costs. For example, when business volume increases, capacity can be expanded by adding servers to improve the parallel processing capabilities of the entire database, while when business volume decreases, capacity can be reduced by removing servers to save operating costs. However, when a distributed database cluster is expanded or shrunk, the page ownership information stored on each server in the cluster must be redistributed within the newly expanded or shrunk cluster to ensure load balancing across the servers in the newly expanded or shrunk cluster.
[0005] To address this situation, the current mainstream database practice is to freeze business requests by applying a global lock to the entire database when redistributing page ownership information until the database expansion or reduction is completed. During this period, business access supported by the database will be stagnant. Although this can avoid page access failures caused by page ownership information migration during the expansion or reduction of the database cluster, it also seriously affects the availability of user services and gives users a poor experience.
[0006] Summary of the Invention
[0007] The present application provides a method, server, and system for implementing page access, which can open client business access rights when adding and / or deleting servers in a distributed database cluster, maintain the availability of user services, and help improve user experience.
[0008] In a first aspect, the present application provides a method for implementing page access, the method comprising: in the case of adding a server and / or deleting a server in a distributed database cluster, the first server sends first page attribution information in a first preset storage space to a second server, deletes the first page attribution information, and marks the status of the first preset storage space in the first server as invalid, the first page attribution information is used to represent the correspondence between at least one page and the server storing the at least one page, each server in the distributed database cluster includes the same number of multiple preset storage spaces, and each preset storage space in the multiple preset storage spaces is only valid on one server; the first server receives a first query request from a third server, the first query request is used to query the page attribution information of the first page, and the first query request carries the identification information of the first page; the first server determines, based on the identification information of the first page, that the preset storage space corresponding to the page attribution information of the first page is the first preset storage space, and obtains that the status of the first preset storage space in the first server is invalid; the first server sends a first query response to the third server, and the query response is used to indicate that the query corresponding to the first query request fails.
[0009] It should be understood that each server in the distributed database cluster includes the same number of multiple preset storage spaces, and the multiple preset storage spaces are used to store page attribution information. In the distributed database cluster, each of the multiple preset storage spaces is only in a valid state on one server. It should be understood that if the preset storage space on a server is in a valid state, it means that the preset storage space is available on the server, and the preset storage space should store or has stored the page attribution information of the specified page. "Should be stored" means that when the server receives a page attribution information query request for a certain page, the preset storage space corresponding to the page on the server is in a valid state, but the page attribution information of the target page has not yet been stored, the server should create the page attribution information of the target page and store it in this preset storage space. "Specified page" refers to a page that should store page attribution information in the preset storage space according to the preset storage rules.
[0010] When a distributed database cluster adds and / or deletes servers, the first page attribution information in the first preset storage space sent by the first server is stored in the first preset storage space in the second server by the second server. After the migration of the first page attribution information from the first preset storage space in the first server to the first preset storage space in the second server is completed, the first preset storage space is marked as invalid on the first server and marked as valid on the second server.
[0011] It should be understood that the reason why the first page attribution information needs to be migrated from the first server to the second server is that when the distributed database cluster adds and / or deletes a server, the page attribution information stored on the original server in the cluster needs to be redistributed on the server after the server is added and / or deleted in the cluster. The redistribution in the embodiment of the present application is based on all the page attribution information in a single preset storage space as the minimum redistribution unit. According to the preset load balancing algorithm, the load balancing calculation is performed on the server after the distributed database cluster is added and / or deleted. The calculation result shows that the first preset storage space should be valid on the second server, that is, the first page attribution information should be stored in the first preset storage space of the second server. Therefore, the first page attribution information in the first preset storage space in the first server is migrated to the first preset storage space in the second server.
[0012] It should also be understood that the first page attribution information refers to all page attribution information in the first preset storage space. The first preset storage space can store page attribution information of one page or page attribution information of multiple pages. This application does not limit this.
[0013] In an embodiment of the present application, after the first server receives the first query request from the third server, it determines that its page ownership information should be stored in the first preset storage space based on the identification information of the first page. However, at this time, the first server has already sent the first page ownership information in the first preset storage space to the second server, and the first preset storage space in the first server is in an invalid state, that is, at this time, there is no page ownership information of the first page in the first preset storage space in the first server. Therefore, the first server returns a first query response to the third server to indicate that the query corresponding to the first query request has failed. The method provided in the present application stores page ownership information by setting a preset storage space, and by marking the status of the preset storage space, when a query request needs to access this preset storage, it can directly know whether the information to be queried is stored in the current preset storage space through its status, without the need for complex searches, which is conducive to improving the efficiency of user page access.
[0014] In combination with the first aspect, in certain implementations of the first aspect, before the first server sends the first page attribution information to the second server, the method also includes: the first server receives a first notification message from the cluster management server, the first notification message is used to notify the distributed database cluster to add servers and / or delete servers, the first notification message includes identification information of multiple first target servers, and the multiple first target servers include the second server; the first server determines to migrate the first page attribution information in the first preset storage space in the first server to the second server based on the identification information of the multiple first target servers.
[0015] It should be understood that the server that assumes the cluster management function in the distributed database cluster can be any one of the first server, the second server and the third server, or it can be a server independently set up outside the first server, the second server and the third server. This application does not limit this.
[0016] In the embodiment of the present application, the cluster management function is assumed by an independently set cluster management server, and the cluster management function is separated from the business carrying function, which is beneficial to saving the computing power consumption of the server that carries client business access and is beneficial to maintaining the stability of the distributed database cluster performance.
[0017] In combination with the first aspect, in certain implementations of the first aspect, the first server determines to migrate the first page ownership information in the first preset storage space in the first server to the second server based on the identification information of the multiple first target servers, including: the first server uses a consistent hash algorithm to calculate the identification information of each server in the multiple first target servers respectively to obtain the hash value of each first server, and maps the hash values of the multiple first target servers to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring; the first server obtains the hash value of each preset storage space in the multiple preset storage spaces, and maps the hash values of the multiple preset storage spaces to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring; the first server determines to migrate the first page ownership information in the first preset storage space in the first server to the second server based on the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring and the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring, the multiple preset storage spaces including the first preset storage space.
[0018] It should be understood that the multiple first target servers refer to the servers in the distributed database cluster after adding and / or deleting servers. The identification information of the multiple first target servers is used to uniquely identify each server and can be a device code, Internet Protocol (IP) address, protocol port number, or a unique number of each server in the cluster, which is not limited in this application.
[0019] It should also be understood that the first server uses the consistent hashing algorithm as the preset load balancing algorithm to allocate on which server each preset storage space in the multiple preset storage spaces should be in a valid state. The preset storage space is set to store page ownership information, so the allocation of multiple preset storage spaces is the allocation of page ownership information stored in the multiple preset storage spaces.
[0020] It is worth noting that the first server determines to allocate the first preset storage space to the second server based on the corresponding positions of the hash values of multiple first target servers on the consistent hash ring and the corresponding positions of the hash values of multiple preset storage spaces on the consistent hash ring, which means that after the distributed database cluster adds and / or deletes servers, the first preset storage space should be in a valid state on the second server. Based on the first notification message, the first server determines that the first preset storage space currently valid on the first server should be valid on the second server after the cluster adds and / or deletes servers, so the first page attribution information in the first preset storage space in the first server is migrated to the second server, thereby realizing the reallocation of the server corresponding to the first page attribution information.
[0021] In an embodiment of the present application, a consistent hashing algorithm is used to load-balance multiple preset storage spaces on each server in the cluster, so that the storage of page ownership information on each server is in a balanced state after adding and / or deleting servers in the cluster, which is beneficial to the stable operation of the cluster and to ensuring the performance of the business carried by the cluster.
[0022] In combination with the first aspect, in certain implementations of the first aspect, before the first server sends the first page attribution information in the first preset storage space to the second server, the method also includes: the first server determines the identification information of multiple second target servers, and the multiple second target servers include the first server; the first server uses the identification information of the multiple second target servers to mark the initial status of all or part of the multiple preset storage spaces in the first server as a valid status, and the all or part of the preset storage spaces include the first preset storage space.
[0023] It should be understood that if the situation of adding and / or deleting servers described in the first aspect is the first time that the distributed database cluster is initialized to add and / or delete servers in the cluster, then all or part of the preset storage space includes the first preset storage space
[0024] In this embodiment of the present application, the first server assumes the cluster management function. The first server obtains the identification information of multiple second target servers through interaction between internal functional modules and performs initialization processing. If the server memory allows, the first server assumes the cluster management function, which helps save equipment costs.
[0025] In combination with the first aspect, in certain implementations of the first aspect, before marking the initial status of all or part of the multiple preset storage spaces in the first server as a valid status, the method also includes: the first server uses a consistent hash algorithm to calculate the identification information of each server in the multiple second target servers respectively to obtain the hash value of each second target server, and maps the hash values of the multiple second target servers to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring; the first server obtains the hash value of each preset storage space in the multiple preset storage spaces, and maps the hash values of the multiple preset storage spaces to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring; the first server determines the initial status of the multiple preset storage spaces based on the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring and the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring.
[0026] It should be understood that when the distributed database cluster is initially set up, the status of multiple preset storage spaces on each server is invalid. After each server obtains the identification information of the multiple second target servers initially included in the cluster, it obtains the hash value of each second target server and maps the hash values of the multiple second target servers to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring. Based on the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring and the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring, the multiple preset storage spaces are allocated to the multiple second target servers. After the first server obtains the initial allocation result of the multiple preset storage spaces based on the identification information of the multiple second target servers, it marks the preset storage space allocated to itself as valid.
[0027] In an embodiment of the present application, multiple preset storage spaces are allocated when the distributed database cluster is pulled up. Each server marks the preset storage space allocated to the server as valid based on the obtained allocation result. This allows the page ownership information assigned to a certain server to be stored in the preset storage space that is in a valid state on the server during the use of the distributed database, which is conducive to maintaining the load balancing state of the servers in the cluster.
[0028] In combination with the first aspect, in certain implementations of the first aspect, the first server determines the identification information of multiple second target servers, including: the first server receives a second notification message from the cluster management server, the second notification message is used to notify the distributed database cluster to perform initialization processing, and the second notification message includes the identification information of multiple second target servers; the first server determines the identification information of multiple second target servers based on the second notification message.
[0029] In combination with the first aspect, in certain implementations of the first aspect, the first server determines that the preset storage space corresponding to the page ownership information of the first page is the first preset storage space based on the identification information of the first page, including: the first server uses a consistent hash algorithm to calculate the identification information of the first page to obtain the hash value of the first page, maps the hash value of the first page to a consistent hash ring, and obtains the position of the first page on the consistent hash ring; the first server determines that the first preset storage space is the preset storage space corresponding to the page ownership information of the first page based on the position of the first page on the consistent hash ring and the positions corresponding to the hash values of multiple preset storage spaces on the consistent hash ring.
[0030] It should be understood that for the first page, its page ownership information is stored according to the mapping relationship between the hash value of the first page and the hash value of the first preset storage space when it is created. The page ownership information of the first page stored in the first preset storage space, the ownership server of the first page contained in its page ownership information can be changed according to the nature of the page access request (read request / write request), but it will always be stored in the first preset storage space. The page ownership information of the first page will be stored in the first preset storage space on the server where the first preset storage space is valid.
[0031] In combination with the first aspect, in certain implementations of the first aspect, after the first server sends the first page attribution information in the first preset storage space to the second server, the first server records the identification information of the second server in the first preset storage space in the first server; after the first server obtains that the status of the first preset storage space in the first server is invalid, the method also includes: the first server obtains the identification information of the second server recorded in the first preset storage space in the first server; the first server sends a first query response to the third server, the query response is used to indicate that the query corresponding to the first query request has failed, and the first query response carries the identification information of the second server.
[0032] In an embodiment of the present application, after the first server sends the first page attribution information in the first preset storage space to the second server, the first server records the identification information of the second server in the first preset storage space, so that when the first query request fails, the identification information of the second server can be carried in the request response and returned to the third server, which is beneficial for the third server to re-determine the server that needs to be accessed.
[0033] In combination with the first aspect, in some implementations of the first aspect, the first page attribution information is stored in a first preset storage space in the first server in the form of a bidirectional linked list, or is stored in the first server in the form of a hash table.
[0034] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: the first server receives a second query request from a third server, the second query request is used to query the page attribution information of the second page, and the second query request carries the identification information of the second page; the first server determines, based on the identification information of the second page, that the preset storage space corresponding to the page attribution information of the second page is the second preset storage space; when the first server finds that the status of the second preset storage space in the first server is a valid state, the first server obtains the page attribution information of the second page from the second preset storage space in the first server; the first server sends the page attribution information of the second page to the third server.
[0035] In the second aspect, the present application further provides a method for realizing page access, including: the second server receives the first page attribution information, stores the first page attribution information in the first preset storage space in the second server, and marks the status of the first preset storage space in the second server as a valid state; the second server receives a first query request from a third server, the first query request is used to query the page attribution information of the first page, and the first query request carries the identification information of the first page; the second server determines that the preset storage space corresponding to the page attribution information of the first page is the first preset storage space based on the identification information of the first page, and obtains that the status of the first preset storage space in the second server is a valid state; the second server obtains the page attribution information of the first page from the first preset storage space in the second server; the second server sends the page attribution information of the first page to the third server.
[0036] It should be understood that the first page attribution information from the first preset storage space of the first server is also stored in the corresponding first preset storage space of the second server. Each server has a first preset storage space, the first preset storage space identification information on each server is the same, and the first preset storage space is only valid on one server.
[0037] In combination with the second aspect, in certain implementations of the second aspect, the first page attribution information is stored in the first preset storage space in the second server in the form of a bidirectional linked list, and the second server obtains the page attribution information of the first page from the first preset storage space in the second server, including: the second server searches the bidirectional linked list in a traversal manner based on the identification information of the first page until the target node where the page attribution information of the first page is stored in the bidirectional linked list is found, and obtains the page attribution information of the first page from the target node.
[0038] It should be understood that a doubly linked list is a data storage structure. When the amount of data in the distributed database is less than a preset threshold, the page ownership information can be stored in multiple preset storage spaces in the form of a doubly linked list. When searching, the page ownership information corresponding to the identifier of the target page can be found by traversing the doubly linked list in the preset storage space.
[0039] In the embodiment of the present application, the method of storing page ownership information in the form of a bidirectional linked list has a higher search efficiency when the amount of data in the distributed database is less than a preset threshold.
[0040] In combination with the second aspect, in certain implementations of the second aspect, the first page ownership information is stored in the second server in the form of a hash table, and the second server obtains the page ownership information of the first page from the first preset storage space in the second server, including: the second server uses a consistent hashing algorithm to calculate the identification information of the first page to obtain the hash value of the first page; the second server searches the hash table for the target array corresponding to the hash value of the first page, and obtains the page ownership information of the first page from the target array.
[0041] It should be understood that when the first page's attribution information is stored in the second server in the form of a hash table, a hash search using the hash value of the first page can very quickly locate the target array corresponding to the hash value of the first page, and then obtain the page attribution information of the first page from the target array. This approach is advantageous when the amount of data in the distributed database is greater than or equal to a preset threshold, and can quickly narrow the search scope to the target array, thereby improving search efficiency.
[0042] In combination with the second aspect, in certain implementations of the second aspect, the method further includes: the second server receives a second query request from the third server, the second query request is used to query the page attribution information of the second page, and the second query request carries the identification information of the second page; the second server determines, based on the identification information of the second page, that the preset storage space corresponding to the page attribution information of the second page is the second preset storage space; when the second server finds that the status of the second preset storage space in the second server is invalid, the second server sends a query response to the third server, and the query response is used to indicate that the query failed.
[0043] In a third aspect, the present application also provides a method for implementing page access, including: a third server receives a first access request from a client, the first access request is used to request access to a first page, and the first access request carries identification information of the first page; the third server determines, based on the identification information of the first page, that the page attribution information of the first page is stored on the first server; the third server sends a first query request to the first server, the first query request is used to query the page attribution information of the first page, and the first query request carries the identification information of the first page; the third server receives the first query response from the first server, the first query response is used to indicate that the query corresponding to the first query request has failed; the third server re-determines, based on a preset redirection algorithm, that the page attribution information of the first page is stored on the second server; the third server sends the first query request to the second server; the third server receives the page attribution information of the first page from the second server.
[0044] It should be understood that the timing at which each server in the cluster receives the notification of adding and / or deleting servers is different, so there is a possible scenario, that is, when the first server has completed the migration of the first page ownership information in the first preset storage space, the third server has not yet received the notification of adding and / or deleting servers in the distributed database cluster. If the third server receives a first page access request from the client at this time, then when the third server determines the server corresponding to the page ownership information of the first page requested to be accessed by the first page access request, it can still only determine it based on the correspondence between the identification information of the first page and the original servers in the cluster. It is determined that the page ownership information of the first page is stored in the first server, so the first query request is sent to the first server, but all information in the first preset storage space storing the page ownership information of the first page in the first server has been migrated to the second server, so the query request of the third server to the first server fails.
[0045] It should be understood that the preset redirection algorithm is triggered after the first query request fails, and is used to re-determine the server where the page ownership information of the first page is located. The determination method includes any one of the following: (1) The third server determines that the page ownership information of the first page is migrated to the second server based on the migration record carried by the first query response; (2) After the third server receives the notification of adding and / or deleting servers from the server that assumes the cluster management function, it determines that the page ownership information of the first page is stored on the second server based on the result of the load balancing calculation of the servers after the addition and / or deletion of the servers; (3) The third server predicts that the page ownership information of the first page is stored on the second server based on the preset load balancing algorithm. The above determination methods are all described in detail in the subsequent embodiments according to different situations and will not be expanded here.
[0046] In an embodiment of the present application, after the query request to the first server fails, the third server can still re-determine the server corresponding to the page attribution information of the first page based on a preset redirection algorithm, which is conducive to obtaining the page attribution information of the first page and implementing the page access request from the client.
[0047] In combination with the third aspect, in certain implementations of the third aspect, the first query response carries identification information of the second server; then the third server re-determines, based on a preset redirection algorithm, that the page attribution information of the first page is stored on the second server, including: the third server determines, based on the first query response, that the first page attribution information is stored on the second server.
[0048] In combination with the third aspect, in some implementations of the third aspect, before the third server receives the first access request from the client, the method further includes: the third server receives a second notification message from the cluster management server, the second notification message is used to notify the distributed database cluster to perform initialization processing, the second notification message includes identification information of multiple second target servers, and the multiple second target servers include the third server; the third server uses a consistent hashing algorithm to calculate the identification information of each server in the multiple second target servers respectively, to obtain the hash value of each second target server, and maps the hash values of the multiple second target servers to a consistent hashing ring to obtain the multiple second The hash value of the target server corresponds to the position on the consistent hash ring; the third server obtains the hash value of each preset storage space in the multiple preset storage spaces, and maps the hash values of the multiple preset storage spaces to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring; the third server determines the initial states of the multiple preset storage spaces based on the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring and the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring; the third server marks the initial states of all or part of the multiple preset storage spaces in the third server as valid states.
[0049] In combination with the third aspect, in certain implementations of the third aspect, the third server determines that the page ownership information of the first page is stored on the first server based on the identification information of the first page, including: the third server uses the consistent hash algorithm to calculate the identification information of the first page to obtain the hash value of the first page, maps the hash value of the first page to a consistent hash ring, and obtains the position of the hash value of the first page on the consistent hash ring; the third server determines that the page ownership information of the first page is stored on the first server based on the position of the hash value of the first page on the consistent hash ring and the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring.
[0050] In combination with the third aspect, in certain implementations of the third aspect, the third server re-determines that the page attribution information of the first page is stored on the second server based on a preset redirection algorithm, including: the third server searches on the consistent hash ring in a preset direction based on the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring, taking the position of the hash value of the first server on the consistent hash ring as the starting point to determine the second server, and the corresponding position of the hash value of the second server on the consistent hash ring is after the first server and adjacent to the first server.
[0051] Optionally, the preset direction may be clockwise or counterclockwise, which is not limited in this application.
[0052] In combination with the third aspect, in certain implementations of the third aspect, after the third server receives the first query response from the first server, the method further includes: the third server receiving a first notification message from a cluster management server, the first notification message being used to notify the distributed database cluster to add a server and / or delete a server, the first notification message including identification information of multiple first target servers, the multiple first target servers including the second server; the third server re-determining, based on a preset redirection algorithm, that the first page attribution information is stored on the second server, It includes: the third server uses a consistent hash algorithm to calculate the identification information of the first page to obtain the hash value of the first page, maps the hash value of the first page to a consistent hash ring, and obtains the position of the hash value of the first page on the consistent hash ring; the third server uses a consistent hash algorithm to calculate the identification information of each server in the multiple first target servers respectively to obtain the hash value of each first target server, and maps the hash values of the multiple first target servers to a consistent hash ring to obtain the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring; the third server determines that the page ownership information of the first page is stored in the second server based on the position of the hash value of the first page on the consistent hash ring and the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring.
[0053] In combination with the third aspect, in some implementations of the third aspect, the method also includes: the third server receives a second access request from the client, the second access request is used to request access to the second page, and the second access request carries the identification information of the second page; the third server determines that the page ownership information of the second page is stored on the second server based on the identification information of the second page; the third server sends a second query request to the second server, the second query request is used to query the page ownership information of the second page, and the second query request carries the identification information of the second page; the third server receives the second query response from the second server, and the second query response is used to indicate that the query corresponding to the second query request has failed; the third server re-determines that the page ownership information of the second page is stored on the first server according to a preset redirection algorithm; the third server sends the second query request to the first server; the third server receives the page ownership information of the second page from the first server.
[0054] In combination with the third aspect, in certain implementations of the third aspect, the third server re-determines that the page ownership information of the second page is stored on the first server based on a preset redirection algorithm, including: the third server uses the consistent hash algorithm to calculate the identification information of the second page to obtain the hash value of the second page, maps the hash value of the second page to the consistent hash ring, and obtains the position of the hash value of the second page on the consistent hash ring; the third server determines that the page ownership information of the second page is stored on the first server based on the position of the hash value of the second page on the consistent hash ring and the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring.
[0055] In a fourth aspect, the present application provides a system for implementing page access, the system comprising a first server for implementing the method for implementing page access described in any possible implementation of the first aspect above, a second server for implementing the method for implementing page access described in any possible implementation of the second aspect above, and a third server for implementing the method for implementing page access described in any possible implementation of the third aspect above.
[0056] In a fifth aspect, the present application provides a distributed database cluster, which includes a first server for implementing the method for page access described in any possible implementation method of the first aspect above, a second server for implementing the method for page access described in any possible implementation method of the second aspect above, and a third server for implementing the method for page access described in any possible implementation method of the third aspect above.
[0057] In a sixth aspect, the present application provides a server comprising a module for executing the method for implementing page access described in any possible implementation of the first aspect, the second aspect, or the third aspect.
[0058] In a seventh aspect, the present application provides a server comprising a processor, wherein the processor is configured to execute the method for implementing page access as described in any possible implementation of the first aspect, the second aspect, or the third aspect.
[0059] Optionally, the server may further include a memory for storing instructions and data. The memory is coupled to the processor, and when the processor executes the instructions stored in the memory, the methods described in the above aspects may be implemented.
[0060] Optionally, the server may further include a transceiver, and the transceiver is used for the server to communicate with hardware or software modules of other servers in the distributed database cluster.
[0061] In an eighth aspect, the present application provides a computer-readable storage medium comprising a computer program, which, when executed on a computer, enables the computer to implement the method in any possible implementation of the first aspect, the second aspect, or the third aspect.
[0062] In the ninth aspect, the present application provides a computer program product, which includes: a computer program (also referred to as code, or instructions), which, when run, enables a computer to execute a method in any possible implementation of the first aspect, the second aspect, or the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] FIG1 is an architecture diagram of a distributed database cluster provided in an embodiment of the present application;
[0064] FIG2 is a schematic flow chart of a method for implementing page access provided in an embodiment of the present application;
[0065] FIG3 is a schematic diagram of a method for allocating multiple preset storage spaces provided in an embodiment of the present application;
[0066] FIG4 is a schematic diagram of the corresponding relationship between pages and servers provided in an embodiment of the present application;
[0067] FIG5 is a schematic diagram of the correspondence between pages and preset storage spaces provided in an embodiment of the present application;
[0068] FIG6 is a schematic flow chart of another method for implementing page access provided in an embodiment of the present application;
[0069] FIG7 is a schematic flow chart of another method for implementing page access provided in an embodiment of the present application;
[0070] FIG8 is an example diagram of a page ownership information migration process from a cluster perspective when adding a server according to an embodiment of the present application;
[0071] FIG9 is an example diagram of a page ownership information migration process from a cluster perspective when a server is deleted according to an embodiment of the present application;
[0072] FIG10 is a schematic block diagram of a server provided in an embodiment of the present application;
[0073] FIG11 is a schematic block diagram of another server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0074] The technical solution in this application will be described below with reference to the accompanying drawings.
[0075] To facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or execution order, and the words "first" and "second" do not necessarily mean different.
[0076] It should be noted that, in this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.
[0077] In addition, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b and c can mean: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c, where a, b, c can be single or multiple.
[0078] A distributed database cluster is a logically unified computer system consisting of multiple physically separate servers containing the same database instance, connected via a network. A shared disk architecture is one type of distributed database architecture. A distributed database cluster in this architecture consists of multiple servers that host client access and a shared disk device. Multiple servers can access the same shared disk device, providing database services to clients through collaborative work between the servers. Distributed databases using shared disk architectures are often used in large-scale enterprise applications and data warehouses due to their high availability, performance, and scalability.
[0079] FIG1 is a shared disk distributed database cluster system architecture 100 applicable to an embodiment of the present application. As shown in FIG1 , the distributed database cluster includes a first server 101, a second server 102, a third server 103, and a shared disk device 104. The client 105 is a program corresponding to the business carried by the distributed database and providing an operating interface for the user, or an electronic device installed with the program. The number of servers and clients shown in FIG1 is only exemplary. In a specific implementation, the number of servers and clients can be infinite, and this application does not limit this.
[0080] Among them, the first server 101, the second server 102 and the third server 103 are servers that carry client access. The first server 101, the second server 102 and the third server 103 include the same database instance. Any of the first server 101, the second server 102 and the third server 103 can access the shared disk device 104. The data in the shared disk device 104 is stored in the format of pages. The first server 101, the second server 102 and the third server 103 are all provided with a page buffer pool for caching pages. Pages designated to belong to a certain server are usually loaded into the page buffer pool of the server. In addition, the page directory (PD) of the specified page is also stored in the page buffer pool of each server. The page directory information is the corresponding relationship information between the page and the server to which it belongs.
[0081] For example, when the second server 102 receives a page access request from the client 105 (taking this page access request as a read request), if the page ownership information corresponding to the page requested by the page access request is stored in the first server 101, the second server must obtain the page ownership information from the first server 101 before it can retrieve the page from the page buffer pool of the server to which the page belongs (e.g., the third server 103) indicated by the page ownership information. Therefore, whether the page can be successfully accessed is closely related to whether the page ownership information can be obtained.
[0082] The scalability of a distributed database under a shared disk architecture is an important indicator for measuring the performance of the database. The so-called "scalability" means that the distributed database can flexibly adjust the number of servers that carry client access according to the business volume it carries. That is, when the business volume carried by the distributed database increases and the capacity of the servers in the cluster cannot support business requirements, new servers can be added to expand the database capacity, ensure the stability of business access, and maintain the performance of each server; conversely, if the business volume decreases and the capacity of the servers in the distributed database cluster exceeds the business requirements, the existing servers in the cluster can be deleted to save operating costs.
[0083] In the process of fulfilling user page access requests, it is necessary to obtain the page ownership information of the accessed page. The server storing this page ownership information must bear this access load. Therefore, when adding and / or removing servers from a distributed database cluster (i.e., when expanding or shrinking the distributed database), in order to achieve load balancing of the access traffic carried by the various servers in the cluster after the addition and / or removal of the server, the page ownership information stored by the original servers in the cluster must be redistributed. Oracle's Real Application Clusters (RAC) is a commonly used shared disk architecture database. During the redistribution of page ownership information (in Oracle RAC, page ownership information is called the global resource directory (GRD)), Oracle RAC freezes database service access by applying a global lock. The global lock is not released until the GRD redistribution is complete. This approach causes user service access to be stalled during this period, preventing users from accessing the database and affecting user service availability.
[0084] In view of this, the present application proposes a method for realizing page access, in which multiple preset storage spaces for storing page ownership information are set in each server of the distributed database cluster, and the status of the preset storage space is marked to distinguish whether the storage space is valid on the server. When the distributed database cluster adds and / or deletes servers, the server marks the preset storage space where the page ownership information has been migrated to other servers as invalid. Correspondingly, the preset storage space on the server that receives the page ownership information is marked as valid. When querying the page ownership information, the server can determine whether the page ownership information of the target page is stored on the server based on the marking status of the preset storage space. If not, the server where the page ownership information of the target page is located can also be found based on the preset redirection algorithm. The method provided by the present application enables users to access the database normally during the expansion or reduction of the distributed database cluster, thereby ensuring the availability of the user's business.
[0085] Fig. 2 is a schematic flow chart of a method 200 for implementing page access according to an embodiment of the present application. The method 200 is applied to a distributed database cluster under a shared disk architecture as shown in Fig. 1 .
[0086] The method 200 includes the following steps:
[0087] S201. A first server sends first page attribution information in a first preset storage space to a second server; correspondingly, the second server receives the first page attribution information.
[0088] S202: The first server deletes the first page ownership information and marks the state of the first preset storage space in the first server as invalid.
[0089] S203: The second server stores the first page attribution information in a first preset storage space in the second server, and marks the status of the first preset storage space in the second server as valid.
[0090] It is worth noting that each server in the distributed database cluster provided by the embodiment of the present application includes the same number of multiple preset storage spaces, and the multiple preset storage spaces are used to store page attribution information. In the distributed database cluster, each of the multiple preset storage spaces is only in a valid state on one server. It should be understood that if the preset storage space on a server is in a valid state, it means that the preset storage space is available on the server, and the preset storage space should store or has stored the page attribution information of the specified page. "Should be stored" means that when the server receives a page attribution information query request for a certain page, the preset storage space corresponding to the page on the server is in a valid state, but the page attribution information of the target page has not yet been stored, the server should create the page attribution information of the target page and store it in this preset storage space. "Specified page" refers to a page that should store page attribution information in the preset storage space according to the preset storage rules.
[0091] When a distributed database cluster adds and / or deletes servers, the first page attribution information in the first preset storage space sent by the first server is stored in the first preset storage space in the second server by the second server. After the migration of the first page attribution information from the first preset storage space in the first server to the first preset storage space in the second server is completed, the first preset storage space is marked as invalid on the first server and marked as valid on the second server.
[0092] It should be understood that the reason why the first page attribution information needs to be migrated from the first server to the second server is that when the distributed database cluster adds and / or deletes a server, the page attribution information stored on the original server in the cluster needs to be redistributed on the server after the server is added and / or deleted in the cluster. The redistribution in the embodiment of the present application is based on all the page attribution information in a single preset storage space as the minimum redistribution unit. According to the preset load balancing algorithm, the load balancing calculation is performed on the server after the distributed database cluster is added and / or deleted. The calculation result shows that the first preset storage space should be valid on the second server, that is, the first page attribution information should be stored in the first preset storage space of the second server. Therefore, the first page attribution information in the first preset storage space in the first server is migrated to the first preset storage space in the second server.
[0093] It should also be understood that the first page attribution information refers to all page attribution information in the first preset storage space. The first preset storage space can store page attribution information of at least one page. The at least one page in the embodiment of the present application includes the first page.
[0094] S204: The third server receives a first page access request from the client, where the first page access request is used to request access to a first page and carries identification information of the first page.
[0095] S205: The third server determines, based on the identification information of the first page, that the page attribution information of the first page is stored in the first server.
[0096] S206. The third server sends a first query request to the first server. The first query request is used to query page attribution information of the first page and carries identification information of the first page. Correspondingly, the first server receives the first query request.
[0097] It should be understood that whether the distributed database cluster adds and / or deletes servers is notified by the server in the cluster that assumes the cluster management function. The timing of receiving the notification by each server in the cluster is different. Therefore, there is a possible scenario, that is, when the first server has completed the migration of the first page ownership information in the first preset storage space, the third server has not yet received the notification of the distributed database cluster adding and / or deleting servers. If the third server receives the first page access request from the client at this time, when the third server determines the server corresponding to the page ownership information of the first page requested to be accessed by the first page access request, it can still only determine it based on the correspondence between the identification information of the first page and the original server in the cluster, and determines that the page ownership information of the first page is stored on the first server, so it sends a first query request to the first server.
[0098] S207: The first server determines, based on the identification information of the first page, that the preset storage space corresponding to the page attribution information of the first page is the first preset storage space, and obtains that the status of the first preset storage space of the first server is invalid.
[0099] S208. The first server sends a first query response to the third server, where the first query response is used to indicate that the query corresponding to the first query request has failed. Correspondingly, the third server receives the first query response.
[0100] It should be understood that the identification information of the first page is used to uniquely identify the page to be accessed by the first access request, and the identification information has a unique corresponding relationship with the first preset storage space among the multiple preset storage spaces.
[0101] After the first server receives the first query request from the third server, it determines that the page ownership information of the first page should be stored in the first preset storage space based on the identification information of the first page. However, at this time, the first server has executed the above S201 to S202, and the first preset storage space in the first server is in an invalid state. That is, at this time, the first preset storage space in the first server no longer has the page ownership information of the first page. Therefore, the first server returns a first query response to the third server, indicating that the query corresponding to the first query request has failed.
[0102] S209: The third server re-determines the page attribution information of the first page according to a preset redirection algorithm and stores it in the second server.
[0103] It should be understood that the preset redirection algorithm is triggered after the failure of the first query request, and is used to re-determine the server where the page ownership information of the first page is located. The determination method includes any one or more of the following: (1) The third server determines that the page ownership information of the first page is migrated to the second server based on the migration record carried by the first query response; (2) After the third server receives the notification of adding and / or deleting servers issued by the server that assumes the cluster management function, it determines that the page ownership information of the first page is stored on the second server based on the result of the load balancing calculation of the servers after the addition and / or deletion of the servers; (3) The third server predicts that the page ownership information of the first page is stored on the second server based on the preset load balancing algorithm. The above determination methods are all described in detail in the subsequent embodiments according to different situations and will not be expanded here.
[0104] S210. The third server sends a first query request to the second server; correspondingly, the second server receives the first query request.
[0105] S211. The second server determines that the preset storage space corresponding to the attribution information of the first page is the first preset storage space based on the identification information of the first page carried in the first query request, and obtains that the status of the first preset storage space in the second service area is valid.
[0106] S212: The second server obtains page attribution information of the first page from a first preset storage space of the second server.
[0107] S213. The second server sends the page attribution information of the first page to the third server; correspondingly, the third server receives the page attribution information of the first page.
[0108] It should be understood that after the third server successfully re-determines that the page ownership information of the first page is stored on the second server based on the preset redirection algorithm, the page ownership information of the first page is obtained from the second server. If the first access request is a read request for the first page, the third server obtains the first page from the server to which the first page belongs, as indicated by the page ownership information of the first page; if the first access request is a write request for the first page, the third server requests the second server to change the ownership server of the first page to the third server, and after the modification is successful, the third server obtains the first page from the shared disk device, completes the write operation on the first page, and stores the updated first page in the page buffer pool of the third server.
[0109] The method provided by the embodiment of the present application sets multiple preset storage spaces for storing page attribution information in each server of the distributed database cluster, and distinguishes whether the storage space is valid on the server by marking the status of the preset storage space. When the distributed database cluster adds and / or deletes a server, after the first server migrates the first page attribution information in the first preset storage space to the first preset storage space of the second server, the first server marks the first preset storage space in the first server as invalid. Correspondingly, after receiving the first page attribution information, the second server marks the first preset storage space in the second server as valid. When the first server receives the first query request from the third server, it can determine that the page attribution information of the first page is not stored in the server based on the marking status of the first preset storage space, and return a query failure response corresponding to the first query request, so that the third server re-determines the server where the page attribution information of the first page is located based on the preset redirection algorithm. The method provided in the embodiment of the present application, when a distributed database cluster adds and / or deletes servers, completes the redistribution of page ownership information with the page ownership information stored in a preset storage space as the minimum migration granularity, while maintaining the openness of user access rights during this period, thereby ensuring the availability of user services during the period when the distributed database cluster adds and / or deletes servers.
[0110] It should be understood that the server that assumes the cluster management function in the distributed database cluster can be any one of the first server, the second server and the third server, or it can be a server independently set up outside the first server, the second server and the third server. This application does not limit this.
[0111] As an optional embodiment, if the server that undertakes the cluster management function in the distributed cluster is an independently set cluster management server, then before the above S201, the method 200 also includes: the first server receives a first notification message from the cluster management server, the first notification message is used to notify the distributed database cluster to add servers and / or delete servers, the first notification message includes identification information of multiple first target servers, and the multiple first target servers include the second server; the first server determines to migrate the first page attribution information in the first preset storage space in the first server to the second server based on the identification information of the multiple first target servers.
[0112] It should be understood that if the server responsible for cluster management in the distributed cluster is an independently set cluster management server, the first server will not know that the distributed database cluster needs to add servers and / or delete servers until it receives the first notification message from the cluster management server.
[0113] It should also be understood that the first notification message sent by the cluster management server is a broadcast message within the current cluster, and the time at which each server in the cluster receives the first notification message may vary. This first notification message may be sent based on an operation by a cluster administrator, or may be sent by the cluster management server based on the load of the current cluster servers, which is not limited in this application.
[0114] In the embodiment of the present application, an independently set cluster management server assumes the cluster management function, separating the cluster management function from the business carrying function, which is beneficial to saving the computing power consumption of the server that carries client business access and is beneficial to maintaining the stability of the distributed database cluster performance.
[0115] As an optional embodiment, a possible implementation method of the above-mentioned "the first server determines to migrate the first page ownership information in the first preset storage space in the first server to the second server based on the identification information of multiple first target servers" includes: the first server uses a consistent hash algorithm to calculate the identification information of each server in the multiple first target servers respectively to obtain the hash value of each first server, and maps the hash values of the multiple first target servers to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring; the first server obtains the hash value of each preset storage space in the multiple preset storage spaces, and maps the hash values of the multiple preset storage spaces to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring; the first server determines to migrate the first page ownership information in the first preset storage space in the first server to the second server based on the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring and the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring, and the multiple preset storage spaces include the first preset storage space.
[0116] It should be understood that the multiple first target servers refer to the servers in the distributed database cluster after adding and / or deleting servers. The identification information of the multiple first target servers is used to uniquely identify each server and can be a device code, Internet Protocol (IP) address, protocol port number, or a unique number of each server in the cluster, which is not limited in this application.
[0117] The first server uses a consistent hashing algorithm as a preset load balancing algorithm to allocate a server on which each preset storage space in the plurality of preset storage spaces should be in an active state. It should be understood that the preset storage space is configured to store page attribution information, so allocating the plurality of preset storage spaces is equivalent to allocating the page attribution information stored in the plurality of preset storage spaces.
[0118] In one possible implementation, each preset storage space in a plurality of preset storage spaces is identified by its unique hash value, and the hash values of the plurality of preset storage spaces are all unsigned integers in the range of [0, N]. The present application does not limit the specific value of N. The first server can organize the hash value space of [0, M] into a virtual consistent hash ring through a consistent hash algorithm, and the consistent hash ring distributes the hash values from 0 to M in a clockwise direction, where M is an unsigned integer greater than or equal to N. The present application does not limit the value of M. The first server calculates the identification information of each server in a plurality of first target servers using a consistent hash algorithm to obtain a hash value of each first target server. It should be understood that the hash value of each first target server obtained by the consistent hash algorithm should be in the range of [0, M]. Therefore, the hash values of the plurality of first target servers each uniquely correspond to a position in the consistent hash ring. Similarly, the hash value of each preset storage space in the plurality of preset storage spaces also uniquely corresponds to a position on the consistent hash ring.
[0119] Taking multiple first target servers including a first server, a second server, and a third server, and multiple preset storage spaces including a first preset storage space, a second preset storage space, a third preset storage space, a fourth preset storage space, a fifth preset storage space, a sixth preset storage space, and a seventh preset storage space as an example, FIG3 exemplarily shows the corresponding positions of the hash values of each server and the hash values of each preset storage space on the consistent hash ring. As shown in FIG3 , A, B, and C in the figure correspond to the positions of the hash values of the second server, the first server, and the third server on the consistent hash ring, respectively, and a, b, c, d, e, f, and g correspond to the positions of the hash values of the first preset storage space, the second preset storage space, the third preset storage space, the fourth preset storage space, the fifth preset storage space, the sixth preset storage space, and the seventh preset storage space on the consistent hash ring, respectively.
[0120] A possible implementation method for the first server to determine on which server each preset storage space in the plurality of preset storage spaces should be in a valid state based on the corresponding positions of the hash values of the plurality of first target servers on the consistent hash ring and the corresponding positions of the hash values of the plurality of preset storage spaces on the consistent hash ring is as follows: on the consistent hash ring shown in FIG3 , with the position a of the hash value of the first preset storage space on the consistent hash ring as the starting point, A is the position of the nearest server in the clockwise direction of the consistent hash ring, and A corresponds to the position of the hash value of the second server on the consistent hash ring, so the first preset storage space is allocated to the second server, that is, after the cluster adds and / or deletes servers, the first preset storage space is in a valid state on the second server. Similarly, the second preset storage space and the third preset storage space corresponding to positions b and c on the arc segment CA on the consistent hash ring can also be allocated to the second server, the fourth preset storage space and the fifth preset storage space corresponding to positions d and e on the arc segment AB can be allocated to the first server, and the sixth preset storage space and the seventh preset storage space corresponding to positions f and g on the arc segment BC can be allocated to the third server.
[0121] Optionally, the counterclockwise direction may be used as a reference direction for allocating multiple preset storage spaces on the consistent hash ring, and this application does not impose any specific limitation on this.
[0122] It should be understood that the first server determines to allocate the first preset storage space to the second server based on the corresponding positions of the hash values of multiple first target servers on the consistent hash ring and the corresponding positions of the hash values of multiple preset storage spaces on the consistent hash ring, which means that after the distributed database cluster adds and / or deletes servers, the first preset storage space should be in a valid state on the second server. Based on the first notification message, the first server determines that the first preset storage space currently valid on the first server should be valid on the second server after the cluster adds and / or deletes servers, so the first page attribution information in the first preset storage space in the first server is migrated to the second server, thereby realizing the reallocation of the server corresponding to the first page attribution information.
[0123] Optionally, the first preset storage space currently in a valid state on the first server may be in a valid state after a server is added and / or deleted from the distributed database cluster, or may be in a state after the cluster is initialized. This application does not limit this. However, regardless of how the valid state of the first preset storage space on the first server is obtained, initialization processing will be performed on multiple preset storage spaces on each server when the cluster is started. The purpose is to determine the initial states of the multiple preset storage spaces on each server so that each server is initially in a load-balanced state.
[0124] As an optional embodiment, before the above S201, the first server performs initialization processing on multiple preset storage spaces, and the method also includes: the first server determines the identification information of multiple second target servers, and the multiple second target servers include the first server; the first server uses the identification information of the multiple second target servers to mark the initial status of all or part of the multiple preset storage spaces in the first server as a valid status.
[0125] In one possible implementation, the first server assumes cluster management functionality and obtains identification information of multiple second target servers from a module within its internal functional modules responsible for cluster management. Based on the identification information of the multiple second target servers, a consistent hashing algorithm is employed to determine all or some of the multiple preset storage spaces that should be valid on the first server, and these all or some of the preset storage spaces are marked as valid.
[0126] Optionally, if the case of adding and / or deleting servers corresponding to the above method 200 is the first time to add and / or delete servers in the cluster after the distributed database cluster is initialized, then all or part of the above preset storage space includes the first preset storage space.
[0127] In another possible implementation, the distributed database cluster includes a cluster management server dedicated to cluster management, and the first server can obtain the identification information of multiple second target servers initially included in the cluster through a notification message from the cluster management server. The specific steps include: the first server receives a second notification message from the cluster management server, the second notification message is used to notify the distributed database cluster to perform initialization processing, and the second notification message includes the identification information of multiple second target servers; the first server determines the identification information of the multiple second target servers based on the second notification message.
[0128] As an optional embodiment, the method for the first server to determine the initial states of multiple preset storage spaces when the distributed database cluster is pulled up includes: the first server uses a consistent hashing algorithm to calculate the identification information of each server in the multiple second target servers respectively, obtains the hash value of each second target server, and maps the hash values of the multiple second target servers to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring; the first server obtains the hash value of each preset storage space in the multiple preset storage spaces, and maps the hash values of the multiple preset storage spaces to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring; the first server determines the initial states of the multiple preset storage spaces based on the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring and the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring.
[0129] It should be understood that when the distributed database cluster is initially pulled up, the status of multiple preset storage spaces on each server is invalid. After each server obtains the identification information of the multiple second target servers initially included in the cluster, it obtains the hash value of each second target server and maps the hash values of the multiple second target servers to the consistent hash ring as shown in Figure 3, obtains the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring, and based on the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring and the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring, allocates the multiple preset storage spaces to the multiple second target servers. After the first server obtains the initial allocation result of the multiple preset storage spaces based on the identification information of the multiple second target servers, it marks the preset storage space allocated to itself as valid.
[0130] In an embodiment of the present application, multiple preset storage spaces are allocated when the distributed database cluster is pulled up. Each server marks the preset storage space allocated to the server as valid based on the obtained allocation result. This allows the page ownership information assigned to a certain server to be stored in the preset storage space that is in a valid state on the server during the use of the distributed database, which is conducive to maintaining the load balancing state of the servers in the cluster.
[0131] As an optional embodiment, a possible implementation method of the above S205 includes: the third server uses a consistent hash algorithm to calculate the identification information of the first page to obtain the hash value of the first page, maps the hash value of the first page to a consistent hash ring, and obtains the position of the hash value of the first page on the consistent hash ring; the third server determines that the page ownership information of the first page is stored on the first server based on the position of the hash value of the first page on the consistent hash ring and the corresponding positions of the hash values of multiple second target servers on the consistent hash ring.
[0132] It should be understood that the above S205 is executed when the third server has not yet received the first notification message sent by the cluster management server. In one possible implementation, the situation of adding and / or deleting servers in the distributed database cluster described in method 200 is the first time that the servers in the cluster are added and / or deleted after the distributed database cluster is initialized. In this case, when the third server determines the server corresponding to the page ownership information of the first page, it refers to the positions of the multiple second target servers distributed on the consistent hash ring when the third server is initialized.
[0133] For example, as shown in FIG4 , the multiple second target servers include a first server and a third server. The hash values of the first server and the third server are located at positions B and C on the consistent hash ring, respectively. The hash value of the first page is located at position h on the consistent hash ring. Starting from position h where the hash value of the first page is located, B is the closest server in the clockwise direction. B corresponds to the position of the hash value of the first server, so it can be determined that the page ownership information of the first page is stored on the first server. Therefore, the third server sends a first query request to the first server.
[0134] It should be understood that when searching for the page ownership information of the server corresponding to the first page, the direction referenced on the consistent hash ring should be consistent with the direction referenced when storing the page ownership information. Optionally, the direction can be clockwise or counterclockwise, which is not limited in this application.
[0135] As an optional embodiment, a possible implementation method for the first server in the above S207 to determine that the preset storage space corresponding to the page ownership information of the first page is the first preset storage space based on the identification information of the first page includes: the first server uses a consistent hash algorithm to calculate the identification information of the first page to obtain the hash value of the first page, maps the hash value of the first page to a consistent hash ring, and obtains the position of the first page on the consistent hash ring; the first server determines that the first preset storage space is the preset storage space corresponding to the page ownership information of the first page based on the position of the first page on the consistent hash ring and the corresponding positions of the hash values of multiple preset storage spaces on the consistent hash ring.
[0136] Exemplarily, the multiple preset storage spaces include a first preset storage space, a second preset storage space, a third preset storage space, a fourth preset storage space, a fifth preset storage space, a sixth preset storage space and a seventh preset storage space. As shown in FIG5 , the position of the hash value of the first page on the consistent hash ring is h, and the positions of the hash values of the first preset storage space, the second preset storage space, the third preset storage space, the fourth preset storage space, the fifth preset storage space, the sixth preset storage space and the seventh preset storage space on the consistent hash ring correspond to a, b, c, d, e, f and g respectively. Starting from the position h of the hash value of the first page on the consistent hash ring, the position a of the first closest preset storage space in the clockwise direction is the first preset storage space where the page attribution information of the first page is located.
[0137] It should be understood that for the first page, its page ownership information is stored according to the rules shown in Figure 5 above when it is created. The page ownership information of the first page stored in the first preset storage space, the ownership server of the first page contained in its page ownership information can be changed according to the nature of the page access request (read request / write request), but it will always be stored in the first preset storage space. The page ownership information of the first page will be stored in the first preset storage space on the server where the first preset storage space is valid.
[0138] After executing S201 to S202 and receiving the first query request, the first server can determine that the first preset storage space is the preset storage space corresponding to the page attribution information of the first page based on the identification information of the first page carried in the first query request. However, at this time, the first preset storage space on the first server is in an invalid state, so the first server returns a query failure response to the third server after the query.
[0139] After the third server receives the first query response from the first server, indicating that the query corresponding to the first query request has failed, the above S209 can be implemented in the following three ways:
[0140] (1) The third server determines that the page ownership information of the first page is migrated to the second server based on the migration record carried by the first query response. After executing S201, the first server records the identification information of the second server in the first preset storage space of the first server, and after receiving the first query request and obtaining that the status of the first preset storage space of the first server is invalid, obtains the identification information of the second server recorded in the first preset storage space of the first server, and returns the identification information of the second server to the third server in the first query response. In this way, the third server can continue to send the first query request to the second server based on the identification information of the second server.
[0141] (2) After the third server receives the notification of adding and / or deleting servers from the server that assumes the cluster management function, it determines that the page ownership information of the first page is stored on the second server based on the result of the load balancing calculation of the servers after the addition and / or deletion of the servers. The third server receives the above-mentioned second notification message, and the second notification message includes the identification information of multiple first target servers, and the multiple first target servers include the second server. After receiving the second notification message, the third server determines the position of the hash values of the multiple first target servers on the consistent hash ring based on the identification information of the multiple first target servers, and then based on the position of the hash value of the first page on the consistent hash ring, re-determines that the page ownership information of the first page is stored on the second server.
[0142] (3) The third server predicts that the page ownership information of the first page is stored on the second server based on the preset load balancing algorithm. In a possible implementation, according to the rules of the consistent hashing algorithm, if the first server is a server that needs to be deleted in the current cluster, then after the first server is deleted, the first preset storage space currently valid on the first server will be allocated to the next server after the first server in the clockwise direction of the consistent hashing ring; if the current cluster needs to add a server, and the position of the new server is located before and adjacent to the first server in the clockwise direction of the consistent hashing ring, then part of the currently valid preset storage space on the first server needs to be allocated to the new server. If the next server in the server deletion scenario and the new server in the server addition scenario is the second server, then the third server can predict the server corresponding to the page ownership information of the first page (the adjacent server before or after the first server in the clockwise direction of the consistent hashing ring) based on the relative position of the server and the first server on the consistent hashing ring.
[0143] It should be understood that in the above method 200, the reason why the third server fails to obtain the page ownership information of the first page from the first server is that the third server believes that the page ownership information of the first page should be stored in the first preset storage space of the first server based on the currently known load balancing status of the server. However, in fact, since the number of servers in the cluster has changed, the page ownership information in the servers is being redistributed, and the page ownership information of the first page has been migrated out of the first preset storage space of the first server. In this case, the third server can re-determine the server corresponding to the first page and that actually stores its page ownership information based on the above method (1), method (2) or method (3).
[0144] In an embodiment of the present application, program codes are provided for implementing the above-mentioned methods (1), (2) and (3). In specific implementation, the third server may randomly select any method or try them one by one until the redirection is successful, that is, the server where the valid first preset storage space is located is found.
[0145] As an optional embodiment, the first page attribution information is stored in the first preset storage space of the second server in the form of a bidirectional linked list. A possible way for the second server to implement the above S212 includes: the second server searches in the bidirectional linked list in a traversal manner based on the identification information of the first page until the target node where the page attribution information of the first page is stored in the bidirectional linked list is found, and obtains the page attribution information of the first page from the target node.
[0146] It should be understood that a doubly linked list is a data storage structure. When the amount of data in the distributed database is less than a preset threshold, the page ownership information can be stored in multiple preset storage spaces in the form of a doubly linked list. When searching, the page ownership information corresponding to the identifier of the target page can be found by traversing the doubly linked list in the preset storage space.
[0147] In the embodiment of the present application, the method of storing page ownership information in the form of a bidirectional linked list has a higher search efficiency when the amount of data in the distributed database is less than a preset threshold.
[0148] As an optional embodiment, the first page ownership information is stored in the second server in the form of a hash table. Another possible way for the second server to implement the above S212 includes: the second server uses a consistent hashing algorithm to calculate the identification information of the first page to obtain the hash value of the first page; the second server searches the hash table for the target array corresponding to the hash value of the first page, and obtains the page ownership information of the first page from the target array.
[0149] It should be understood that when the first page's attribution information is stored in the second server in the form of a hash table, a hash search using the hash value of the first page can very quickly locate the target array corresponding to the hash value of the first page, and then obtain the page attribution information of the first page from the target array. This approach is advantageous when the amount of data in the distributed database is greater than or equal to a preset threshold, and can quickly narrow the search scope to the target array, thereby improving search efficiency.
[0150] Figure 6 is a schematic flow chart of a method 600 for implementing page access provided in an embodiment of the present application. The method 600 can be implemented after the above method 200 or can be implemented separately, which is not limited in the present application.
[0151] Method 600 specifically includes the following steps:
[0152] S601: A third server receives a second access request from a client, where the second access request is used to request access to a second page and carries identification information of the second page.
[0153] S602: The third server determines, based on the identification information of the second page, that the page attribution information of the second page is stored in the second server.
[0154] S603. The third server sends a second query request to the second server. The second query request is used to query the page attribution information of the second page and carries the identification information of the second page. Correspondingly, the second server receives the second query request.
[0155] S604: The second server determines, based on the identification information of the second page, that the preset storage space corresponding to the page attribution information of the second page is the second preset storage space, and obtains that the status of the second preset storage space is invalid.
[0156] S605. The second server sends a second query response to the third server, where the second query response is used to indicate that the query corresponding to the second query request fails. Correspondingly, the third server receives the second query response.
[0157] S606: The third server re-determines the page attribution information of the second page according to a preset redirection algorithm and stores it in the first server.
[0158] S607: The third server sends a second query request to the first server; correspondingly, the first server receives the first query request.
[0159] S608: The first server determines, based on the identification information of the second page, that the preset storage space corresponding to the attribution information of the second page is the second preset storage space, and obtains that the status of the second preset storage space of the first server is valid.
[0160] S609: The first server obtains page attribution information of the second page from the second preset storage space.
[0161] S610: The first server sends page attribution information of the second page to the third server.
[0162] The implementation scenario of method 600 is as follows: the cluster management server has broadcast a first notification message, the first notification message is used to notify the distributed database cluster to add and / or delete servers, and the first server and the third server have received the notification. Based on the identification information of the multiple first target servers included in the notification, after the distributed database cluster is added and / or deleted through the consistent hashing algorithm, the second preset storage space should be in a valid state on the second server. However, since the migration of page ownership information takes time, when the second server receives the second query request from the third server, it has not yet received the page ownership information in the second preset storage space from the first server. Therefore, when the second server receives the second query request, the second preset storage space on the second server is temporarily in an invalid state, so the second server returns a query failure response to the third server.
[0163] A possible implementation method of the above S604 includes: the third server determines that the server where the page ownership information of the second page is located before the third server receives the first notification message is the first server based on the positions of the hash values of multiple second target servers included in the cluster initialization on the consistent hash ring, and the position of the hash value of the second page on the consistent hash ring, the third server sends a second query request to the first server, and receives the page ownership information of the second page found by the first server in the second preset storage space.
[0164] The method for determining the server where the page ownership information of the second page is located based on the identification information of the second page in the above S602 is similar to the method for determining the server where the page ownership information of the first page is located based on the identification information of the first page; the method for determining the preset storage space corresponding to the page ownership information of the second page based on the identification information of the second page in the above S604 and S608 is also similar to the method for determining the preset storage space corresponding to the page ownership information of the first page based on the identification information of the first page, and they will not be repeated here.
[0165] The method provided in the embodiment of the present application enables a distributed database cluster to, when adding and / or deleting servers, if the second query request sent by the third server to the second server fails because the page attribution information in the second preset storage space should be migrated from the first server to the second server but has not yet been migrated to the second server, the page attribution information of the second page can be re-stored on the first server based on a preset redirection method. This is beneficial to speeding up the efficiency of obtaining the attribution information of the second page, reducing the probability of failure of the second page access request, and ensuring the availability of user services.
[0166] Optionally, the "multiple preset storage spaces" described in the above embodiment can be a bucket array, or any other data structure that can implement the functions described in this application. This application does not limit the specific structure of the multiple preset storage spaces.
[0167] Below, a method for implementing page access described in an embodiment of the present application is described in detail by taking multiple preset storage spaces as a bucket array as an example.
[0168] Figure 7 is a schematic flow chart of a method 700 for implementing page access, provided in an embodiment of the present application. This method 700 is applied to a scenario where the number of servers supporting client service access in a distributed database cluster increases from two to three, and the cluster is provided with an independent cluster management server. The distributed database cluster is initially configured with two servers supporting client service access. When the cluster is established, the cluster management server broadcasts the initial cluster view within the initial cluster. For example, the initial cluster includes a first server and a third server.
[0169] It should be understood that the bucket array provided in the embodiment of the present application is included in each server of the distributed database cluster, and is the program code in the page buffer pool included in the database instance of the server. During runtime, the program code runs in the server host memory and / or graphics processing unit (GPU) memory.
[0170] In one possible implementation, the valid and invalid states of each bucket in the bucket array can be marked with valid and invalid, respectively. A usage lock can be set for each bucket in the bucket array. When migrating the page ownership information stored in the bucket, a dedicated lock with a mutually exclusive property (usage lock exclusive) is added to prevent any request from accessing the bucket at this time; when the bucket is accessed, a shared lock (usage lock share) is added to identify its usage status. At this time, the bucket can be accessed by multiple requests at the same time. Each bucket in the bucket array can calculate a specific hash value through the hash function hash(key), and the hash value of each bucket is used as its identification information, recorded as bucket id, where key can be a unique code for each bucket in the bucket array. Each bucket in the bucket array includes a bidirectional linked list of a preset length, and each node in the bidirectional linked list can store the page ownership information of a page.
[0171] Method 700 specifically includes the following steps:
[0172] S701: The cluster management server broadcasts an initial cluster view, where the initial cluster view includes a version number of the initial cluster and identification information of a first server and a third server.
[0173] Optionally, the version number of the initial cluster in the initial cluster view can be 100, and the identification information corresponding to the servers included in the initial cluster can be the unique number of each server in the cluster, the first server is numbered 0, and the third server is numbered 1, but this application is not limited to this.
[0174] S702. The first server receives the initial cluster view, and based on the consistent hashing algorithm, calculates the identification information of the first server and the third server included in the initial cluster view, obtains the positions of the hash values of the first server and the third server on the consistent hashing ring, and determines the first allocation result of the bucket array on the first server and the third server based on the positions of the hash values of each bucket in the bucket array on the consistent hashing ring, and marks the status of the bucket allocated to the first server as valid.
[0175] In a possible implementation, the storage space (bucket) array includes multiple buckets with bucket IDs from 0 to m, where m is an unsigned integer. The first allocation result is to allocate the buckets with bucket IDs from 0 to k in the bucket array to the first server, and allocate the buckets with bucket IDs from k + 1 to m to the third server, where 0 < k < m and k is also an unsigned integer.
[0176] Optionally, the multiple buckets with bucket IDs from 0 to m can be m consecutive buckets with bucket IDs, or multiple buckets within the range of bucket IDs from 0 to m but with a number of buckets less than m. This application does not limit this. In the embodiments of this application, an example is given where the bucket array includes m buckets with bucket IDs from 0 to m.
[0177] The specific implementation process of the above S702 includes: The first server starts from the bucket with bucket ID 0, and逐个 judges whether the current bucket is allocated to the first server in the first allocation result. If so, it marks the status of the bucket as valid. If not, it skips the bucket and continues to judge the next one until it traverses each bucket in the bucket array.
[0178] S703. The third server receives the initial cluster view, calculates the positions of the identification information of the first server and the third server included in the initial cluster view on the consistent hash ring based on the consistent hash algorithm, determines the first allocation result of the bucket array on the first server and the third server based on the positions of the hash values of each bucket in the bucket array on the consistent hash ring, and marks the status of the buckets allocated to the third server as the valid status.
[0179] It should be understood that the logic for the third server to execute S703 is the same as that for the first server to execute S702, and will not be elaborated here.
[0180] At this point, the cluster initialization is completed. After that, the cluster management server determines, based on the load conditions of the first server and the second server, that a server needs to be added to the current cluster, for example, it can be the second server. Therefore, it broadcasts the first cluster view in the cluster including the second server.
[0181] S704: The cluster management server broadcasts the first cluster view, where the first cluster view includes a version number of the first cluster view and identification information corresponding to the first server, the third server, and the second server.
[0182] Optionally, the version number of the first cluster view may be 101, and the identification information corresponding to the servers included in the cluster may be number 0 of the first server, number 1 of the third server, and number 2 of the second server.
[0183] It should be understood that the purpose of the cluster management server including the cluster view version number in each broadcast of the cluster view is to prevent the server from receiving migration information corresponding to a cluster view version that is lower than the locally stored version. It can also prevent the cluster view version on some servers that have completed the latest expansion or contraction from rolling back when the next expansion or contraction is started before the previous expansion or contraction in the cluster is completely completed.
[0184] S705. The first server receives the first cluster view, and calculates the identification information of the first server, the third server, and the second server included in the first cluster view based on the consistent hashing algorithm to obtain the positions of the hash values of the first server, the third server, and the second server on the consistent hashing ring, and determines the second allocation results of the bucket array on the first server, the third server, and the second server based on the positions of the hash values of each bucket in the bucket array on the consistent hashing ring, and determines the target bucket in the first server that is currently in a valid state but is allocated to other servers in the second allocation result.
[0185] Exemplarily, the other server may be a second server.
[0186] S706. The first server sends the page ownership information in the target bucket to the second server, where the target bucket includes the bucket with bucket ID q. Correspondingly, the second server receives the page ownership information.
[0187] The specific implementation process of S705-S706 above includes: starting with the bucket with bucket ID 0, the first server determines whether the current bucket's status is valid and whether it is allocated to the second server in the second allocation result; if so, it obtains the usage lock exclusive lock for the current bucket, sends the page ownership information in the current bucket to the second server, and after receiving the lock release request from the second server, marks the status of the bucket as invalid and releases the usage lock to the idle state; if not, it skips the bucket and continues to determine the next one, until every bucket in the bucket array is traversed. After receiving the page ownership information, the second server obtains the usage lock exclusive lock for the bucket corresponding to the first server, populates the received page ownership information into the bucket, marks the bucket's status as valid, and releases the usage lock to the idle state.
[0188] It's worth noting that when a usage lock exclusive is applied to a bucket, access requests to the bucket will be temporarily blocked by the usage lock exclusive lock until the usage lock is released to the idle state. At this point, the request can obtain the valid or invalid status of the bucket. This prevents access requests to the bucket from continuously reporting errors due to inaccessible data during the bucket migration process, thus saving signaling overhead between servers.
[0189] FIG. 8 is a schematic diagram of the migration process of page attribution information from the perspective of the cluster. As shown in FIG. 8, the first allocation result of multiple buckets in the bucket array on the first server and the third server is represented by row L1. Among them, the buckets with bucket IDs from 0 to k in the shaded part of the bucket array corresponding to the first server represent the buckets allocated to the first server, and the buckets with bucket IDs from k + 1 to m in the shaded part corresponding to the third server represent the buckets allocated to the third server; row L2 represents the migration process of S706 - S707 above. As the first server traverses the local buckets, the page attribution information in the buckets with bucket IDs from 0 to q has been migrated to the newly added second server, and the bucket allocation result on the third server in the first allocation result and the second allocation result remains unchanged; row L3 indicates that the migration of the page attribution information in the buckets with bucket IDs from 0 to s in the first server that should be migrated to the second server has been completed, and the load balancing state corresponding to the second allocation result has been reached. Among them, 0 < q < s < k < m, and 0, q, s, k, and m are all unsigned integers.
[0190] In the embodiment of the present application, the access right of the client to the server is always open. A possible scenario is that during the process of the first server migrating the page attribution information in the corresponding bucket based on the second allocation result (when reaching the state shown in L2 of FIG. 8), the third server receives a page access request from the client.
[0191] S707. The third server receives a first access request from the client, which is used to request access to page I and carries the identification information of page I.
[0192] S708. The third server determines that the page attribution information of page I is stored on the first server according to the positions of the hash values of the first server and the third server determined during the initialization of the cluster on the consistent hash ring.
[0193] S709. The third server sends a first query request to the first server, which is used to request the page attribution information of page I and carries the identification information of page I; correspondingly, the first server receives this first query request.
[0194] S710. Based on the identification information of page I, the first server determines that the bucket ID corresponding to the page attribution information of page I is q, and queries that the status of the bucket with bucket ID q in the first server is the invalid state (invalid).
[0195] S711. The first server sends a first query response to the third server, indicating that the first query request fails. Correspondingly, the third server receives the first query response.
[0196] Optionally, the first query response carries identification information of the second target server.
[0197] S712. The third server re-determines that the corresponding server is the second server based on the identification information of the second target server carried in the first query response, or based on the received first cluster view, or based on the initial cluster view, according to the identification information of page I.
[0198] It should be understood that the reason why the third server cannot correctly determine the server corresponding to the page ownership information of page I is that the timing at which the third server and other servers receive the broadcast notification from the cluster management server is different. Therefore, as long as the third server is synchronized with the server information it wants to access, that is, the obtained cluster view is consistent, the third server can always determine the server where the page ownership information of page I is located based on the cluster view.
[0199] S713. The third server sends a first query request to the second server, requesting the page ownership information of page I, carrying the identification information of page I; correspondingly, the second server receives the first query request.
[0200] S714. The second server determines that the page ownership information of page I corresponds to bucket ID q based on the identification information of page I, queries the status of the bucket with bucket ID q in the first server and finds that it is valid. The second server obtains the page ownership information of page I from the bucket.
[0201] S715. The second server sends the page attribution information of page I to the third server; correspondingly, the third server receives the page attribution information of page I.
[0202] In one possible implementation, the page ownership information including page I is stored in the form of a bidirectional linked list in the bucket with bucket ID q on the second server. Based on the identification information of page I, the second server searches the bidirectional linked list in the bucket in a traversal manner until the page ownership information of page I corresponding to the identification information of page I is found.
[0203] In another possible implementation, a pointer (hash entry) pointing to the corresponding location of the target page ownership information in the hash table is stored in the bucket with bucket ID q on the second server in the form of a doubly linked list. The page ownership information is stored in the server in the form of a hash table (key-value). When searching for the page ownership information of page I, a consistent hash calculation is performed on the identification information of page I. The resulting hash value of page I uniquely corresponds to a target array in the hash table. The page ownership information of page I is then searched from the target array based on the identification information of page I. The hash entry stored in the doubly linked list can be used to retrieve all the page ownership information corresponding to the entire bucket from the hash table one by one when migrating the bucket. In addition, storing page ownership information in a hash table does not occupy the bucket capacity to store page ownership information, and the page ownership information storage will not reach the bucket capacity limit due to bucket capacity limitations, which is conducive to improving the flexibility of the distributed database cluster.
[0204] In one possible scenario, after the distributed database cluster reaches the load-balanced state corresponding to L3 in FIG8 , the cluster includes a first server, a second server, and a third server. After a period of use, the cluster management server determines, based on the load of each server, that the three servers are overpowered at the current business scale and that one server needs to be deleted. For example, if the second server needs to be deleted, the migration process of the second server's page ownership information from the cluster perspective can be shown in FIG9 . L1 in FIG9 shows the allocation of the bucket arrays on each server before the second server is deleted. Buckets with bucket IDs 0 to s are valid on the second server, bucket IDs s+1 to k are valid on the first server, and bucket IDs k+1 to m are valid on the third server. L2 in FIG9 shows the process of migrating page ownership information from valid buckets on the second server to the first server. L3 in FIG9 shows the new load-balanced state of the distributed database cluster after all page ownership information on the second server has been migrated.
[0205] Optionally, in the scenario of deleting a server, the cluster management server can broadcast the cluster view twice. The first broadcast is used to indicate the identification information of the server to be deleted, and each server in the cluster calculates the distribution of the bucket array on the remaining servers based on the identification information of the remaining servers; the second broadcast is used to indicate the identification information of the servers included in the cluster after the server is deleted, so as to determine that the servers in the current cluster have reached the expected load balancing status.
[0206] It should be understood that when the distributed database cluster deletes the second server, the migration process of the page ownership information on the second server is similar to the execution process of the first server in the above method 200 or 600, and will not be repeated here.
[0207] The above text describes in detail the method for page access provided by the embodiment of the present application in conjunction with Figures 2 to 9. The following text describes in detail the server of the embodiment of the present application in conjunction with Figures 10 and 11.
[0208] FIG10 is a schematic block diagram of a server 1000 provided in an embodiment of the present application. Referring to FIG10 , the server 1000 includes a transceiver module 1001 and a processing module 1002 .
[0209] In a possible implementation, the server 1000 is used to execute the processes and steps corresponding to the first server in the above method embodiment.
[0210] The transceiver module 1001 is used to: send the first page attribution information in the first preset storage space to the second server; the processing module 1002 is used to: delete the first page attribution information, mark the status of the first preset storage space in the first server as invalid, the first page attribution information is used to represent the correspondence between at least one page and the server storing at least one page, each server in the distributed database cluster includes the same number of multiple preset storage spaces, and each preset storage space in the multiple preset storage spaces is only valid on one server; the transceiver module 1001 is also used to: receive a first query request from a third server, the first query request is used to query the page attribution information of the first page, and the first query request carries the identification information of the first page; the processing module 1002 is also used to: determine, based on the identification information of the first page, that the preset storage space corresponding to the page attribution information of the first page is the first preset storage space, and obtain that the status of the first preset storage space in the first server is invalid; and, send a first query response to the third server, the query response is used to indicate that the query corresponding to the first query request has failed.
[0211] Optionally, the transceiver module 1001 is also used to: receive a first notification message from the cluster management server, the first notification message is used to notify the distributed database cluster to add a server and / or delete a server, the first notification message includes identification information of multiple first target servers, and the multiple first target servers include a second server; the processing module 1002 is also used to: based on the identification information of multiple first target servers, determine to migrate the first page attribution information in the first preset storage space in the first server to the second server.
[0212] Optionally, the processing module 1002 is specifically used to: use a consistent hash algorithm to calculate the identification information of each server in multiple first target servers respectively to obtain the hash value of each first server, and map the hash values of the multiple first target servers to a consistent hash ring to obtain the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring; obtain the hash value of each preset storage space in multiple preset storage spaces, and map the hash values of the multiple preset storage spaces to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring; based on the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring and the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring, determine to migrate the first page attribution information in the first preset storage space in the first server to the second server, and the multiple preset storage spaces include the first preset storage space.
[0213] Optionally, the processing module 1002 is also used to: determine the identification information of multiple second target servers, the multiple second target servers including the first server; and use the identification information of the multiple second target servers to mark the initial status of all or part of the multiple preset storage spaces in the first server as a valid status, all or part of the preset storage spaces including the first preset storage space.
[0214] Optionally, the processing module 1002 is also used to: use a consistent hash algorithm to calculate the identification information of each server in the multiple second target servers respectively, obtain the hash value of each second target server, and map the hash values of the multiple second target servers to a consistent hash ring to obtain the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring; obtain the hash value of each preset storage space in the multiple preset storage spaces, and map the hash values of the multiple preset storage spaces to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring; determine the initial states of the multiple preset storage spaces based on the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring and the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring.
[0215] Optionally, the transceiver module 1001 is also used to: receive a second notification message from the cluster management server, the second notification message is used to notify the distributed database cluster to perform initialization processing, and the second notification message includes identification information of multiple second target servers; the processing module 1002 is also specifically used to: determine the identification information of multiple second target servers based on the second notification message.
[0216] Optionally, the processing module 1002 is further specifically used to: calculate the identification information of the first page using a consistent hash algorithm to obtain the hash value of the first page, map the hash value of the first page to a consistent hash ring, and obtain the position of the first page on the consistent hash ring; based on the position of the first page on the consistent hash ring and the corresponding positions of the hash values of multiple preset storage spaces on the consistent hash ring, determine that the first preset storage space is the preset storage space corresponding to the page attribution information of the first page.
[0217] Optionally, the processing module 1002 is also used to: record the identification information of the second server in the first preset storage space in the first server; obtain the identification information of the second server recorded in the first preset storage space in the first server; the transceiver module 1001 is also used to: send a first query response to the third server, the query response is used to indicate that the query corresponding to the first query request has failed, and the first query response carries the identification information of the second server.
[0218] Optionally, the first page attribution information is stored in a first preset storage space in the first server in the form of a bidirectional linked list, or is stored in the first server in the form of a hash table.
[0219] The transceiver module 1001 is also used to: receive a second query request from a third server, the second query request is used to query the page ownership information of the second page, and the second query request carries the identification information of the second page; the processing module 1002 is also used to: based on the identification information of the second page, determine that the preset storage space corresponding to the page ownership information of the second page is the second preset storage space; and, when the status of the second preset storage space in the first server is queried to be a valid state, obtain the page ownership information of the second page from the second preset storage space in the first server; the transceiver module 1001 is also used to: send the page ownership information of the second page to the third server.
[0220] In a possible implementation, the server 1000 is further configured to execute the processes and steps corresponding to the second server in the above method embodiment.
[0221] The transceiver module 1001 is used to: receive the first page attribution information, store the first page attribution information in the first preset storage space in the second server, and mark the status of the first preset storage space in the second server as a valid state; and, receive a first query request from the third server, the first query request is used to query the page attribution information of the first page, and the first query request carries the identification information of the first page; the processing module 1002 is used to: based on the identification information of the first page, determine that the preset storage space corresponding to the page attribution information of the first page is the first preset storage space, and obtain the status of the first preset storage space in the second server as a valid state; and, obtain the page attribution information of the first page from the first preset storage space in the second server; the transceiver module 1001 is also used to: send the page attribution information of the first page to the third server.
[0222] Optionally, the first page attribution information is stored in the first preset storage space in the second server in the form of a bidirectional linked list, and the second server obtains the page attribution information of the first page from the first preset storage space in the second server. The processing module 1002 is also used to: based on the identification information of the first page, search in the bidirectional linked list in a traversal manner until the target node where the page attribution information of the first page is stored in the bidirectional linked list is found, and obtain the page attribution information of the first page from the target node.
[0223] Optionally, the first page ownership information is stored in the second server in the form of a hash table, and the second server obtains the page ownership information of the first page from the first preset storage space in the second server. The processing module 1002 is also used to: use a consistent hash algorithm to calculate the identification information of the first page to obtain the hash value of the first page; find the target array corresponding to the hash value of the first page in the hash table, and obtain the page ownership information of the first page from the target array.
[0224] Optionally, the transceiver module 1001 is also used to: receive a second query request from a third server, the second query request is used to query the page attribution information of the second page, and the second query request carries the identification information of the second page; the processing module 1002 is also used to: based on the identification information of the second page, determine that the preset storage space corresponding to the page attribution information of the second page is the second preset storage space; and, in the case that the status of the second preset storage space in the second server is an invalid state, send a query response to the third server, and the query response is used to indicate that the query failed.
[0225] In a possible implementation, the server 1000 is further configured to execute the processes and steps corresponding to the third server in the above method embodiment.
[0226] The transceiver module 1001 is used to: receive a first access request from a client, the first access request is used to request access to a first page, and the first access request carries identification information of the first page; the processing module 1002 is used to: determine, based on the identification information of the first page, that the page ownership information of the first page is stored on the first server; the transceiver module 1001 is also used to: send a first query request to the first server, the first query request is used to query the page ownership information of the first page, and the first query request carries the identification information of the first page; and, receive a first query response from the first server, the first query response is used to indicate that the query corresponding to the first query request has failed; the processing module 1002 is also used to: re-determine, according to a preset redirection algorithm, that the page ownership information of the first page is stored on the second server; the transceiver module 1001 is also used to: send a first query request to the second server; and, receive the page ownership information of the first page from the second server.
[0227] Optionally, the first query response carries identification information of the second server; the processing module 1002 is further configured to: based on the first query response, determine that the first page attribution information is stored in the second server.
[0228] Optionally, the transceiver module 1001 is also used to: receive a second notification message from the cluster management server, the second notification message is used to notify the distributed database cluster to perform initialization processing, the second notification message includes identification information of multiple second target servers, and the multiple second target servers include a third server; the processing module 1002 is also used to: use a consistent hash algorithm to calculate the identification information of each server in the multiple second target servers respectively, to obtain the hash value of each second target server, and map the hash values of the multiple second target servers to a consistent hash ring, to obtain the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring; obtain the hash value of each preset storage space in the multiple preset storage spaces, and map the hash values of the multiple preset storage spaces to the consistent hash ring, to obtain the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring; based on the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring and the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring, determine the initial states of the multiple preset storage spaces; and mark the initial states of all or part of the multiple preset storage spaces in the third server as valid states.
[0229] Optionally, the processing module 1002 is specifically used to: use a consistent hash algorithm to calculate the identification information of the first page to obtain the hash value of the first page, map the hash value of the first page to a consistent hash ring, and obtain the position of the hash value of the first page on the consistent hash ring; and, based on the position of the hash value of the first page on the consistent hash ring and the corresponding positions of the hash values of multiple second target servers on the consistent hash ring, determine that the page ownership information of the first page is stored on the first server.
[0230] Optionally, the processing module 1002 is also specifically used to: based on the corresponding positions of the hash values of multiple second target servers on the consistent hash ring, take the position of the hash value of the first server on the consistent hash ring as the starting point, and search on the consistent hash ring in a preset direction to determine the second server, and the corresponding position of the hash value of the second server on the consistent hash ring is after the first server and adjacent to the first server.
[0231] Optionally, the transceiver module 1001 is also used to: receive a first notification message from the cluster management server, the first notification message is used to notify the distributed database cluster to add a server and / or delete a server, the first notification message includes identification information of multiple first target servers, and the multiple first target servers include a second server; the processing module 1002 is also specifically used to: use a consistent hash algorithm to calculate the identification information of the first page to obtain the hash value of the first page, map the hash value of the first page to a consistent hash ring, and obtain the position of the hash value of the first page on the consistent hash ring; use a consistent hash algorithm to calculate the identification information of each server in the multiple first target servers separately to obtain the hash value of each first target server, and map the hash values of the multiple first target servers to the consistent hash ring to obtain the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring; and, based on the position of the hash value of the first page on the consistent hash ring and the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring, determine that the page ownership information of the first page is stored on the second server.
[0232] Optionally, the transceiver module 1001 is also used to: receive a second access request from the client, the second access request is used to request access to the second page, and the second access request carries identification information of the second page; the processing module 1002 is also used to: based on the identification information of the second page, determine that the page ownership information of the second page is stored on the second server; the transceiver module 1001 is also used to: send a second query request to the second server, the second query request is used to query the page ownership information of the second page, and the second query request carries the identification information of the second page; and, receive a second query response from the second server, the second query response is used to indicate that the query corresponding to the second query request has failed; the processing module 1002 is also used to: according to a preset redirection algorithm, re-determine that the page ownership information of the second page is stored on the first server; the transceiver module 1001 is also used to: send a second query request to the first server; and, receive the page ownership information of the second page from the first server.
[0233] Optionally, the processing module 1002 is also specifically used to: use a consistent hash algorithm to calculate the identification information of the second page to obtain the hash value of the second page, map the hash value of the second page to a consistent hash ring, and obtain the position of the hash value of the second page on the consistent hash ring; and based on the position of the hash value of the second page on the consistent hash ring and the corresponding positions of the hash values of multiple second target servers on the consistent hash ring, determine that the page ownership information of the second page is stored on the first server.
[0234] It should be understood that the server 1000 here is embodied in the form of a functional module. The term "module" here may refer to an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a proprietary processor or a group processor, etc.) and a memory for executing one or more software or firmware programs, a merged logic circuit and / or other suitable components that support the described functions. In an optional example, those skilled in the art will understand that the server 1000 may be specifically the first server, the second server or the third server in the above-mentioned embodiment, and the server 1000 may be used to execute the various processes and / or steps corresponding to the first server, the second server or the third server in the above-mentioned method embodiment. To avoid repetition, they will not be described here.
[0235] The server 1000 has the function of implementing the corresponding steps performed by the first server, the second server, or the third server in the above method; the above functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. For example, the transceiver module 1001 may include a sending module and a receiving module. The sending module can be used to implement the various steps and / or processes for performing the sending action corresponding to the above transceiver module, and the receiving module can be used to implement the various steps and / or processes for performing the receiving action corresponding to the above transceiver module. The sending module can be replaced by a transmitter, and the receiving module can be replaced by a receiver, respectively performing the sending and receiving operations and related processing operations in each method embodiment.
[0236] In the embodiment of the present application, the server 1000 in FIG10 may also be a chip or a chip system, such as a system on chip (SOC). Correspondingly, the transceiver module 1001 may be a transceiver circuit of the chip, which is not limited here.
[0237] Figure 11 is a schematic block diagram of a server 1100 provided in an embodiment of the present application. The server 1100 includes a processor 1101, a transceiver 1102, and a memory 1103. The processor 1101, the transceiver 1102, and the memory 1103 communicate with each other via an internal connection path. The memory 1103 is used to store instructions, and the processor 1101 is used to execute the instructions stored in the memory 1103. The transceiver 1102 can be used to send requests or receive responses to other servers in a distributed database cluster. Exemplarily, the transceiver 1102 reads the instructions stored in the memory 1103 and sends the instructions to the processor 1101.
[0238] It should be understood that the server 1100 can be specifically the first server, the second server, or the third server in the above-mentioned embodiment, and can be used to execute the various steps and / or processes corresponding to any server in the above-mentioned method embodiment. Optionally, the memory 1103 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type. The processor 1101 can be used to execute instructions stored in the memory, and when the processor 1101 executes instructions stored in the memory, the processor 1101 is used to execute the various steps and / or processes of the above-mentioned method embodiment.
[0239] It should be noted that the above-described method embodiments can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above-described method embodiments can be completed by hardware integrated logic circuits within the processor or by software instructions. The above-described processor may be a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules within the decoding processor. The software modules can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in a memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above-described method.
[0240] The memory in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Wherein, the non-volatile memory can be a ROM, PROM, EPROM, EEPROM or flash memory. The volatile memory can be a RAM, which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (doubledata rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM) and direct memory bus random access memory (direct rambus RAM, DR RAM). It should be noted that the memory of the system and method described herein is intended to include but is not limited to these and any other suitable types of memory.
[0241] The present application also provides a distributed database cluster, which includes at least the above-mentioned first server, second server and third server, and is used to execute the method corresponding to the above-mentioned method embodiment.
[0242] The present application also provides a system for implementing page access, which includes at least the above-mentioned first server, second server and third server, and is used to execute the method corresponding to the above-mentioned method embodiment.
[0243] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instructions), which, when executed, enables a computer to execute any of the methods executed by the server in the above method embodiments.
[0244] The present application also provides a computer-readable storage medium storing a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by any server in the above method embodiments.
[0245] The method provided in the above embodiment can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product may include one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic disk), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)).
[0246] Those skilled in the art will appreciate that the various method steps and units described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0247] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0248] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.
[0249] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.
[0250] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.
[0251] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0252] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.
Claims
1. A method for implementing page access, characterized in that: The method comprises: In the case where a server is added and / or a server is deleted in the distributed database cluster, the first server sends first page attribution information in a first preset storage space to the second server, deletes the first page attribution information, and marks the state of the first preset storage space in the first server as invalid, wherein the first page attribution information is used to indicate a corresponding relationship between at least one page and a server storing the at least one page, each server in the distributed database cluster includes a plurality of preset storage spaces of the same number, and each of the plurality of preset storage spaces is in a valid state only on one server; The first server receives a first query request from a third server, where the first query request is used to query page attribution information of a first page, and the first query request carries identification information of the first page; The first server determines, based on the identification information of the first page, that the preset storage space corresponding to the page attribution information of the first page is the first preset storage space, and obtains that the state of the first preset storage space in the first server is an invalid state; The first server sends a first query response to the third server, where the query response is used to indicate that a query corresponding to the first query request fails.
2. The method according to claim 1, characterized in that Before the first server sends the first page attribution information to the second server, the method further includes: The first server receives a first notification message from the cluster management server, the first notification message is used to notify the distributed database cluster to add a server and / or delete a server, the first notification message includes identification information of multiple first target servers, and the multiple first target servers include the second server; The first server determines, based on the identification information of the plurality of first target servers, to migrate the first page attribution information in a first preset storage space in the first server to the second server.
3. The method according to claim 2, characterized in that The first server determines, based on the identification information of the plurality of first target servers, to migrate the first page attribution information in the first preset storage space in the first server to the second server, including: The first server calculates the identification information of each server in the multiple first target servers by using a consistent hash algorithm to obtain a hash value of each first server, and maps the hash values of the multiple first target servers to a consistent hash ring to obtain corresponding positions of the hash values of the multiple first target servers on the consistent hash ring; The first server obtains a hash value of each preset storage space in the multiple preset storage spaces, and maps the hash values of the multiple preset storage spaces to the consistent hash ring to obtain corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring; The first server determines to migrate the first page attribution information in the first preset storage space in the first server to the second server based on the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring and the corresponding positions of the hash values of multiple preset storage spaces on the consistent hash ring, and the multiple preset storage spaces include the first preset storage space.
4. The method according to any one of claims 1 to 3, characterized in that Before the first server sends the first page attribution information in the first preset storage space to the second server, the method further includes: The first server determines identification information of a plurality of second target servers, where the plurality of second target servers include the first server; The first server uses the identification information of the multiple second target servers to mark the initial status of all or part of the multiple preset storage spaces in the first server as valid status, and the all or part of the preset storage spaces include the first preset storage space.
5. The method according to claim 4, characterized in that Before marking the initial status of all or part of the preset storage spaces in the plurality of preset storage spaces in the first server as a valid status, the method further includes: The first server calculates the identification information of each server in the multiple second target servers by using a consistent hash algorithm to obtain a hash value of each second target server, and maps the hash values of the multiple second target servers to a consistent hash ring to obtain the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring; The first server obtains a hash value of each preset storage space in the multiple preset storage spaces, and maps the hash values of the multiple preset storage spaces to the consistent hash ring to obtain corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring; The first server determines the initial states of the multiple preset storage spaces based on the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring and the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring.
6. The method according to claim 4 or 5, characterized in that: The first server determines identification information of multiple second target servers, including: The first server receives a second notification message from the cluster management server, where the second notification message is used to notify the distributed database cluster to perform initialization processing, and the second notification message includes identification information of multiple second target servers; The first server determines identification information of multiple second target servers based on the second notification message.
7. The method according to any one of claims 1 to 6, characterized in that The first server determines, based on the identification information of the first page, that a preset storage space corresponding to the page attribution information of the first page is a first preset storage space, including: The first server calculates the identification information of the first page by using a consistent hash algorithm to obtain a hash value of the first page, maps the hash value of the first page to a consistent hash ring, and obtains a position of the first page on the consistent hash ring; The first server determines that the first preset storage space is the preset storage space corresponding to the page attribution information of the first page based on the position of the first page on the consistent hash ring and the corresponding positions of the hash values of multiple preset storage spaces on the consistent hash ring.
8. The method according to any one of claims 1 to 7, characterized in that After the first server sends the first page attribution information in the first preset storage space to the second server, the first server records the identification information of the second server in the first preset storage space of the first server; After the first server acquires that the state of the first preset storage space in the first server is invalid, the method further includes: The first server acquires identification information of the second server recorded in a first preset storage space in the first server; The first server sends a first query response to the third server, where the query response is used to indicate that a query corresponding to the first query request fails, and the first query response carries identification information of the second server.
9. The method according to any one of claims 1 to 8, characterized in that The first page attribution information is stored in a first preset storage space in the first server in the form of a bidirectional linked list, or is stored in the first server in the form of a hash table.
10. The method according to any one of claims 1 to 9, characterized in that The method further comprises: The first server receives a second query request from a third server, where the second query request is used to query page attribution information of a second page, and the second query request carries identification information of the second page; The first server determines, based on the identification information of the second page, that the preset storage space corresponding to the page attribution information of the second page is the second preset storage space; When the first server finds that the state of the second preset storage space in the first server is valid, the first server obtains the page attribution information of the second page from the second preset storage space in the first server; The first server sends page attribution information of the second page to the third server.
11. A method for implementing page access, characterized in that: include: The second server receives the first page attribution information, stores the first page attribution information in a first preset storage space in the second server, and marks the state of the first preset storage space in the second server as a valid state; The second server receives a first query request from a third server, where the first query request is used to query page attribution information of the first page, and the first query request carries identification information of the first page; The second server determines, based on the identification information of the first page, that the preset storage space corresponding to the page attribution information of the first page is the first preset storage space, and obtains that the status of the first preset storage space in the second server is valid. state; The second server obtains the page attribution information of the first page from a first preset storage space in the second server; The second server sends the page attribution information of the first page to the third server.
12. The method according to claim 11, characterized in that The first page attribution information is stored in a first preset storage space in the second server in the form of a bidirectional linked list, and the second server obtains the page attribution information of the first page from the first preset storage space in the second server, including: Based on the identification information of the first page, the second server searches the bidirectional linked list in a traversal manner until a target node where the page ownership information of the first page is stored in the bidirectional linked list is found, and obtains the page ownership information of the first page from the target node.
13. The method according to claim 11, characterized in that The first page attribution information is stored in the second server in the form of a hash table, and the second server obtains the page attribution information of the first page from a first preset storage space in the second server, including: The second server calculates the identification information of the first page using a consistent hash algorithm to obtain a hash value of the first page; The second server searches the hash table for a target array corresponding to the hash value of the first page, and obtains page attribution information of the first page from the target array.
14. The method according to any one of claims 11 to 13, characterized in that The method further comprises: The second server receives a second query request from the third server, where the second query request is used to query page attribution information of a second page, and the second query request carries identification information of the second page; The second server determines, based on the identification information of the second page, that the preset storage space corresponding to the page attribution information of the second page is the second preset storage space; When the second server finds that the state of the second preset storage space in the second server is invalid, the second server sends a query response to the third server, where the query response is used to indicate that the query fails.
15. A method for implementing page access, characterized in that: include: The third server receives a first access request from the client, where the first access request is used to request access to a first page, and the first access request carries identification information of the first page; The third server determines, based on the identification information of the first page, that the page attribution information of the first page is stored in the first server; The third server sends a first query request to the first server, where the first query request is used to query page attribution information of the first page, and the first query request carries identification information of the first page; The third server receives the first query response from the first server, where the first query response is used to indicate that a query corresponding to the first query request fails; The third server re-determines the page attribution information of the first page according to a preset redirection algorithm and stores it in the second server; The third server sends the first query request to the second server; The third server receives page attribution information of the first page from the second server.
16. The method according to claim 15, characterized in that The first query response carries identification information of the second server; Then the third server re-determines the page attribution information of the first page according to a preset redirection algorithm and stores it in the second server, including: The third server determines, based on the first query response, that the first page attribution information is stored in the second server.
17. The method according to claim 15 or 16, characterized in that Before the third server receives the first access request from the client, the method further includes: The third server receives a second notification message from the cluster management server, where the second notification message is used to notify the distributed database cluster to perform initialization processing, and the second notification message includes identification information of multiple second target servers, where the multiple second target servers include the third server; The third server calculates the identification information of each server in the multiple second target servers by using a consistent hash algorithm to obtain a hash value of each second target server, and maps the hash values of the multiple second target servers to a consistent hash ring to obtain the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring; The third server obtains a hash value of each preset storage space in a plurality of preset storage spaces, and maps the hash values of the plurality of preset storage spaces to the consistent hash ring to obtain corresponding positions of the hash values of the plurality of preset storage spaces on the consistent hash ring; The third server determines the initial states of the multiple preset storage spaces based on the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring and the corresponding positions of the hash values of the multiple preset storage spaces on the consistent hash ring; The third server marks the initial status of all or part of the preset storage spaces among the plurality of preset storage spaces in the third server as a valid status.
18. The method according to claim 17, characterized in that The third server determines, based on the identification information of the first page, that the page attribution information of the first page is stored in the first server, including: The third server calculates the identification information of the first page by using the consistent hash algorithm to obtain a hash value of the first page, maps the hash value of the first page to a consistent hash ring, and obtains a position of the hash value of the first page on the consistent hash ring; The third server determines that the page attribution information of the first page is stored in the first server based on the position of the hash value of the first page on the consistent hash ring and the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring.
19. The method according to claim 18, characterized in that The third server re-determines the page attribution information of the first page according to a preset redirection algorithm and stores it in the second server, including: The third server searches on the consistent hash ring in a preset direction based on the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring and taking the position of the hash value of the first server on the consistent hash ring as a starting point to determine the second server, wherein the corresponding position of the hash value of the second server on the consistent hash ring is after the first server and adjacent to the first server.
20. The method according to claim 15, characterized in that After the third server receives the first query response from the first server, the method further includes: The third server receives a first notification message from the cluster management server, the first notification message is used to notify the distributed database cluster to add a server and / or delete a server, the first notification message includes identification information of multiple first target servers, and the multiple first target servers include the second server; The third server re-determines, according to a preset redirection algorithm, that the first page attribution information is stored in the second server, including: The third server calculates the identification information of the first page by using a consistent hash algorithm to obtain a hash value of the first page, maps the hash value of the first page to a consistent hash ring, and obtains a position of the hash value of the first page on the consistent hash ring; The third server calculates the identification information of each server in the multiple first target servers by using a consistent hash algorithm to obtain a hash value of each first target server, and maps the hash values of the multiple first target servers to a consistent hash ring to obtain the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring; The third server determines that the page attribution information of the first page is stored in the second server based on the position of the hash value of the first page on the consistent hash ring and the corresponding positions of the hash values of the multiple first target servers on the consistent hash ring.
21. The method according to claim 20, characterized in that The method further comprises: The third server receives a second access request from the client, where the second access request is used to request access to a second page, and the second access request carries identification information of the second page; The third server determines, based on the identification information of the second page, that the page attribution information of the second page is stored in the second server; The third server sends a second query request to the second server, wherein the second query request is used to query the second Page attribution information of the page, the second query request carries identification information of the second page; The third server receives the second query response from the second server, where the second query response is used to indicate that a query corresponding to the second query request fails; The third server re-determines the page attribution information of the second page according to a preset redirection algorithm and stores it in the first server; The third server sends the second query request to the first server; The third server receives page attribution information of the second page from the first server.
22. The method according to claim 21, characterized in that The third server re-determines the page attribution information of the second page according to a preset redirection algorithm and stores it in the first server, including: The third server calculates the identification information of the second page by using the consistent hash algorithm to obtain a hash value of the second page, maps the hash value of the second page to the consistent hash ring, and obtains a position of the hash value of the second page on the consistent hash ring; The third server determines that the page attribution information of the second page is stored in the first server based on the position of the hash value of the second page on the consistent hash ring and the corresponding positions of the hash values of the multiple second target servers on the consistent hash ring.
23. A system for implementing page access, characterized in that: include: A first server for implementing the method as claimed in any one of claims 1 to 10, a second server for implementing the method as claimed in any one of claims 11 to 14, and a third server for implementing the method as claimed in any one of claims 15 to 22.
24. A distributed database cluster, characterized in that: include: A first server for implementing the method according to any one of claims 1 to 10, a second server for implementing the method according to any one of claims 11 to 14, and a third server for implementing the method according to any one of claims 15 to 22.
25. A server, characterized in that: include: A processor, wherein the processor is coupled to a memory, wherein the memory is used to store a computer program, and when the processor calls the computer program, the server executes the method as claimed in any one of claims 1 to 10, or executes the method as claimed in any one of claims 11 to 14, or executes the method as claimed in any one of claims 15 to 22.
26. A computer-readable storage medium, characterized in that: Used to store a computer program, the computer program comprising instructions for implementing the method according to any one of claims 1 to 10, or instructions for implementing the method according to any one of claims 11 to 14, or instructions for implementing the method according to any one of claims 15 to 22.
27. A computer program product, comprising instructions, characterized in that: When the instructions are executed on a computer, the computer is enabled to implement the method according to any one of claims 1 to 10, or the method according to any one of claims 11 to 14, or the method according to any one of claims 15 to 22.
Citation Information
Patent Citations
Method, server and system for realizing page access
CN119622129A
Distributed content scheduling method, scheduling system and central server
CN113238836A
Stateful service migration method and device, computer equipment and storage medium
CN113254159A
Hash algorithm-based service access method, apparatus and device, and medium
CN115840639A
Data processing method and device and storage medium
CN116028196A