Database middleware fragment migration method and computer equipment

Abstract

The invention relates to a fragment migration method for database middleware and computer equipment. In response to a migration event sent by the management and control node, entering an emigration persistence state, modifying and storing the state of a target fragment of an emigration node as an emigration state; if the emigration state is successfully stored, the state machine enters an emigration state and sends an emigration request to an emigration node, enters an immigration persistence state under the condition that an emigration success event is received, and modifies and stores the state of the target fragment as an immigration state; and if the immigration state is successfully stored, the state machine enters the immigration state, sends a first immigration request to the immigration node, enters a migration ending persistence state under the condition that a first immigration success event is received, and modifies and stores the state of the target fragment as a migration ending state. According to the embodiment of the invention, through the cooperative work of the state machine, the immigration node and the immigration node, the migration of the fragments between the service nodes is completed, the delay of the high-load service node is reduced, and the data access efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a shard migration method and computer equipment for database middleware. Background Art

[0002] Database middleware is located between the database and the client application system. It is mainly used to coordinate and manage database access and is a bridge for communication between the client and the database. At certain times, due to problems such as read and write hot spots, some of the service nodes in the database middleware may experience heavy request traffic, high load, and high latency, resulting in a poor user experience.

[0003] In order to balance the pressure of each service node, how to transfer part of the shard holding rights of the high-load service node to other service nodes to improve data access efficiency is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0004] Based on this, it is necessary to provide a shard migration method and computer device for database middleware that can transfer partial shard holding rights of high-load service nodes to other service nodes to improve data access efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a shard migration method for a database middleware, wherein the database middleware includes a management and control node and multiple service nodes, wherein the multiple service nodes include a migration-out node and a migration-in node, and the method is applied to a state machine in the management and control node; the method includes:

[0006] In response to the migration event sent by the control node, the migration-out persistence state is entered, and the state of the target shard of the migrated node is modified and stored as the migrated-out state;

[0007] If the migration status is successfully stored, the state machine enters the migration state and sends a migration request to the migration node, and enters the migration persistence state upon receiving a migration success event, and modifies and stores the state of the target shard as the migration state; the migration success event is sent after the migration node successfully sets the target shard to read-only mode based on the migration request;

[0008] If the migration status is stored successfully, the state machine enters the migration state and sends a first migration request to the migration node, and enters the migration end persistence state upon receiving the first migration success event, modifies and stores the status of the target shard as the migration end state; the first migration success event is sent after the migration node successfully loads the target shard in read-write mode based on the first migration request.

[0009] In one embodiment, the method further comprises:

[0010] If the migration end state is stored successfully, the state machine enters the migration end state and sends a first uninstallation request to the migration node, and enters the normal persistence state upon receiving the first uninstallation success event, modifies and stores the state of the target shard as a normal state; the first uninstallation success event is sent by the migration node when the target shard is successfully uninstalled based on the first uninstallation request;

[0011] If the normal state is stored successfully, the state machine enters the normal state.

[0012] In one embodiment, the method further comprises:

[0013] When receiving a retryable failure event sent by the migration node, the migration request or the first uninstallation request is resent to the migration node.

[0014] In one embodiment, the method further comprises:

[0015] When the state machine is in the migration state, if a non-retryable failure event sent by the migration node is received, the state machine enters the loading state and sends a loading request to the migration node; the loading request is used to instruct the migration node to reload the target shard.

[0016] In one embodiment, the migration node includes a first migration node and a second migration node; the method further includes:

[0017] If the state machine is in the migration state and receives a non-retryable failure event sent by the first migration node, or if the state machine is in the migration process and receives a migration node crash event sent by the first migration node, or if the state machine is in the migration process and receives a migration node crash event sent by the first migration node, then enter the migration persistence state to send a second migration request to the second migration node, and enter the migration end persistence state when receiving the second migration success event, and modify and store the state of the target shard as the migration end state;

[0018] Among them, the migration process includes the migration persistence state and the migration state, and the migration process includes the migration persistence state and the migration state; the second migration success event is sent after the second migration node successfully loads the target shard in read-write mode based on the second migration request.

[0019] In one embodiment, the number of the migrated node is one, and the method further includes:

[0020] If the state machine is in the migration state and receives a non-retryable failure event sent by the migration node, or if the state machine is in the migration process and receives a migration node crash event sent by the migration node, or if the state machine is in the migration end process and receives a migration node crash event sent by the migration node, then it enters the migration end abort state to send a second uninstallation request to the migration node, and enters the initial persistence state when receiving a second uninstallation success event, modifies and stores the state of the target shard as the initial state; the second uninstallation success event is sent by the migration node when the target shard is successfully uninstalled based on the second uninstallation request;

[0021] If the initial state is stored successfully, the state machine enters the initial state; the initial state is used to reallocate the target shard;

[0022] The migration end process includes the migration end persistence state and the migration end state.

[0023] In one embodiment, the method further comprises:

[0024] If the state machine is in the migration process and receives a migration node crash event sent by the migration node, it enters a migration aborted state to send a third uninstallation request to the migration node, and enters the initial persistence state upon receiving a third uninstallation success event, modifies and stores the state of the target shard as the initial state; the third uninstallation success event is sent by the migration node when it successfully uninstalls the target shard based on the third uninstallation request.

[0025] In one embodiment, the method further comprises:

[0026] If the state machine is in the process of migrating out and receives a migration node downtime event sent by the migration node, it enters the initial persistence state.

[0027] In one embodiment, the method further comprises:

[0028] If the state machine is in the migration end state and receives a migration node downtime event sent by the migration node, it enters the normal persistence state.

[0029] In a second aspect, the present application also provides a shard migration device for a database middleware, comprising:

[0030] A response module, configured to enter a migration-out persistence state in response to a migration event sent by the control node, and modify and store the state of the target shard of the migrated node as a migration-out state;

[0031] A migration module, for if the migration state is successfully stored, the state machine enters the migration state and sends a migration request to the migration node, and enters the migration persistence state upon receiving a migration success event, and modifies and stores the state of the target shard as the migration state; the migration success event is sent after the migration node successfully sets the target shard to read-only mode based on the migration request;

[0032] The first migration module is used to, if the migration status is stored successfully, then the state machine enters the migration state and sends the first migration request to the migration node, and enters the migration end persistence state upon receiving the first migration success event, modifies and stores the status of the target shard as the migration end state; the first migration success event is sent after the migration node successfully loads the target shard in read-write mode based on the first migration request.

[0033] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the method steps provided in the first aspect when executing the computer program.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method steps provided in the first aspect.

[0035] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the method steps provided in the first aspect when executed by a processor.

[0036] The shard migration method and computer device of the above database middleware enter the migration-out persistence state in response to the migration event sent by the control node, modify and store the state of the target shard of the migration-out node as the migration-out state; if the migration-out state is stored successfully, the state machine enters the migration-out state and sends a migration-out request to the migration-out node, and enters the migration-in persistence state when receiving the migration-out success event, modifies and stores the state of the target shard as the migration-in state; if the migration-in state is stored successfully, the state machine enters the migration-in state and sends the first migration-in request to the migration-in node, and enters the migration-end persistence state when receiving the first migration-in success event, modifies and stores the state of the target shard as the migration-end state; the migration-out success event is sent after the migration-out node successfully sets the target shard to read-only mode based on the migration-out request; the first migration-in success event is sent after the migration-in node successfully loads the target shard in read-write mode based on the first migration-in request. In the embodiment of the present application, the migration of shards between service nodes is effectively completed through the collaborative work of the state machine and the migration-in node, the delay of the high-load service node is reduced, and the data access efficiency is improved. Moreover, the service unavailability time is shortened. During the migration process, the data can be read almost all the time, and only part of the time it cannot be written, which reduces the client's perception of the migration process. At the same time, when multiple service nodes work together to complete this complex process, the completeness and correctness of the migration process are guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 An application environment diagram of a shard migration method of a database middleware in one embodiment;

[0039] Figure 2 It is an application environment diagram of a shard migration method of a database middleware in another embodiment;

[0040] Figure 3 A schematic diagram of a process flow of a shard migration method of a database middleware in one embodiment;

[0041] Figure 4 It is a flowchart of a shard migration method of a database middleware in another embodiment;

[0042] Figure 5 It is a flowchart of a shard migration method of a database middleware in another embodiment;

[0043] Figure 6 It is a structural block diagram of a shard migration device of a database middleware in one embodiment;

[0044] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] In the related art, the actual data of the data shard is stored on the service node. When the shard is migrated, the corresponding data needs to be migrated to another service node. In this case, while the data is being transmitted, the new write request is written to the new service node, and the new read request is read from the new service node. If the data is not found, the old node is accessed for search. However, this method requires the data to be migrated as well.

[0047] Alternatively, the database middleware service node periodically polls the load and other information of the underlying database to determine whether the database node needs to migrate shards to reduce the read and write pressure, and if necessary, sends migration instructions, etc. However, this method mainly solves the problem of high load of the underlying database, and migrates the data actually held by the underlying database, but does not solve the problem of high load of the database middleware service node and other problems that need to be migrated.

[0048] The shard migration method of the database middleware provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown in the figure, the database middleware is located between the database and the client application system. It is mainly used to shield the underlying details of the heterogeneous database and is a bridge for the client and the database to communicate. The user triggers the application system on the client to access the data stored in the database storage node to obtain the required data. Figure 2As shown in the figure, the database middleware includes a control cluster, a service node, and a status information node. Only one control node runs at the same time node. Among them, the service node holds the read and write permissions of each shard. When receiving the read and write request from the client, it performs the actual read and write operation on the corresponding database storage node according to the request, and then returns the operation result to the client. The control cluster contains multiple control nodes, which run in the master-slave mode. They are responsible for managing the shards and health status of each service node, checking and adjusting the delay and traffic of the service node to a reasonable range, and ensuring the availability of the database system. The status information node persistently stores the current status of each shard and the allocation relationship between each shard and the service node, and will not lose information due to service node downtime, power failure, etc. It is mainly used to restore the migration task after the control node downtime and other failures. The status information of a single shard includes: shard status, shard ID, original node information, new node information, etc. A state machine is a mathematical model used to describe the behavior of a system or program transitioning between different states. It consists of a set of states, transition conditions, and actions.

[0049] In an exemplary embodiment, Figure 3 As shown, a sharding migration method for database middleware is provided, and the method is applied to Figure 2 The state machine in the control node is used as an example to illustrate, including the following S301 to S303. Among them:

[0050] S301, in response to a migration event sent by a control node, entering a migration-out persistence state, and modifying and storing the state of a target shard of the migrated-out node as a migration-out state.

[0051] Among them, data sharding refers to dispersing the data stored in a single database into multiple databases or tables according to a certain dimension to achieve the effect of improving performance bottlenecks and availability. In this application, sharding refers to the data sharding obtained after dividing by key values, that is, the target sharding of this application is horizontal sharding.

[0052] In the embodiment of the present application, the control node monitors the traffic of multiple service nodes in real time, and when the traffic of a service node is detected to be greater than a threshold, a migration event is sent to the state machine in a normal state. Figure 4 As shown, when the state machine receives a migration event, it enters the migration persistence state. When the state machine is in the migration persistence state, the state of the target shard of the migration node is modified to the migration state, and the migration state of the target shard is written to the state information node for persistent storage.

[0053] S302, if the migration status is stored successfully, the state machine enters the migration state and sends a migration request to the migration node, and enters the migration persistence state after receiving the migration success event, modifies and stores the state of the target shard as the migration state; the migration success event is sent after the migration node successfully sets the target shard to read-only mode based on the migration request.

[0054] In an embodiment of the present application, after the state information node successfully writes the migration status of the target shard, it sends a persistence success event to the state machine. The state machine enters the migration state after receiving the persistence success event sent by the state information. When the state machine is in the migration state, a migration request is sent to the migration node. Based on the migration request, the migration node sets the read and write permissions of the target shard to read-only mode, and sends a migration success message to the control node after the change is successful. The control node converts and translates the migration success message to obtain a migration success event that can drive the state machine.

[0055] After receiving the migration success event, the state machine enters the migration persistence state and changes the state of the target shard to the migration state.

[0056] S303, if the migration status is stored successfully, the state machine enters the migration state and sends the first migration request to the migration node, and enters the migration end persistence state when receiving the first migration success event, modifies and stores the status of the target shard as the migration end state; the first migration success event is sent after the migration node successfully loads the target shard in read-write mode based on the first migration request.

[0057] In an embodiment of the present application, after the state information node successfully writes the migration status of the target shard, it sends a persistence success event to the state machine. The state machine enters the migration state after receiving the persistence success event sent by the state information. When the state machine is in the migration state, a migration request is sent to the migration node. Based on the migration request, the migration node sets the read and write permissions of the target shard to read-write mode, loads the target shard in read-write mode, and sends a first migration success message to the management and control node after the loading is successful. The management and control node converts and translates the first migration success message to obtain the first migration success event that can drive the state machine.

[0058] After receiving the first successful migration event, the state machine enters the migration end persistence state and changes the state of the target shard to the migration end state.

[0059] In the shard migration method of the above database middleware, in response to the migration event sent by the control node, the migration event enters the migration-out persistence state, and the state of the target shard of the migration-out node is modified and stored as the migration-out state; if the migration-out state is stored successfully, the state machine enters the migration-out state and sends a migration request to the migration-out node, and enters the migration-in persistence state when receiving the migration-out success event, and modifies and stores the state of the target shard as the migration-in state; if the migration-in state is stored successfully, the state machine enters the migration-in state and sends the first migration-in request to the migration-in node, and enters the migration-end persistence state when receiving the first migration-in success event, and modifies and stores the state of the target shard as the migration-end state; the migration-out success event is sent after the migration-out node successfully sets the target shard to read-only mode based on the migration-out request; the first migration-in success event is sent after the migration-in node successfully loads the target shard in read-write mode based on the first migration-in request. In the embodiment of the present application, the migration of shards between service nodes is effectively completed through the collaborative work of the state machine and the migration-in node, and the migration-in node reduces the delay of the high-load service node and improves the data access efficiency. Moreover, the service unavailability time is shortened. During the migration process, the data can be read almost all the time, and only part of the time it cannot be written, which reduces the client's perception of the migration process. At the same time, when multiple service nodes work together to complete this complex process, the completeness and correctness of the migration process are guaranteed.

[0060] In an exemplary embodiment, the method also includes: if the migration end state is stored successfully, the state machine enters the migration end state and sends a first uninstallation request to the migration node, and enters the normal persistence state upon receiving the first uninstallation success event, modifies and stores the state of the target shard as a normal state; the first uninstallation success event is sent when the migration node successfully uninstalls the target shard based on the first uninstallation request; if the normal state is stored successfully, the state machine enters the normal state.

[0061] In the present embodiment, continue as above Figure 4 As shown, after the state information node successfully writes the migration end state of the target shard, it sends a persistence success event to the state machine. The state machine enters the migration end state after receiving the persistence success event sent by the state information. When the state machine is in the migration end state, it sends a first uninstall request to the migration-out node. The migration-out node uninstalls the target shard based on the first uninstall request, and sends a first uninstall success message to the control node after the uninstall is successful. The control node converts and translates the first uninstall success message to obtain a first uninstall success event that can drive the state machine.

[0062] After receiving the first unloading success event, the state machine enters the normal persistence state and changes the state of the target shard to the normal state. After the state information node successfully writes the normal state of the target shard, it sends a persistence success event to the state machine. After receiving the persistence success event sent by the state information, the state machine enters the normal state.

[0063] In an embodiment of the present application, if the migration end state is stored successfully, the state machine enters the migration end state and sends a first uninstall request to the migration-out node, and enters the normal persistence state upon receiving the first uninstall success event, modifies and stores the state of the target shard as a normal state, thereby making the shard migration of the database middleware more complete.

[0064] In a complex environment with distributed multiple nodes, in addition to the above normal operation process, various abnormal problems may also occur. In order to ensure the correctness of the migration event, this application sets up processing methods for various abnormal events that occur at each moment of the migration process in the state machine:

[0065] In an exemplary embodiment, Figure 5 As shown, the method further includes: upon receiving a retryable failure event sent by the migrated node, resending a migrate request or a first uninstall request to the migrated node.

[0066] In the embodiments of the present application, Figure 5 As shown, in any of the above states, if a retry failure event sent by the migration node is received, the state machine jumps to the corresponding state and sends a request to the migration node.

[0067] Furthermore, the method also includes: when the state machine is in the migration state, if a non-retryable failure event sent by the migration node is received, entering the loading state and sending a loading request to the migration node; the loading request is used to instruct the migration node to reload the target shard.

[0068] In the present embodiment, continue as Figure 5 As shown in the figure, in the migration state, the state machine may receive a non-retryable failure event returned by the migration node. The reason for this is that the migration node may have changed the target shard to read-only mode, but due to timeout and other reasons, the migration success message was not sent back to the control node, so that the state machine did not receive the migration success event sent by the control node; it may also be that the migration node has not changed the read and write permissions of the target shard to read-only mode, or has not received the migration request sent by the state machine.

[0069] To ensure the correctness of the service and to guarantee the service availability as much as possible, after receiving the migration failure event, the state machine jumps to the loading state and sends a loading request to the migration node, requesting the migration node to reload. In this way, regardless of whether the migration node has migrated out of the target shard, it can be reloaded and continue to provide services for the target shard.

[0070] Further, the method also includes: if the state machine is in a migration state and receives a non-retryable failure event sent by a first migration node, or if the state machine is in a migration process and receives a migration node downtime event sent by the first migration node, or if the state machine is in a migration process and receives a migration node downtime event sent by the first migration node, then entering a migration persistence state to send a second migration request to a second migration node, and entering a migration end persistence state when receiving a second migration success event, and modifying and storing the state of the target shard as a migration end state;

[0071] Among them, the migration process includes migration into persistent state and migration into state, and the migration out process includes migration out persistent state and migration out state; the second migration success event is sent after the second migration node successfully loads the target shard in read-write mode based on the second migration request.

[0072] In the present embodiment, continue as Figure 5 As shown, if the state machine is in the migration state and receives a non-retryable failure event sent by the first migration node, or if the state machine is in the migration process and receives a migration node downtime event sent by the first migration node, or if the state machine is in the migration process and receives a migration node downtime event sent by the first migration node, then update the migration task information to the second migration node and re-enter the migration persistence state. Write the second migration node into the state of the target shard, that is, the new service node in the migration state of the target shard becomes the second migration node. After that, the state machine runs to the migration state and repeats the migration operation.

[0073] In a possible implementation, if the state machine is in the process of migration out and receives the migration node downtime event sent by the first migration node, the first migration node is not performing any operation at this time, and the migration node can be directly changed to the second migration node. For the convenience of operation, this event can also be postponed to the migration process.

[0074] Further, the method also includes: if the state machine is in a migration state and receives a non-retryable failure event sent by the migration node, or if the state machine is in a migration process and receives a migration node downtime event sent by the migration node, or if the state machine is in a migration end process and receives a migration node downtime event sent by the migration node, then entering a migration end suspension state to send a second uninstallation request to the migration node, and entering an initial persistence state when receiving a second uninstallation success event, modifying and storing the state of the target shard as an initial state; the second uninstallation success event is sent when the migration node successfully uninstalls the target shard based on the second uninstallation request; if the initial state is stored successfully, the state machine enters the initial state; the initial state is used to reallocate the target shard;

[0075] The migration end process includes the migration end persistence state and the migration end state.

[0076] In the present embodiment, continue as Figure 5 As shown, in the migration state, the state machine receives a non-retryable migration failure event returned by the migration node. If there is no alternative migration node, it jumps to the migration end suspension state. Since the migration node has not loaded the corresponding shard at this time, it only needs to make the migration node also unload the corresponding shard to ensure that no service node holds this shard. Therefore, in the migration end suspension state, the second unloading request is sent to the migration node through the control node, and the migration node unloads the target shard. After the unloading is successful, the target shard has no belonging relationship, and the state machine returns to the initial state, waiting for the next scheduling to reallocate the target shard.

[0077] Furthermore, during the migration process, if the state machine receives a migration node downtime event sent by the migration node, and there is no alternative migration node at this time, the state machine jumps to the migration end abort state, and sends a second uninstall request to the migration node through the control node, and the migration node uninstalls the target shard. After the uninstallation is successful, no service node holds this target shard, and the state machine directly jumps to the initial persistence state. After persistently storing the state information corresponding to the target shard, the state machine returns to the initial state and waits to be reallocated to a service node and loaded.

[0078] Similarly, when the state machine is in the migration end process and receives a migration node crash event sent by the migration node, it also enters the migration end suspension state and sends a second uninstallation request to the migration node through the control node to return to the initial state.

[0079] Furthermore, the method also includes: if the state machine is in the migration process and receives a migration node crash event sent by the migration node, it enters a migration abort state to send a third unloading request to the migration node, and enters the initial persistence state when receiving a third unloading success event, modifies and stores the state of the target shard as the initial state; the third unloading success event is sent when the migration node successfully unloads the target shard based on the third unloading request.

[0080] In the present embodiment, continue as Figure 5 As shown, during the migration process, the state machine receives a migration node crash event sent by the migration node. At this time, it is highly likely that the migration operation is not completed. In this case, the state machine jumps to the migration suspension state and sends a third unloading request to the migration node. If the migration node has already loaded the target shard, the unloading operation is performed. After the unloading is successful, the third unloading success event is sent to the state machine; if the migration node has not actually loaded the target shard, the third unloading success event is directly sent to the state. After that, no service node holds this target shard, and the state machine directly jumps to the initial persistence state. After persistently storing the state information corresponding to the target shard, the state machine waits in the initial state to be reallocated to a service node and loaded.

[0081] Furthermore, the method further includes: if the state machine is in a migration process and receives a migration node downtime event sent by the migration node, then entering an initial persistence state.

[0082] In the present embodiment, continue as Figure 5 As shown, during the migration process, the state machine receives a migration node crash event sent by the migration node, and the migration node loses the right to hold the target shard. At this time, it can be considered that no service node holds this target shard. The state machine directly jumps to the initial persistence state, and after persistently storing the state information corresponding to the target shard, the state machine waits in the initial state to be reallocated to a service node and loaded.

[0083] Furthermore, the method further includes: if the state machine is in the migration end state and receives a migration node downtime event sent by the migration node, entering a normal persistence state.

[0084] In the present embodiment, continue as Figure 5 As shown in the figure, during the migration end process, the state machine receives the node downtime event sent by the migration node, which is equivalent to the migration node uninstalling all shards, that is, the migration node completes the target shard uninstallation task, and there is no need to resend the uninstallation request to the migration node. The state machine directly jumps to the normal persistence state according to the successful migration process. After the state information corresponding to the target shard is persistently stored, the state machine jumps to the normal state.

[0085] The embodiment of the present application can cover and handle various possible abnormal situations when multiple service nodes work together to complete the complex process of smooth migration, ensuring the completeness and correctness of the migration process and ensuring that the system provides database functions normally. Moreover, the state machine can reliably support smooth migration sharding, effectively reducing the client's perception of the migration process and improving the availability of the application system.

[0086] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0087] Based on the same inventive concept, the embodiment of the present application also provides a shard migration device for database middleware for implementing the shard migration method for database middleware involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the shard migration device for one or more database middleware provided below can refer to the limitations of the shard migration method for database middleware above, and will not be repeated here.

[0088] In an exemplary embodiment, Figure 6 As shown, a shard migration device for database middleware is provided, comprising: a response module 11, a migration module 12 and a migration module 13, wherein:

[0089] A response module 11 is used to enter a migration-out persistence state in response to a migration event sent by a control node, and to modify and store the state of a target shard of the migrated node as a migration-out state;

[0090] The migration module 12 is used for, if the migration state storage is successful, the state machine enters the migration state and sends a migration request to the migration node, and enters the migration persistence state upon receiving a migration success event, and modifies and stores the state of the target shard as the migration state; the migration success event is sent after the migration node successfully sets the target shard to read-only mode based on the migration request;

[0091] The first migration module 13 is used to, if the migration status is successfully stored, the state machine enters the migration state and sends the first migration request to the migration node, and enters the migration end persistence state upon receiving the first migration success event, modifies and stores the status of the target shard as the migration end state; the first migration success event is sent after the migration node successfully loads the target shard in read-write mode based on the first migration request.

[0092] In one embodiment, the shard migration device of the database middleware further includes:

[0093] A first unloading module, which is used for, if the migration end state is successfully stored, the state machine enters the migration end state and sends a first unloading request to the migration-out node, and enters the normal persistence state upon receiving the first unloading success event, and modifies and stores the state of the target shard as a normal state; the first unloading success event is sent when the migration-out node successfully unloads the target shard based on the first unloading request;

[0094] The first entry module is used for the state machine to enter the normal state if the normal state storage is successful.

[0095] In one embodiment, the shard migration device of the database middleware further includes:

[0096] The second unloading module is used to resend the migration request or the first unloading request to the migration node when receiving a retryable failure event sent by the migration node.

[0097] In one embodiment, the shard migration device of the database middleware further includes:

[0098] The loading module is used to enter the loading state and send a loading request to the migrating node when the state machine is in the migrating state and receives a non-retryable failure event sent by the migrating node; the loading request is used to instruct the migrating node to reload the target shard.

[0099] In one embodiment, the shard migration device of the database middleware further includes:

[0100] A second migration module is used to enter a migration persistence state if the state machine is in a migration state and receives a non-retryable failure event sent by the first migration node, or if the state machine is in a migration process and receives a migration node crash event sent by the first migration node, or if the state machine is in a migration process and receives a migration node crash event sent by the first migration node, to send a second migration request to the second migration node, and enter a migration end persistence state when receiving a second migration success event, and modify and store the state of the target shard as a migration end state;

[0101] Among them, the migration process includes migration into persistent state and migration into state, and the migration out process includes migration out persistent state and migration out state; the second migration success event is sent after the second migration node successfully loads the target shard in read-write mode based on the second migration request.

[0102] In one embodiment, the shard migration device of the database middleware further includes:

[0103] A third unloading module is used to enter a migration end suspension state if the state machine is in a migration state and receives a non-retryable failure event sent by the migration node, or if the state machine is in a migration process and receives a migration node crash event sent by the migration node, or if the state machine is in a migration end process and receives a migration node crash event sent by the migration node, to send a second unloading request to the migration node, and enter an initial persistence state upon receiving a second unloading success event, modify and store the state of the target shard as the initial state; the second unloading success event is sent when the migration node successfully unloads the target shard based on the second unloading request;

[0104] The second entry module is used for the state machine to enter the initial state if the initial state storage is successful; the initial state is used to reallocate the target shard;

[0105] The migration end process includes the migration end persistence state and the migration end state.

[0106] In one embodiment, the shard migration device of the database middleware further includes:

[0107] The fourth unloading module is used to enter the migration suspension state if the state machine is in the migration process and receives the migration node downtime event sent by the migration node, so as to send the third unloading request to the migration node, and enter the initial persistence state when receiving the third unloading success event, modify and store the state of the target shard as the initial state; the third unloading success event is sent when the migration node successfully unloads the target shard based on the third unloading request.

[0108] In one embodiment, the shard migration device of the database middleware further includes:

[0109] The third entry module is used to enter the initial persistence state if the state machine is in the migration process and receives a migration node downtime event sent by the migration node.

[0110] In one embodiment, the shard migration device of the database middleware further includes:

[0111] The fourth entry module is used to enter the normal persistence state if the state machine is in the migration end state and receives the migration node downtime event sent by the migration node.

[0112] Each module in the shard migration device of the above database middleware can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0113] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store relevant data of the sharding migration method of the database middleware. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a sharding migration method of the database middleware is implemented.

[0114] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0115] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of any of the above method embodiments when executing the computer program.

[0116] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above method embodiments are implemented.

[0117] In one embodiment, a computer program product is provided, including a computer program, which implements the steps of any of the above method embodiments when executed by a processor.

[0118] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0119] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0121] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A shard migration method for database middleware, characterized in that: The database middleware includes a management and control node and multiple service nodes, the multiple service nodes include an outgoing node and an incoming node, and the method is applied to a state machine in the management and control node; the method includes: In response to the migration event sent by the control node, the migration-out persistence state is entered, and the state of the target shard of the migrated node is modified and stored as the migrated-out state; If the migration status is successfully stored, the state machine enters the migration state and sends a migration request to the migration node, and enters the migration persistence state upon receiving a migration success event, and modifies and stores the state of the target shard as the migration state; the migration success event is sent after the migration node successfully sets the target shard to read-only mode based on the migration request; If the migration status is stored successfully, the state machine enters the migration state and sends a first migration request to the migration node, and enters the migration end persistence state upon receiving the first migration success event, modifies and stores the status of the target shard as the migration end state; the first migration success event is sent after the migration node successfully loads the target shard in read-write mode based on the first migration request.

2. The method according to claim 1, characterized in that The method further comprises: If the migration end state is stored successfully, the state machine enters the migration end state and sends a first uninstallation request to the migration node, and enters the normal persistence state upon receiving the first uninstallation success event, modifies and stores the state of the target shard as a normal state; the first uninstallation success event is sent by the migration node when the target shard is successfully uninstalled based on the first uninstallation request; If the normal state is stored successfully, the state machine enters the normal state.

3. The method according to claim 2, characterized in that The method further comprises: When receiving a retryable failure event sent by the migration node, the migration request or the first uninstallation request is resent to the migration node.

4. The method according to claim 1, characterized in that: The method further comprises: When the state machine is in the migration state, if a non-retryable failure event sent by the migration node is received, the state machine enters the loading state and sends a loading request to the migration node; the loading request is used to instruct the migration node to reload the target shard.

5. The method according to claim 1, characterized in that The migration node includes a first migration node and a second migration node; the method further includes: If the state machine is in the migration state and receives a non-retryable failure event sent by the first migration node, or if the state machine is in the migration process and receives a migration node crash event sent by the first migration node, or if the state machine is in the migration process and receives a migration node crash event sent by the first migration node, then enter the migration persistence state to send a second migration request to the second migration node, and enter the migration end persistence state when receiving the second migration success event, and modify and store the state of the target shard as the migration end state; Among them, the migration process includes the migration persistence state and the migration state, and the migration process includes the migration persistence state and the migration state; the second migration success event is sent after the second migration node successfully loads the target shard in read-write mode based on the second migration request.

6. The method according to claim 5, characterized in that The number of the migrated node is one, and the method further includes: If the state machine is in the migration state and receives a non-retryable failure event sent by the migration node, or if the state machine is in the migration process and receives a migration node crash event sent by the migration node, or if the state machine is in the migration end process and receives a migration node crash event sent by the migration node, then it enters the migration end abort state to send a second uninstallation request to the migration node, and enters the initial persistence state when receiving a second uninstallation success event, modifies and stores the state of the target shard as the initial state; the second uninstallation success event is sent by the migration node when the target shard is successfully uninstalled based on the second uninstallation request; If the initial state is stored successfully, the state machine enters the initial state; the initial state is used to reallocate the target shard; The migration end process includes the migration end persistence state and the migration end state.

7. The method according to claim 6, characterized in that The method further comprises: If the state machine is in the migration process and receives a migration node crash event sent by the migration node, it enters a migration aborted state to send a third uninstallation request to the migration node, and enters the initial persistence state upon receiving a third uninstallation success event, modifies and stores the state of the target shard as the initial state; the third uninstallation success event is sent by the migration node when it successfully uninstalls the target shard based on the third uninstallation request.

8. The method according to claim 6, characterized in that The method further comprises: If the state machine is in the process of migrating out and receives a migration node downtime event sent by the migration node, it enters the initial persistence state.

9. The method according to claim 2, characterized in that: The method further comprises: If the state machine is in the migration end state and receives a migration node downtime event sent by the migration node, it enters the normal persistence state.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images