A method for selecting an optimal transmission client based on a cluster file transmission system

Abstract

The present invention discloses a method for selecting an optimal transmission client based on a cluster file transmission system. Through the information interaction between the system client and the control server, the present invention can select a control client with the optimal performance, avoid problems caused by insufficient performance of the control client, and still quickly select an optimal control client in the case of abnormal operation or disconnection of the control client, ensuring the normal operation of the system and improving the stability and continuity of file transmission in the cluster file transmission system.

Description

Technical Field

[0001] The present invention relates to the technical field of cluster file transfer, and in particular to a method for selecting an optimal transfer client based on a cluster file transfer system. Background Art

[0002] In current highly available cluster file transfer systems, due to simple design, a client or a server only has one of the two functions of a transfer type and a control type. Whether it is a control type or a transfer type is determined during system deployment and will not change afterwards. In practical applications, it is found that in existing highly available cluster file transfer systems, transfer type clients are overly dependent on control type clients. Coupled with the unstable environment in which the clients are deployed, when uncontrollable external factors such as power outages, network disconnections, and poor machine performance occur in the client environment, if the affected clients include control type clients, then all transfer type clients cannot execute transfer tasks normally due to the lack of scheduling by the control type clients, and the entire file transfer system will be in a stagnant state.

[0003] Therefore, there are defects in the prior art and improvements are needed. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: to provide a method for selecting an optimal transfer client based on a cluster file transfer system, through negotiation of transfer capabilities between a client and a control type server, and dynamically select whether the client is of a transfer type or a control type by the control type server, so as to improve the stability and continuity of file transfer in the cluster file transfer system.

[0005] The technical solution of the present invention is as follows: to provide a method for selecting an optimal transfer client based on a cluster file transfer system, wherein the cluster file transfer system includes: a system client and a system server.

[0006] The system client includes: a control type client and at least one transfer type client; the control type client is used to manage the transfer type clients, including: transfer task status and transfer type client running status; the transfer type client is used to transfer files, report task transfer situations, and its own status.

[0007] The system server includes: a redis server, a transfer type server, and at least one control type server; the redis server is used to store tasks of all nodes and status information of all nodes, and all nodes include all nodes of the system client and all nodes of the system server; the control type server is used to manage the transfer type server and manage the function selection of the system client; the transfer type server is used to transfer files, report task transfer situations, and its own status.

[0008] A method for selecting an optimal transmission client based on a cluster file transfer system, comprising the following steps.

[0009] S1: After a transmission client is started, it registers its own attributes with the control server, including: whether it has the ability of the control class, IP address, listening port, and hardware information; the hardware information includes: the number of CPU cores, memory size, hard disk speed, and remaining available hard disk space.

[0010] S2: The control server processes the information reported by the transmission client; if the reported information does not have the ability to become a control client, no processing is done; if the reported information has the ability to become a control client, the information reported by the transmission client and the reporting time are inserted into the candidate control client queue, and sorted according to the hardware information reported by the transmission client; the sorting priority is the number of CPU cores > memory size > hard disk speed > remaining available hard disk space.

[0011] S3: All transmission clients go through steps S1 and S2 to ensure that transmission clients with control class capabilities report information to the control server; at the same time, the current control client will regularly report its own running status to the control server, including normal running and abnormal running. The range of the timing in this step is 20s - 1min.

[0012] S4: The control server regularly judges the status of the current control client and processes it; the range of the timing in this step is 20s - 1min.

[0013] If the current control client is in a normal running state and is ranked in the top twenty percent in the candidate control client queue, no processing is done.

[0014] If the current control client is in other states, then search for candidate control clients from the first one in the candidate control client queue downwards, select a candidate control client that meets the conditions, then set this candidate control client as the new control client, and then notify the current control client and all transmission clients.

[0015] S5: When the current control client receives a new control client, it switches its own status to a transmission client, no longer has control class functions, and registers with the new control client. At the same time, this transmission client enters steps S1 and S2; when a candidate control client receives that it has become the current control client, it changes its own status to a control client and provides control class functions; when a transmission client receives a new control client, it will register with the new control client.

[0016] By negotiating the transmission capabilities between the system client and the control server, the performance and status of the control client can be obtained in a timely manner. The control server dynamically selects whether the client is a transmission client or a control client, and promptly replaces the control server with poor performance and bad status, ensuring the normal operation of the system and enhancing the stability and continuity of file transmission in the cluster file transmission system.

[0017] In step S4, the types of other states of the current control client include: the current control client is in a normal running state, but the current control client is not in the top 20% of the candidate control client queue; or, the current control client is in an abnormal running state; or, the current control client has not reported its own running state for a long time, that is, it is in a disconnected state, and the long time is not less than 1 minute, preferably 3 minutes.

[0018] In step S4, when the time from when the candidate control client reports its own information to the current time is greater than one-third of the time from when the current control client last reported its own information to the current time, it is a qualified candidate control client. For example, the time when the candidate control client reports its own information is T1, the current time is T2, and the time when the current control client last reported its own information is T3. The time difference between T1 and T2 is t1, and the time difference between T3 and T2 is t2, and t1 > 1 / 3 * t2. This condition can exclude the candidate control client that has been newly added for a short time. The candidate control client that has been newly added for a short time is prone to instability, that is, the performance of the qualified candidate control client is stable and reliable enough.

[0019] Adopting the above solution, the present invention provides a method for selecting the optimal transmission client based on a cluster file transmission system. Through the information exchange between the system client and the control server, the control client with the optimal performance can be selected, avoiding the problems caused by insufficient performance of the control client. Even in the case of abnormal operation or disconnection of the control client, the optimal control client can still be quickly selected, ensuring the normal operation of the system and enhancing the stability and continuity of file transmission in the cluster file transmission system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Please refer to Figure 1 , this embodiment provides a method for selecting the optimal transmission client based on a cluster file transmission system, wherein the cluster file transmission system includes: a system client and a system server.

[0023] The system client includes: a control client and at least one transmission client; the control client is used to manage the transmission client, including: the transmission task status and the running status of the transmission client; the transmission client is used to transmit files, report the task transmission situation and its own status.

[0024] The system server includes: a redis server, a transmission server and at least one control server; the redis server is used to store the tasks of all nodes and the status information of all nodes, and all nodes include all nodes of the system client and all nodes of the system server; the control server is used to manage the transmission server and the function selection of the system client; the transmission server is used to transmit files, report the task transmission situation and its own status.

[0025] A method for selecting an optimal transmission client based on a cluster file transmission system includes the following steps.

[0026] S1: After the transmission client is started, it registers its own attributes with the control server, which include: whether it has the ability of the control class, IP address, listening port, hardware information; the hardware information includes: the number of CPU cores, memory size, hard disk speed, and remaining available hard disk space.

[0027] S2: The control server processes according to the information reported by the transmission client; if the reported information does not have the ability to become a control client, no processing is performed; if the reported information has the ability to become a control client, the information reported by the transmission client and the reporting time are inserted into the candidate control client queue, and sorted according to the hardware information reported by the transmission client; the sorting priority is the number of CPU cores > memory size > hard disk speed > remaining available hard disk space.

[0028] S3: All transmission clients go through steps S1 and S2 to ensure that the transmission clients with control class capabilities report information to the control server; at the same time, the current control client will regularly report its own running status to the control server, including normal running and abnormal running. The range of the regular time in this step is 20s - 1min.

[0029] S4: The control server regularly judges the status of the current control client and processes it; the range of the regular time in this step is 20s - 1min.

[0030] If the current control client is in a normal running state and is ranked in the top twenty percent in the candidate control client queue, no processing is performed.

[0031] If the current control client is in other states, search for candidate control clients from the first one in the candidate control client queue downwards, select a candidate control client that meets the conditions, then set this candidate control client that meets the conditions as the new control client, and then notify the current control client and all transfer clients.

[0032] S5: When the current control client receives a new control client, it switches its own state to a transfer client, no longer has control functions, and registers with the new control client. At the same time, this transfer client enters steps S1 and S2; when the candidate control client receives that it becomes the current control client, it changes its own state to a control client and provides control functions; when the transfer client receives a new control client, it will register with the new control client.

[0033] Through the negotiation of transmission capabilities between the system client and the control server, the performance and status of the control client can be obtained in a timely manner. The control server dynamically selects whether the client is a transfer client or a control client, and replaces the control server with poor performance and bad status in a timely manner, ensuring that the system can run normally and improving the stability and continuity of file transmission in the cluster file transmission system.

[0034] In step S4, the types of other states of the current control client include: the current control client is in a normal running state, but the current control client is not in the top 20% of the candidate control client queue; or, the current control client is in an abnormal running state; or, the current control client has not reported its own running state for a long time, that is, it is in a disconnected state. The long time is not less than 1 minute, preferably 3 minutes.

[0035] In step S5, it further includes: after the current control client becomes a transfer client, this transfer client enters steps S1 and S2.

[0036] In step S4, when the time since the candidate control client reported its own information is greater than one-third of the time since the current control client reported its own information for the last time, it is a candidate control client that meets the conditions.

[0037] In summary, the present invention provides a method for selecting the optimal transfer client based on a cluster file transmission system. Through the information interaction between the system client and the control server, the control client with the optimal performance can be selected, avoiding problems caused by insufficient performance of the control client. In the case of abnormal operation or disconnection of the control client, the optimal control client can still be quickly selected, ensuring that the system can run normally and improving the stability and continuity of file transmission in the cluster file transmission system.

[0038] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for selecting an optimal transmission client based on a cluster file transmission system, characterized in that, The cluster file transfer system includes: a system client and a system server; The system client includes: a control client and at least one transfer client; The system server includes: a redis server, a transfer server, and at least one control server; It includes the following steps: S1: After the transfer client starts, it registers its own attributes with the control server, including: whether it has the ability of a control client and hardware information; the hardware information includes: the number of CPU cores, the memory size, the hard disk speed, and the remaining available space of the hard disk; S2: The control server processes according to the information reported by the transfer client; if the reported information does not have the ability to become a control client, no processing is done; if the reported information has the ability to become a control client, the information reported by the transfer client is inserted into the candidate control client queue and sorted according to the hardware information reported by the transfer client; S3: All transfer clients go through steps S1 and S2 to ensure that transfer clients with control client capabilities report information to the control server; at the same time, the current control client will regularly report its own running status to the control server, including normal running and abnormal running; S4: The control server regularly judges the status of the current control client and processes it; If the current control client is in a normal running state and is ranked in the top twenty percent in the candidate control client queue, no processing is done; If the current control client is in other states, it searches for candidate control clients from the first one in the candidate control client queue downwards, selects a candidate control client that meets the conditions, then sets this candidate control client as the new control client, and then notifies the current control client and all transfer clients; S5: When the current control client receives a new control client, it switches its own status to a transfer client, no longer has control client functions, and registers with the new control client; when the candidate control client receives that it becomes the current control client, it changes its own status to a control client and provides control client functions; when the transfer client receives a new control client, it registers with the new control client.

2. The method for selecting an optimal transmission client based on a cluster file transmission system according to claim 1, wherein In step S4, the types of other states of the current control client include: the current control client is in a normal running state, but the current control client is not in the top twenty percent of the candidate control client queue; or, the current control client is in an abnormal running state; or, the current control client has not reported its own running status for a long time, and the long time is not less than 1 minute.

3. The method for selecting an optimal transmission client based on a cluster file transmission system according to claim 1, characterized in that In step S5, it also includes: after the current control client becomes a transfer client, this transfer client enters steps S1 and S2.

4. A method for selecting an optimal transmission client based on a cluster file transmission system according to claim 1, characterized in that In step S3, it also includes: if the reported information has the ability to become a control client, the reporting time of the transfer client is inserted into the candidate control client queue.

5. The method for selecting an optimal transmission client based on a cluster file transmission system according to claim 4, wherein In step S4, when the time from when the candidate control class client reports its own information to the current time is greater than one-third of the time from when the current control class client last reported its own information to the current time, it is a qualified candidate control class client.

6. The method for selecting an optimal transmission client based on a cluster file transmission system according to claim 1, wherein In step S2, the sorting priority is cpu core count > memory size > hard disk speed > available hard disk space.

7. A method for selecting an optimal transmission client based on a cluster file transmission system according to claim 1, wherein The self-attributes of the transport class client also include: ip address, listening port.

8. The method for selecting an optimal transmission client based on a cluster file transmission system according to claim 1, wherein In step S3, the timing range in this step is 20s - 1min; in step S4, the timing range in this step is 20s - 1min.

Images