Management node negotiation method for small-scale embedded computing cluster systems

Abstract

The present invention discloses a management node negotiation method suitable for a small-scale embedded computing cluster system. In the method, the BMC in the health subsystem is used to monitor the online status of the embedded computer single board, and the management node negotiation is dynamically triggered according to the registration and offline events of the computer single board. The health management master node SMC specifies the management board in the system according to the registration order of the online computer single board, and notifies the cluster management software to set the operation mode to "management node", thereby realizing the dynamic generation of management nodes in the initial power-on, hot plug and other scenarios, ensuring the high availability of the system, while reducing the board types and reducing the maintenance cost.

Description

Technical Field

[0001] The invention belongs to the field of computer cluster management software, and relates to a management node negotiation method for a computing cluster system, and in particular to a management node negotiation method suitable for a small-scale embedded computing cluster system. Background Art

[0002] With the development of edge computing technology, the demand for small-scale embedded computing cluster systems in application scenarios such as vehicle-mounted, airborne, and single-soldier backpacks is increasing. A typical small-scale embedded computing cluster system generally includes at most more than ten single-board computer modules, 1 storage module, 1-2 power modules, 1 network switching module, bottom board, chassis or rack and other components. The single-board computer modules are interconnected through high-speed Ethernet, one of which is a management board and the others are computing boards. The management board dispatches task requests to different computing boards to achieve cluster computing. Each module in the system has a health management daughter card onboard that can communicate with the CPU. The daughter card is equipped with health management firmware and can work independently. It forms a health subnet through network interfaces such as CAN or IIC, forming an independent health subsystem. The health subsystem is configured with one health management master node (SMC) by default, and the other nodes are health management slave nodes (BMC). BMC is responsible for collecting health information of the module, receiving and executing instructions issued by SMC, and controlling the startup and shutdown of the single-board computer. SMC is responsible for aggregating and analyzing health information, issuing instructions to BMC, and providing network interfaces to interact with other systems.

[0003] Due to the limited computing power of embedded computers, the overall performance of small-scale embedded computing clusters is not strong. At the same time, in typical application scenarios, there are special requirements for system real-time performance and support for computing resources of multiple architectures, which makes it difficult to deploy popular commercial cluster management software such as K8S, and a simplified cluster management software architecture must be adopted. Usually, in order to avoid implementing complex algorithms such as management board election, the method of pre-dividing the single-board computer board into two types of boards, management board and computing board, is adopted, and even the slots and IP addresses are specified for the management board and computing board. The simplified solution means that when assembling and maintaining the system, it is necessary to confirm that the single-board computer matches the slot before inserting it into the cluster, otherwise the cluster may not work. At the same time, due to the increase in the physical types of boards, the types and quantities of spare parts for maintenance also increase, which increases the maintenance cost. Summary of the invention

[0004] The purpose of the present invention is to provide a method for dynamically managing and adjusting the board type of a single-board computer module and dynamically generating management nodes by negotiating with a health management subsystem and cluster management software, in order to solve the problems of low assembly efficiency and a large number of types and quantities of maintenance spare parts in the current small-scale embedded computing cluster system using static board types, slots and IPs as fixed management node methods.

[0005] The technical solution to achieve the purpose of the present invention is: a management node negotiation method suitable for a small-scale embedded computing cluster system, the method comprising:

[0006] Computer board online status monitoring: Use the health subsystem to monitor the online status of computer boards;

[0007] Management board negotiation: The health management master node, SMC, dynamically specifies the management board based on the online status of the computer board.

[0008] Furthermore, the computer board online status monitoring specifically includes the following steps:

[0009] (1) The health management master node, i.e., SMC, is deployed on other modules other than computer boards, and the health management slave node, i.e., BMC, is deployed on computer boards;

[0010] (2) After the BMC is powered on, it registers with the SMC. After successful registration, it controls the computer module on this board to start up.

[0011] (3) The cluster management software periodically queries the BMC of the board for the board type. The BMC determines whether the cluster management software is online based on the query message interval.

[0012] (4) The BMC periodically reports the online status of the cluster management software to the SMC. The SMC determines the online status of the computer board based on the communication interval with the BMC or the cluster management software offline message.

[0013] Further, the SMC determines the online status of the computer board according to the communication interval with the BMC or the offline message of the cluster management software, specifically:

[0014] If the SMC does not receive the "cluster management software online" message reported by the BMC within the specified time, or directly receives the "cluster management software offline" message, the SMC determines that the computer board is offline, otherwise it determines that the computer board is online.

[0015] Furthermore, the management board negotiation specifically includes the following steps:

[0016] (1) Initial power-on

[0017] The SMC designates a certain computer board as a management board and the others as computing boards based on the BMC registration status;

[0018] The SMC notifies the BMC of the assigned board type through the health bus. The BMC updates the board type information of the current board, starts the single board computer, and the cluster management software is automatically executed.

[0019] After the cluster management software is started, it obtains the board type from the onboard BMC and sets the software running mode according to the board type. That is, the cluster software on the computer board of the management board type runs in the "management node" mode, and the cluster software on the computer board of the computing board type runs in the "computing node" mode.

[0020] The cluster management software in "management node" mode regularly broadcasts its management node identity over the network;

[0021] (2) Management board offline

[0022] The SMC determines that the management board is offline and modifies the slot registration sequence list;

[0023] The SMC selects the next management board from the slot registration order list according to the management board negotiation rules;

[0024] SMC notifies BMC to update the board type;

[0025] The cluster management software periodically queries the onboard BMC for the board type. When it finds that the board type has changed from "computing board" to "management board", it changes the operation mode to "management node" and broadcasts its own identity.

[0026] (3) The computing board is offline

[0027] The SMC determines that the computing board is offline and modifies the slot registration sequence list;

[0028] The SMC notifies the "management node" that the cluster management software can now calculate the resource slot number;

[0029] The “management node” cluster manages the software update resource pool;

[0030] (4) Calculation board online

[0031] SMC determines that the computing board is online and modifies the slot registration sequence list;

[0032] The SMC notifies the "management node" that the cluster management software can now calculate the resource slot number;

[0033] The "management node" cluster manages the software update resource pool.

[0034] Furthermore, the SMC determines whether the single board computer is offline or online, specifically:

[0035] The cluster management software periodically queries the onboard BMC for the board type;

[0036] The BMC determines whether the single board computer is offline or online according to the number of query messages received within a specified time. Specifically, if the number of query messages received within a specified time is greater than a preset threshold, the computer is determined to be online, otherwise the computer is determined to be offline.

[0037] The BMC periodically reports the board type status information to the SMC.

[0038] Further, the SMC determines that the management board is offline, and modifies the slot registration sequence list, specifically: the SMC deletes the slot corresponding to the management board in the "slot registration sequence list", sends a corresponding offline processing command to the BMC, and initiates a system resource update;

[0039] The SMC determines that the computing board is offline and modifies the slot registration sequence list, specifically: the SMC sends the current online status information of each slot to the "management node", and the "management node" deletes the corresponding computing resources;

[0040] The SMC determines that the computing board is online and modifies the slot registration sequence list. Specifically, the SMC sends the current online status information of each slot to the "management node", and the "management node" adds the corresponding computing resources.

[0041] Furthermore, the next management board is selected according to the management board negotiation rules, specifically: according to the "slot registration order linked list", the BMC on the single-board computer in the first slot of the SMC in the linked list sends a board type change message to update the board type from "computing board" to "management board".

[0042] Compared with the prior art, the present invention has the following significant advantages: it makes full use of the existing health subsystem in the cluster system and simplifies the management node assistance mechanism of the cluster management software. When the single-board computer is connected to the cluster system, the onboard BMC registers with the SMC, and the SMC dynamically allocates the board type of "management board" or "computing board" to the single-board computer according to a certain algorithm. The cluster management software dynamically configures it to work in the "management node" or "computing node" mode by regularly querying the board type of this board. While realizing the dynamic configuration of the cluster "management node", it will not significantly increase the difficulty of cluster management software development and performance overhead, and reduce the number of physical types of boards, significantly improving assembly efficiency and maintenance costs.

[0043] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a typical small-scale embedded computing cluster system software deployment diagram of the present invention in one embodiment.

[0045] Figure 2 The figure is a sequence diagram of an online negotiation algorithm for a single board computer in one embodiment.

[0046] Figure 3 It is a sequence diagram of the management node offline negotiation algorithm in one embodiment.

[0047] Figure 4 The figure is a sequence diagram of a computing node offline negotiation algorithm in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] In one embodiment, in combination Figure 1 The management node negotiation method for a small-scale embedded computing cluster system provided by the present invention involves a typical deployment relationship among cluster management software, SMC, and BMC, including:

[0050] Cluster management software: Deployed on the management board and computing board. When running on the management board, it works in the "management node" mode; when running on the computing board, it works in the "computing node" mode. The cluster management software can communicate with the onboard BMC through the internal interface inside the module, and can be connected to the outside through the LAN and can communicate with the SMC.

[0051] SMC: Typically deployed on the switch board onboard health daughter card, with health bus interface and network communication interface. It can communicate with BMC through the health bus interface, and can also interact with cluster management software through the network interface.

[0052] BMC: Typically deployed on an onboard health daughter card other than the switch board, with a health bus interface that can communicate with the SMC. It also has an internal interface that communicates with the CPU, enabling interaction with the computer on this board.

[0053] The implementation of this method involves the collaboration of two software, namely the negotiation algorithm between the health management firmware running in the health management subsystem and the cluster management software running on the single-board computer. The negotiation algorithm defaults to SMC going online first, and the SMC should be mounted on the module that has priority access to the system, such as the storage module, network switching module or bottom board, and the access address should be fixed. The SMC should not be mounted on the computer single board.

[0054] The specific steps of the collaborative interaction of the two software are as follows:

[0055] In the first step, when the single-board computer module is powered on, the health subsystem is powered on first. After the health subsystem enters the working state, the onboard BMC first reads the slot number information of this board and converts the health subnet address of the local BMC according to the slot number. The BMC registers the board type with the SMC through the health subnet, including the slot number, single-board computer and other information. When the SMC receives the registration message sent by the onboard BMC of the single-board computer, it adds the slot number to the end of the "slot registration sequence list". When the "slot registration sequence list" is updated, the SMC sends a notification to the onboard BMC of the single-board computer on the first slot in the "slot registration sequence list" to inform that the board type is "management board". The SMC notifies the onboard BMC of the single-board computer on the other slots except the first one in the "slot registration sequence list" that the board type is "computing board". The SMC always maintains the "slot registration sequence list". After receiving the notification of the specified board type, the BMC controls the single-board computer to power on and start.

[0056] In the second step, after the single-board computer is powered on, the cluster management software automatically runs. The cluster management software first communicates with the onboard BMC to obtain the slot number, converts the unique IP address according to the slot number, and configures the IP. Then it communicates with the onboard BMC to obtain the board type. If the board type received in the reply is legal and is a "management board", the cluster management software sets its working mode to "management node" and runs; if the board type received in the reply is legal and is a "computing board", the cluster management software sets its working mode to "computing node" and runs; if no legal board type is received, the cluster management software suspends or performs other processing.

[0057] In the third step, the "management node" regularly broadcasts the slot information of the management board in the local area network. The "computing node" converts the IP address of the management board according to the received slot information, establishes a network link with the "management node", and interacts with the "management node".

[0058] Step 4: The "management node" periodically queries the BMC of this board for the board type. If the onboard BMC receives the query command within the specified time, it will immediately send an "online message" to the SMC to notify the SMC that the single-board computer in the slot is online. If the onboard BMC does not receive the board type query message from the cluster management software within the specified time, or detects that the health status of the single-board computer is abnormal, it will immediately send an "offline message" to the SMC to notify the SMC that the single-board computer in the slot is offline.

[0059] Step 5: If the SMC does not receive an "online message" from the BMC on a certain slot for a period of time, or if the SMC receives an "offline message" from the BMC on a certain slot, the SMC deletes the unit corresponding to the slot in the "slot registration sequence list" and initiates a system resource update. If the slot was originally a computing board, the online status information of the current slots is sent to the "management node" so that the "management node" can delete the corresponding computing resources; if the slot was originally a management board, according to the "slot registration sequence list", the SMC onboard BMC on the first slot in the list sends a board type change message to update the board type from "computing board" to "management board".

[0060] Step 6: When the "management node" receives the message that the system resource pool status has been updated, it adjusts the task scheduling strategy and no longer schedules tasks to the offline computing board; when the "computing node" queries the BMC and finds that the board type has changed from "computing board" to "management board", it switches to the "management node" working mode and broadcasts its slot number regularly. Other computing boards receive the broadcast and immediately obey the management of the new management board.

[0061] Specifically, in one embodiment, a management node negotiation method applicable to a small-scale embedded computing cluster system is provided, the method comprising the following steps:

[0062] (1) Combination Figure 2 , single board computer online negotiation:

[0063] The onboard BMC runs and obtains the slot number of the current slot through the hardware identification on the bottom board slot.

[0064] The onboard BMC initiates a registration request to the SMC and sends the slot number and card type of the board to the SMC through the health bus.

[0065] After receiving the registration request, the SMC parses the request message. If the board type is "single board computer", the slot number is added to the end of the "slot registration sequence list". The slot registration sequence list is a one-way linked list. When the list is empty, the newly inserted slot number is in the first unit of the list.

[0066] The SMC replies to the BMC that initiated the registration with the board "logical" type. If the slot is in the first unit of the "slot registration sequence list", it replies "management board", otherwise it replies "computing board".

[0067] After the BMC receives the board type reply, it considers the registration successful, updates the board type information of the board, starts the single board computer, and automatically executes the cluster management software.

[0068] The cluster management software first sends a query message to the BMC to query the slot number of the board. After receiving the slot number replied by the BMC, it calculates the IP address of the board and performs network IP configuration.

[0069] The cluster management software then sends a query to the BMC to inquire about the board type. After receiving the board type reply from the BMC, the cluster management software configures the operating mode according to the board type. If the received board type is "management board", the cluster management software's working mode is set to "management node"; if the received board type is "computing board", the cluster management software's working mode is set to "computing node".

[0070] (2) Combination Figure 3 , the management board negotiates offline to generate a new management board

[0071] Regardless of the working mode, the cluster management software will periodically query the onboard BMC for status information such as the board type of the board. Based on the number of query messages received within a certain period of time, the BMC determines whether the single board computer is offline or online, and regularly reports the status to the SMC.

[0072] When the SMC receives a message that the single board computer in a slot is online, no processing is required.

[0073] When the SMC receives a message that the single-board computer in a slot is offline, or receives a specified number of status report messages from the BMC in the slot within a certain period of time, it deletes the corresponding unit of the corresponding slot number in the "slot registration sequence list" and sends the corresponding offline processing command to the BMC. The BMC executes the command. After the board is offline, it needs to be re-registered before it can enter the online state.

[0074] If the SMC determines that the offline slot is a "management board", it reads the slot number information of the first unit in the "slot registration sequence list" and notifies it to update the board type to "management board".

[0075] The cluster management software periodically queries the onboard BMC for status information such as the board type. When it finds that the board type is updated to "management board", it switches the software operation mode to "management node".

[0076] The "management node" regularly broadcasts information such as the local slot number in the local area network. After other "computing nodes" receive the broadcast and find that the "management node" has been updated, they re-establish communication links with the "management node".

[0077] (3) Combination Figure 4 , computing boards negotiate offline to update system computing resources

[0078] Regardless of the working mode, the cluster management software will periodically query the onboard BMC for status information such as the board type. Based on the number of query messages received within a certain period of time, the BMC determines whether the single board computer is offline or online, and regularly reports the status to the SMC.

[0079] When the SMC receives a message that the single board computer in a slot is online, no processing is required.

[0080] When the SMC receives a message that the single-board computer in a slot is offline, or receives a specified number of status report messages from the BMC in the slot within a certain period of time, it deletes the corresponding unit of the corresponding slot number in the "slot registration sequence list" and sends the corresponding offline processing command to the BMC. The BMC executes the command. After the board is offline, it needs to be re-registered before it can enter the online state.

[0081] If the SMC determines that the offline slot is a "computing board", it will send the online status information of all current slots to the "management node" through the local area network.

[0082] After receiving the online status information of all current slots, the "management node" updates the system resource pool, changes the scheduling strategy, and no longer allocates tasks to the "computing nodes" in the offline slots.

[0083] In summary, the method of the present invention uses the BMC in the health subsystem to monitor the online status of the embedded computer board, and dynamically triggers the management node negotiation according to the computer board registration and offline events. The BMC specifies the management board in the system according to the registration order of the online computer board, and notifies the cluster management software to set the operation mode to "management node", thereby realizing the dynamic generation of management nodes in the initial power-on, hot plug and other scenarios, ensuring the high availability of the system, while reducing the board types and maintenance costs.

[0084] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A management node negotiation method suitable for a small-scale embedded computing cluster system, characterized in that: The method comprises: Computer board online status monitoring: Use the health subsystem to monitor the online status of computer boards; Management board negotiation: The health management master node, SMC, dynamically designates the management board based on the online status of the computer board; The computer board online status monitoring specifically comprises the following steps: (1) The health management master node, i.e., SMC, is deployed on other modules other than computer boards, and the health management slave node, i.e., BMC, is deployed on computer boards; (2) After the BMC is powered on, it registers with the SMC. After successful registration, it controls the computer module on this board to start up. (3) The cluster management software periodically queries the BMC of the board for the board type. The BMC determines whether the cluster management software is online based on the query message interval. (4) The BMC periodically reports the online status of the cluster management software to the SMC. The SMC determines the online status of the computer board based on the communication interval with the BMC or the cluster management software offline message. The management board negotiation specifically includes the following steps: (1) Initial power-on The SMC designates a certain computer board as a management board and the others as computing boards based on the BMC registration status; The SMC notifies the BMC of the assigned board type through the health bus. The BMC updates the board type information of the current board, starts the single board computer, and the cluster management software is automatically executed. After the cluster management software is started, it obtains the board type from the onboard BMC and sets the software running mode according to the board type. That is, the cluster software runs in the "management node" mode on the computer board of the management board type, and the cluster software runs in the "computing node" mode on the computer board of the computing board type. The cluster management software in "management node" mode regularly broadcasts its management node identity over the network; (2) Management board offline The SMC determines that the management board is offline and modifies the slot registration sequence list; The SMC selects the next management board from the slot registration order list according to the management board negotiation rules; SMC notifies BMC to update the board type; When the cluster management software periodically queries the onboard BMC for the board type, if it finds that the board type has changed from "computing board" to "management board", it changes the operation mode to "management node" and broadcasts its own identity; (3) The computing board is offline The SMC determines that the computing board is offline and modifies the slot registration sequence list; SMC notifies the "management node" that the cluster management software can now calculate the resource slot number; "Management Node" cluster management software update resource pool; (4) Calculation board online SMC determines that the computing board is online and modifies the slot registration sequence list; SMC notifies the "management node" that the cluster management software can now calculate the resource slot number; The "management node" cluster manages the software update resource pool.

2. The management node negotiation method applicable to a small-scale embedded computing cluster system according to claim 1, characterized in that: The SMC determines the online status of the computer board according to the communication interval with the BMC or the offline message of the cluster management software, specifically: If the SMC does not receive the "cluster management software online" message reported by the BMC within the specified time, or directly receives the "cluster management software offline" message, the SMC determines that the computer board is offline, otherwise it determines that the computer board is online.

3. The management node negotiation method applicable to a small-scale embedded computing cluster system according to claim 1, characterized in that: The SMC determines whether the single board computer is offline or online. Specifically: The cluster management software periodically queries the onboard BMC for the board type; The BMC determines whether the single board computer is offline or online according to the number of query messages received within a specified time. Specifically, if the number of query messages received within a specified time is greater than a preset threshold, the computer is determined to be online, otherwise the computer is determined to be offline. The BMC periodically reports the board type status information to the SMC.

4. The management node negotiation method applicable to a small-scale embedded computing cluster system according to claim 1, characterized in that: The SMC determines that the management board is offline and modifies the slot registration sequence list, specifically: the SMC deletes the slot corresponding to the management board in the "slot registration sequence list", sends a corresponding offline processing command to the BMC, and initiates a system resource update; The SMC determines that the computing board is offline and modifies the slot registration sequence list, specifically: the SMC sends the current online status information of each slot to the "management node", and the "management node" deletes the corresponding computing resources; The SMC determines that the computing board is online and modifies the slot registration sequence list. Specifically, the SMC sends the current online status information of each slot to the "management node", and the "management node" adds the corresponding computing resources.

5. The management node negotiation method applicable to a small-scale embedded computing cluster system according to claim 1, characterized in that: The next management board is selected according to the management board negotiation rules, specifically: according to the "slot registration order linked list", the onboard BMC of the single-board computer in the first slot in the linked list sends a board type change message to update the board type from "computing board" to "management board".

Images