Server monitoring system and server monitoring method

Abstract

To continue operation of a server system without isolating a housing where a failure has occurred in system control of the housing.SOLUTION: A server system having a building block configuration is composed of housings 10-1 to 10-4 with an identical configuration, each of which has a function to independently serve as a server device. The housing 10-1 detects a failure in the housing 10-3, acquires control rights of the housing 10-3 when the failure is detected and controls the housing 10-3 on its behalf, acts on its behalf for control to recover the housing 10-3 to normal, and returns the control rights to the housing 10-3 when the housing 10-3 recovers to normal.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a server monitoring system and a server monitoring method. [Background technology]

[0002] 2. Description of the Related Art In an information processing system in which a plurality of information processing devices are connected, several techniques are known in which the information processing devices mutually monitor for occurrence of failures (see, for example, Patent Documents 1 and 2).

[0003] For example, a technique is known in which operating system instances executed in multiple partitions obtained by dividing system resources have a means for continuously monitoring each other's activities to detect malfunctions of the operating system instances. Also, a technique is known in which, when a failure is detected in one of multiple loosely coupled processors, an alternative processor is selected and output waiting data of the failed processor is output from a shared input / output device under the management of the alternative processor. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 11-316747 [Patent Document 2] Japanese Patent Application Publication No. 6-282452 Summary of the Invention [Problem to be solved by the invention]

[0005] There is known a building block server system in which multiple chassis, each of which functions as a server device on its own, are connected together and operated as a single server system. In the following explanation, the building block configuration will be referred to as the "BB configuration."

[0006] In a server system with a BB configuration, chassis system control and total system control are performed as system controls for stable operation of the server system. Chassis system control is performed for each of the multiple chassis that make up the server system, and monitors and controls each part of the chassis itself. For one of the multiple chassis, chassis system control for itself as well as total system control are performed. Total system control is system control for the entire server system, and integrates the chassis system control performed for each chassis.

[0007] In the case where a failure occurs in the cabinet system control performed in one of the cabinets of a server system with a BB configuration, the fact that the cabinet is in an abnormal state can be detected by the other cabinet system controls. When such a failure occurs, the monitoring and control of each part of the cabinet itself provided by the failed cabinet system control becomes impossible, so the total system control performs a process in which the cabinet system control separates the failed cabinet from the system.

[0008] In one aspect, the present invention has an object to continue operation of a server system without isolating a cabinet in which a failure has occurred in cabinet system control. [Means for solving the problem]

[0009] In one proposal, the server monitoring system includes a control unit that detects a failure in a case that is a server device, and when the control unit detects a failure, it acquires control of the case and takes over control of the case, takes over control to restore the case in which the failure was detected to normal, and returns control to the server device when the case returns to normal. Effect of the Invention

[0010] According to one aspect, the operation of the server system can be continued without isolating a chassis in which a failure has occurred in chassis system control. [Brief description of the drawings]

[0011] [Figure 1A]FIG. 10 is a diagram illustrating a first method for controlling the server system. [Figure 1B] FIG. 11 is a diagram illustrating a second method for controlling the server system. [Diagram 2] FIG. 1 illustrates a first configuration example of a server system in a BB configuration. [Diagram 3] FIG. 2 is a diagram showing an example of a detailed configuration of a housing. [Figure 4] FIG. 2 is a diagram showing a first example of a connection configuration between housings. [Diagram 5] FIG. 13 is a diagram for explaining a method of reusing the housing system control. [Figure 6A] FIG. 13 is a diagram illustrating a first example of a configuration that enables proxy control of a target control device. [Figure 6B] FIG. 13 is a diagram illustrating a second example of a configuration that enables proxy control of a target control device. [Figure 6C] FIG. 13 is a diagram illustrating a third example of a configuration that enables proxy control of a target control device. [Figure 7] 10 is a flowchart showing the process contents of proxy processing of enclosure system control. [Figure 8] FIG. 13 is a diagram showing a second example of a connection configuration between housings. [Figure 9] FIG. 11 illustrates a second configuration example of a server system in a BB configuration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments will be described in detail with reference to the drawings.

[0013] First, two methods for controlling a server system will be explained.

[0014] The first method of controlling the server system is to control the entire server system by a single system control. First, this first method will be described with reference to FIG. 1A.

[0015] FIG. 1A shows the configuration of a server system in which a single housing 1 houses a plurality of CPU modules 2, each equipped with a CPU (Central Processing Unit) and the like, and a system controller 3.

[0016] In FIG. 1A, two system controls 3 are housed in the housing 1. This indicates that the system controls 3 are redundant in the configuration of FIG. 1A, and the black and white circles attached to each of the two system controls 3 indicate the operational system and the standby system.

[0017] The contents of the control over the CPU module 2 include, for example, control of power on and off, control of the number of revolutions of the cooling fan, control of disconnection of the CPU module 2 when a failure occurs, and control of collection and analysis of monitoring logs. In the first method, such control over a plurality of CPU modules 2 is handled by one system control 3. Therefore, in the first method, the load on the system control 3 becomes very high. For this reason, in order to build a system with high responsiveness in terms of system startup time, detection time from the occurrence of a failure, collection of failure logs, etc., the system control 3 is required to have large-scale and highly functional control. In addition, a large number of control elements and wiring are required to build such a system control 3, which leads to large implementation problems such as component costs and complicated wiring.

[0018] In the first method, redundancy of the system control 3 as shown in Fig. 1A is effective for improving the fault tolerance of the system control 3. However, the standby system constructed by redundancy becomes an extra resource that is not used under normal circumstances.

[0019] Next, a second method of controlling a server system will be described. This second method is a method in which each of the chassis 1 in a server system composed of multiple chassis 1 individually controls its own system, and is also used for system control in supercomputers. This second method will be described with reference to FIG. 1B.

[0020] 1B shows the configuration of a server system in which multiple chassis 1, each housing a CPU module 2 and a system control 3, are connected to a management terminal 4. The management terminal 4 is used to give instructions to the system control 3 of each chassis 1.

[0021] Note that two management terminals 4 are shown in Figure 1B, which indicates that the management terminals 4 are redundant in the configuration of Figure 1B, and the black and white circles attached to each of the two management terminals 4 represent the operational system and the standby system.

[0022] In the configuration of FIG. 1B, the system control 3 of each housing 1 only controls the power on and off of the housing 1 in which it is housed and the CPU module 2 housed in the housing 1 in response to instructions from the management terminal 4, and does not control other housings 1. Therefore, for example, even if a failure occurs in a housing 1, each housing 1 responds to the failure independently. Therefore, even if the failure affects other housings 1 and causes abnormalities in multiple housings 1, no processing is performed to determine which housing 1 is responsible for the abnormality. That is, for example, even if an abnormality is detected in a CPU in another housing 1 that was communicating with the housing 1 due to a power failure in the housing 1, it may not be possible to determine whether the cause of the abnormality is in the CPU or in the communication partner of the CPU.

[0023] In the second method, the management terminal 4 issues instructions to turn on and off the power supply for the system control 3 of each case 1 and collects monitoring logs via a communication network such as a LAN (Local Area Network). Therefore, if an abnormality occurs in the system control 3 of a case 1, the management terminal 4 cannot communicate with the system control 3 in which the abnormality occurred. As a measure to prevent the influence of this abnormality from spreading, it is possible to block the interfaces of the other cases 1 by instructions from the management terminal 4 and to isolate the case 1 in which the abnormality occurred from the system. However, it is not possible to cut off the power supply for the case 1 in which the abnormality occurred from the management terminal 4.

[0024] Next, a method for controlling a server system in the BB configuration will be described.

[0025] Fig. 2 shows a first example of a server system with a BB configuration. This first example shows a configuration in which a plurality of (four in Fig. 2) chassis 1, each housing a CPU module 2 and a system control 3, are interconnected. Each of these chassis 1 functions as a server device independently. In addition, the system control 3 of each chassis 1 performs chassis system control to monitor and control each part in its own chassis 1. Specifically, the chassis system control monitors the validity of the operating state, such as detecting power supply abnormalities, temperature abnormalities, and component failures, controls the supply and stop of power to the CPU module 2, controls the number of revolutions of the cooling fan of the chassis 1, and controls the restart of a CPU in which a failure has been detected.

[0026] In the configuration example of FIG. 2, the housing 1-1, which is one of the multiple housings 1, is connected one-to-one with each of the other housings 1. The system control 3 of the housing 1-1 performs overall system control that controls the entire server system by integrating the housing system control performed for each housing 1, together with the housing system control for itself. In the overall system control, the entire server system is controlled using the above-mentioned monitoring and control status shared between the housing 1-1 and each of the other housings 1 by exchanging various information through mutual communication between the housing 1-1 and each of the other housings 1. Since the system control 3 of the housing 1-1 performs this overall system control together with the housing system control for itself, there is no need to provide the server system with a dedicated component for performing the overall system control.

[0027] 2 further shows that the housing 1-2, which is one of the multiple housings 1, is connected one-to-one to each of the housings 10 other than the housing 1-1. This is to allow the housing 1-2 to function as a backup for the housing 1-1. That is, in this configuration example, when an abnormality occurs in the operation of the housing 1-1, the system control 3 marked with a white circle can take over the overall system control that integrates the housing system control for each housing 1, which was previously performed by the system control 3 marked with a black circle.

[0028] In this way, in the BB configuration illustrated in Fig. 2, the control processing for the server system is distributed to each case 1. Therefore, compared to the system control 3 that handles the control for all cases 1 in the first method described above, the processing load for the total system control performed by the system control 3 of the case 1-1 is small, and the reaction time to the monitoring control is short. Also, in this BB configuration, the total system control by the system control 3 of the case 1-1 unifies the case system control for each case 1. Therefore, when the influence of a failure that occurs in one case 1 spreads and an abnormality is detected in multiple cases 1, it is easier to identify the case 1 that is the root cause of the abnormality than in the control in the second method described above.

[0029] In a server system with a BB configuration, if a failure occurs in the cabinet system control in any of the cabinets 1, the cabinet 1 controlled by the cabinet system control where the failure occurred will become uncontrollable. The response of the total system control in this case will be explained using the configuration example in FIG.

[0030] Even if such a failure occurs in the housing system control, housing 1-1, which is connected one-to-one with the uncontrollable housing 1, is able to detect the abnormal state of the housing system control in which the failure has occurred. Therefore, when housing 1-1 recognizes the detection of the abnormal state of housing 1, it can control the entire system to separate the uncontrollable housing 1 from the server system in order to continue the operation of the server system. However, such separation control alone is not enough to easily maintain the provision of the server function that was being provided by the uncontrollable housing 1, and housing 1-1 cannot turn off the power supply of the uncontrollable housing 1.

[0031] Instead of having housing 1-1 perform such separation control as the entire system control, it is possible to consider a method of making the housing system control redundant for each housing 1. However, this method would result in each housing 1 becoming larger in size and increasing costs.

[0032] Therefore, in the embodiment described below, a server system with a BB configuration that can continue operation without isolating a case 1 in which a failure has occurred in the case system control will be described. More specifically, in this server system, when a failure occurs in the case system control of any of the cases 1, case 1-1 acquires control rights for the case 1 that was the control target of the case system control in which the failure has occurred, and takes over the case system control for that case 1.

[0033] Although details will be described later, in this embodiment, the configuration of each housing 1 is the same, so that housing 1-1 can easily take over housing system control for other housings 1 by utilizing its own housing system control. This delegation of housing system control maintains the operation of the housing 1 under the control of housing 1-1, so that the operation of the server system can be continued without disconnecting the housing 1. Also, if necessary, control to cut off the power supply to the housing 1 can be safely performed from housing 1-1 as total system control.

[0034] In addition, if housing 1-1 fails, housing 1-2, which serves as a backup for housing 1-1, can take over the control performed by housing 1-1 and perform overall system control for the entire server system and housing system control for housing 1-1.

[0035] First, a detailed configuration example of the chassis 10 used in the server system of the BB configuration according to this embodiment will be described with reference to FIG.

[0036] The housing 10 includes a system control processor 20, a PSU 30, a FAN-C 40, a system control device 50, a CPU 60, a DDC 70, an ADC 80, and a target control device 90. The CPU module 2 in the housing 1 shown in Fig. 2 includes the CPU 60, the DDC 70, and the ADC 80 in Fig. 3. The system control 3 in Fig. 2 is composed of the system control processor 20, the system control device 50, and the target control device 90 in Fig. 3.

[0037] The system control processor 20 is an information processing device that executes system control software to control the system control device 50 and the target control device 90, thereby realizing the above-mentioned housing system control.

[0038] Furthermore, when the case 10 is used as the case 1-1 in FIG. 2 described above to control the entire server system, the system control processor 20 also performs the above-mentioned overall system control by executing this system control software.

[0039] This system control software is stored in advance, for example, in a memory (not shown) connected to the system control processor 20. When power is supplied to the system control processor 20 and it starts operating, it reads out this system control software from the memory and executes it to provide the functions of controlling the housing system and the entire system.

[0040] The PSU (Power Supply Unit) 30 is a power supply device that supplies power to each component of the housing 10 .

[0041] FAN-C (Fan Controller) 40 is a control device for a cooling fan provided in the housing 10.

[0042] Both of these PSU 30 and FAN-C 40 are examples of the first control target in the housing 10.

[0043] The system control device 50 provides data communication interfaces such as I2C (Inter-Integrated Circuit), JTAG (Joint test Action Group), and SPI (Serial Peripheral Interface). The system control device 50 includes a device control unit 51, a reset control unit 52, inter-device IFs 53 and 54, and an IF control unit 55 between devices in other housings. Note that "IF" is an abbreviation for interface.

[0044] The device control unit 51 monitors and controls the PSU 30 and FAN-C 40 in accordance with control-related instructions from the system control processor 20. The system control device 50 equipped with this device control unit 51 is an example of a first control device that controls a first control target in the housing 10.

[0045] The reset control unit 52 provides a reset function for the system control processor 20 .

[0046] The inter-device IF 53 provides a bidirectional communication interface function with the target control device 90 and relays communication between the system control processor 20 and the target control device 90 .

[0047] The inter-device IF 54 provides a bidirectional communication interface function with a control device included in another housing 10, enabling communication with that control device.

[0048] The other housing inter-device IF control unit 55 controls the inter-device IF 54 under the management of the housing system control.

[0049] The system control device 50 has an internal register. By accessing this internal register and reading and writing values, the system control device 50, including the monitoring and control of each component included in the system control device 50, is monitored and controlled.

[0050] The CPU 60 is a processor that provides the server functions of the housing 10 .

[0051] The DDC 70 is a DC-DC converter that converts the output voltage from the PSU 30 into an operating voltage for the CPU 60 and supplies power to the CPU 60. Note that "DC" is an abbreviation for direct current.

[0052] The ADC 80 is an AD converter including a measurement device, and outputs digital data representing measurement values ​​such as the operating temperature of the CPU 60 and the voltage supplied from the DDC 70 to the CPU 60. Note that "AC" is an abbreviation for Alternating Current.

[0053] The CPU 60, the DDC 70, and the ADC 80 are all examples of a second control target in the housing 10.

[0054] The target control device 90 provides a data communication interface such as I2C, JTAG, SPI, etc., similar to the system control device 50. The target control device 90 includes a CPU control unit 91, a device control unit 92, and inter-device IFs 93 and 94.

[0055] The CPU control unit 91 monitors and controls the CPU 60 in accordance with control-related instructions given from the system control processor 20 via the system control device 50 .

[0056] The device control unit 92 monitors and controls the DDC 70 and ADC 80 in accordance with control-related instructions given from the system control processor 20 via the system control device 50 .

[0057] The target control device 90 including the CPU control unit 91 and the device control unit 92 is an example of a second control device that controls a second control target in the housing 10.

[0058] The inter-device IF 93 provides a bidirectional communication interface function with the system control device 50 that relays communication between the system control processor 20 and the target control device 90 .

[0059] The inter-device IF 94 provides a bidirectional communication interface function with a control device provided in another housing 10. This inter-device IF 94 is used to monitor and control the CPU 60, DDC 70, and ADC 80 from the other housing 10 when the system control device 50 becomes inoperable due to a malfunction or the like.

[0060] The target control device 90 also has an internal register, similar to the system control device 50. By accessing this internal register and reading and writing values, the target control device 90, including the monitoring and control of each of the above-mentioned components of the target control device 90, is monitored and controlled.

[0061] Next, an example of a connection configuration between the chassis 10 in the case where a server system of the BB configuration is configured using a plurality of chassis 10 of FIG. 3 will be described with reference to FIG.

[0062] The first example of the connection configuration shown in FIG. 4 is a connection example in which a server system having a BB configuration is configured using four chassis 10 having the configuration shown in FIG. 3, and corresponds to the configuration example shown in FIG. 2.

[0063] In Fig. 4, the cases 10-1, 10-2, 10-3, and 10-4 all have the same configuration as the case 10 shown in Fig. 3. However, in the configuration example of Fig. 4, the system control device 50 has three inter-device IFs 54 (inter-device IFs 54-1, 54-2, and 54-3), and the target control device 90 has two inter-device IFs 94 (inter-device IFs 94-1 and 94-2).

[0064] For simplicity, Fig. 4 shows only the system control device 50 and the target control device 90 among the components of the housing 10. Fig. 4 also shows only the inter-device IFs 53, 54-1, 54-2, and 54-3 among the components of the system control device 50, and shows only the inter-device IFs 93, 94-1, and 94-2 among the components of the target control device 90.

[0065] In FIG. 4, the inter-device IF 54-1 of the system control device 50 of the housing 10-1 and the inter-device IF 54-1 of the system control device 50 of the housing 10-2 are connected via an inter-device path 101. Also, the inter-device IFs 54-2 and 54-3 of the system control device 50 of the housing 10-1 and the inter-device IFs 54-1 of the system control devices 50 of the housings 10-3 and 10-4 are connected via an inter-device path 102. That is, the inter-device paths 101 and 102 connect the housing 10-1 to each of the other housings 10-2, 10-3, and 10-4 in a one-to-one relationship. Therefore, direct communication is possible between the system control processor 20 of the housing 10-1 and the system control processors 20 of the housings 10-2, 10-3, and 10-4.

[0066] The example in Fig. 4 has the above-mentioned connection configuration, and the system control processor 20 of the housing 10-1 can perform overall system control that integrates the housing system control performed in each of the housings 10-1, 10-2, 10-3, and 10-4. In other words, the housing 10-1 can function as a server monitoring system that monitors the entire server system of the BB configuration configured as shown in Fig. 4. This connection configuration also allows the system control processor 20 of the housing 10-1 to access the internal registers of the system control devices 50 of the housings 10-2, 10-3, and 10-4.

[0067] 4, the inter-device IFs 54-2 and 54-3 of the system control device 50 in the housing 10-2 are connected to the inter-device IFs 54-2 of the system control devices 50 in the housings 10-3 and 10-4 via an inter-device path 103. Therefore, the housing 10-2 is connected one-to-one to each of the other housings 10-1, 10-3, and 10-4 by the inter-device paths 101 and 103. This connection is used when the housing 1-2 marked with a white circle functions as a backup for the housing 1-1 marked with a black circle.

[0068] Next, a description will be given of functions that enable the cabinet system control of a cabinet 10 in which a failure has occurred to be taken over by the other cabinet 10 connected one-to-one with the cabinet 10 in question. In this embodiment, the cabinet system control that is responsible for the overall system control is provided with the four functions described below.

[0069] First, the housing system control of the housing 10 has a first function of monitoring the normality of the housing system control of the other housing 10 connected one-to-one with the housing 10 and detecting an abnormality.

[0070] In this embodiment, the housing system control of the housing 10 controls the system control device 50 of the housing 10 itself to communicate at regular time intervals with the housing system control of the other housing 10 that is connected one-to-one. In this communication, if no notification is received from the other housing even after a predetermined time has elapsed, the housing system control of the housing 10 determines that an abnormality has occurred in the housing system control of the other housing 10 and that the other housing 10 has broken down.

[0071] Also, the housing system control of the housing 10 has a second function of resetting the housing system control of the other housing 10 in which an abnormality has been detected.

[0072] 4, communication is possible between the system control processor 20 of the case 10 and the system control processor 20 of the other case 10. Therefore, the case system control of the case 10 first issues a reset request to the system control processor 20 of the other case 10 through this communication, and attempts to reset the system control processor 20 of the other case 10.

[0073] 4, the system control processor 20 of the housing 10 can access the internal registers of the system control device 50 of the other housing 10 via the system control device 50 of the housing 10 itself. The housing system control of the housing 10 then uses this access to control the system control device 50 of the other housing 10 to cause the reset control unit 52 to function, thereby attempting to reset the system control processor 20 of the other housing 10.

[0074] Furthermore, the housing system control of the housing 10 is provided with a third function of acquiring control rights to control the other housing 10, and a fourth function of controlling the other housing 10 on behalf of the other housing 10 when the control rights are acquired.

[0075] First, the housing system control of the housing 10 attempts to control the system control device 50 of the other housing 10 to obtain control. That is, the housing system control of the housing 10 accesses the internal register of the other system control device 50 and performs control to inhibit access to the internal register from the system control processor 20 of the other housing 10. More specifically, the housing system control of the housing 10 accesses the internal register and attempts control such as resetting, stopping the clock supply, and blocking the signal path to the interface circuit (not shown) to the system control processor 20. If these controls succeed in inhibiting access and the control of the other housing 10 is obtained, the housing system control of the housing 10 takes over and performs control of the other housing 10.

[0076] As described above, in the connection configuration shown in Fig. 4, each housing 10 has the same configuration, which is similar to the housing 10 shown in Fig. 3, and the housing system control in each housing 10 is also the same. Therefore, a housing 10 can use the housing system control for its own control to act as a proxy for the control of the other housing 10. The method of using this housing system control will be described with reference to Fig. 5.

[0077] 5, it is assumed that a housing 10 including a system control processor 20-1 and a system control device 50-1 and a counterpart housing 10 including a system control processor 20-2 and a system control device 50-2 have the same configuration. It is also assumed that the system control device 50-1 has internal registers 56-1 and 56-2, and the system control device 50-2 has internal registers 56-3 and 56-4.

[0078] Here, a common address is assigned to the internal registers 56-1 and 56-3 that provide the same functions in the case 10 and the other case 10. Similarly, a common address is assigned to the internal registers 56-2 and 56-4. In this way, the internal registers 56-3 and 56-4 can be operated by the case system control of the case 10 itself, simply by modifying the addresses indicating the internal registers 56-1 and 56-2 to give them a predetermined offset. Therefore, it is easy to use the case system control of the case 10 itself to act as a proxy for the control of the other case 10.

[0079] However, depending on the abnormality of the housing system control of the other housing 10, the housing system control of the housing 10 may not be able to control the system control device 50 of the other housing 10. In this case, the housing system control of the housing 10 may control the target control device 90 of the other housing 10 to attempt to obtain the control right.

[0080] For this purpose, the chassis system control of the chassis 10 accesses the internal register of the opponent's target control device 90 and inhibits access to the internal register from the opponent's system control processor 20 via the system control device 50. If this access inhibition is successful and control of the opponent's target control device 90 can be obtained, the chassis system control of the chassis 10 takes over and monitors and controls the CPU 60, DDC 70, and ADC 80 in the opponent's chassis 10.

[0081] However, it may not be possible to access the internal register of the target control device 90 of the other party by the case system control of the case 10 via the out-of-control system control device 50 of the other party. Therefore, in order to take over the control of the target control device 90 of the other party's case 10, a configuration may be provided that enables the case system control of the case 10 to control the target control device 90 without going through the system control device 50 of the other party's case 10. Next, three configuration examples as specific examples of this configuration will be described with reference to Figures 6A, 6B, and 6C, respectively.

[0082] First, common points among the configuration examples of FIG. 6A, FIG. 6B, and FIG. 6C will be described.

[0083] The configurations shown in Figures 6A, 6B, and 6C are all added to the connection configuration shown in Figure 4. For simplicity, these figures only show the configuration between housing 10-1 and housing 10-3, but a configuration similar to this configuration can be provided between the housings 10 that are connected one-to-one in Figure 4.

[0084] 4, the inter-housing path 102 representing the one-to-one connection between the housing 10-1 and the housing 10-3 is represented by a bidirectional arrow, but in Fig. 6A, Fig. 6B, and Fig. 6C, it is represented as an inter-housing path 102-1 and an inter-housing path 102-2. The inter-housing path 102-1 is a transmission path for signals from the housing 10-1 to the housing 10-3, and the inter-housing path 102-2 is a transmission path for signals from the housing 10-3 to the housing 10-1.

[0085] The inter-housing paths 102-1 and 102-2 are integrated into one cable 110. The cable 110 is connected to a connector 120 provided in the housing 10. The inter-housing paths 102-1 and 102-2 are led to a circuit board 57 on which the system control device 50 is mounted via the connector 120. The inter-housing paths 102-1 and 102-2 are connected to an inter-device IF 54-2 provided in the system control device 50 by a wiring that connects the system control device 50 mounted on the circuit board 57 and the connector 120.

[0086] The circuit board 57 is further provided with a connector 130 to which a circuit board 95 on which the target control device 90 is mounted is connected. The inter-device IF 53 of the system control device 50 and the inter-device IF 93 of the target control device 90 are connected via this connector 130.

[0087] In the following description, a case will be described in which an abnormality occurs in the case system control of the case 10-3, and the case system control of the case 10-1 takes over the control of the target control device 90 of the case 10-3.

[0088] First, a first example of a configuration that enables proxy control of the target control device 90 will be described with reference to Fig. 6A. This first example is a configuration in which an inter-device IF 54 provided in the system control device 50 of the housing 10 is added to directly connect the system control device 50 of the housing 10-1 and the target control device 90 of the housing 10-3.

[0089] In the example of Fig. 6A, three inter-device IFs 54 are added to the system control device 50 in order to directly connect the system control device 50 in the case 10-1 to the target control devices 90 in the case 10-2, 10-3, and 10-4 in the connection configuration of Fig. 4. Note that Fig. 6A only shows the connection between the inter-device IF 54-5 of the added inter-device IFs 54-4, 54-5, and 54-6 and the inter-device IF 94-1 of the target control device 90 in the case 10-3.

[0090] 6A, the inter-device IF 54-5 of the system control device 50 in the housing 10-1 and the inter-device IF 94-1 of the target control device 90 in the housing 10-3 are connected by inter-device paths 104-1 and 104-2. The inter-device paths 104-1 and 104-2 may be integrated into one cable 110 together with the inter-device paths 102-1 and 102-2.

[0091] In the configuration of this first example, the target control device 90 of the housing 10-3 can be controlled by the housing system control of the housing 10-1, without going through the system control device 50 of the housing 10-3. Therefore, even if the system control device 50 of the housing 10-3 becomes uncontrollable, the housing system control of the housing 10-1 can take over control of the target control device 90 of the housing 10-3.

[0092] Next, a second example of a configuration that enables proxy control of the target control device 90 will be described with reference to Fig. 6B. This second example is a configuration that does not require a large-scale expansion of hardware resources, such as the expansion of the inter-device IF 54 as in the first example.

[0093] In this second example, a selector 140 is provided on the circuit board 57, and the inter-chassis paths 102-1 and 102-2 are each branched by the selector 140. The branched paths 111-1 and 111-2 are connected to the inter-device IF 94-1 of the target control device 90.

[0094] In addition, in this second example, a selection signal line 141 is further provided to connect the system control device 50 of the housing 10-1 and the selector 140. This selection signal line 141 may be integrated into one cable 110 together with the inter-housing paths 102-1 and 102-2.

[0095] In the configuration of this second example, a control procedure by the case system control of the case 10-1 for acquiring the control right of the target control device 90 in the case 10-3 will be described.

[0096] The housing system control of the housing 10-1 first controls its own system control device 50 to output a selection instruction signal to the selector 140 to the selection signal line 141. This selection instruction signal causes the selector 140 to disconnect the inter-housing paths 102-1 and 102-2 from the inter-device IF 54-1 provided in the system control device 50 of the housing 10-3 and connect them to the paths 111-1 and 111-2, respectively. Then, the inter-device IF 54-2 of the system control device 50 of the housing 10-1 and the inter-device IF 94-1 of the target control device 90 of the housing 10-3 are connected. As a result, the system control processor 20 of the housing 10-1 can directly communicate with the target control device 90 of the housing 10-3 without going through the system control device 50 of the housing 10-3, and the target control device 90 can be controlled.

[0097] Next, a third example of a configuration that enables proxy control of the target control device 90 will be described with reference to Fig. 6C. This third example is a configuration that does not require any additional connection lines other than the inter-chassis paths 102-1 and 102-2 as connection lines between the chassis 10-1 and the chassis 10-3.

[0098] In this third example, the inter-chassis paths 102-1 and 102-2 are each branched on the circuit board 57 without providing a selector 140 as in the second example shown in FIG. 6B. The branched paths 112-1 and 112-2 are connected to the inter-device IF 94-1 of the target control device 90. However, a switch 150 is inserted in the inter-chassis path 102-2 between the branch point and the inter-device IF 54-1 of the system control device 50 on the circuit board 57. Furthermore, a switch 160 is inserted in the path 112-2 between the branch point on the inter-chassis path 102-2 and the inter-device IF 94-1 of the target control device 90.

[0099] In addition, in this third example, switching signal lines 142-1 and 142-2 are further provided which connect these switches 150 and 160, respectively, to the target control device 90 of the housing 10-3.

[0100] In the configuration of this third example, a control procedure for the case 10-1 to acquire the control right of the target control device 90 in the case 10-3 will be described.

[0101] First, as a normal operation, inter-chassis communication is performed at regular time intervals between the chassis 10-1 and the chassis 10-3. In the chassis 10-1, this inter-chassis communication is performed under the control of the entire system. In the chassis 10-3, the chassis system control monitors whether this inter-chassis communication is being received by the target control device 90.

[0102] Thereafter, if an abnormality in the case system control in case 10-3 is detected and an attempt to reset the case system control does not resolve the abnormality, case 10-1 transmits a transmission suppression instruction to case 10-3 as control by the entire system control. In case 10-3, this transmission suppression instruction is received by both the system control device 50 and the target control device 90.

[0103] When the system control device 50 receives this transmission suppression instruction, it attempts to suppress the transmission of signals from the system control device 50 itself to the inter-chassis path 102-2.

[0104] However, if the system control device 50 is out of control, the transmission of signals to the inter-housing path 102-2 may not be suppressed. Therefore, when the target control device 90 receives this signal suppression signal, it first outputs a switching signal to the switching signal line 142-1 to switch the switch 150 and disconnects the system control device 50 from the inter-housing path 102-2.

[0105] Next, the target control device 90 outputs a switching signal to the switching signal line 142-2 to switch the switch 160 and connect the inter-device IF 94-1 to the path 112-2. As a result, the system control processor 20 in the case 10-1 can communicate with the target control device 90 in the case 10-3 without going through the system control device 50 in the case 10-3, and can control the target control device 90.

[0106] Next, a description will be given of proxy processing for the housing system control of the other housing 10, which is performed in the housing system control of the housing 10. Fig. 7 is a flow chart showing the processing contents of this proxy processing.

[0107] When the processing of FIG. 7 is started, first, in S101, a failure detection process is performed to detect a failure in the other casing 10, and then, in S102, a process is performed to determine whether a failure in the other casing 10 has been detected by the failure detection process of S101.

[0108] As described above, in the failure detection process, the system control device 50 of the case 10 itself is controlled to communicate with the case system control of the other case 10 connected one-to-one at regular time intervals. In the process of S102, the reception of a notification from the other case 10 in the communication by the failure detection process is monitored, and it is determined whether the notification is received before a predetermined period (a period equal to or longer than the time interval of the communication) has elapsed.

[0109] As a result of this monitoring, if no notification is received from the other casing 10 even after the predetermined period has elapsed, the determination result of S102 becomes YES, it is determined that a failure of the other casing 10 has been detected, and the process proceeds to S103. On the other hand, if as a result of this monitoring, a notification is received from the other casing 10 before the predetermined period has elapsed, the determination result of S102 becomes NO, it is determined that a failure of the other casing 10 has not been detected, the process returns to S101, and the execution of the failure detection process continues.

[0110] Next, in S103, a reset process is performed to attempt to reset the housing system control of the other housing 10 in which the failure was detected, and then in S104, a process is performed to determine whether the reset was successful.

[0111] In the reset process, various reset attempts are made to the system control processor 20 as a case control unit that provides case system control of the other case 10, and the system control device 50. For example, as described above, first, a reset request is issued to the other case's system control processor 20 by utilizing communication between the system control processor 20 of the case 10 and the system control processor 20 of the other case 10. If the other case's system control processor 20 responds to the issued reset request, and the reset of the other case's system control is successful and normal operation is restored, the determination result in S104 is YES.

[0112] On the other hand, if the system control processor 20 of the other party does not respond to the reset request, a reset of the chassis system control is requested to the system control device 50 of the other party's chassis 10. In other words, an attempt is made to reset the system control processor 20 by accessing the internal register of the system control device 50 of the other party's chassis 10 to cause the reset control unit 52 to function.

[0113] Here, when software reset and hardware reset are available as methods for resetting the system control processor 20 by the reset control unit 52 of the system control device 50, the software reset may be attempted first.

[0114] A software reset is a process in which the software itself resets the program counter, etc., triggered by an interrupt from an external signal or a watchdog timer, thereby restarting the program and eliminating an abnormal state, such as a hang-up, of the system control processor 20. A software reset has the advantages that the settings inside the system control processor 20 are maintained and that the time required for restart is shorter than that of a hardware reset.

[0115] If such a software reset does not restore normal operation of the other chassis' system control, a hardware reset is attempted. A hardware reset is performed by initializing all registers of the system control processor 20 using an external signal, and then restarting the system control processor 20. A software reset has the advantage that there is a higher possibility that the abnormal state of the system control processor 20 will be resolved than a software reset.

[0116] The determination result in S104 will also be YES if the reset attempt by the reset control unit 52 of the system control device 50 to the system control processor 20 results in a successful reset of the chassis system control of the other party and normal operation is restored.

[0117] If the determination in S104 is YES, the process returns to S101, and the failure detection process is started again for the other case 10 whose case system control has recovered normal operation. On the other hand, if the reset of the case system control of the other case 10 fails in any reset attempt in the reset process in S103, the determination in S104 is NO, and the process proceeds to S105.

[0118] In S105, a control right acquisition process is performed to try to acquire the control right of the other enclosure 10 in which a failure has been detected, and in the following S106, a process is performed to determine whether the control right has been acquired.

[0119] In the control right acquisition process, as described above, the internal register of the other party's system control device 50 is accessed, and control is performed to inhibit access to the internal register from the system control processor 20 of the other party's case 10. As a result, if the access is successfully inhibited and the control right of the other party's case 10 can be acquired, the determination result in S106 becomes YES, and the process proceeds to S107. On the other hand, if the access cannot be inhibited and the control right of the other party's case 10 cannot be acquired, the determination result in S106 becomes NO, and the process proceeds to S110.

[0120] In S107, a substitute reset control process is performed to control the other party's housing 10 on behalf of the other party's housing system control and to attempt to recover control of the housing system, and then in S108, a process is performed to determine whether control of the other party's housing system has been recovered.

[0121] In the substitute reset control process, the aforementioned software reset and hardware reset of the other party's system control processor 20 are attempted while preventing access from the other party's system control processor 20 to the system control device 50. Also, an attempt may be made to interrupt and resume the power supply to each unit by the PSU 30 of the other party's case 10. As a result, if the other party's case system control is recovered and the other party's case 10 returns to normal, the determination result in S108 becomes YES, and the process proceeds to S109. On the other hand, if the other party's case system control is not recovered, the determination result in S108 becomes NO, and the process proceeds to S111.

[0122] In S109, a control right return process is performed to convert the control right of the other party's case 10 to the restored case system control of the other party. In this process, the internal register of the other party's system control device 50 is accessed, and control is performed to restore the access to the internal register from the system control processor 20 of the other party's case 10, which was inhibited in the control right acquisition process of S105, to a permitted state. After this process, the process returns to S101, and the failure detection process for the other party's case 10, whose case system control has recovered normal operation, is started again.

[0123] On the other hand, if the judgment result in S106 is NO, in S110, maintenance work is performed on the failed counterpart housing 10. This work is performed by a maintenance worker, and the counterpart housing 10 is restored to normal operation by replacing or resetting the hardware that provides the housing system control of the counterpart housing 10, stopping and restarting the power supply to the counterpart housing 10, etc. After the housing system control of the counterpart housing 10 is restored by this work, the process returns to S101, and the failure detection process is started again for the counterpart housing 10 whose housing system control has been restored to normal operation.

[0124] Before performing the maintenance work of S110, the housing system control of the housing 10 may control the target control device 90 of the other housing 10 to perform a process of trying to acquire the control right of the target control device 90. That is, as described above, the housing system control of the housing 10 may access the internal register of the other target control device 90 and perform a process of inhibiting the access to the internal register made by the other system control processor 20. If the inhibition of this access is successful and the control right of the other target control device 90 is acquired, for example, the target control device 90 may be controlled to stop the supply of power to the CPU 60 by the DDC 70, and then the maintenance work of S110 may be performed. In this way, it is possible to reduce the possibility that the CPU 60, which is a key part for providing the function as a server device by the housing 10, may be accidentally damaged during a failure of the housing 10 or during maintenance work for recovery from the failure.

[0125] Also, when the determination result in S108 described above is NO, in S111, the maintenance worker performs the maintenance work on the failed other casing 10 in the same manner as in S110. However, this maintenance work in S111 is performed when the control right of the other casing 10 is held. Therefore, after the process of S111 is completed, the process returns to S109, and the control right return process described above is performed.

[0126] The above processing is performed by the housing system control of the housing 10, thereby acting as the housing system control of the other housing 10.

[0127] Although the disclosed embodiments and their advantages have been described in detail above, it will be appreciated that those skilled in the art may make various modifications, additions, and omissions therein without departing from the scope of the present invention as clearly set forth in the claims.

[0128] For example, in the detailed configuration example of the housing 10 shown in FIG. 3, the housing 10 is provided with a system control device 50 and a target control device 90, and communication between them is possible via the inter-device IFs 53 and 93. Instead of this, the housing 10 may be provided with an integrated device of the system control device 50 and the target control device 90. That is, for example, as shown in FIG. 8 showing a second example of the connection configuration between the housings, the system control device 50 may have all the functions of the target control device 90, and this system control device 50 may be provided in each housing 10. Note that the connection configuration of the housings 10-1, 10-2, 10-3, and 10-4 in the second example shown in FIG. 8 may be the same as that of the first example shown in FIG. 4.

[0129] Furthermore, the server system of the BB configuration is not limited to the first configuration example shown in FIG. 2, but may be, for example, the second configuration example shown in FIG.

[0130] In a server system with a BB configuration, communication for the operation of the server system is performed directly between the CPU modules 2 provided in each housing 1. For this reason, the CPU module 2 in each housing 1 is connected one-to-one to each of the CPU modules 2 in all other housings 1. For example, when a server system is configured with two housings 1, only one connection cable is required for one-to-one connection between the CPU modules 2. However, when the number of housings 1 that configure the server system increases to 3, 4, 5, ..., the number of connection cables for one-to-one connection between the CPU modules 2 increases explosively to 3, 6, 10, ....

[0131] In the second configuration example shown in FIG. 9, a multiple connection housing 5 is used instead of the above-mentioned connection cable to perform one-to-one connection between the CPU modules 2.

[0132] In Figure 9, the multiple housings 1 each containing a CPU module 2 and a system control 3 are similar to those in the first configuration example shown in Figure 2, and each of these housings 1 has the function of a server device on its own.

[0133] The multi-connection housing 5 houses an inter-CPU communication IF 6, a crossbar switch 7, and a system control 8.

[0134] The inter-CPU communication IF 6 provides a communication interface with the CPU module 2 .

[0135] The crossbar switch 7 provides one-to-one communication paths between the CPU modules 2 included in each of the multiple chassis 1. Each CPU module 2 is connected to the crossbar switch 7 via an inter-CPU communication IF 6.

[0136] In the configuration example of Fig. 9, the system control 8 of the multi-connection chassis 5-1 performs overall system control that controls the entire server system by integrating the chassis system controls performed by the system controls 3 of the multiple chassis 1. In the overall system control, the entire server system is controlled using the above-mentioned monitoring and control status that is shared with the chassis system control by exchanging various information through mutual communication with the system control 3 of the chassis 1.

[0137] 9, the system control 8 of the multi-connection housing 5-2, which is connected one-to-one with the system control 8 of the multi-connection housing 5-1, is also connected to the system control 3 of each of the multiple housings 1. This is to allow the multi-connection housing 5-2 to function as a backup for the multi-connection housing 5-1. That is, in the configuration example of FIG. 9, when an abnormality occurs in the operation of the multi-connection housing 5-1, the system control 8 marked with a white circle can take over the overall housing system control for each housing 1, which was previously performed by the system control 8 marked with a black circle.

[0138] Even in a server system having such a configuration, by following the procedure described above, it becomes possible for the multi-connection chassis 5 to acquire control of the chassis 1 that was the target of control by the chassis system control that has failed, and to take over the chassis system control for that chassis 1. [Explanation of symbols]

[0139] 1, 1-1, 1-2, 10, 10-1, 10-2, 10-3, 10-4 Case 2 CPU Module 3, 8 System Control 4. Management terminal 5, 5-1, 5-2 Multi-connection housing 6 CPU communication interface 7 Crossbar Switch 20, 20-1, 20-2 System control processor 30 PSU 40 FAN-C 50, 50-1, 50-2 System Control Device 51, 92 Device control section 52 Reset control section 53, 54, 54-1, 54-2, 54-3, 54-4, 54-5, 54-6, 93, 94, 94-1, 94-2 Device IF 55 IF control section between other devices 56-1, 56-2, 56-3, 56-4 Internal Registers 57, 95 Circuit board 60 CPU 70 DDC 80 ADC 90 Target Control Device 91 CPU control unit 101, 102, 102-1, 102-2, 103, 104-1, 104-2 Inter-chassis route 110 Cable 111-1, 111-2, 112-1, 112-2 Routes 120, 130 Connectors 140 Selector 141 Selection signal line 142-1, 142-2 Switching signal line 150, 160 Switch

Claims

1. Detects failures in the server device enclosure, When the failure is detected, the control right of the housing is acquired and the control of the housing is substituted; performing control to restore the housing in which the failure was detected to normal; When the case returns to normal, the control right is returned to the case. A server monitoring system comprising a control unit that performs processing.

2. The housing includes: A control device that controls the housing; A processor that gives instructions regarding control of the housing to the control device; Equipped with The control unit acquires the control right by controlling the control device to inhibit access from the processor for providing the instruction.

2. The server monitoring system according to claim 1.

3. The control device comprises: A first control device that controls a first control object in the housing; a second control device that controls a second control object different from the first control object in the housing, the second control device receiving the instruction from the processor via the first control device; Equipped with the processor providing the instructions for the second control device to the first control device; The control unit acquires the control right for the second control target in the housing by controlling the second control device to inhibit access from the first control device for transferring the instruction for the second control device to the second control device.

3. The server monitoring system according to claim 2.

4. The control unit is When a failure of the housing is detected, a housing control unit that controls the housing is requested to reset the control of the housing; When the reset of the control of the housing fails, the control right is acquired.

2. The server monitoring system according to claim 1.

5. The housing control unit is a processor that executes a program to perform a control process for controlling the housing; a control device that controls the housing in response to an instruction from the processor; Equipped with When the control unit detects a failure of the housing, it requests the processor executing the program to reset the control process, and if the reset of the control process fails, it requests the control device to perform a software reset of the processor, and if the software reset fails, it requests the control device to perform a hardware reset of the processor, and if the hardware reset fails, it acquires the control right.

5. The server monitoring system according to claim 4.

6. 2. The server monitoring system according to claim 1, wherein the control unit performs control to reset the operation of the case as the control to return the case to normal.

7. the server monitoring system has the same hardware configuration as the chassis, The control unit controls the chassis by utilizing the same configuration that the server monitoring system has.

7. The server monitoring system according to claim 1, wherein the server monitoring system is a server monitoring system.

8. Detects failures in the server device enclosure, When the failure is detected, the control right of the housing is acquired and the control of the housing is substituted; performing control to restore the housing to normal by taking over the control; When the case returns to normal, the control right is returned to the case. A server monitoring method, characterized in that the processing is performed by an information processing device.

Images