Monitoring method and device and baseboard management controller
By configuring the number of monitoring threads according to the dynamic and target monitoring interval relationship of the GPU, the problems of serial access of a single thread and excessive thread resource occupation are solved, and the efficiency and processor performance of GPU monitoring are improved.
Patent Information
- Application Number
- CN202411144920.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-29
AI Technical Summary
When monitoring multiple GPUs using the Baseboard Management Controller (BMC), a single thread serial access results in low timeliness of information acquisition, and setting independent threads for each GPU takes up more processor resources, resulting in a degradation of processor performance.
According to the dynamic information monitoring interval of each GPU and the target dynamic information monitoring interval relationship, the number of monitoring threads is flexibly configured, and corresponding monitoring threads are generated for GPU monitoring, avoiding serial access and excessive thread resource occupation.
It improves the timeliness of information acquisition, saves time to obtain information, avoids degradation in processor performance, and improves GPU monitoring efficiency.
Smart Images

Figure CN119046094A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of data monitoring, and in particular to a monitoring method, device and baseboard management controller (BMC). Background Art
[0002] With the development of information technology, servers are increasingly used in all walks of life. However, server management issues also arise. As an important management component of the server, BMC has functions such as remote management, hardware monitoring, and power control. It can also monitor multiple GPUs in the server.
[0003] When using BMC to monitor multiple GPUs in a GPU server, if a single thread is used to monitor all GPUs, each round of monitoring process requires serial access to each GPU until the BMC obtains the parameters to be obtained for all GPUs, resulting in low timeliness of obtaining information; if an independent thread is set for each GPU to monitor, too many threads will occupy more processor resources, resulting in reduced processor performance. Summary of the invention
[0004] In order to solve the above-mentioned problems in the prior art, the example of the present application provides a monitoring method, device and baseboard management controller, which can flexibly configure the number of monitoring threads according to the information of multiple GPUs in the GPU server, thereby realizing the monitoring of the GPU server.
[0005] In a first aspect, an embodiment of the present application provides a monitoring method, which is applied to a baseboard management controller BMC, where the BMC is connected to a plurality of GPUs; the method comprises:
[0006] Determine a dynamic information monitoring interval of each GPU among the multiple GPUs; the dynamic information monitoring interval is used to characterize the time length used to obtain the to-be-obtained parameters of the corresponding GPU;
[0007] Based on the quantitative relationship between the dynamic information monitoring interval and the target dynamic information monitoring interval, a first quantity is determined, and at least one monitoring thread having the same number as the first quantity is generated; the target dynamic information monitoring interval is used to characterize a target duration for obtaining the to-be-obtained parameters of the corresponding GPU;
[0008] The multiple GPUs are monitored using the at least one monitoring thread.
[0009] By comprehensively considering the dynamic information monitoring intervals of multiple GPUs and the target dynamic information monitoring intervals, the number of monitoring threads is determined. This can avoid the problem of low timeliness of obtaining information caused by using a single monitoring thread to serially monitor each GPU, thereby saving the time for obtaining information; it can also avoid setting up an independent thread for monitoring each GPU to occupy more processor resources, resulting in a decrease in processor performance, thereby improving the monitoring efficiency of the GPU.
[0010] In a possible implementation, determining the dynamic information monitoring interval of each GPU among the multiple GPUs includes:
[0011] The following operations are performed on a first GPU, where the first GPU is any one of the multiple GPUs:
[0012] Starting a timer corresponding to the first GPU, and obtaining a value of a parameter to be obtained from the first GPU;
[0013] The values of the parameters to be obtained obtained from the first GPU are saved in a first file; the first file includes a plurality of parameters to be obtained in the configuration file;
[0014] The timer stops timing until the multiple parameters to be obtained in the first file correspond to the values of the parameters to be obtained one by one, and a first duration is obtained; the configuration file is saved in the BMC;
[0015] The first duration is used as a dynamic information monitoring interval of the first GPU.
[0016] In the above implementation, the timer corresponding to the first GPU is used to turn on the timer when the first parameter is obtained, and to turn off the timer after the last parameter is obtained, so as to determine the dynamic information monitoring interval corresponding to the first GPU. Limiting the time nodes for turning on and off the timer can make the determined dynamic information monitoring interval more accurate.
[0017] In a possible implementation manner, the quantitative relationship between the dynamic information monitoring interval and the target dynamic information monitoring interval is determined in the following manner:
[0018] The dynamic information monitoring interval is multiplied by the number of the multiple GPUs, and the multiplied result is divided by the target dynamic information monitoring interval to obtain the quantitative relationship representing the multiple relationship between the dynamic information monitoring interval for monitoring the multiple GPUs and the target dynamic information monitoring interval.
[0019] Through the above implementation, the quantitative relationship between the dynamic information monitoring interval and the target dynamic information monitoring interval can be accurately determined according to different dynamic information monitoring intervals or different target dynamic information monitoring intervals, that is, in various different situations, which is beneficial to the subsequent steps.
[0020] In a possible implementation manner, determining the first quantity based on the quantitative relationship between the dynamic information monitoring interval and the target dynamic information monitoring interval includes:
[0021] If the quantity relationship is an integer multiple relationship, the value of the quantity relationship is used as the first quantity;
[0022] If the quantity relationship is a non-integer multiple relationship, the integer part of the value of the quantity relationship is increased by a set value as the first quantity.
[0023] Through the above implementation, different first quantities can be determined according to different quantitative relationships. The first quantity determines the number of monitoring threads. Therefore, by accurately determining the quantitative relationship and then determining the corresponding first quantity through the quantitative relationship, the number of monitoring threads can be determined more reasonably, and the relationship between the time to obtain the data to be obtained and the resource occupancy rate can be better balanced.
[0024] In a possible implementation, monitoring the multiple GPUs by using the at least one monitoring thread includes:
[0025] generating a number of components equal to the first number;
[0026] Select a component to which a GPU is not assigned from each component;
[0027] If the number of GPUs of the unallocated components is greater than a second number, the GPUs of the second number of unallocated components are allocated to the selected component, and the step of selecting a component to which the GPUs are unallocated from each component is returned; the second number is used to represent the maximum number of GPUs that can be accommodated in the selected component; the second number is determined based on the target dynamic information monitoring interval and the dynamic information monitoring interval;
[0028] If the number of GPUs not assigned components is not greater than the second number, assigning all GPUs not assigned components to the selected component;
[0029] The generated components are monitored using the at least one monitoring thread.
[0030] Through the above implementation, GPUs are allocated to different components, and the monitoring of multiple GPUs using monitoring threads is converted into monitoring the same number of components using monitoring threads. Since each monitoring thread monitors one component, each monitoring thread can better determine the GPU that needs to be monitored, that is, the GPU located in the corresponding component, which simplifies the monitoring task and reduces the complexity and burden of the monitoring thread.
[0031] In a possible implementation, the components correspond to the monitoring threads one by one; and the monitoring the generated components by using the at least one monitoring thread includes:
[0032] A static time length is set for the target component; the number of GPUs included in the target component is less than the number of GPUs included in any non-target component; the static time length is the time length used to monitor the difference GPUs in the target component and any non-target component;
[0033] The at least one monitoring thread is used to simultaneously monitor the generated components, and after obtaining the GPU parameters included in the target component, the GPU parameters included in the target component are obtained again after waiting for the static time period.
[0034] Through the above implementation, the monitoring duration of components including different numbers of GPUs can be balanced, so that the progress of each monitoring thread in monitoring each component is synchronized, thereby ensuring the accuracy of monitoring.
[0035] In a possible implementation manner, the simultaneously starting to monitor the generated components includes:
[0036] The at least one monitoring thread is used to simultaneously obtain a plurality of GPU parameters included in the generated component, and if any parameter of the plurality of GPU parameters exceeds a corresponding parameter threshold, an alarm is issued to a display device connected to the BMC.
[0037] Through the above implementation, the monitoring process is improved. By setting a parameter threshold for each parameter, an alarm signal can be sent to the central processor in time when the parameter exceeds the parameter threshold, allowing the user or relevant staff to process the fault parameter as soon as possible according to the alarm signal.
[0038] In a second aspect, an embodiment of the present application provides a monitoring device, the device comprising:
[0039] A monitoring interval determining unit, configured to determine a dynamic information monitoring interval of each GPU among the plurality of GPUs; the dynamic information monitoring interval is used to characterize a time length used to obtain a parameter to be obtained of the corresponding GPU;
[0040] a thread quantity determination unit, configured to determine a first quantity based on a quantity relationship between the dynamic information monitoring interval and a target dynamic information monitoring interval, and generate at least one monitoring thread having the same quantity as the first quantity; the target dynamic information monitoring interval is used to characterize a target duration for obtaining a parameter to be obtained of a corresponding GPU;
[0041] The monitoring thread monitoring unit is configured to monitor the multiple GPUs using the at least one monitoring thread.
[0042] In a third aspect, an embodiment of the present application provides a BMC, including a memory and a processor, wherein the memory stores a computer program running on the processor, and when the computer program is executed by the processor, the monitoring method in the first aspect is implemented.
[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the monitoring method in the first aspect is implemented.
[0044] The present application example provides a monitoring method, device and baseboard management controller, which can determine the dynamic information monitoring interval of each GPU used to characterize the time taken to obtain the parameters to be obtained of the corresponding GPU, and determine the number of monitoring threads based on the dynamic information monitoring interval of each GPU and the target dynamic information monitoring interval and generate corresponding monitoring threads, so that the baseboard management controller can use monitoring threads to monitor multiple GPUs. By comprehensively considering the dynamic information monitoring intervals and target dynamic information monitoring intervals of multiple GPUs to determine the number of monitoring threads, it is possible to avoid the problem of low timeliness of obtaining information due to serial monitoring of each GPU using a single monitoring thread, thereby saving time for obtaining information; it is also possible to avoid the problem of setting up an independent thread for monitoring each GPU to occupy more processor resources, resulting in a decrease in processor performance, thereby improving the efficiency of monitoring the GPU. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0046] Figure 1 An application scenario diagram of a monitoring method provided in an embodiment of the present application;
[0047] Figure 2 A flowchart of an execution monitoring method provided in an embodiment of the present application;
[0048] Figure 3 A flowchart for determining a dynamic information monitoring interval for each GPU in a plurality of GPUs provided in an embodiment of the present application;
[0049] Figure 4 A flowchart of a method for allocating a GPU to each component provided in an embodiment of the present application;
[0050] Figure 5 A specific flow chart of an execution monitoring method provided in an embodiment of the present application;
[0051] Figure 6 A structural block diagram of a monitoring device provided in an embodiment of the present application;
[0052] Figure 7 A structural block diagram of a BMC provided in an embodiment of the present application. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0054] It should be noted that the terms "including" and "having" and their variations involved in the documents of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0055] The terms "first" and "second" in this document are used for descriptive purposes only and should not be understood as explicitly or implicitly indicating relative importance or implicitly indicating the number of technical features indicated. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.
[0056] The word “exemplary” is used hereinafter to mean “serving as an example, example, or illustration.” Any embodiment described as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0057] The present application is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0058] Figure 1 An application scenario diagram of a monitoring method provided by an embodiment of the present application is shown, see Figure 1 As shown, the application scenario includes a central processing unit (CPU) 100, a baseboard management controller (BMC) 200, and multiple graphics processing units (GPUs) including GPU1 310, GPU2 320, and GPU3 330, wherein the present application does not limit the number of GPUs, and multiple GPUs can form a GPU server 300, and in most cases the GPUs included in the GPU server 300 are the same GPUs, and the following description will take the GPU server including the same GPU as an example.
[0059] The monitoring method provided in the embodiment of the present application can be applied to BMC200, and information can be exchanged between CPU100, BMC200 and GPU server 300 through the bus. It should be noted that in some possible embodiments, the monitoring method provided in the embodiment of the present application can also be applied to some other hardware devices or software to achieve monitoring of GPU server 300. The following description still takes the application of the monitoring method provided in the embodiment of the present application to BMC200 as an example.
[0060] Figure 2 A flow chart of a monitoring method provided by an embodiment of the present application is shown. Figure 2 As shown, the method may include the following steps:
[0061] Step S201 , determining a dynamic information monitoring interval of each GPU among a plurality of GPUs.
[0062] In a possible embodiment, before executing step S201, a configuration file stored in the BMC may also be read. By reading the configuration file, the model of the GPU to be monitored, the parameters to be obtained, and the target dynamic information monitoring interval corresponding to the GPU to be monitored may be parsed, wherein the parameters to be obtained may include information such as the temperature of the GPU, and the target dynamic information monitoring interval is used to characterize the target duration for obtaining the parameters to be obtained of the corresponding GPU, and the value of the target dynamic information monitoring interval may be customized according to user needs. It should be noted that in addition to being stored in the BMC, the configuration file may also be stored in a memory or other component that the BMC can read, and this application is not limited here. After completing the reading and parsing of the configuration file, the parsed information may be written into the monitoring process shared data, so that the monitoring thread generated in the subsequent steps can more easily monitor the GPU server.
[0063] In a possible embodiment, after reading the configuration file, the status of the GPU server can be monitored. If the GPU server changes from a powered-on state to a powered-off state, the previously generated monitoring thread can be deleted. If the GPU server changes from a powered-off state to a powered-on state, it can be further determined whether the GPU server changes from a powered-off state to a powered-on state for the first time. If this is the first time that the GPU server changes from a powered-off state to a powered-on state, step S201 can be executed. If this is not the first time that the GPU server changes from a powered-off state to a powered-on state, but it has changed from a powered-off state to a powered-on state before, it can be further determined whether the GPU server has changed compared to before. If the GPU server changes from a powered-off state to a powered-on state, step S202 can be executed. If there is no change compared to the previous one, the same monitoring thread can be regenerated according to the previously generated monitoring thread to monitor the GPU server; if the GPU server is changed compared to the previous one, for example, an additional GPU is added to the original GPU server, or the number of GPUs is reduced, it can be further determined whether the change of the GPU server complies with the change rules. If the change of the GPU server is unreasonable, the change information for the GPU server is cleared in the configuration file, and the same monitoring thread is regenerated according to the previously generated monitoring thread to monitor the GPU server; if the change of the GPU server complies with the change rules, the remaining monitoring thread is deleted, and then step S201 is executed.
[0064] In one possible embodiment, Figure 3 A flow chart of determining a dynamic information monitoring interval of each GPU in a plurality of GPUs provided by an embodiment of the present application is shown. Figure 3 As shown, the following operations may be performed for any GPU in the GPU server to determine the dynamic information monitoring interval of the GPU, and any GPU is named as the first GPU:
[0065] Step S2011, start a timer corresponding to the first GPU, and obtain the parameters to be obtained from the first GPU.
[0066] In a possible embodiment, the time node at which the BMC starts to obtain the parameters to be obtained from the first GPU can be used as the time node for starting the timer corresponding to the first GPU. It should be noted that if the two time nodes cannot be unified, the shorter the time interval between the two time nodes, the smaller the error in determining the dynamic information monitoring interval of each GPU in the multiple GPUs.
[0067] In step S2012, the parameter values to be obtained from the first GPU are correspondingly saved in the first file.
[0068] In a possible embodiment, the value of the parameter to be obtained obtained from the first GPU can be saved in a first file accordingly, wherein the first file includes multiple parameters to be obtained in the configuration file, such as temperature, occupancy rate and other parameters. The value of the parameter to be obtained obtained from the first GPU can be saved in a position in the first file corresponding to the parameter to be obtained. For example, if the obtained temperature is 50°C, the value of 50°C can be saved corresponding to the temperature.
[0069] Step S2013, determining whether the multiple parameters to be obtained in the first file correspond to the parameter values to be obtained one by one, if so, executing step S2014; if not, returning to executing step S2012.
[0070] In a possible embodiment, judging whether the multiple parameters to be obtained in the first file correspond to the parameter values to be obtained one-to-one is to determine whether the BMC has completed obtaining all the parameter values to be obtained from the first GPU. If the BMC has completed obtaining all the parameter values to be obtained from the first GPU, then the multiple parameters to be obtained in the first file should correspond to the parameter values to be obtained one-to-one; conversely, if the BMC has not completed obtaining all the parameter values to be obtained from the first GPU, then the multiple parameters to be obtained in the first file cannot correspond to the parameter values to be obtained one-to-one.
[0071] Step S2014, stop the timer, and use the obtained first duration as the dynamic information monitoring interval of the first GPU.
[0072] In a possible embodiment, the multiple parameters to be obtained in the first file correspond to the values of the parameters to be obtained one by one, which can indicate that the BMC has completed obtaining all the values of the parameters to be obtained from the first GPU. Then, the time node when the BMC completes obtaining the last value of the parameter to be obtained from the first GPU can be used as the time node for stopping the timer. Similarly, if the two time nodes cannot be unified, the shorter the time interval between the two time nodes, the smaller the error in determining the dynamic information monitoring interval of each GPU in the multiple GPUs. The first time length obtained from starting the timing to stopping the timing can be used as the dynamic information monitoring interval of the first GPU.
[0073] Step S202: determine a first quantity based on a quantitative relationship between a dynamic information monitoring interval and a target dynamic information monitoring interval, and generate at least one monitoring thread having the same quantity as the first quantity.
[0074] In a possible embodiment, the dynamic information monitoring interval can be multiplied by the number of multiple GPUs, and the multiplied result can be divided by the target dynamic information monitoring interval to obtain a quantitative relationship. For example, if the dynamic information monitoring interval is 100ms, the number of GPUs is 6, and the target dynamic information monitoring interval is 200ms, then the quantitative relationship can be 3, where the quantitative relationship is used to characterize the multiple relationship between the dynamic information monitoring interval for monitoring multiple GPUs and the target dynamic information monitoring interval.
[0075] In a possible embodiment, the first quantity can be determined according to a quantitative relationship. If the quantitative relationship is an integer multiple relationship, the value of the quantitative relationship is used as the first quantity. For example, if the dynamic information monitoring interval is 100ms, the number of GPUs is 6, and the target dynamic information monitoring interval is 200ms, then the quantitative relationship can be 3, and the value of the quantitative relationship can be used as the first quantity, that is, the first quantity is 3; if the quantitative relationship is a non-integer multiple relationship, the integer part of the value of the quantitative relationship is increased by a set value as the first quantity, wherein the set value is generally 1. For example, if the dynamic information monitoring interval is 100ms, the number of GPUs is 7, and the target dynamic information monitoring interval is 200ms, then the quantitative relationship can be 3.5, and the integer part of the value of the quantitative relationship can be increased by 1 as the first quantity, that is, the first quantity is 4.
[0076] In a possible embodiment, after the first number is determined, monitoring threads having the same number as the first number may be created.
[0077] Step S203: monitor multiple GPUs using at least one monitoring thread.
[0078] In a possible embodiment, the same number of components as the first number can be generated, that is, the number of components and the number of monitoring threads created are the same, and each monitoring thread created can monitor one of the components. Therefore, Figure 4 The method shown in FIG. 1 allocates GPUs to each component, such as Figure 4 As shown, the following steps are included:
[0079] Step S2031: Select a component to which a GPU is not allocated from each component.
[0080] Step S2032, determining whether the number of GPUs without components allocated is greater than the second number, if so, executing step S2033; if not, executing step S2034.
[0081] In a possible embodiment, the second number is the maximum number of GPUs that can be accommodated in a component, and can be determined by dividing the target dynamic information monitoring interval by the dynamic information monitoring interval. For example, if the target dynamic information monitoring interval is 300ms and the dynamic information monitoring interval is 100ms, then the second number may be 300 / 100=3, that is, the maximum number of GPUs that can be accommodated in a component is 3. After the second number is determined, the relationship between the number of GPUs that are not allocated to the component and the second number may be determined.
[0082] Step S2033, allocating the GPUs of the second number of unallocated components to the selected components, and returning to execute step S2031.
[0083] In a possible embodiment, if the number of GPUs of unallocated components is greater than the second number, the second number of GPUs of unallocated components may be allocated to the selected component. For example, if the second number is 3 and the number of GPUs of unallocated components is 9, then 3 GPUs may be selected from the 9 GPUs of unallocated components and allocated to the selected component. After the allocation, the number of GPUs of unallocated components becomes 6. After the GPUs are allocated according to the above method, the process may return to step S2031 and select a component of unallocated GPUs from each component again.
[0084] Step S2034: assign all GPUs that have not been assigned components to the selected components.
[0085] In a possible embodiment, if the number of GPUs in unallocated components is not greater than the second number, all GPUs in unallocated components can be allocated to the selected component. For example, if the second number is 3 and there are 2 GPUs in unallocated components, then the 2 GPUs in unallocated components can be allocated to the selected component. After the allocation, it can be determined that the step of allocating GPUs to each component is completed.
[0086] In a possible embodiment, after the allocation of GPUs to each component is completed, at least one monitoring thread can be used to monitor the generated components. Before monitoring, it is also necessary to determine whether the component contains the target component, wherein the number of GPUs contained in the target component is less than the number of GPUs contained in any non-target component. It should be noted that there may be two situations when allocating GPUs to each component. In case 1, the number of GPUs contained in each component is the same; in case 2, the number of GPUs contained in one component is less than the number of GPUs contained in other components. For the component with a smaller number of GPUs in case 2, it can be considered as the target component. For example, the number of GPUs contained in the components is 3, 3, and 2 respectively, then the component with 2 GPUs is the target component.
[0087] In a possible embodiment, a static time period may be set for the target component, and the static time period is the time period used to monitor the difference GPUs in the target component and any non-target component. For example, if the number of GPUs accommodated in the target component is 2 and the number of GPUs accommodated in any non-target component is 5, then the static time period may be the time period used to monitor 5-2=3 GPUs.
[0088] In a possible embodiment, at least one monitoring thread is used to simultaneously monitor each generated component, and after obtaining the GPU parameters contained in the target component, the GPU parameters contained in the target component can be re-obtained after waiting for a static time. For example, there are 3 components corresponding to 3 monitoring threads, and the number of GPUs contained in these 3 components is 3, 3, and 2, respectively. For any non-target component, the monitoring thread can re-obtain the GPU parameters contained in the component after completing the acquisition of the parameters for 3 of the GPUs, and for the target component, since the number of GPUs contained in the target component is 1 less than that of the non-target component, after the monitoring thread corresponding to the target component completes the acquisition of the GPU parameters contained in the target component, it can wait for the time used to monitor 1 GPU, and then re-obtain the GPU parameters contained in the target component. Through the above implementation, the monitoring time of components containing different numbers of GPUs can be balanced, so that the progress of each monitoring thread monitoring each component is synchronized, ensuring the accuracy of monitoring.
[0089] In a possible embodiment, after the static time is set for the target component, at least one monitoring thread can be used to simultaneously obtain multiple GPU parameters contained in the generated component, and the obtained parameter values can be directly sent to a display device connected to the BMC, and the obtained parameter values, configuration information in the configuration file, and other contents can be displayed on the display device. It should be noted that on the display device, a monitoring interface can be implemented through a web interface that can provide an intuitive graphical user interface to display the parameter values, or a monitoring interface can be provided through the Intelligent Platform Management Interface (IPMI) standard protocol to implement hardware-level remote management and monitoring functions, and a unified monitoring interface can also be provided through the Redfish interface, which is not limited in this application. In the monitoring interface, the user can set and adjust various parameters in the configuration file, such as the target dynamic information monitoring interval, the parameter to be monitored, and the threshold of the parameter to be monitored, and a recommended value can be provided for the threshold of the parameter to be monitored to help the user better complete the editing of the configuration file.
[0090] In a possible embodiment, if any parameter of the GPU parameters obtained by the monitoring thread exceeds the corresponding parameter threshold, an alarm prompt is issued to the display device connected to the BMC. For example, if the temperature threshold is 60°C, and the temperature of the GPU parameters obtained by the monitoring thread is 66°C, then the BMC can issue an alarm prompt to the display device connected to the BMC, where the alarm prompt can include the parameter value, the GPU corresponding to the alarm event, and the corresponding solution, etc., which is not limited in this application.
[0091] In a specific implementation, the process of a monitoring method provided in the embodiment of the present application can be as follows: Figure 5 As shown, the following steps are included:
[0092] Step S501: start a timer corresponding to a first GPU, and obtain parameters to be obtained from the first GPU.
[0093] Step S502: The parameter values to be obtained from the first GPU are correspondingly saved in the first file.
[0094] Step S503, determining whether the multiple parameters to be obtained in the first file correspond to the parameter values to be obtained one by one, if so, executing step S504; if not, returning to executing step S502.
[0095] Step S504: stop the timer and use the obtained first duration as a dynamic information monitoring interval of the first GPU.
[0096] Step S505 , multiplying the dynamic information monitoring interval by the number of the plurality of GPUs, and dividing the multiplied result by the target dynamic information monitoring interval to obtain a quantitative relationship.
[0097] Step S506, determine whether the quantitative relationship is an integer multiple relationship, if so, execute step S507; if not, execute step S508.
[0098] Step S507: taking the value of the quantity relationship as the first quantity.
[0099] Step S508: increasing the integer part of the value of the quantity relationship by a set value as the first quantity.
[0100] Step S509: Generate at least one monitoring thread of the same number as the first number, generate components of the same number as the first number, and select a component to which a GPU is not allocated from each component.
[0101] Step S510, determining whether the number of GPUs without components allocated is greater than a second number, if so, executing step S511; if not, executing step S512.
[0102] Step S511, allocating the GPUs of the second number of unallocated components to the selected components, and returning to execute step S509.
[0103] Step S512: Allocate all GPUs that have not been assigned components to the selected components.
[0104] Step S513, setting a static duration for the target component, and using at least one monitoring thread to simultaneously obtain multiple GPU parameters contained in the generated component.
[0105] Step S514: If any parameter among the multiple GPU parameters exceeds the corresponding parameter threshold, an alarm is sent to a display device connected to the BMC.
[0106] Step S515, after obtaining the GPU parameters included in the target component, wait for the static time and then re-obtain the GPU parameters included in the target component.
[0107] Based on the same inventive concept, the present application embodiment provides a monitoring device, please refer to Figure 6 , the monitoring device 600 comprises:
[0108] A monitoring interval determining unit 601 is used to determine a dynamic information monitoring interval of each GPU among the multiple GPUs; the dynamic information monitoring interval is used to represent the time length used to obtain the to-be-obtained parameters of the corresponding GPU;
[0109] The thread quantity determining unit 602 is used to determine a first quantity based on a quantity relationship between the dynamic information monitoring interval and a target dynamic information monitoring interval, and generate at least one monitoring thread having the same quantity as the first quantity; the target dynamic information monitoring interval is used to represent a target duration for obtaining a parameter to be obtained for a corresponding GPU;
[0110] The monitoring thread monitoring unit 603 is configured to monitor the multiple GPUs using the at least one monitoring thread.
[0111] In a possible implementation, the monitoring interval determining unit 601 is configured to perform the following operations on a first GPU, where the first GPU is any one of the multiple GPUs:
[0112] Starting a timer corresponding to the first GPU, and obtaining a value of a parameter to be obtained from the first GPU;
[0113] The values of the parameters to be obtained obtained from the first GPU are saved in a first file; the first file includes a plurality of parameters to be obtained in the configuration file;
[0114] The timer stops timing until the multiple parameters to be obtained in the first file correspond to the values of the parameters to be obtained one by one, and a first duration is obtained; the configuration file is saved in the BMC;
[0115] The first duration is used as a dynamic information monitoring interval of the first GPU.
[0116] In a possible implementation manner, the thread quantity determining unit 602 is configured to determine the quantity relationship between the dynamic information monitoring interval and the target dynamic information monitoring interval in the following manner:
[0117] The dynamic information monitoring interval is multiplied by the number of the multiple GPUs, and the multiplied result is divided by the target dynamic information monitoring interval to obtain the quantitative relationship representing the multiple relationship between the dynamic information monitoring interval for monitoring the multiple GPUs and the target dynamic information monitoring interval.
[0118] In a possible implementation manner, the thread quantity determination unit 602 is configured to use the value of the quantity relationship as the first quantity if the quantity relationship is an integer multiple relationship;
[0119] If the quantity relationship is a non-integer multiple relationship, the integer part of the value of the quantity relationship is increased by a set value as the first quantity.
[0120] In a possible implementation, the monitoring thread monitoring unit 603 is configured to generate components of the same number as the first number;
[0121] Select a component to which a GPU is not assigned from each component;
[0122] If the number of GPUs of the unallocated components is greater than a second number, the GPUs of the second number of unallocated components are allocated to the selected component, and the step of selecting a component to which the GPUs are unallocated from each component is returned; the second number is used to represent the maximum number of GPUs that can be accommodated in the selected component; the second number is determined based on the target dynamic information monitoring interval and the dynamic information monitoring interval;
[0123] If the number of GPUs not assigned components is not greater than the second number, assigning all GPUs not assigned components to the selected component;
[0124] The generated components are monitored using the at least one monitoring thread.
[0125] In a possible implementation, the monitoring thread monitoring unit 603 is used to set a static time length for a target component; the number of GPUs included in the target component is less than the number of GPUs included in any non-target component; the static time length is the time length used to monitor the difference GPU between the target component and any non-target component;
[0126] The at least one monitoring thread is used to simultaneously monitor the generated components, and after obtaining the GPU parameters included in the target component, the GPU parameters included in the target component are obtained again after waiting for the static time period.
[0127] In a possible implementation, the monitoring thread monitoring unit 603 is used to use the at least one monitoring thread to simultaneously obtain multiple GPU parameters included in the generated component, and if any parameter of the multiple GPU parameters exceeds the corresponding parameter threshold, an alarm prompt is issued to a display device connected to the BMC.
[0128] Based on the same inventive concept, the embodiment of the present application provides a BMC, which can realize the functions of the monitoring device discussed above, please refer to Figure 7 The BMC 700 includes a memory 701 , a processor 702 and a bus 703 .
[0129] The memory 701 is used to store computer programs executed by the processor 702. The memory 701 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.
[0130] The memory 701 may be a volatile memory, such as a random-access memory (RAM); the memory 701 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 701 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 701 may be a combination of the above memories.
[0131] The processor 702 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 702 is used to implement the monitoring method in the above embodiment when calling the computer program stored in the memory 701 .
[0132] The specific connection medium between the memory 701 and the processor 702 is not limited in the embodiment of the present application. Figure 7 In the embodiment, the memory 701 and the processor 702 are connected via a bus 703. The bus 703 is connected to the processor 702 via a bus 703. Figure 7 The connections between other components are shown in bold lines, which are only for illustration and are not intended to be limiting. Bus 703 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0133] Based on the same inventive concept, the embodiment of the present application provides a computer-readable storage medium, and the computer program product includes: computer program code, when the computer program code is executed on a computer, the computer executes any monitoring method discussed above. Since the principle of solving the problem by the above-mentioned computer-readable storage medium is similar to that of the monitoring method, the implementation of the above-mentioned computer-readable storage medium can refer to the implementation of the method, and the repeated parts will not be repeated.
[0134] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0135] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0136] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0138] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A monitoring method, characterized in that: Applied to a baseboard management controller BMC, the BMC is connected to multiple GPUs; the method includes: Determine a dynamic information monitoring interval of each GPU among the multiple GPUs; the dynamic information monitoring interval is used to characterize the time length used to obtain the to-be-obtained parameters of the corresponding GPU; Based on the quantitative relationship between the dynamic information monitoring interval and the target dynamic information monitoring interval, a first quantity is determined, and at least one monitoring thread having the same number as the first quantity is generated; the target dynamic information monitoring interval is used to characterize a target duration for obtaining the to-be-obtained parameters of the corresponding GPU; The multiple GPUs are monitored using the at least one monitoring thread.
2. The method according to claim 1, characterized in that Determining the dynamic information monitoring interval of each GPU among the multiple GPUs includes: The following operations are performed on a first GPU, where the first GPU is any one of the multiple GPUs: Starting a timer corresponding to the first GPU, and obtaining a value of a parameter to be obtained from the first GPU; The values of the parameters to be obtained obtained from the first GPU are saved in a first file; the first file includes a plurality of parameters to be obtained in the configuration file; The timer stops timing until the multiple parameters to be obtained in the first file correspond to the values of the parameters to be obtained one by one, and a first duration is obtained; the configuration file is saved in the BMC; The first duration is used as a dynamic information monitoring interval of the first GPU.
3. The method according to claim 1, characterized in that The quantitative relationship between the dynamic information monitoring interval and the target dynamic information monitoring interval is determined by: The dynamic information monitoring interval is multiplied by the number of the multiple GPUs, and the multiplied result is divided by the target dynamic information monitoring interval to obtain the quantitative relationship representing the multiple relationship between the dynamic information monitoring interval for monitoring the multiple GPUs and the target dynamic information monitoring interval.
4. The method according to claim 1, characterized in that: The determining the first quantity based on the quantitative relationship between the dynamic information monitoring interval and the target dynamic information monitoring interval includes: If the quantity relationship is an integer multiple relationship, the value of the quantity relationship is used as the first quantity; If the quantity relationship is a non-integer multiple relationship, the integer part of the value of the quantity relationship is increased by a set value as the first quantity.
5. The method according to any one of claims 1 to 4, characterized in that: The monitoring of the multiple GPUs by using the at least one monitoring thread includes: generating a number of components equal to the first number; Select a component to which a GPU is not assigned from each component; If the number of GPUs of the unallocated components is greater than a second number, the GPUs of the second number of unallocated components are allocated to the selected component, and the step of selecting a component to which the GPUs are unallocated from each component is returned; the second number is used to represent the maximum number of GPUs that can be accommodated in the selected component; the second number is determined based on the target dynamic information monitoring interval and the dynamic information monitoring interval; If the number of GPUs not assigned components is not greater than the second number, assigning all GPUs not assigned components to the selected component; The generated components are monitored using the at least one monitoring thread.
6. The method according to claim 5, characterized in that The components correspond to the monitoring threads one by one; and the monitoring of the generated components by using the at least one monitoring thread includes: A static time length is set for the target component; the number of GPUs included in the target component is less than the number of GPUs included in any non-target component; the static time length is the time length used to monitor the difference GPUs in the target component and any non-target component; The at least one monitoring thread is used to simultaneously monitor the generated components, and after obtaining the GPU parameters included in the target component, the GPU parameters included in the target component are obtained again after waiting for the static time period.
7. The method according to claim 6, characterized in that The using the at least one monitoring thread to simultaneously start monitoring each generated component comprises: The at least one monitoring thread is used to simultaneously obtain a plurality of GPU parameters included in the generated component, and if any parameter of the plurality of GPU parameters exceeds a corresponding parameter threshold, an alarm is issued to a display device connected to the BMC.
8. A monitoring device, characterized in that: The device comprises: A monitoring interval determining unit, configured to determine a dynamic information monitoring interval of each GPU among the plurality of GPUs; the dynamic information monitoring interval is used to characterize a time length used to obtain a parameter to be obtained of the corresponding GPU; a thread quantity determination unit, configured to determine a first quantity based on a quantity relationship between the dynamic information monitoring interval and a target dynamic information monitoring interval, and generate at least one monitoring thread having the same quantity as the first quantity; the target dynamic information monitoring interval is used to characterize a target duration for obtaining a parameter to be obtained of a corresponding GPU; The monitoring thread monitoring unit is configured to monitor the multiple GPUs using the at least one monitoring thread.
9. A BMC, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program to be run on the processor, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
Citation Information
Cited By
Information processing method and electronic equipment
CN120821634A
