A fan speed control method, device and apparatus

Abstract

The application provides a fan rotating speed control method, device and equipment, the method comprising: determining an initial duty cycle currently adopted by each fan; based on a target sequence of multiple fans, sequentially traversing each fan as a candidate fan; adjusting the initial duty cycle corresponding to the candidate fan multiple times to obtain different multiple candidate duty cycles; for each candidate duty cycle, obtaining reward and punishment data of a network device under the candidate duty cycle, and determining a reward and punishment parameter value corresponding to the candidate duty cycle based on the reward and punishment data; selecting an optimal reward and punishment parameter value from the reward and punishment parameter values corresponding to the multiple candidate duty cycles, and determining the candidate duty cycle corresponding to the optimal reward and punishment parameter value as a target duty cycle corresponding to the candidate fan; after obtaining the target duty cycle corresponding to each fan, controlling the fan based on the rotating speed corresponding to the target duty cycle of each fan. Through the technical scheme, the rotating speed of each fan is effectively controlled, and the power consumption and noise of the fan are reduced.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a fan speed control method, device, and equipment. Background Technology

[0002] Network devices (such as routers and switches) can include multiple boards (such as main control boards, interface boards, and service boards). During the operation of these boards, the power consumption of different boards varies greatly. For example, the power consumption of a board may be only 100W, while the power consumption of a board may exceed 600W. When multiple boards are operating at high power consumption, the temperature of the network device will be very high, which will make the network device prone to failure.

[0003] To cool network devices, multiple fans are typically deployed. Higher fan speeds result in better cooling, while lower speeds lead to poorer cooling. Higher fan speeds also increase noise and power consumption, resulting in energy waste. Conversely, lower fan speeds reduce noise and power consumption, minimizing energy waste.

[0004] In summary, it can be seen that the higher the fan speed, the better the cooling effect of the network device, but the louder the fan noise and the higher the power consumption. Conversely, the lower the fan speed, the worse the cooling effect of the network device, but the quieter the fan noise and the lower the power consumption. Therefore, it is necessary to control the fan speed to avoid the network device from overheating while ensuring low fan noise and low power consumption, thus achieving the goal of energy saving and noise reduction.

[0005] However, there is no effective way in the relevant technology to control the fan speed to avoid the network equipment from overheating, and to ensure that the fan noise and power consumption are low. Summary of the Invention

[0006] This application provides a fan speed control method applied to a network device, the network device including multiple fans, wherein when the network device is in an adjustment state, the method includes: Determine the initial duty cycle currently being used by each fan; Based on the target order of the multiple fans, each fan is sequentially traversed as a candidate fan; The initial duty cycle corresponding to the candidate fan is adjusted multiple times to obtain multiple different candidate duty cycles. For each candidate duty cycle, the reward and penalty data of the network device under the candidate duty cycle is obtained, and the reward and penalty parameter value corresponding to the candidate duty cycle is determined based on the reward and penalty data. The optimal reward and penalty parameter value is selected from the reward and penalty parameter values ​​corresponding to the multiple candidate duty cycles, and the candidate duty cycle corresponding to the optimal reward and penalty parameter value is determined as the target duty cycle corresponding to the candidate fan. After obtaining the target duty cycle for each fan, if the network device is in a stable state, the fan is controlled based on the rotational speed corresponding to the target duty cycle for each fan.

[0007] This application provides a fan speed control device applied to a network device, the network device including multiple fans, and the device comprising: when the network device is in an adjustment state. The determination module is used to determine the initial duty cycle currently being used by each fan; The processing module is configured to: sequentially traverse each fan as a candidate fan based on the target order of the multiple fans; adjust the initial duty cycle corresponding to the candidate fan multiple times to obtain multiple different candidate duty cycles; for each candidate duty cycle, obtain the reward / penalty data of the network device under the candidate duty cycle, determine the reward / penalty parameter value corresponding to the candidate duty cycle based on the reward / penalty data; select the optimal reward / penalty parameter value from the reward / penalty parameter values ​​corresponding to the multiple candidate duty cycles, and determine the candidate duty cycle corresponding to the optimal reward / penalty parameter value as the target duty cycle corresponding to the candidate fan; The control module is used to control the fan based on the rotational speed corresponding to the target duty cycle of each fan, if the network device is in a stable state, after obtaining the target duty cycle of each fan.

[0008] This application provides a network device, including: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the fan speed control method of the above example.

[0009] As can be seen from the above technical solutions, in this embodiment, the initial duty cycle corresponding to the fan is adjusted multiple times to obtain several different candidate duty cycles. The optimal reward / penalty parameter value is selected from the reward / penalty parameter values ​​corresponding to these candidate duty cycles, and the candidate duty cycle corresponding to the optimal reward / penalty parameter value is determined as the target duty cycle for the fan, thus obtaining the target duty cycle for each fan. Based on the target duty cycle for each fan, the fan speed is determined, thereby effectively controlling the speed of each fan, reducing fan power consumption and noise. While avoiding excessively high temperatures in the network equipment, this ensures low fan noise and low fan power consumption, achieving the purpose of energy saving and noise reduction. This meets the energy saving and noise reduction requirements of network equipment, reduces noise pollution, reduces the inhalation of harmful substances in the air such as dust, and helps reduce corrosion damage to network equipment. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings of the embodiments of this application.

[0011] Figure 1 This is a flowchart illustrating a fan speed control method according to one embodiment of this application; Figure 2 This is a schematic diagram of fan speed control in one embodiment of this application; Figure 3 This is a schematic diagram of the DDPG algorithm model in one embodiment of this application; Figure 4 This is a schematic diagram of the fan speed control device in one embodiment of this application; Figure 5 This is a hardware structure diagram of a network device according to one embodiment of this application. Detailed Implementation

[0012] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” as used in this application and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0013] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0014] This application proposes a fan speed control method, which can be applied to network devices (such as routers, switches, etc.). These network devices may include multiple fans. (See also...) Figure 1 The diagram shown is a flowchart of the fan speed control method, which may include the following steps: Step 101: When the network device is in the adjustment state, determine the initial duty cycle currently being used by each fan. For example, if at the current moment, a fan is being controlled using a speed corresponding to a certain duty cycle, then that duty cycle can be used as the initial duty cycle currently being used by that fan.

[0015] Step 102: Based on the target order of multiple fans, traverse each fan in turn as a candidate fan.

[0016] Step 103: Adjust the initial duty cycle of the candidate fan being traversed multiple times to obtain multiple different candidate duty cycles; for each candidate duty cycle, obtain the reward and penalty data of the network device under that candidate duty cycle, and determine the reward and penalty parameter value corresponding to that candidate duty cycle based on the reward and penalty data; select the optimal reward and penalty parameter value from the reward and penalty parameter values ​​corresponding to multiple candidate duty cycles, and determine the candidate duty cycle corresponding to the optimal reward and penalty parameter value as the target duty cycle corresponding to that candidate fan.

[0017] For example, determining the reward / penalty parameter value corresponding to the candidate duty cycle based on the reward / penalty data may include, but is not limited to: determining the total system power and the maximum duty cycle value based on the reward / penalty data; wherein the reward / penalty data includes the duty cycle corresponding to each fan, and the maximum duty cycle value is the maximum value among all duty cycles; if the reward / penalty data includes power supply power, then the total system power is determined based on the power supply power; if the reward / penalty data includes the power corresponding to each board and the power corresponding to each fan, then the total system power is determined based on the power corresponding to all boards and the power corresponding to all fans. Then, the target noise value is determined based on the maximum duty cycle value, and the reward / penalty parameter value is determined based on the total system power and the first weighting coefficient corresponding to the total system power, the target noise value and the second weighting coefficient corresponding to the target noise value.

[0018] For example, the sum of the first weighting coefficient and the second weighting coefficient can be a fixed value (such as 1); if energy saving is more important than noise reduction, the first weighting coefficient can be greater than the second weighting coefficient; if noise reduction is more important than energy saving, the first weighting coefficient can be less than the second weighting coefficient.

[0019] Step 104: After obtaining the target duty cycle for each fan, if the network device is in a stable state, control the fan based on the rotational speed corresponding to the target duty cycle for each fan.

[0020] In one possible implementation, after obtaining the target duty cycle for each fan, if the network device is still in an unadjusted state, the target duty cycle for each fan can be updated to the initial duty cycle for that fan, and the operation of iterating through each fan as a candidate fan based on the target order of multiple fans can be returned, i.e., the execution of step 102 can be returned. When re-executing step 102, the target order of the multiple fans may change, or it may remain unchanged.

[0021] For example, if the total number of fans that meet the target conditions is not less than a preset threshold (which can be configured empirically, such as the total number of fans * a%, where a can be 100, 90, etc.), then the network device can be determined to be in a stable state. Otherwise, if the total number of fans that meet the target conditions is less than the preset threshold, then the network device can be determined to be in a state awaiting adjustment. Specifically, for each fan, if the target duty cycle corresponding to the fan is the same as the initial duty cycle corresponding to the fan, then the fan has met the target conditions (i.e., the fan has been adjusted to a locally optimal state). Otherwise, if the target duty cycle corresponding to the fan is the same as the initial duty cycle corresponding to the fan, then the fan does not meet the target conditions.

[0022] In one possible implementation, after the network device powers on or triggers a temperature alarm, the duty cycle to be adjusted for each fan can be determined based on the temperature value of each board in the network device. When controlling the fan based on the rotational speed corresponding to the duty cycle to be adjusted for each fan, the absolute value of the difference between the temperature value of each board and the target temperature value can be made less than a temperature threshold. Once the absolute value of the difference between the temperature value of each board and the target temperature value is less than the temperature threshold, the network device can be determined to be in an adjustment state. The duty cycle to be adjusted for each fan is then updated to the initial duty cycle corresponding to that fan, and the fan is controlled based on the rotational speed corresponding to the initial duty cycle.

[0023] In one possible implementation, after the network device reaches a stable state, detection state data corresponding to the network device can be acquired and input into the trained target action model to obtain the output duty cycle for each fan; wherein, the output duty cycles for different fans may be the same or different. A first reward / penalty parameter value corresponding to the output duty cycle is determined based on the reward / penalty data of the network device under each fan's output duty cycle; a second reward / penalty parameter value corresponding to the target duty cycle is determined based on the reward / penalty data of the network device under each fan's target duty cycle. If the first reward / penalty parameter value is better than the second reward / penalty parameter value, the network device is determined to be in an adjustment state, the output duty cycle for each fan is updated to the initial duty cycle for that fan, and the execution returns to the target order based on multiple fans, sequentially traversing each fan as a candidate fan, i.e., returning to step 102.

[0024] For example, the detection status data corresponding to the network device may include, but is not limited to: sensor data corresponding to each board within the network device, system data corresponding to the network device, and target duty cycle corresponding to each fan. Specifically, the sensor data corresponding to each board includes at least one of the following: junction temperature data corresponding to the chip within the board, maximum temperature corresponding to each temperature measurement point within the board, and power corresponding to the board; the system data includes at least one of the following: ambient temperature, power supply power, and power corresponding to each fan.

[0025] For example, before inputting the detected state data into the target action model, the target action model can be trained. The training process of the target action model may include: after the network device is in a stable state, acquiring sample state data corresponding to the network device; inputting the sample state data into the initial action model to obtain the sample duty cycle corresponding to each fan; inputting the sample state data and the sample duty cycle corresponding to each fan into the initial evaluation model to obtain the state action value corresponding to the sample state data; determining the reward / penalty parameter value corresponding to the state action value based on the sample state data; and training the initial action model and the initial evaluation model based on the reward / penalty parameter value and the state action value to obtain the trained target action model and target evaluation model.

[0026] For example, training an initial action model and an initial evaluation model based on reward / penalty parameter values ​​and state / action values ​​to obtain a trained target action model and a target evaluation model may include, but is not limited to: determining a first error value corresponding to the initial action model based on the state / action value; adjusting the network parameters of the initial action model based on the first error value to obtain an adjusted action model; determining a second error value corresponding to the initial evaluation model based on the reward / penalty parameter value and the state / action value; adjusting the network parameters of the initial evaluation model based on the second error value to obtain an adjusted evaluation model; if the adjusted action model and the adjusted evaluation model have converged, the adjusted action model can be determined as the target action model, and the adjusted evaluation model can be determined as the target evaluation model; if the adjusted action model and / or the adjusted evaluation model have not converged, the adjusted action model can be determined as the initial action model, and the adjusted evaluation model can be determined as the initial evaluation model, and the operation of inputting sample state data into the initial action model can be returned.

[0027] For example, determining the first error value corresponding to the initial action model based on the state action value may include, but is not limited to, determining the first error value based on the following formula: L(Θ) v ) = The second error value corresponding to the initial evaluation model is determined based on the reward / penalty parameter value and the state action value. This can include, but is not limited to, determining the second error value based on the following formula: L(Θ) u ) = 1 / 2 In the above formula, L(Θ) v L(Θ) represents the first error value. u ) indicates the second error value. s Represents sample state data. a Indicates the sample duty cycle. Indicates the state action value. This indicates the reward / penalty parameter value.

[0028] As can be seen from the above technical solutions, in this embodiment, the initial duty cycle corresponding to the fan is adjusted multiple times to obtain several different candidate duty cycles. The optimal reward / penalty parameter value is selected from the reward / penalty parameter values ​​corresponding to these candidate duty cycles, and the candidate duty cycle corresponding to the optimal reward / penalty parameter value is determined as the target duty cycle for the fan, thus obtaining the target duty cycle for each fan. Based on the target duty cycle for each fan, the fan speed is determined, thereby effectively controlling the speed of each fan, reducing fan power consumption and noise. While avoiding excessively high temperatures in the network equipment, this ensures low fan noise and low fan power consumption, achieving the purpose of energy saving and noise reduction. This meets the energy saving and noise reduction requirements of network equipment, reduces noise pollution, reduces the inhalation of harmful substances in the air such as dust, and helps reduce corrosion damage to network equipment.

[0029] The technical solutions of the embodiments of this application will be described below in conjunction with specific application scenarios.

[0030] Network devices (such as routers and switches; switches can be chassis switches) may include multiple boards (such as main control boards, interface boards, and service boards) and multiple fans. During the operation of these boards, the temperature of the network device rises. Fans are used to cool the network device to prevent overheating and thus avoid malfunctions. Obviously, the higher the fan speed, the better the cooling effect; conversely, the lower the fan speed, the worse the cooling effect. However, higher fan speeds also result in higher fan noise and higher power consumption, leading to energy waste. Lower fan speeds, on the other hand, result in lower fan noise and lower power consumption, reducing energy waste.

[0031] In summary, it can be seen that the higher the fan speed, the better the cooling effect of the network device, but the louder the fan noise and the higher the power consumption. Conversely, the lower the fan speed, the worse the cooling effect of the network device, but the quieter the fan noise and the lower the power consumption. Therefore, it is necessary to control the fan speed to avoid the network device from overheating while ensuring low fan noise and low power consumption, thus achieving the goal of energy saving and noise reduction.

[0032] However, there is no effective way in the relevant technology to control the fan speed to avoid the network equipment from overheating, and to ensure that the fan noise and power consumption are low.

[0033] For example, you can try to reduce the fan speed to reduce fan noise and power consumption. However, if the fan speed is too low, the temperature of the network device will increase. For the high-power chips inside the network device, as the temperature of the network device increases, the power of the high-power chips will increase with the temperature, that is, the overall power consumption of the network device will be greater. In other words, reducing the fan speed will actually lead to greater overall power consumption.

[0034] For example, an empirical temperature can be set. When the network device's temperature exceeds this empirical temperature, the fan speed increases; when the network device's temperature falls below this empirical temperature, the fan speed decreases. However, it's difficult to set the empirical temperature accurately. If the empirical temperature is inappropriate, it can cause the fan speed to fluctuate repeatedly, resulting in oscillating temperatures of the network device. For instance, increasing the fan speed might cause the network device's temperature to fall below the empirical temperature, leading to a decrease in fan speed. Conversely, decreasing the fan speed might cause the network device's temperature to rise above the empirical temperature, requiring an increase in fan speed, thus causing the fan speed to oscillate.

[0035] In response to the above findings, and to meet the energy-saving and noise-reduction requirements of network devices, this application proposes a fan speed control method that combines PID (Proportion Integral Differential) algorithm, Greedy algorithm, and DDPG (Deep Deterministic Policy Gradient) algorithm to control fan speed. This method effectively controls the speed of each fan, reduces fan power consumption and noise, and prevents network devices from overheating.

[0036] See Figure 2 As shown, after the network device powers on, it is determined that the network device is in the power-on startup state. While in this state, a PID algorithm is used to control the fan speed until the absolute value of the difference between the temperature value of each board and the target temperature value is less than the temperature threshold. At this point, the network device is considered to be in a state awaiting adjustment. While in this state, a Greedy algorithm is used to control the fan speed until a local optimum solution for each fan (i.e., the local optimum duty cycle, denoted as the target duty cycle) is found. The target duty cycle of each fan is then used to control the fan speed, and the network device is determined to be in a stable state, i.e., it has reached a locally optimal stable state.

[0037] The data collected during the Greedy algorithm process can be used to train the DDPG model. When the network device is in a stable state, the DDPG model can be used to determine whether the Greedy algorithm needs to be restarted. For example, if the environment and power in the stable state are determined to change based on the DDPG model, the Greedy algorithm is triggered to find the next local optimum. In other words, the network device can be re-determined to be in an adjustment state, and the Greedy algorithm can be used to control the fan speed until the target duty cycle of each fan is found. The target duty cycle of each fan is then used to control the fan speed, and the network device is determined to be in a stable state.

[0038] Based on the above technical concept, the technical solution of the embodiments of this application may involve the following process: First, after the network device powers on, based on the temperature value of each board, the duty cycle to be adjusted for each fan is determined. The fan speed is then controlled based on the speed corresponding to that duty cycle (e.g., controlling fan 1 based on its speed, controlling fan 2 based on its speed, and so on). When controlling the fan based on its speed, the duty cycle can be continuously adjusted to ensure that the absolute value of the difference between the board's temperature and the target temperature is less than a temperature threshold (configurable empirically). In other words, when the absolute value of the difference is greater than the temperature threshold, the fan's duty cycle can be increased to decrease the board's temperature; conversely, when the absolute value is less than the temperature threshold, the fan's duty cycle can be decreased to increase the board's temperature. Clearly, by repeatedly adjusting the duty cycle of each fan, the absolute value of the difference between the board's temperature and the target temperature is eventually made less than the temperature threshold.

[0039] In one possible implementation, based on the temperature value of each board of the network device, a PID algorithm is used to determine the duty cycle to be adjusted for each fan. The duty cycle to be adjusted for each fan is continuously adjusted based on the PID algorithm, so that the absolute value of the difference between the temperature value of each board and the target temperature value is less than the temperature threshold. The process of using the PID algorithm to determine the duty cycle to be adjusted is not limited in this embodiment.

[0040] In summary, in this embodiment, after the network device is powered on, the temperature and power consumption change rapidly. Therefore, in order to ensure the safety of the network device, a PID algorithm can be used to continuously adjust the duty cycle of each fan, so that the temperature value of each board reaches a stable value. The stable temperature value of a board means that the absolute value of the difference between the temperature value of the board and the target temperature value is less than the temperature threshold, that is, the temperature value of the board is near the target temperature value and fluctuates slightly around the target temperature value.

[0041] For example, once the absolute value of the difference between the temperature value of each board and the target temperature value is less than the temperature threshold, i.e., when the temperature value of each board reaches a stable value, it can be determined that the network device is in a state awaiting adjustment. When the network device is in a state awaiting adjustment, the Greedy algorithm can be used to control the fan speed, and the control process is described in subsequent embodiments, instead of using the PID algorithm to control the fan speed.

[0042] When the network device is in the adjustment state, the duty cycle to be adjusted for each fan is updated to the initial duty cycle for that fan. The fan is controlled based on the speed corresponding to the initial duty cycle. For example, the speed of fan 1 is controlled based on the speed corresponding to the initial duty cycle of fan 1, the speed of fan 2 is controlled based on the speed corresponding to the initial duty cycle of fan 2, and so on.

[0043] In one possible implementation, if the network device does not start the AI ​​fan speed control algorithm after startup, the algorithm of this embodiment can be omitted to control the fan speed. The PID algorithm is used to control the fan speed only after the network device starts the AI ​​fan speed control algorithm.

[0044] Second, when network devices are in a state of readiness for adjustment, the Greedy algorithm is used to control fan speed until the target duty cycle for each fan is found, and then the fan speed is controlled using the target duty cycle for each fan. For example, the target duty cycle for each fan can be obtained using the following steps: Step S11: Determine the initial duty cycle currently being used by each fan.

[0045] Step S12: Based on the target order of multiple fans, traverse each fan in turn as a candidate fan.

[0046] For example, multiple fans can be randomly sorted, and the random order can be used as the target order of the multiple fans. Assuming the target order is fan 1, fan 2, fan 3, then first traverse to select fan 1 as a candidate fan, then traverse to select fan 2 as a candidate fan, and then traverse to select fan 3 as a candidate fan.

[0047] Step S13: For each candidate fan, adjust the initial duty cycle multiple times to obtain several different candidate duty cycles. For each candidate duty cycle, obtain the reward / penalty data of the network device under that candidate duty cycle, and determine the reward / penalty parameter value corresponding to that candidate duty cycle based on the reward / penalty data. Select the optimal reward / penalty parameter value from the reward / penalty parameter values ​​corresponding to the multiple candidate duty cycles, and determine the candidate duty cycle corresponding to the optimal reward / penalty parameter value as the target duty cycle corresponding to that candidate fan.

[0048] For example, suppose fan 1 corresponds to initial duty cycle 1, fan 2 to initial duty cycle 2, and fan 3 to initial duty cycle 3. When selecting fan 1 as a candidate fan, the initial duty cycle 1 corresponding to fan 1 is adjusted multiple times to obtain candidate duty cycles 1-1, 1-2, etc. Based on candidate duty cycles 1-1, initial duty cycles 2, and initial duty cycles 3, reward and penalty data is obtained, and the reward and penalty parameter value corresponding to candidate duty cycle 1-1 is determined based on this data. Based on candidate duty cycles 1-2, initial duty cycles 2, and initial duty cycles 3, reward and penalty data is obtained, and the reward and penalty parameter value corresponding to candidate duty cycle 1-2 is determined based on this data, and so on. Assuming the reward and penalty parameter value corresponding to candidate duty cycle 1-2 is the optimal reward and penalty parameter value, then candidate duty cycle 1-2 is taken as the target duty cycle corresponding to fan 1, denoted as target duty cycle 1-2.

[0049] When selecting fan 2 as a candidate fan, the initial duty cycle 2 corresponding to fan 2 is adjusted multiple times to obtain candidate duty cycles 2-1, 2-2, etc. Based on the target duty cycle 1-2 (the initial duty cycle 1 of fan 1 has been adjusted to the target duty cycle 1-2), candidate duty cycles 2-1, and initial duty cycle 3, reward / penalty data is obtained. Based on this data, the reward / penalty parameter value corresponding to candidate duty cycle 2-1 is determined. This process is repeated to obtain the reward / penalty parameter value for each candidate duty cycle. Assuming the reward / penalty parameter value for candidate duty cycle 2-1 is the optimal value, then candidate duty cycle 2-1 is taken as the target duty cycle for fan 2. This process is repeated to obtain the target duty cycle for each fan.

[0050] For example, obtaining reward and penalty data based on candidate duty cycle 1-1, initial duty cycle 2, and initial duty cycle 3 means obtaining the reward and penalty data corresponding to the network device based on controlling the speed of fan 1 with candidate duty cycle 1-1 (as the initial duty cycle), controlling the speed of fan 2 with initial duty cycle 2, and controlling the speed of fan 3 with initial duty cycle 3.

[0051] For example, adjusting the initial duty cycle 1 multiple times to obtain multiple candidate duty cycles means: In the first adjustment, the initial duty cycle 1 can be adjusted upwards (or downwards) to obtain candidate duty cycle 1-1. In the second adjustment, if the reward / penalty parameter value corresponding to the initial duty cycle 1 is better than the reward / penalty parameter value corresponding to candidate duty cycle 1-1, it means that adjusting upwards results in a worse reward / penalty parameter value. Therefore, the initial duty cycle 1 is adjusted downwards to obtain candidate duty cycle 1-2. If the reward / penalty parameter value corresponding to candidate duty cycle 1-1 is better than the reward / penalty parameter value corresponding to the initial duty cycle 1, it means that adjusting upwards results in a better reward / penalty parameter value. Therefore, the initial duty cycle 1 is adjusted upwards again to obtain candidate duty cycle 1-2. This process continues, adjusting the duty cycle in the direction that improves the reward / penalty parameter value in each adjustment, ultimately obtaining a local optimum, i.e., the target duty cycle.

[0052] For example, the reward and penalty data corresponding to network devices may include, but are not limited to: the duty cycle corresponding to each fan (i.e., the duty cycle currently in use; if the initial duty cycle corresponding to the fan is adjusted to the candidate duty cycle, the candidate duty cycle is used as reward and penalty data; if the initial duty cycle corresponding to the fan is adjusted to the target duty cycle, the target duty cycle is used as reward and penalty data; if the initial duty cycle corresponding to the fan is not adjusted, the initial duty cycle is used as reward and penalty data); power supply; power corresponding to all boards and power corresponding to all fans.

[0053] For example, the total system power and maximum duty cycle value can be determined based on the reward / penalty data. This reward / penalty data includes the duty cycle corresponding to each fan, and the maximum duty cycle value can be the highest among all duty cycles. If the reward / penalty data includes power supply power, the total system power can be determined based on the power supply power. If the reward / penalty data includes the power corresponding to each board and the power corresponding to each fan, the total system power can be determined based on the power corresponding to all boards and the power corresponding to all fans. Then, the target noise value is determined based on the maximum duty cycle value, and the reward / penalty parameter value is determined based on the total system power and its corresponding first weighting coefficient, the target noise value and its corresponding second weighting coefficient.

[0054] For example, the reward and punishment parameter values ​​can be determined using the following formula, and there are no restrictions on the method of determination.

[0055] N(S) In the above formula, r Indicates the reward and punishment parameter value. Represents the total power of the system E The corresponding first weight coefficient, E This represents the total system power, which can be determined based on the power supply. E That is, the total power of the system EThe total system power can be determined based on the power consumption of all individual boards and the power consumption of all fans. E That is, the total power of the system E This is the sum of the power corresponding to all individual boards and the power corresponding to all fans.

[0056] In the above formula, This represents the second weighting coefficient corresponding to the target noise value N(S). S This represents the maximum duty cycle, which is the maximum duty cycle among all fan duty cycles, also known as the maximum duty cycle. N(S) represents the target noise value, and N represents the configured function. There are no restrictions on this configured function N; it can be configured based on experience. The configured function N represents the functional relationship between the maximum duty cycle and the target noise value. That is, the input of the configured function N is the maximum duty cycle, and the output of the configured function N is the target noise value. Therefore, the maximum duty cycle can be substituted into the configured function N to obtain the target noise value.

[0057] As can be seen from the above formula, the reward and penalty parameter values ​​can be determined based on the total system power and the first weighting coefficient corresponding to the total system power, the target noise value and the second weighting coefficient corresponding to the target noise value.

[0058] In one possible implementation, the first weighting coefficient corresponding to the total system power can be adjusted based on the customer's needs for energy saving and noise reduction. The second weighting coefficient corresponding to the target noise value First weighting coefficient The second weighting coefficient represents the magnitude of the energy-saving performance. This indicates the weight of the noise reduction performance.

[0059] For example, the first weighting coefficient With the second weighting coefficient The sum can be a fixed value (such as 1), that is Based on customers' needs for energy saving and noise reduction, the first weighting coefficient was adjusted. Second weighting coefficient For example, if energy saving is more important than noise reduction, then the first weighting coefficient... It can be greater than the second weighting coefficient. If noise reduction is more important than energy saving, then the first weighting coefficient is... It can be less than or greater than the second weighting coefficient. Regarding the first weighting coefficient Second weighting coefficient The value of is not limited in this embodiment, and the first weighting coefficient can be configured based on experience. Second weighting coefficient .

[0060] In summary, each time the initial duty cycle of a candidate fan is adjusted to obtain a candidate duty cycle, reward / penalty data under that candidate duty cycle can be collected, thereby obtaining the reward / penalty parameter value corresponding to that candidate duty cycle. This process yields reward / penalty parameter values ​​for multiple candidate duty cycles. The optimal reward / penalty parameter value, such as the minimum reward / penalty parameter value, is selected from these multiple candidate duty cycles. The candidate duty cycle corresponding to the optimal reward / penalty parameter value is then used as the target duty cycle, thus obtaining the target duty cycle for each fan.

[0061] Step S14: After obtaining the target duty cycle for each fan, count the total number of fans that have met the target conditions. Fans that have met the target conditions are those whose duty cycles have not changed, meaning that the target duty cycle is already locally optimal and there is no need to adjust the target duty cycle of this fan.

[0062] For example, for each fan, if the target duty cycle and the initial duty cycle are the same, then the fan meets the target condition; if the target duty cycle and the initial duty cycle are different, then the fan does not meet the target condition. After performing the above processing on each fan, the total number of fans that meet the target condition can be obtained.

[0063] Step S15: Determine whether the total number of fans that have met the target conditions is not less than a preset threshold (e.g., the total number of all fans * a%, where a can be 100, 90, etc.). If yes, proceed to step S16; otherwise, proceed to step S17.

[0064] Step S16: Determine that the network device is in a stable state, that is, the target duty cycle for each fan has been obtained, and control the fan based on the speed corresponding to the target duty cycle for each fan.

[0065] For example, if the total number of fans that have met the target conditions is greater than or equal to a preset threshold, it means that the target duty cycle of all or most fans is the same as the initial duty cycle, that is, the duty cycle has not changed. Therefore, the target duty cycle of these fans is already a local optimum, the optimal reward and penalty parameter value has been found, there is no need to continue to adjust the duty cycle, the network device is considered to have reached a stable state, and the duty cycle is no longer adjusted.

[0066] In summary, by obtaining the target duty cycle for each fan, the fan can be controlled based on the speed corresponding to the target duty cycle. This process will not be elaborated further.

[0067] Step S17: Determine that the network device is in an adjustment state (i.e., the state has not changed), update the target duty cycle corresponding to each fan to the initial duty cycle corresponding to that fan, and return to execute step S12.

[0068] For example, if the total number of fans that have met the target conditions is less than a preset threshold, it means that the target duty cycle of most fans is different from the initial duty cycle, that is, the duty cycle has changed. Therefore, the target duty cycle of these fans is not a local optimum, and the optimal reward / penalty parameter value has not yet been found, so there is a need to continue to adjust the duty cycle.

[0069] While continuing to adjust the duty cycle, the target duty cycle for each fan can be updated to the initial duty cycle for that fan. Based on these initial duty cycles, step S12 is re-executed. When re-executing step S12, the target order (i.e., random order) of multiple fans can be redefined. The target order of multiple fans can change, or it can remain unchanged.

[0070] In summary, by repeatedly executing steps S12-S17, the initial duty cycle corresponding to each fan can be iterated multiple times until the total number of fans that meet the target conditions is greater than or equal to a preset threshold. At this point, the network device is considered to have reached a stable state, and the target duty cycle corresponding to each fan is then output.

[0071] Third, during the operation of network equipment, to ensure its safety, temperature monitoring can be performed on each temperature control point. If the temperature value at any control point exceeds a preset temperature threshold, it indicates that the network equipment is in a high-temperature danger state, i.e., a temperature alarm is triggered. When a temperature alarm occurs, based on the temperature value of each board in the network equipment, a PID algorithm can be used to determine the duty cycle to be adjusted for each fan. Based on the speed corresponding to the duty cycle to be adjusted for each fan, the fan speed is controlled so that the absolute value of the difference between the temperature value of each board and the target temperature value is less than the temperature threshold, thereby ensuring the safety of the network equipment. The specific process can be found in point one and will not be repeated here.

[0072] Fourth, after the network device is in a stable state, its operating environment can be periodically monitored. If the operating environment changes, the network device can be switched from a stable state to a state requiring adjustment. In this state, the Greedy algorithm is reused to control the fan speed until the target duty cycle for each fan is found. The specific process of the Greedy algorithm can be found in point two and will not be elaborated here.

[0073] Specifically, if there is a single-board insertion or removal operation on the network device, it is determined that the operating environment of the network device has changed. Alternatively, if the ambient temperature of the network device changes significantly, it is determined that the operating environment of the network device has changed. Or, if the forwarding traffic of the network device increases significantly, it is determined that the operating environment of the network device has changed. Or, if the forwarding traffic of the network device decreases significantly, it is determined that the operating environment of the network device has changed. Of course, these are just a few examples and are not considered as a limitation.

[0074] Because network devices operate in a relatively stable environment that is not easily changed, they can remain in a stable state for a considerable period of time without repeatedly adjusting the target duty cycle of each fan.

[0075] For example, referring to step S16, after the network device is in a stable state, the reward and penalty data of the network device under the target duty cycle (i.e., the reward and penalty data includes the target duty cycle of each fan) can be obtained, and the reward and penalty parameter value corresponding to the target duty cycle can be determined based on the reward and penalty data, which is denoted as reward and penalty parameter value X1.

[0076] After the network device is in a stable state, the reward and penalty data corresponding to the network device can be obtained periodically (although the duty cycle in the reward and penalty data has not changed, the power supply, the power of the board, the power of the fan, etc. may change), and the reward and penalty parameter value X2 can be determined based on the reward and penalty data.

[0077] If the reward / penalty parameter value X2 is better than the reward / penalty parameter value X1, and the difference between the reward / penalty parameter value X1 and the reward / penalty parameter value X2 is relatively large (such as greater than a certain threshold), that is, the reward / penalty parameter value has changed significantly, then the network device can be adjusted from a stable state to a state to be adjusted. When the network device is in the state to be adjusted, the Greedy algorithm is used again to control the fan speed until the target duty cycle of each fan is found.

[0078] If the reward / penalty parameter value X1 is better than the reward / penalty parameter value X2, or if the reward / penalty parameter value X2 is better than the reward / penalty parameter value X1, but the difference between the reward / penalty parameter value X1 and the reward / penalty parameter value X2 is relatively small, then the network equipment can be kept in a stable state without adjusting the target duty cycle of each fan.

[0079] Fifth, acquire sample state data corresponding to the network devices, and train the target action model and target evaluation model based on the sample state data. The sample state data may include sample state data when the network device is in an adjustment state, and / or sample state data when the network device is in a stable state.

[0080] For example, the target action model and target evaluation model can be trained using the following steps: Step S21: Obtain sample status data corresponding to the network device.

[0081] For example, the sample status data corresponding to the network device may include, but is not limited to, at least one of the following: sensor data corresponding to each board within the network device, system data corresponding to the network device, and duty cycle corresponding to each fan (i.e., the duty cycle of the fan during its current operation). The sensor data corresponding to each board may include, but is not limited to, at least one of the following: junction temperature data corresponding to the chip within the board, maximum temperature corresponding to each temperature measurement point within the board, and power corresponding to the board; the system data may include, but is not limited to, at least one of the following: ambient temperature, power supply, and power corresponding to each fan.

[0082] For example, when the network device is in a state of adjustment or a stable state, sample status data of the network device can be obtained in each collection cycle. The interval between two adjacent collection cycles can be configured arbitrarily, such as 3 seconds, 5 seconds, 8 seconds, etc., without any restrictions.

[0083] For each board, if the board may include at least one of the following chips: MAC chip, CPU (Central Processing Unit) chip, FPGA (Field Programmable Gate Array) chip, AI (Artificial Intelligent) chip, or optical module chip, then the junction temperature data (i.e., actual operating temperature) of these chips can be collected, and the junction temperature data of these chips can be used as the sample status data of the network device.

[0084] For each board, if the board includes at least one temperature measurement point, the temperature value corresponding to each temperature measurement point in the board can be collected, and the maximum value among these temperature values ​​can be selected as the maximum temperature corresponding to each temperature measurement point in the board. This maximum temperature can then be used as the sample status data corresponding to the network device.

[0085] For each board, the power corresponding to that board can be collected, that is, the current power used by that board, and the power corresponding to that board can be used as the sample status data of the network device.

[0086] For network devices, the ambient temperature and power consumption can be collected and used as sample status data for the network devices.

[0087] For each fan, the power corresponding to that fan can be collected, that is, the current power used by that fan, and the power corresponding to that fan can be used as the sample status data of the network device.

[0088] For each fan, the duty cycle can be collected, which represents the current operating speed of the fan. For example, the duty cycle indicates the fan speed. It can be an integer value between the minimum and maximum duty cycles. For instance, if the minimum duty cycle is 20 and the maximum is 100, the duty cycle can be an integer value between 20 and 100, such as 20, 25, 50, or 100. Assuming the fan's maximum speed is k (i.e., the maximum speed capability the fan supports) and the duty cycle is m, the fan speed can be m%*k, meaning the fan speed is m% of the maximum speed k. For example, a duty cycle of 20 means the fan speed is 20% of the maximum speed k.

[0089] Step S22: Obtain the configured initial action model and the configured initial evaluation model.

[0090] In this embodiment, an initial action model can be pre-configured. The input data of the initial action model is the sample state data corresponding to the network device, and the output data of the initial action model is the duty cycle corresponding to all fans (for ease of distinction, the duty cycle is referred to as the sample duty cycle). In other words, the sample duty cycle corresponding to all fans of the network device is taken as a whole and used as the output data of the initial action model.

[0091] For example, the initial action model can be a deep learning model or a neural network model. There are no restrictions on the structure of the initial action model, and it can be configured arbitrarily, as long as the initial action model can convert the sample state data corresponding to the network device into the sample duty cycle corresponding to all fans.

[0092] In this embodiment, an initial evaluation model can be pre-configured. The input data of the initial evaluation model are the sample state data corresponding to the network device and the sample duty cycle corresponding to all fans (i.e., the output data of the initial action model). The output data of the initial evaluation model is the state-action value. This state-action value reflects the value of the state and action. Here, the state can be the sample state data, and the action can be the sample duty cycle corresponding to all fans. In other words, the state-action value reflects the value of the sample state data and the sample duty cycle corresponding to all fans.

[0093] For example, the initial evaluation model can be a deep learning model or a neural network model. There are no restrictions on the structure of the initial evaluation model, and it can be configured arbitrarily, as long as the initial evaluation model can convert the sample state data and the sample duty cycle corresponding to all fans into state action values.

[0094] In one possible implementation, the initial action model and the initial evaluation model can be network models based on the DDPG (Deep Deterministic Policy Gradient) algorithm, or network models based on other algorithms. There is no limitation on this. For ease of description, this embodiment takes the initial action model and the initial evaluation model based on the DDPG algorithm as an example.

[0095] For example, see Figure 3 The diagram shows an example of an initial action model and an initial evaluation model based on the DDPG algorithm. The Actor network is the initial action model, and the Critit network is the initial evaluation model. S represents the input data for the initial action model, and S represents the input data for the initial evaluation model. In this embodiment, S represents the sample state data corresponding to the network device. A represents the output data for the initial action model, and A represents the input data for the initial evaluation model. In this embodiment, A represents the sample duty cycle (i.e., the Action of the Actor network) corresponding to all fans. Q represents the output data for the initial evaluation model. In this embodiment, Q represents the state action value.

[0096] The Deep Deterministic Policy Gradient (DDPG) algorithm is a deep deterministic policy gradient algorithm proposed to solve the continuous action control problem. DDPG addresses the discrete action space by tackling the problem of discrete actions. The deterministic policy in DDPG is contrasted with stochastic policies. For some action sets, the values ​​may be continuous or high-dimensional discrete values, resulting in a large action space. DDPG uses a deterministic policy to simplify this problem. The determinism in DDPG means that the output of a continuous action is a specific value. When actions are discrete, the probability of each action occurring is output based on the goal of maximizing long-term reward. When actions are continuous, under the goal of maximizing long-term reward, the output can only be a specific numerical value representing a specific action, thus becoming a deterministic policy.

[0097] Based on the aforementioned principles of the DDPG algorithm, this embodiment can implement the initial action model and the initial evaluation model using the DDPG algorithm. The DDPG algorithm is based on the Actor-Critict architecture and processes continuous action spaces. Therefore, the Actor network can be used as the initial action model, and the Critict network as the initial evaluation model. In this embodiment, the continuous action space refers to the sample duty cycle corresponding to all fans. For each fan's sample duty cycle, this sample duty cycle can be a continuous integer value between the minimum and maximum duty cycles, rather than a discrete integer value between the minimum and maximum duty cycles. For example, the sample duty cycle can be a continuous integer value of 20, 21, 22, ... 100, rather than a discrete integer value of 20, 25, 30, 35, ... 100.

[0098] Step S23: Input the sample state data into the initial action model to obtain the sample duty cycle (i.e., the Action of the initial action model) for each fan. Input the sample state data and the sample duty cycle for each fan into the initial evaluation model to obtain the state action value corresponding to the sample state data.

[0099] For example, the sample state data S corresponding to the network device can be used as the input data of the initial action model. The initial action model processes the sample state data S to obtain the sample duty cycle A corresponding to each fan. The sample duty cycle A corresponding to all fans is the Action of the initial action model.

[0100] The sample state data S corresponding to the network device and the sample duty cycle A corresponding to each fan can be used as the input data of the initial evaluation model. The initial evaluation model processes the sample state data S and the sample duty cycle A corresponding to each fan to obtain the state action value Q corresponding to the sample state data S. The state action value Q can also be called the Q function value, which is the output data of the initial evaluation model.

[0101] Step 304: Determine the reward / penalty parameter value corresponding to the state action value based on the sample state data.

[0102] For example, the total system power and the maximum duty cycle value can be determined based on the sample state data. This sample state data may include the duty cycle corresponding to each fan, and the maximum duty cycle value can be the highest among all duty cycles. If the sample state data includes power supply power, the total system power can be determined based on this power supply power. If the sample state data includes the power corresponding to each board and the power corresponding to each fan, the total system power can be determined based on the power corresponding to all boards and the power corresponding to all fans. Then, the target noise value can be determined based on the maximum duty cycle value. Then, the reward / penalty parameter value corresponding to the state action value can be determined based on the total system power and the first weighting coefficient corresponding to the total system power, the target noise value and the second weighting coefficient corresponding to the target noise value. The process of determining the reward / penalty parameter value based on the total system power and the maximum duty cycle value can be found in step S13, and will not be repeated here.

[0103] Step S25: Based on the state action value, determine the first error value corresponding to the initial action model. Based on the first error value, adjust the network parameters of the initial action model to obtain the adjusted action model. Based on the reward / penalty parameter value and the state action value, determine the second error value corresponding to the initial evaluation model. Based on the second error value, adjust the network parameters of the initial evaluation model to obtain the adjusted evaluation model.

[0104] For example, an error function (i.e., loss function) corresponding to the initial action model can be configured. This error function can be configured empirically, and there are no restrictions on it. For instance, an example of this error function could be: L(Θ v ) = Of course, the above formula is just an example of an error function. There are no restrictions on this error function, as long as it is related to the state action value.

[0105] Based on this, the first error value can be determined using the following formula: L(Θ) v ) = In the above formula, L(Θ) v () indicates the first error value. s Represents sample state data. a Indicates the sample duty cycle. This represents the state action value. In summary, when using sample state data... s After inputting the data into the initial motion model, the duty cycle of the sample for each fan can be obtained. a Then, the sample state data s and the sample duty cycle corresponding to each fan a Input the values ​​into the initial evaluation model to obtain state-action values. After obtaining the state action value Then, the state action value can be... Substituting into the above formula, we obtain the first error value L(Θ). v Clearly, the state action value. When the value is larger, the first error value L(Θ) v The smaller the value, the higher the state action value. The smaller the value, the lower the first error value L(Θ). v The larger it is.

[0106] After obtaining the first error value L(Θ) v After that, we can base our analysis on the first error value L(Θ). v The network parameters of the initial action model are adjusted. This adjustment process is not restricted, and the goal is to optimize the first error value L(Θ). v The error value L(Θ) gets smaller and smaller, that is, the first error value L(Θ) is found. v The minimum value of ) is used to obtain the adjusted action model.

[0107] For example, an error function (i.e., loss function) corresponding to the initial evaluation model can be configured. This error function can be configured empirically, and there are no restrictions on it. For instance, an example of this error function could be: L(Θ u ) = 1 / 2 Of course, the above formula is just an example of an error function, as long as the error function is related to the reward / penalty parameter value and the state action value.

[0108] Based on this, the second error value is determined according to the following formula: L(Θ) u ) = 1 / 2 In the above formula, L(Θ) u The second error value is indicated by ) s Represents sample state data. a Indicates the sample duty cycle. Indicates the state action value. This represents the reward / penalty parameter value. In summary, when using sample state data... s After inputting the data into the initial motion model, the duty cycle of the sample for each fan can be obtained. a Then, the sample state data s and the sample duty cycle corresponding to each fan a Input the values ​​into the initial evaluation model to obtain state-action values. After obtaining the state action value Then, the state action value can be... and reward / punishment parameter values Substituting into the above formula, we obtain the second error value L(Θ). u ).

[0109] Clearly, state action value When the value is larger, the second error value L(Θ) u The smaller the value, the higher the state action value. The smaller the value, the greater the second error value L(Θ). u The larger the value, the greater the reward / penalty parameter value. When the value is larger, the second error value L(Θ) u The larger the value, the greater the reward / penalty parameter value. The smaller the value, the greater the second error value L(Θ). u The smaller the value.

[0110] After obtaining the second error value L(Θ) u After that, we can base our analysis on the second error value L(Θ). u The network parameters of the initial evaluation model are adjusted, and the adjustment process is not restricted. The adjustment objective is to make the second error value L(Θ) more accurate. u The error value L(Θ) gets smaller and smaller, that is, the second error value L(Θ) is found. u The minimum value of ) is used to obtain the adjusted evaluation model.

[0111] For example, suppose the first acquisition cycle acquires sample state data a1, the second acquisition cycle acquires sample state data a2, and the third acquisition cycle acquires sample state data a3. Sample state data a1 (i.e., the current state) is used as the sample state data. s When the sample state data a2 (i.e., the next state) is used as the sample state data, then the sample state data is used as the next state data. s Sample state data a2 is used as sample state data s At that time, sample state data a3 is used as sample state data. s Based on the above formula N(S) is used to obtain sample state data. s Corresponding reward and punishment parameter values Obtain sample state data s Corresponding reward and punishment parameter values In summary, when determining the second error value, the reward / penalty parameter value... It is the reward / penalty parameter value corresponding to the next state of the current state.

[0112] Step S26: Determine whether the adjusted action model and the adjusted evaluation model have converged.

[0113] If the adjusted action model and the adjusted evaluation model have converged, proceed to step S27.

[0114] If the adjusted action model and / or the adjusted evaluation model do not converge, proceed to step S28.

[0115] For example, if the first error value meets the first convergence condition, it can be determined that the adjusted motion model has converged; if the first error value does not meet the first convergence condition, it can be determined that the adjusted motion model has not converged. The first convergence condition is configured based on experience and is not restricted. For example, if the first error value is within a certain preset value range, the first error value meets the first convergence condition; otherwise, the first error value does not meet the first convergence condition.

[0116] If the second error value satisfies the second convergence condition, the adjusted evaluation model can be determined to have converged; if the second error value does not satisfy the second convergence condition, the adjusted evaluation model can be determined to have not converged. The second convergence condition can be configured empirically and is not restricted. For example, if the second error value is within a certain preset numerical range, the second error value satisfies the second convergence condition; otherwise, the second error value does not satisfy the second convergence condition.

[0117] For example, if the number of iterations of the initial action model reaches the first threshold (which can be configured based on experience and is not restricted), it can be determined that the adjusted action model has converged. If the number of iterations of the initial action model does not reach the first threshold, it can be determined that the adjusted action model has not converged.

[0118] If the number of iterations of the initial evaluation model reaches the second threshold (which can be configured based on experience and is not restricted), it can be determined that the adjusted evaluation model has converged. If the number of iterations of the initial evaluation model does not reach the second threshold, it can be determined that the adjusted evaluation model has not converged.

[0119] For example, if the iteration duration of the initial action model reaches the first duration threshold (which can be configured based on experience and is not restricted), it can be determined that the adjusted action model has converged. If the iteration duration of the initial action model does not reach the first duration threshold, it can be determined that the adjusted action model has not converged.

[0120] If the iteration duration of the initial evaluation model reaches the second duration threshold (which can be configured based on experience and is not restricted), it can be determined that the adjusted evaluation model has converged. If the iteration duration of the initial evaluation model does not reach the second duration threshold, it can be determined that the adjusted evaluation model has not converged.

[0121] Of course, the above are just a few examples of how to determine whether the adjusted action model and the adjusted evaluation model have converged. There are no restrictions on the judgment method, and the judgment method can be set arbitrarily according to actual needs.

[0122] Step S27: Determine the adjusted action model as the target action model and the adjusted evaluation model as the target evaluation model. This completes the training process for the initial action model and the initial evaluation model, resulting in the trained target action model and the trained target evaluation model.

[0123] Step S28: Determine the adjusted motion model as the initial motion model and the adjusted evaluation model as the initial evaluation model, then return to execute steps S23, S25 and S26.

[0124] Sixth, after the network device is in a stable state, the DDPG model (Target Action Model and Target Evaluation Model) can be used periodically to determine whether the Greedy algorithm needs to be restarted. If so, the network device is adjusted from a stable state to a state awaiting adjustment. While the network device is in the state awaiting adjustment, the Greedy algorithm is used again to control the fan speed until the target duty cycle for each fan is found. The target duty cycle for each fan is then used to control the fan speed. The specific process of the Greedy algorithm can be found in point two and will not be repeated here. If not, the network device remains in a stable state, and it is not necessary to redetermine the target duty cycle for each fan. That is, the target duty cycle for each fan remains unchanged (i.e., the target duty cycle determined in point two is maintained), and the next detection cycle is waited for. The DDPG model is then used again to determine whether the Greedy algorithm needs to be restarted. For example, the following steps can be used to determine whether to adjust the target duty cycle: Step S31: After the network device is in a stable state, obtain the detection status data corresponding to the network device.

[0125] For example, the detection status data corresponding to the network device includes, but is not limited to, at least one of the following: sensor data corresponding to each board within the network device, system data corresponding to the network device, and duty cycle corresponding to each fan (i.e., the duty cycle during current operation, i.e., the target duty cycle). The sensor data corresponding to each board may include, but is not limited to, at least one of the following: junction temperature data corresponding to the chip within the board, maximum temperature corresponding to each temperature measurement point within the board, and power corresponding to the board; the system data may include, but is not limited to, at least one of the following: ambient temperature, power supply, and power corresponding to each fan.

[0126] For each circuit board, if it includes at least one of the following chips: MAC chip, CPU chip, FPGA chip, AI chip, or optical module chip, then the junction temperature data (i.e., actual operating temperature) of these chips is collected and used as the detection status data for the network device. If the circuit board includes at least one temperature measuring point, then the temperature value corresponding to each measuring point on the circuit board can be collected, and the maximum value among these temperature values ​​is selected as the maximum temperature corresponding to each measuring point on the circuit board. This maximum temperature is used as the detection status data for the network device. The power of the circuit board, i.e., the current power consumption of the circuit board, can also be collected and used as the detection status data for the network device.

[0127] For network devices, the ambient temperature and power consumption can be collected and used as the corresponding detection status data of the network devices.

[0128] For each fan, the power corresponding to that fan can be collected, that is, the current power used by that fan, and the power corresponding to that fan can be used as the detection status data of the network device.

[0129] For each fan, the duty cycle corresponding to that fan can be collected, indicating that the fan is rotating at the speed corresponding to that duty cycle. This duty cycle can be used as the detection status data for the network device. For example, the duty cycle represents the fan speed. The duty cycle can be an integer value between the minimum and maximum duty cycles. Assuming the maximum fan speed is k and the duty cycle is m, then the fan speed can be m%*k, meaning the fan speed is m% of the maximum speed k.

[0130] Step S32: Input the detection state data into the trained target action model to obtain the output duty cycle corresponding to each fan; wherein, the output duty cycles corresponding to different fans may be the same or different.

[0131] For example, the input data of the initial action model is sample state data, and the output data of the initial action model is the sample duty cycle corresponding to all fans. That is, the sample duty cycle corresponding to all fans of the network device is treated as a whole and used as the output data of the initial action model. Based on this, after training the target action model, the input data of the target action model is the detection state data, and the output data of the target action model is the output duty cycle corresponding to all fans (for ease of distinction, the duty cycle output by the target action model is called the output duty cycle). In other words, the output duty cycle corresponding to all fans of the network device can be treated as a whole and used as the output data of the target action model.

[0132] In step S32, the detection status data corresponding to the network device can be input to the target action model, which processes the detection status data. There are no restrictions on this processing procedure, and the output duty cycle corresponding to each fan can be obtained. The output duty cycles corresponding to different fans may be the same or different.

[0133] For example, after inputting the detection status data into the target motion model, we can obtain the output duty cycle corresponding to fan 1, the output duty cycle corresponding to fan 2, and so on.

[0134] Step S33: Determine the first reward / penalty parameter value corresponding to the output duty cycle based on the reward / penalty data of the network device for each fan corresponding to the output duty cycle; determine the second reward / penalty parameter value corresponding to the target duty cycle based on the reward / penalty data of the network device for each fan corresponding to the target duty cycle.

[0135] For example, reward and penalty data for the network device at its output duty cycle can be obtained (i.e., the reward and penalty data includes the output duty cycle corresponding to each fan, the power consumption, the power consumption of the board, the power consumption of the fan, etc., which correspond to the output duty cycle), and a first reward and penalty parameter value corresponding to the output duty cycle can be determined based on this reward and penalty data. Reward and penalty data for the network device at a target duty cycle can be obtained (i.e., the reward and penalty data includes the target duty cycle corresponding to each fan, the power consumption, the power consumption of the board, the power consumption of the fan, etc., which correspond to the target duty cycle), and a second reward and penalty parameter value corresponding to the target duty cycle can be determined based on this reward and penalty data. The process of determining the first or second reward and penalty parameter value based on the reward and penalty data can be found in step S13, and will not be repeated here.

[0136] Step S34: Determine whether the first reward / penalty parameter value is better than the second reward / penalty parameter value.

[0137] If the first reward / penalty parameter value is better than the second reward / penalty parameter value, then proceed to step S35.

[0138] If the first reward / penalty parameter value is not better than the second reward / penalty parameter value, then proceed to step S36.

[0139] Step S35: Determine that the network device is in the state of being adjusted, update the output duty cycle of each fan to the initial duty cycle of that fan, and re-use the Greedy algorithm to control the fan speed until the target duty cycle of each fan is found. Use the target duty cycle of each fan to control the fan speed. The specific process of the Greedy algorithm can be found in point 2, and will not be repeated here.

[0140] For example, after the network device is in a stable state, the output duty cycle of each fan can be obtained based on the target action model. If the first reward parameter value corresponding to the output duty cycle is better than the second reward parameter value corresponding to the target duty cycle, it means that the target duty cycle needs to be optimized. The network device can be adjusted from a stable state to a state to be adjusted. When the network device is in the state to be adjusted, the Greedy algorithm is used again to control the fan speed until the target duty cycle of each fan is found.

[0141] Step S36: Determine that the network device is in a stable state, restore the output duty cycle of each fan to the target duty cycle of that fan, and continue to use the target duty cycle to control the fan speed.

[0142] For example, after the network device is in a stable state, the output duty cycle corresponding to each fan can be obtained based on the target action model. If the first reward parameter value corresponding to the output duty cycle is not better than the second reward parameter value corresponding to the target duty cycle, it means that the target duty cycle does not need to be optimized. The network device is kept in a stable state, and the output duty cycle corresponding to each fan is restored to the target duty cycle corresponding to that fan. The target duty cycle corresponding to each fan is then used to control the fan speed.

[0143] As can be seen from the above technical solutions, the embodiments of this application can effectively control the speed of each fan, reduce fan power consumption and noise, and ensure low fan noise and low fan power consumption while avoiding excessive temperature of network equipment, thus achieving the purpose of energy saving and noise reduction, meeting the energy saving and noise reduction requirements of network equipment, reducing noise pollution, reducing the inhalation of harmful substances in the air such as dust, and helping to reduce the corrosion damage to network equipment. Based on the customer's needs for energy saving and noise reduction, the weight coefficients are adjusted independently, and a suitable reward and punishment function is set to obtain the reward and punishment parameter values. To ensure the safety of network equipment, the temperature control points of the network equipment can be detected. If they are in a dangerous high-temperature state, the system is forced to switch to the PID algorithm to ensure the safety of the network equipment. The Greedy algorithm can be used to find the locally optimal stable state, and the DDPG algorithm can be used to infer new perturbation values ​​to the network equipment, which is beneficial to finding the globally optimal solution of the network equipment.

[0144] Based on the same concept as the above method, this application proposes a fan speed control device for use in a network device, which includes multiple fans. See [link to relevant documentation]. Figure 4 The diagram shown is a structural schematic of the device. When the network device is in an adjustment state, the device may include: Module 41 is used to determine the initial duty cycle currently being used by each fan; Processing module 42 is configured to: sequentially traverse each fan as a candidate fan based on the target order of the plurality of fans; adjust the initial duty cycle corresponding to the candidate fan multiple times to obtain different candidate duty cycles; for each candidate duty cycle, obtain the reward / penalty data of the network device under the candidate duty cycle, determine the reward / penalty parameter value corresponding to the candidate duty cycle based on the reward / penalty data; select the optimal reward / penalty parameter value from the reward / penalty parameter values ​​corresponding to the plurality of candidate duty cycles, and determine the candidate duty cycle corresponding to the optimal reward / penalty parameter value as the target duty cycle corresponding to the candidate fan; The control module 43 is used to control the fan based on the rotation speed corresponding to the target duty cycle of each fan after obtaining the target duty cycle of each fan, if the network device is in a stable state.

[0145] For example, after obtaining the target duty cycle for each fan, the processing module 42 is further configured to: if the network device is in a state to be adjusted, update the target duty cycle for each fan to the initial duty cycle for that fan, and return to execute the operation of sequentially traversing each fan as a candidate fan based on the target order of the multiple fans; wherein, if the total number of fans that have met the target conditions is not less than a preset threshold, the network device is in a stable state; otherwise, the network device is in a state to be adjusted; for each fan, if the target duty cycle for that fan is the same as the initial duty cycle for that fan, then that fan has met the target conditions; otherwise, that fan does not meet the target conditions.

[0146] For example, when the processing module 42 determines the reward / penalty parameter value corresponding to the candidate duty cycle based on the reward / penalty data, it specifically performs the following: determining the total system power and the maximum duty cycle value based on the reward / penalty data; the reward / penalty data includes the duty cycle corresponding to each fan, and the maximum duty cycle value is the maximum value among all duty cycles; if the reward / penalty data includes power supply power, determining the total system power based on the power supply power; if the reward / penalty data includes the power corresponding to each board and the power corresponding to each fan, determining the total system power based on the power corresponding to all boards and the power corresponding to all fans; determining the target noise value based on the maximum duty cycle value; and determining the reward / penalty parameter value based on the total system power and the first weighting coefficient corresponding to the total system power, the target noise value and the second weighting coefficient corresponding to the target noise value.

[0147] For example, the sum of the first weighting coefficient and the second weighting coefficient is a fixed value; if energy saving is more important than noise reduction, then the first weighting coefficient is greater than the second weighting coefficient; if noise reduction is more important than energy saving, then the first weighting coefficient is less than the second weighting coefficient.

[0148] For example, after the network device powers on or a temperature alarm is triggered, the determining module 41 is further configured to determine the duty cycle to be adjusted for each fan based on the temperature value of each board of the network device; wherein, when controlling the fan based on the speed corresponding to the duty cycle to be adjusted for each fan, the absolute value of the difference between the temperature value of each board and the target temperature value is less than a temperature threshold; the determining module 41 is further configured to determine that the network device is in an adjustment state after the absolute value of the difference between the temperature value of each board and the target temperature value is less than the temperature threshold, and update the duty cycle to be adjusted for each fan to the initial duty cycle corresponding to that fan; the control module 43 is further configured to control the fan based on the speed corresponding to the initial duty cycle corresponding to each fan.

[0149] For example, after the network device is in a stable state, the processing module 42 is further configured to acquire the detection state data corresponding to the network device; input the detection state data into the trained target action model to obtain the output duty cycle corresponding to each fan; wherein, the output duty cycles corresponding to different fans are the same or different; determine the first reward and penalty parameter value corresponding to the output duty cycle based on the reward and penalty data of the network device under the output duty cycle corresponding to each fan; determine the second reward and penalty parameter value corresponding to the target duty cycle based on the reward and penalty data of the network device under the target duty cycle corresponding to each fan; if the first reward and penalty parameter value is better than the second reward and penalty parameter value, then determine that the network device is in an adjustment state, update the output duty cycle corresponding to each fan to the initial duty cycle corresponding to that fan, and return to execute the operation of sequentially traversing each fan as a candidate fan based on the target order of the multiple fans.

[0150] For example, the detection status data corresponding to the network device includes: sensor data corresponding to each board in the network device, system data corresponding to the network device, and target duty cycle corresponding to each fan; wherein, the sensor data corresponding to each board includes at least one of the following: junction temperature data corresponding to the chip in the board, maximum temperature corresponding to each temperature measuring point in the board, and power corresponding to the board; the system data includes at least one of the following: ambient temperature, power supply, and power corresponding to each fan.

[0151] For example, the fan speed control device further includes: a training module, used to train the target action model; specifically, when the training module trains the target action model, it is used to: after the network device is in a stable state, acquire sample state data corresponding to the network device; input the sample state data into an initial action model to obtain the sample duty cycle corresponding to each fan; input the sample state data and the sample duty cycle corresponding to each fan into an initial evaluation model to obtain the state action value corresponding to the sample state data; determine the reward / penalty parameter value corresponding to the state action value based on the sample state data; and train the initial action model and the initial evaluation model based on the reward / penalty parameter value and the state action value to obtain the trained target action model and target evaluation model.

[0152] For example, when the training module trains the initial action model and the initial evaluation model based on the reward / penalty parameter value and the state / action value to obtain the trained target action model and target evaluation model, it specifically performs the following steps: determining a first error value corresponding to the initial action model based on the state / action value; adjusting the network parameters of the initial action model based on the first error value to obtain an adjusted action model; determining a second error value corresponding to the initial evaluation model based on the reward / penalty parameter value and the state / action value; adjusting the network parameters of the initial evaluation model based on the second error value to obtain an adjusted evaluation model; if the adjusted action model and the adjusted evaluation model have converged, then the adjusted action model is determined as the target action model, and the adjusted evaluation model is determined as the target evaluation model; if the adjusted action model and / or the adjusted evaluation model have not converged, then the adjusted action model is determined as the initial action model, and the adjusted evaluation model is determined as the initial evaluation model, and the process returns to execute the operation of inputting the sample state data to the initial action model.

[0153] Based on the same application concept as the above method, this application proposes a network device, see [link to application]. Figure 5 As shown, the network device includes a processor 51 and a machine-readable storage medium 52, the machine-readable storage medium 52 storing machine-executable instructions that can be executed by the processor 51; the processor 51 is used to execute the machine-executable instructions to implement the fan speed control method disclosed in the above example of this application.

[0154] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the fan speed control method disclosed in the above examples of this application.

[0155] The aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0156] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0157] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0158] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0159] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0160] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0161] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0162] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A fan speed control method, characterized in that, Applied to a network device including multiple fans, the method includes, when the network device is in an adjustment state: Determine the initial duty cycle currently being used by each fan; Based on the target order of the multiple fans, each fan is sequentially traversed as a candidate fan; The initial duty cycle corresponding to the candidate fan is adjusted multiple times to obtain multiple different candidate duty cycles. For each candidate duty cycle, the reward and penalty data of the network device under the candidate duty cycle is obtained, and the reward and penalty parameter value corresponding to the candidate duty cycle is determined based on the reward and penalty data. The optimal reward and penalty parameter value is selected from the reward and penalty parameter values ​​corresponding to the multiple candidate duty cycles, and the candidate duty cycle corresponding to the optimal reward and penalty parameter value is determined as the target duty cycle corresponding to the candidate fan. After obtaining the target duty cycle for each fan, if the network device is in a stable state, the fan is controlled based on the rotational speed corresponding to the target duty cycle for each fan. The step of determining the reward / penalty parameter value corresponding to the candidate duty cycle based on the reward / penalty data includes: determining the total system power and the maximum duty cycle value based on the reward / penalty data; wherein the reward / penalty data includes the duty cycle corresponding to each fan, and the maximum duty cycle value is the maximum value among all duty cycles; if the reward / penalty data includes power supply power, then the total system power is determined based on the power supply power; if the reward / penalty data includes the power corresponding to each board and the power corresponding to each fan, then the total system power is determined based on the power corresponding to all boards and the power corresponding to all fans; determining the target noise value based on the maximum duty cycle value; and determining the reward / penalty parameter value based on the total system power and the first weighting coefficient corresponding to the total system power, the target noise value and the second weighting coefficient corresponding to the target noise value.

2. The method according to claim 1, characterized in that, After obtaining the target duty cycle for each fan, the method further includes: if the network device is in an adjustment state, then update the target duty cycle for each fan to the initial duty cycle for that fan, and return to execute the operation of sequentially traversing each fan as a candidate fan based on the target order of the multiple fans; If the total number of fans that have met the target conditions is not less than a preset threshold, then the network device is determined to be in a stable state; otherwise, the network device is determined to be in a state to be adjusted. For each fan, if the target duty cycle corresponding to the fan is the same as the initial duty cycle corresponding to the fan, then the fan has met the target condition; otherwise, the fan does not meet the target condition.

3. The method according to claim 1, characterized in that, The sum of the first weighting coefficient and the second weighting coefficient is a fixed value; If energy saving is more important than noise reduction, the first weighting coefficient is greater than the second weighting coefficient. If noise reduction is more important than energy saving, the first weighting coefficient is less than the second weighting coefficient.

4. The method according to claim 1, characterized in that, After the network device powers on or generates a temperature alarm, the method further includes: Based on the temperature value of each board of the network device, the duty cycle to be adjusted for each fan is determined; wherein, when controlling the fan based on the speed corresponding to the duty cycle to be adjusted for each fan, the absolute value of the difference between the temperature value of each board and the target temperature value can be made less than the temperature threshold. Once the absolute value of the difference between the temperature value of each board and the target temperature value is less than the temperature threshold, the network device is determined to be in an adjustment state. The duty cycle to be adjusted for each fan is updated to the initial duty cycle for that fan, and the fan is controlled based on the speed corresponding to the initial duty cycle for each fan.

5. The method according to claim 1, characterized in that, After the network device reaches a stable state, the method further includes: Obtain the detection status data corresponding to the network device; The detected state data is input into the trained target action model to obtain the output duty cycle for each fan; wherein, the output duty cycles for different fans may be the same or different. Based on the reward and penalty data of the network device at the output duty cycle corresponding to each fan, a first reward and penalty parameter value corresponding to the output duty cycle is determined; based on the reward and penalty data of the network device at the target duty cycle corresponding to each fan, a second reward and penalty parameter value corresponding to the target duty cycle is determined; If the first reward / penalty parameter value is better than the second reward / penalty parameter value, it is determined that the network device is in a state to be adjusted. The output duty cycle corresponding to each fan is updated to the initial duty cycle corresponding to that fan. Then, the operation of traversing each fan as a candidate fan based on the target order of the multiple fans is returned.

6. The method according to claim 5, characterized in that, The detection status data corresponding to the network device includes: sensor data corresponding to each board in the network device, system data corresponding to the network device, and target duty cycle corresponding to each fan; The sensor data corresponding to each board includes at least one of the following: junction temperature data corresponding to the chip in the board, maximum temperature corresponding to each temperature measuring point in the board, and power corresponding to the board; the system data includes at least one of the following: ambient temperature, power supply, and power corresponding to each fan.

7. The method according to claim 5, characterized in that, Before inputting the detection state data into the trained target action model, the training process of the target action model includes: After the network device is in a stable state, obtain the sample state data corresponding to the network device; The sample state data is input into the initial action model to obtain the sample duty cycle for each fan; and the sample state data and the sample duty cycle for each fan are input into the initial evaluation model to obtain the state action value corresponding to the sample state data. Determine the reward / penalty parameter value corresponding to the state action value based on the sample state data; The initial action model and the initial evaluation model are trained based on the reward and punishment parameter values ​​and the state action values ​​to obtain the trained target action model and target evaluation model.

8. The method according to claim 7, characterized in that, The step of training the initial action model and the initial evaluation model based on the reward / penalty parameter values ​​and the state / action values ​​to obtain the trained target action model and target evaluation model includes: Based on the state action values, a first error value corresponding to the initial action model is determined. Based on the first error value, the network parameters of the initial action model are adjusted to obtain the adjusted action model. Based on the reward / penalty parameter values ​​and the state action values, a second error value corresponding to the initial evaluation model is determined. Based on the second error value, the network parameters of the initial evaluation model are adjusted to obtain the adjusted evaluation model. If the adjusted action model and the adjusted evaluation model have converged, then the adjusted action model is determined as the target action model, and the adjusted evaluation model is determined as the target evaluation model. If the adjusted action model and / or the adjusted evaluation model fail to converge, the adjusted action model will be determined as the initial action model, the adjusted evaluation model will be determined as the initial evaluation model, and the operation of inputting the sample state data into the initial action model will be returned.

9. A fan speed control device, characterized in that, Applied to network devices, the network devices including multiple fans, the device includes, when the network devices are in an adjustment state: The determination module is used to determine the initial duty cycle currently being used by each fan; The processing module is configured to: sequentially traverse each fan as a candidate fan based on the target order of the multiple fans; adjust the initial duty cycle corresponding to the candidate fan multiple times to obtain multiple different candidate duty cycles; for each candidate duty cycle, obtain the reward / penalty data of the network device under the candidate duty cycle, determine the reward / penalty parameter value corresponding to the candidate duty cycle based on the reward / penalty data; select the optimal reward / penalty parameter value from the reward / penalty parameter values ​​corresponding to the multiple candidate duty cycles, and determine the candidate duty cycle corresponding to the optimal reward / penalty parameter value as the target duty cycle corresponding to the candidate fan; The control module is used to control the fan based on the rotation speed corresponding to the target duty cycle of each fan after obtaining the target duty cycle of each fan, if the network device is in a stable state. When the processing module determines the reward / penalty parameter value corresponding to the candidate duty cycle based on the reward / penalty data, it specifically performs the following: determining the total system power and the maximum duty cycle value based on the reward / penalty data; wherein, the reward / penalty data includes the duty cycle corresponding to each fan, and the maximum duty cycle value is the maximum value among all duty cycles; if the reward / penalty data includes power supply power, then the total system power is determined based on the power supply power; if the reward / penalty data includes the power corresponding to each board and the power corresponding to each fan, then the total system power is determined based on the power corresponding to all boards and the power corresponding to all fans; determining the target noise value based on the maximum duty cycle value; and determining the reward / penalty parameter value based on the total system power and the first weighting coefficient corresponding to the total system power, the target noise value and the second weighting coefficient corresponding to the target noise value.

10. The apparatus according to claim 9, characterized in that, After obtaining the target duty cycle for each fan, the processing module is further used to: if the network device is in an adjustment state, update the target duty cycle for each fan to the initial duty cycle for that fan, and return to execute the operation of iterating through each fan as a candidate fan based on the target order of the multiple fans. If the total number of fans that have met the target conditions is not less than a preset threshold, the network device is in a stable state; otherwise, the network device is in a state to be adjusted. For each fan, if the target duty cycle corresponding to the fan is the same as the initial duty cycle corresponding to the fan, then the fan has met the target condition; otherwise, the fan does not meet the target condition.

11. The apparatus according to claim 9, characterized in that, After the network device is powered on or a temperature alarm is triggered, the determining module is further configured to determine the duty cycle to be adjusted for each fan based on the temperature value of each board of the network device; when controlling the fan based on the speed corresponding to the duty cycle to be adjusted for each fan, the absolute value of the difference between the temperature value of each board and the target temperature value is less than the temperature threshold. The determining module is also used to determine that the network device is in an adjustment state after the absolute value of the difference between the temperature value of each board and the target temperature value is less than the temperature threshold, and to update the duty cycle to be adjusted for each fan to the initial duty cycle corresponding to that fan. The control module is also used to control the fan based on the speed corresponding to the initial duty cycle of each fan.

12. The apparatus according to claim 9, characterized in that, After the network device is in a stable state, the processing module is further configured to acquire the detection status data corresponding to the network device; The detected state data is input into the trained target action model to obtain the output duty cycle for each fan; wherein, the output duty cycles for different fans may be the same or different. Based on the reward and penalty data of the network device at the output duty cycle corresponding to each fan, a first reward and penalty parameter value corresponding to the output duty cycle is determined; based on the reward and penalty data of the network device at the target duty cycle corresponding to each fan, a second reward and penalty parameter value corresponding to the target duty cycle is determined; If the first reward / penalty parameter value is better than the second reward / penalty parameter value, then the network device is determined to be in an adjustment state. The output duty cycle corresponding to each fan is updated to the initial duty cycle corresponding to that fan. Then, the operation of traversing each fan as a candidate fan based on the target order of the multiple fans is returned.

13. The apparatus according to claim 12, characterized in that, Also includes: The training module is used to train the target action model. The training module, when training to obtain the target action model, is specifically used for: After the network device is in a stable state, obtain the sample state data corresponding to the network device; The sample state data is input into the initial motion model to obtain the sample duty cycle for each fan. The sample state data and the sample duty cycle corresponding to each fan are input into the initial evaluation model to obtain the state action value corresponding to the sample state data. Determine the reward / penalty parameter value corresponding to the state action value based on the sample state data; The initial action model and the initial evaluation model are trained based on the reward and punishment parameter values ​​and the state action values ​​to obtain the trained target action model and target evaluation model.

14. The apparatus according to claim 13, characterized in that, The training module trains the initial action model and the initial evaluation model based on the reward / penalty parameter values ​​and the state / action values ​​to obtain the trained target action model and target evaluation model. Specifically, this is used for: Based on the state action values, a first error value is determined corresponding to the initial action model. Based on the first error value, the network parameters of the initial action model are adjusted to obtain the adjusted action model. Based on the reward / penalty parameter values ​​and the state action values, a second error value is determined corresponding to the initial evaluation model. Based on the second error value, the network parameters of the initial evaluation model are adjusted to obtain the adjusted evaluation model. If the adjusted action model and the adjusted evaluation model have converged, then the adjusted action model is determined as the target action model, and the adjusted evaluation model is determined as the target evaluation model. If the adjusted action model and / or the adjusted evaluation model fail to converge, the adjusted action model is determined as the initial action model, the adjusted evaluation model is determined as the initial evaluation model, and the operation of inputting the sample state data into the initial action model is returned.

15. A network device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the steps of the method according to any one of claims 1-8.

Images