Power conversion control method and system for chip

Abstract

The present invention provides a power conversion control method and system for a chip, which relates to the technical field of power management, and comprises: constructing a power conversion optimization model, randomly generating model training samples for pre-training, generating network parameters, migrating to an actual control environment, and obtaining a trained power conversion optimization model; obtaining working state parameters and an external power supply voltage of a chip to be tested, generating an optimal conversion control strategy by forward reasoning, adjusting the working voltage of the chip to be tested, judging whether a significant change occurs, and if so, regenerating the optimal conversion control strategy, otherwise packaging and generating an operation log; performing feature engineering, determining association rules, generating an optimization control rule set, training base learners and combining them to obtain an integrated optimization model, fusing them with a power conversion optimization model, obtaining candidate conversion models for optimization effect evaluation, and using the parameters of the candidate conversion model with the highest optimization effect as optimal parameters for feedback and power conversion.

Description

Technical Field

[0001] The present invention relates to the technical field of power management, and in particular to a power conversion control method and system for a chip. Background Art

[0002] With the continuous development of integrated circuit technology, the functions of chips are becoming increasingly complex, and the requirements for power supply are becoming higher and higher. In order to meet the power supply needs of chips, it is usually necessary to use a power conversion circuit to convert the external power supply into the working voltage required by the chip. However, the traditional power conversion control method has some limitations and is difficult to adapt to the high performance, low power consumption, high reliability and other requirements of modern chips.

[0003] In the prior art, power conversion control usually adopts a fixed control strategy, such as a fixed conversion frequency, a fixed duty cycle, etc. The existing control method lacks flexibility and cannot be dynamically adjusted according to the actual working state of the chip, resulting in low power conversion efficiency and large energy loss. At the same time, the fixed control strategy is difficult to cope with changes in the chip working environment, such as temperature fluctuations, load changes, etc., which may cause unstable power output and affect the normal operation of the chip;

[0004] Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0005] The embodiments of the present invention provide a power conversion control method and system for a chip, which can at least solve some of the problems existing in the prior art.

[0006] According to a first aspect of an embodiment of the present invention, a power conversion control method for a chip is provided, comprising:

[0007] Based on the proximal strategy optimization algorithm, combined with the temporal difference learning and multi-step bootstrap return technology, a power conversion optimization model based on deep reinforcement learning is constructed. The state space, action function and reward value of the power conversion optimization model are determined in combination with the power conversion control characteristics of the chip. Model training samples are randomly generated and pre-trained on the power conversion optimization model in combination with the proximal strategy optimization algorithm to generate initial strategy network parameters and initial value network parameters. The parameters obtained by training are transferred to the actual control environment in combination with the transfer learning technology to obtain the trained power conversion optimization model.

[0008] Obtain the working state parameters of the chip under test and the external power supply voltage and add them to the power conversion optimization model, generate an optimal conversion control strategy through forward reasoning, adjust the external power supply voltage through the conversion execution unit based on the optimal conversion control strategy and the intelligent zero-crossing detection algorithm, and adjust the working voltage of the chip under test in combination with the closed-loop feedback control mechanism to generate a second working state parameter, extract the state change characteristics corresponding to the second working state parameter and determine whether a significant change occurs. If so, regenerate the optimal conversion control strategy, otherwise package the second working state parameter and the optimal conversion control strategy to generate a running log;

[0009] For the operation log, semantic parsing and key information extraction are performed through natural language processing technology, key parameters are determined, and feature engineering is performed on the second working state parameters through a time series analysis algorithm, the timing characteristics are determined, and the association rules are determined through an association rule mining algorithm, and an optimization control rule set is generated. Based on the optimization control rule set, a pre-set base learner is trained and combined to obtain an integrated optimization model, and the integrated optimization model is merged with the power conversion optimization model. In combination with an active learning mechanism, chip working conditions are selected according to prediction uncertainty and incremental learning is performed to obtain a candidate conversion model and evaluate the optimization effect through a test data set in an expert knowledge base. The evaluation is repeated until a preset maximum number of iterations is reached, and the parameters of the candidate conversion model with the highest optimization effect are fed back as the optimal parameters and power conversion is performed on the current chip to be tested.

[0010] In an optional embodiment,

[0011] Based on the proximal strategy optimization algorithm, combined with the time difference learning and multi-step bootstrap return technology, a power conversion optimization model based on deep reinforcement learning is constructed. The state space, action function and reward value of the power conversion optimization model are determined in combination with the power conversion control characteristics of the chip. Model training samples are randomly generated and pre-trained on the power conversion optimization model in combination with the proximal strategy optimization algorithm to generate initial strategy network parameters and initial value network parameters. The parameters obtained by training are transferred to the actual control environment in combination with the transfer learning technology. The trained power conversion optimization model includes:

[0012] Based on the proximal policy optimization algorithm, a power conversion optimization model based on deep reinforcement learning is constructed in combination with temporal difference learning and multi-step bootstrap return technology, and a policy network, a value network and an experience playback buffer are set in the power conversion optimization model;

[0013] A state space is set based on a state variable of the power system, an action function is defined based on the operation control, a performance indicator is set based on the power conversion requirement, and a reward value is determined based on the performance indicator, wherein the state space includes voltage, current, temperature and load conditions, the operation control includes voltage conversion curve parameters and conversion timing control parameters, and the performance indicator includes chip performance, energy consumption level and chip temperature rise;

[0014] A power conversion simulation environment is constructed based on chip information, and state parameters, power conversion control parameters and performance indicators corresponding to the chip are collected in the power conversion simulation environment. Initial training samples are randomly generated, and random disturbances are added to the initial training samples through data enhancement technology, and adjacent sample data are interpolated to obtain a constructed model training sample set;

[0015] Pre-training the power conversion optimization model based on the model training sample set, sampling the model training sample set through the policy network to obtain multiple trajectory data, wherein the trajectory data includes multiple states, actions and reward information, calculating the advantage function value and the importance sampling ratio for the state and action in each trajectory, truncating the importance sampling ratio, calculating the loss function value of the policy network based on the truncated importance sampling ratio and the advantage function value, and obtaining the gradient of the policy network parameters based on the loss function corresponding to the policy network and updating the parameters in combination with the optimizer to obtain the initial policy network parameters;

[0016] Use the current value network to estimate the value function value corresponding to each state in the model training sample set, combine the temporal difference learning principle, calculate the temporal difference learning target value through the immediate reward of each state and the estimated value of the next state, use the temporal difference learning target value as the learning target of the state value function, solve the gradient of the value network based on the value function value and the temporal difference learning target value, and update the parameters in combination with the optimizer to obtain the initial value network parameters;

[0017] Based on the initial strategy network parameters and the initial value network parameters, they are loaded into the model of the actual control environment as initial parameters through transfer learning, the power control system is operated in the actual control environment, data is collected and target domain training samples are generated, the strategy network and the value network are trained through the target domain training samples, and the training is repeated until a preset maximum number of training times is reached or the model performance meets the standard, the optimized model parameters are obtained and added to the power conversion optimization model, and a trained power conversion optimization model is obtained.

[0018] In an optional embodiment,

[0019] The loss function value of the policy network is calculated based on the truncated importance sampling ratio and the advantage function value as shown in the following formula:

[0020] ;

[0021] in, L ( i ) represents the loss function value, i represents the current parameters of the policy network, N represents the sampled state-action pair, A i Indicates i The advantage function value of a state-action pair, Indicates that in the current strategy π θ Next, given the state s i Take action when a i The probability of Indicates the strategy in the last iteration Next, given the state s i Take action when a i The probability of e Represents a hyperparameter that controls the cutoff range of the importance sampling ratio.

[0022] In an optional embodiment,

[0023] The working state parameters of the chip under test and the external power supply voltage are obtained and added to the power conversion optimization model, and an optimal conversion control strategy is generated through forward reasoning. Based on the optimal conversion control strategy and the intelligent zero-crossing detection algorithm, the external power supply voltage is adjusted through the conversion execution unit, and the working voltage of the chip under test is adjusted in combination with the closed-loop feedback control mechanism to generate a second working state parameter, extract the state change characteristics corresponding to the second working state parameter and determine whether a significant change occurs. If so, the optimal conversion control strategy is regenerated, otherwise the second working state parameter and the optimal conversion control strategy are packaged, and the generated operation log includes:

[0024] The working state parameters of the chip to be tested and the real-time voltage value of the external power supply are collected in real time through a pre-set data acquisition module, wherein the working state parameters include the voltage, current, temperature and load conditions of the chip, the real-time voltage value is used as the reference voltage for conversion control, the collected working state parameters are feature extracted and normalized, converted into the input format of the power conversion optimization model, and the converted working state parameters and the external power supply voltage are combined into an input vector, which is added to the power conversion optimization model;

[0025] Based on the input vector, the strategy network in the power conversion optimization model generates an action probability distribution through forward propagation calculation, samples the action probability distribution to generate a conversion control action and uses it as the optimal conversion control strategy in the current state, decodes the optimal conversion control strategy into a voltage conversion curve and a conversion timing control parameter, determines the time and position of the voltage zero-crossing point based on an intelligent zero-crossing detection algorithm, generates a power conversion control instruction sequence in combination with the voltage conversion curve and the conversion timing control parameter, and performs switch switching and parameter adjustment at the voltage zero-crossing point according to the power conversion control instruction sequence. Combined with a closed-loop feedback control mechanism, the chip's operating voltage is monitored in real time and compared with the target voltage, the voltage error is calculated and a feedback control signal is generated, the feedback control signal is merged with the power conversion control instruction sequence, and the control parameters of the conversion execution unit are dynamically adjusted;

[0026] After the closed-loop feedback control mechanism is completed, the latest working state parameter of the chip is obtained as the second working state parameter and the state change characteristic is extracted, wherein the state change characteristic includes voltage and current skewness, and the state change characteristic is compared with a preset fluctuation threshold. If the state change characteristic is greater than the fluctuation threshold, it is considered that the working state of the chip has changed significantly, and the second working state is added to the power conversion optimization model and forward propagated to generate the optimal conversion control strategy. Otherwise, the current optimal conversion control strategy is retained, and the second working state parameter and the current optimal conversion control strategy are packaged to generate an operation log.

[0027] In an optional embodiment,

[0028] The latest working state parameter of the chip is obtained as the second working state parameter and the state change characteristics are extracted as shown in the following formula:

[0029] ;

[0030] in, t Indicates the current time point. g represents any time point within the time window, M Indicates the time window size, V ( g ) indicates a time point g The voltage value, I ( g ) indicates a time point g The current value, m V ( t ) represents the weighted voltage mean in the time window, m I ( t) represents the weighted current mean value in the time window, s V ( t ) represents the weighted voltage standard deviation within the time window, s I ( t ) represents the weighted current standard deviation in the time window, w g Indicates time point g The weight of .

[0031] In an optional embodiment,

[0032] For the operation log, semantic parsing and key information extraction are performed through natural language processing technology, key parameters are determined, and feature engineering is performed on the second working state parameters through a time series analysis algorithm, time series features are determined, and association rules are determined through an association rule mining algorithm, and an optimization control rule set is generated. A pre-set base learner is trained based on the optimization control rule set and combined to obtain an integrated optimization model, and the integrated optimization model is integrated with the power conversion optimization model. In combination with an active learning mechanism, a chip operating condition is selected according to prediction uncertainty and incremental learning is performed to obtain a candidate conversion model and to evaluate the optimization effect through a test data set in an expert knowledge base, and the evaluation is repeated until a preset maximum number of iterations is reached, and the parameters of the candidate conversion model with the highest optimization effect are fed back as optimal parameters and power conversion is performed on the current chip to be tested, including:

[0033] Preprocess the operation log, perform semantic analysis in combination with natural language processing technology, identify key entities, attributes and relationships, perform semantic association and reasoning on the extracted entities and relationships in combination with domain ontology and knowledge graph, construct a semantic representation of the operation log, extract key parameters and key information from the semantic representation in combination with keyword extraction and text summarization technology, perform structured representation and storage, and obtain structured features of the operation log;

[0034] Performing time series decomposition on the second working state parameter, extracting trend terms, periodic terms and random terms, determining long-term trends, calculating partial autocorrelation coefficients of the second working state parameter in combination with a partial autocorrelation analysis algorithm and determining short-term dependencies, judging the stationarity of the second working state parameter through a stationarity test and selecting a differential order for stationarization processing according to the stationarity, performing feature extraction on the second working state parameter after stationarization processing, constructing time series features corresponding to the second working state parameter in combination with a time delay technology, and obtaining the time series features;

[0035] An expert knowledge base is constructed based on the structural features and the time series features, frequent item sets and association rules between operation log features are determined through an association rule mining algorithm, the association rules are evaluated and screened, strong association rules are selected and semantically interpreted and optimized, optimization control pairs are determined, conflict detection and consistency verification are performed on each optimization control rule, and the optimization control rule set is obtained;

[0036] Based on the optimization control rule set, multiple heterogeneous base learners are trained in combination with the timing features and the structured features, the base learners are combined through an ensemble learning algorithm to obtain a candidate ensemble model, hyperparameter optimization and model selection are performed on the candidate ensemble model to obtain an ensemble optimization model with the best performance, the ensemble optimization model is merged with the power conversion optimization model, and the chip operating conditions are sorted and screened in combination with an active learning mechanism and the prediction uncertainty recorded during the operation of the power conversion optimization model, and the operating conditions with the highest prediction uncertainty are selected as samples to be labeled, and the selected samples are manually labeled to obtain an incremental learning data set, and the incremental learning data set is added to the original training data set, and the fused integrated optimization model and power conversion optimization model are incrementally learned and parameter adjusted to obtain the candidate conversion model;

[0037] For the candidate conversion model, the optimization effect is evaluated through the test data set in the expert knowledge base, the data corresponding to each evaluation indicator is calculated to obtain the evaluation result, and the evaluation result is compared with the performance benchmark in the expert knowledge base. If the evaluation result is greater than the performance benchmark, the current candidate conversion model is used as the optimal conversion model, and the parameters of the current candidate conversion model are used as the optimal parameters. The power conversion of the current chip to be tested is performed through the optimal conversion model, and the optimal parameters are used as feedback information to perform feedback adjustment on the power conversion optimization model.

[0038] In an optional embodiment,

[0039] The partial autocorrelation coefficient of the second working state parameter is calculated by combining the partial autocorrelation analysis algorithm as shown in the following formula:

[0040] ;

[0041] in, PACF ( k ) indicates that the lag order is k The partial autocorrelation coefficient under the condition of H represents the total number of samples, j represents the sample index, X t ( j ) is the time series at time jThe observed value, which means at the time point t The actual data, Indicates that the lag order is k Under the conditions of X t ( j ), X t-k ( j ) is the time series at time j Lag k The observed value at the time point tk The actual data, Indicates that the lag order is k Under the conditions of X t-k ( j )’s predicted value.

[0042] According to a second aspect of an embodiment of the present invention, a power conversion control system for a chip is provided, comprising:

[0043] The first unit is used to construct a power conversion optimization model based on deep reinforcement learning based on the proximal strategy optimization algorithm, combined with temporal difference learning and multi-step bootstrap return technology, determine the state space, action function and reward value of the power conversion optimization model in combination with the power conversion control characteristics of the chip, randomly generate model training samples and pre-train the power conversion optimization model in combination with the proximal strategy optimization algorithm, generate initial strategy network parameters and initial value network parameters, and transfer the trained parameters to the actual control environment in combination with the transfer learning technology to obtain the trained power conversion optimization model;

[0044] The second unit is used to obtain the working state parameters of the chip under test and the external power supply voltage and add them to the power conversion optimization model, generate an optimal conversion control strategy through forward reasoning, adjust the external power supply voltage through the conversion execution unit based on the optimal conversion control strategy and the intelligent zero-crossing detection algorithm, and adjust the working voltage of the chip under test in combination with the closed-loop feedback control mechanism to generate a second working state parameter, extract the state change characteristics corresponding to the second working state parameter and determine whether a significant change occurs. If so, regenerate the optimal conversion control strategy, otherwise package the second working state parameter and the optimal conversion control strategy to generate a running log;

[0045] The third unit is used to perform semantic analysis and key information extraction on the operation log through natural language processing technology, determine key parameters and perform feature engineering on the second working state parameters through a time series analysis algorithm, determine timing features and determine association rules through an association rule mining algorithm, generate an optimization control rule set, train a pre-set base learner based on the optimization control rule set and combine them to obtain an integrated optimization model, merge the integrated optimization model with the power conversion optimization model, combine with an active learning mechanism, select chip conditions according to prediction uncertainty and perform incremental learning, obtain candidate conversion models, and evaluate the optimization effect through a test data set in an expert knowledge base, repeat the evaluation until a preset maximum number of iterations is reached, and use the parameters of the candidate conversion model with the highest optimization effect as the optimal parameters for feedback and perform power conversion on the current chip to be tested.

[0046] According to a third aspect of the embodiments of the present invention,

[0047] An electronic device is provided, comprising:

[0048] processor;

[0049] a memory for storing processor-executable instructions;

[0050] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0051] According to a fourth aspect of the embodiments of the present invention,

[0052] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0053] In the present invention, by introducing deep reinforcement learning technology, an intelligent power conversion optimization model is constructed, which can be dynamically adjusted according to the actual working state of the chip, realize intelligent optimization of power conversion, improve conversion efficiency, and reduce energy loss. The proximal strategy optimization algorithm, timing difference learning and multi-step bootstrap return technology are combined to enable the power conversion optimization model to have adaptive control decision-making capabilities, which can ensure the stability of power output and improve the reliability of the chip. Through time series analysis and association rule mining and other technologies, the working state of the chip can be fully perceived, richer and more comprehensive state information can be obtained, and sufficient data support can be provided for control decisions to achieve more accurate power conversion optimization. The ability of continuous learning and optimization enables the power conversion control to always remain efficient and reliable. By dynamically adjusting the power conversion strategy, the aging process of the chip can be delayed, its service life can be extended, and the reliability and stability of the chip can be improved. In summary, the present invention significantly improves the power conversion efficiency and reliability, extends the service life of the chip, and provides an efficient, reliable and intelligent solution for chip power management. It has broad application prospects and can promote the further development and optimization of chip technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flow chart of a power conversion control method for a chip according to an embodiment of the present invention;

[0055] Figure 2 The figure is a schematic diagram of the structure of a power conversion control system for a chip according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0058] Figure 1 FIG. 1 is a flow chart of a power conversion control method for a chip according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0059] S1. Based on the proximal strategy optimization algorithm, combined with the time difference learning and multi-step bootstrap return technology, a power conversion optimization model based on deep reinforcement learning is constructed. The state space, action function and reward value of the power conversion optimization model are determined in combination with the power conversion control characteristics of the chip. Model training samples are randomly generated and pre-trained on the power conversion optimization model in combination with the proximal strategy optimization algorithm to generate initial strategy network parameters and initial value network parameters. The parameters obtained by training are transferred to the actual control environment in combination with the transfer learning technology to obtain the trained power conversion optimization model;

[0060] The proximal policy optimization is a deep reinforcement learning algorithm used to train a policy network (i.e., the decision-making policy of an intelligent agent) and to update the policy by optimizing an objective function called the "cut objective". The temporal difference learning is an algorithm based on reinforcement learning and is used to evaluate the value of a policy and to update the value function by calculating the temporal difference error. The multi-step bootstrap return technique is an extension of temporal difference learning and is used to improve the stability and efficiency of learning through multi-step returns. The policy network parameters are parameters in a neural network used to determine the policy, i.e., the probability of taking a specific action in a given state. The value network parameters are parameters in a neural network used to estimate the value of a state or a state-action pair. The transfer learning is a machine learning method that aims to apply knowledge acquired from one domain (source domain) to another related domain (target domain).

[0061] In an optional embodiment,

[0062] Based on the proximal strategy optimization algorithm, combined with the time difference learning and multi-step bootstrap return technology, a power conversion optimization model based on deep reinforcement learning is constructed. The state space, action function and reward value of the power conversion optimization model are determined in combination with the power conversion control characteristics of the chip. Model training samples are randomly generated and pre-trained on the power conversion optimization model in combination with the proximal strategy optimization algorithm to generate initial strategy network parameters and initial value network parameters. The parameters obtained by training are transferred to the actual control environment in combination with the transfer learning technology. The trained power conversion optimization model includes:

[0063] Based on the proximal policy optimization algorithm, a power conversion optimization model based on deep reinforcement learning is constructed in combination with temporal difference learning and multi-step bootstrap return technology, and a policy network, a value network and an experience playback buffer are set in the power conversion optimization model;

[0064] A state space is set based on a state variable of the power system, an action function is defined based on the operation control, a performance indicator is set based on the power conversion requirement, and a reward value is determined based on the performance indicator, wherein the state space includes voltage, current, temperature and load conditions, the operation control includes voltage conversion curve parameters and conversion timing control parameters, and the performance indicator includes chip performance, energy consumption level and chip temperature rise;

[0065] A power conversion simulation environment is constructed based on chip information, and state parameters, power conversion control parameters and performance indicators corresponding to the chip are collected in the power conversion simulation environment. Initial training samples are randomly generated, and random disturbances are added to the initial training samples through data enhancement technology, and adjacent sample data are interpolated to obtain a constructed model training sample set;

[0066] Pre-training the power conversion optimization model based on the model training sample set, sampling the model training sample set through the policy network to obtain multiple trajectory data, wherein the trajectory data includes multiple states, actions and reward information, calculating the advantage function value and the importance sampling ratio for the state and action in each trajectory, truncating the importance sampling ratio, calculating the loss function value of the policy network based on the truncated importance sampling ratio and the advantage function value, and obtaining the gradient of the policy network parameters based on the loss function corresponding to the policy network and updating the parameters in combination with the optimizer to obtain the initial policy network parameters;

[0067] Use the current value network to estimate the value function value corresponding to each state in the model training sample set, combine the temporal difference learning principle, calculate the temporal difference learning target value through the immediate reward of each state and the estimated value of the next state, use the temporal difference learning target value as the learning target of the state value function, solve the gradient of the value network based on the value function value and the temporal difference learning target value, and update the parameters in combination with the optimizer to obtain the initial value network parameters;

[0068] Based on the initial strategy network parameters and the initial value network parameters, they are loaded into the model of the actual control environment as initial parameters through transfer learning, the power control system is operated in the actual control environment, data is collected and target domain training samples are generated, the strategy network and the value network are trained through the target domain training samples, and the training is repeated until a preset maximum number of training times is reached or the model performance meets the standard, the optimized model parameters are obtained and added to the power conversion optimization model, and a trained power conversion optimization model is obtained.

[0069] The state space is a set used to describe the environment in reinforcement learning, which includes all possible environmental states. The operational control refers to the process of the intelligent agent selecting and executing actions in a specific state. The data enhancement technology is used to generate more training samples by performing various transformations on the original data, thereby improving the generalization ability and robustness of the model. The interpolation calculation is used to estimate the value of unknown data points between known data points. The trajectory data refers to the record of the movement of an object along a path in a time series. The advantage function value is used to measure the degree of goodness of taking a certain action relative to other actions in a specific state. The importance sampling ratio is used to correct the estimation bias introduced by policy changes. The truncation is a technology for limiting the importance sampling ratio to prevent excessive updates from causing training instability. The target domain training samples refer to data samples used to train the model in the target domain (i.e., new environment or task).

[0070] The proximal strategy optimization algorithm is used as the basic algorithm, combined with temporal difference learning and multi-step bootstrap return technology to improve the learning efficiency and stability of the algorithm. The strategy network, value network and experience playback buffer are set in the model for decision-making, evaluation and storage of experience data.

[0071] The state space is set based on the state variables of the power system, including voltage, current, temperature and load conditions, etc. The action function is defined based on the operation control, including voltage conversion curve parameters and conversion timing control parameters, etc. The performance indicators are set based on the power conversion requirements, including chip performance, energy consumption level and chip temperature rise, etc. The reward value is determined based on the performance indicators to guide the model to learn the optimization strategy, and a power conversion simulation environment is built based on chip information to simulate the real power conversion scenario. The state parameters, power conversion control parameters and performance indicators corresponding to the chip are collected in the simulation environment, and the initial training samples are randomly generated as the basic data for model training. Random disturbances are added to the initial training samples through data enhancement technology to increase data diversity, and interpolation calculations are performed on adjacent sample data to further expand the training sample set;

[0072] The power conversion optimization model is pre-trained based on the model training sample set. The policy network is used to sample the model training sample set to obtain multiple trajectory data. Each trajectory data contains multiple states, actions, and reward information. The advantage function value and importance sampling ratio are calculated for the states and actions in each trajectory. The importance sampling ratio is truncated to improve the training stability. The loss function value of the policy network is calculated based on the truncated importance sampling ratio and advantage function value. The gradient of the policy network parameters is calculated based on the loss function corresponding to the policy network. The parameters are updated in combination with the optimizer to obtain the initial policy network parameters.

[0073] Use the current value network to estimate the value function value corresponding to each state in the model training sample set. Combined with the principle of temporal difference learning, calculate the temporal difference learning target value through the immediate reward of each state and the estimated value of the next state. Take the temporal difference learning target value as the learning target of the state value function. Solve the gradient of the value network based on the value function value and the temporal difference learning target value. Combined with the optimizer, update the value network parameters to obtain the initial value network parameters.

[0074] Based on the initial strategy network parameters and the initial value network parameters, they are loaded into the model of the actual control environment as initial parameters through transfer learning, the power control system is run in the actual control environment, data is collected and target domain training samples are generated, the strategy network and the value network are trained through the target domain training samples, and the training is repeated until the preset maximum number of training times is reached or the model performance meets the standard, the optimized model parameters are obtained, and added to the power conversion optimization model to obtain the power conversion optimization model after pre-training, transfer learning and optimization.

[0075] In this embodiment, through the deep reinforcement learning model, the system can autonomously learn and optimize the power conversion strategy and parameters according to the real-time status and performance feedback of the power system, adapt to different chip types, workloads and environmental conditions, and realize adaptive optimization control of power conversion. By optimizing the power conversion strategy, the energy consumption level and temperature rise of the chip can be reduced while meeting the functional requirements of the chip, and the performance of the chip, such as computing speed, data throughput, etc., can be improved to enhance the comprehensive competitiveness of the chip. The introduction of pre-training and transfer learning technology based on the simulation environment can significantly accelerate the convergence speed of the power conversion optimization model, reduce the training time and cost in the actual environment, and improve the optimization efficiency. In summary, this embodiment integrates multiple technologies such as deep reinforcement learning, transfer learning, simulation optimization, etc. to construct an adaptive, efficient and robust power conversion optimization model, which can significantly improve the power conversion efficiency, enhance chip performance, accelerate the optimization speed, and provide intelligent decision-making support.

[0076] In an optional embodiment,

[0077] The loss function value of the policy network is calculated based on the truncated importance sampling ratio and the advantage function value as shown in the following formula:

[0078] ;

[0079] in, L ( i ) represents the loss function value, i represents the current parameters of the policy network, N represents the sampled state-action pair, A i Indicatesi The advantage function value of a state-action pair, Indicates that in the current strategy π θ Next, given the state s i Take action when a i The probability of Indicates the strategy in the last iteration Next, given the state s i Take action when a i The probability of e Represents a hyperparameter that controls the cutoff range of the importance sampling ratio.

[0080] In this embodiment, by introducing the importance sampling ratio and truncation operation, the step size of the policy update can be effectively controlled to avoid drastic changes in the policy that lead to performance collapse. By weighting the loss function with the advantage function value, the sampled data can be more effectively utilized to speed up the policy learning. By adjusting the truncation range of the importance sampling ratio, a balance can be achieved between exploration and utilization to improve the efficiency and stability of policy learning. Through the importance sampling and truncation operations, the variance of the policy gradient estimation is effectively reduced, and the instability in the training process is reduced. In summary, this embodiment can achieve stable, efficient and robust policy learning, improve the training quality and convergence speed of the power conversion optimization model, and provide strong algorithmic support for building a high-performance power conversion optimization model.

[0081] S2. Obtain the working state parameters of the chip under test and the external power supply voltage and add them to the power conversion optimization model, generate the optimal conversion control strategy through forward reasoning, adjust the external power supply voltage through the conversion execution unit based on the optimal conversion control strategy and the intelligent zero-crossing detection algorithm, adjust the working voltage of the chip under test in combination with the closed-loop feedback control mechanism, generate the second working state parameter, extract the state change characteristics corresponding to the second working state parameter and determine whether a significant change occurs, if so, regenerate the optimal conversion control strategy, otherwise package the second working state parameter and the optimal conversion control strategy, and generate a running log;

[0082] The external power supply voltage refers to the voltage value provided to the device or circuit by the external power supply system, which is usually used to provide the necessary energy for the electronic device. The intelligent zero-crossing detection algorithm is used to accurately detect the zero-crossing point in the AC power supply signal, that is, the moment when the voltage signal crosses the zero level, and is usually used for control and signal processing of power electronic equipment. The conversion execution unit refers to the component in the power system or electronic equipment that is responsible for converting the input signal (such as voltage, current) into the output signal (such as control instructions, actions). The closed-loop feedback control mechanism is a control system design method that realizes automatic adjustment of the system behavior by feeding back the system output to the input end.

[0083] In an optional embodiment,

[0084] The working state parameters of the chip under test and the external power supply voltage are obtained and added to the power conversion optimization model, and an optimal conversion control strategy is generated through forward reasoning. Based on the optimal conversion control strategy and the intelligent zero-crossing detection algorithm, the external power supply voltage is adjusted through the conversion execution unit, and the working voltage of the chip under test is adjusted in combination with the closed-loop feedback control mechanism to generate a second working state parameter, extract the state change characteristics corresponding to the second working state parameter and determine whether a significant change occurs. If so, the optimal conversion control strategy is regenerated, otherwise the second working state parameter and the optimal conversion control strategy are packaged, and the generated operation log includes:

[0085] The working state parameters of the chip to be tested and the real-time voltage value of the external power supply are collected in real time through a pre-set data acquisition module, wherein the working state parameters include the voltage, current, temperature and load conditions of the chip, the real-time voltage value is used as the reference voltage for conversion control, the collected working state parameters are feature extracted and normalized, converted into the input format of the power conversion optimization model, and the converted working state parameters and the external power supply voltage are combined into an input vector, which is added to the power conversion optimization model;

[0086] Based on the input vector, the strategy network in the power conversion optimization model generates an action probability distribution through forward propagation calculation, samples the action probability distribution to generate a conversion control action and uses it as the optimal conversion control strategy in the current state, decodes the optimal conversion control strategy into a voltage conversion curve and a conversion timing control parameter, determines the time and position of the voltage zero-crossing point based on an intelligent zero-crossing detection algorithm, generates a power conversion control instruction sequence in combination with the voltage conversion curve and the conversion timing control parameter, and performs switch switching and parameter adjustment at the voltage zero-crossing point according to the power conversion control instruction sequence. Combined with a closed-loop feedback control mechanism, the chip's operating voltage is monitored in real time and compared with the target voltage, the voltage error is calculated and a feedback control signal is generated, the feedback control signal is merged with the power conversion control instruction sequence, and the control parameters of the conversion execution unit are dynamically adjusted;

[0087] After the closed-loop feedback control mechanism is completed, the latest working state parameter of the chip is obtained as the second working state parameter and the state change characteristic is extracted, wherein the state change characteristic includes voltage and current skewness, and the state change characteristic is compared with a preset fluctuation threshold. If the state change characteristic is greater than the fluctuation threshold, it is considered that the working state of the chip has changed significantly, and the second working state is added to the power conversion optimization model and forward propagated to generate the optimal conversion control strategy. Otherwise, the current optimal conversion control strategy is retained, and the second working state parameter and the current optimal conversion control strategy are packaged to generate an operation log.

[0088] The action probability distribution is used in reinforcement learning to describe the probability distribution of different actions selected by an agent under a given state. The conversion control action refers to the specific action selected and executed by an agent or a control system in a control system in order to achieve a specific conversion task (such as voltage conversion, power regulation, etc.). The voltage conversion curve describes the output voltage variation law of a power supply system or a converter under different input voltage conditions. The timing control parameter is a parameter used in a control system to manage and adjust system behavior, involving a time-related control process. The power conversion control instruction sequence refers to a series of control instructions used to guide the power supply system to perform specific operations during the conversion process. The fluctuation threshold is a parameter used to define the fluctuation range allowed by the system or signal.

[0089] The working state parameters of the chip to be tested and the real-time voltage value of the external power supply are collected in real time through a pre-set data acquisition module. The working state parameters include the voltage, current, temperature and load conditions of the chip. The real-time voltage value is used as the reference voltage for conversion control. The collected working state parameters are feature extracted and normalized, and converted into the input format of the power conversion optimization model. The converted working state parameters and the external power supply voltage are combined into an input vector and added to the power conversion optimization model.

[0090] Based on the input vector, the strategy network in the power conversion optimization model generates an action probability distribution through forward propagation calculation, samples the action probability distribution to generate a conversion control action, and uses it as the optimal conversion control strategy under the current state. The optimal conversion control strategy is decoded into a voltage conversion curve and conversion timing control parameters, and the time and position of the voltage zero-crossing point are determined based on the intelligent zero-crossing detection algorithm. The power conversion control instruction sequence is generated by combining the voltage conversion curve and the conversion timing control parameters;

[0091] The conversion execution unit performs switch switching and parameter adjustment at the voltage zero crossing point according to the power conversion control instruction sequence, monitors the chip's operating voltage in real time in combination with the closed-loop feedback control mechanism, compares it with the target voltage, calculates the voltage error and generates a feedback control signal, merges the feedback control signal with the power conversion control instruction sequence, and dynamically adjusts the control parameters of the conversion execution unit;

[0092] After the closed-loop feedback control mechanism is completed, the latest working state parameters of the chip are obtained as the second working state parameters, and the state change characteristics, including voltage and current skewness, are extracted. The state change characteristics are compared with the preset fluctuation threshold. If the state change characteristics are greater than the fluctuation threshold, it is considered that the working state of the chip has changed significantly. The second working state is added to the power conversion optimization model, and a new optimal conversion control strategy is generated through forward propagation. If the state change characteristics are less than or equal to the fluctuation threshold, the current optimal conversion control strategy is retained.

[0093] The second working state parameter and the current optimal conversion control strategy are packaged to generate an operation log to record the chip working state and the corresponding conversion control strategy.

[0094] In this embodiment, by collecting the working state parameters of the chip and the external power supply voltage in real time and inputting them into the power conversion optimization model, the optimal power conversion control strategy can be dynamically generated according to the real-time working conditions of the chip, which can effectively cope with the fluctuations and changes in the working state of the chip, and ensure that stable and efficient power conversion can be provided under different conditions. The voltage zero crossing point is determined by the intelligent zero-crossing detection algorithm, and an accurate power conversion control instruction sequence is generated in combination with the voltage conversion curve and the conversion timing control parameters, which can minimize the energy loss and noise interference in the power conversion process and improve the power conversion efficiency and quality. By fusing the feedback control signal with the power conversion control instruction sequence, the control parameters of the conversion execution unit can be dynamically adjusted to achieve real-time optimization and error correction of power conversion, and improve the robustness and stability of the system. The intelligent strategy update mechanism can adapt to the long-term change trend of the chip working state and continuously optimize the power conversion performance. In summary, this embodiment can dynamically adjust the power conversion strategy according to the real-time working state of the chip, maximize the power conversion efficiency, reduce energy consumption, and improve chip performance. At the same time, it has good adaptability and scalability, and provides a novel and reliable solution for chip power management.

[0095] In an optional embodiment,

[0096] The latest working state parameter of the chip is obtained as the second working state parameter and the state change characteristics are extracted as shown in the following formula:

[0097] ;

[0098] in, t Indicates the current time point. g represents any time point within the time window, M Indicates the time window size, V ( g ) indicates a time point g The voltage value, I ( g ) indicates a time point g The current value, m V ( t ) represents the weighted voltage mean in the time window, m I ( t ) represents the weighted current mean value in the time window, s V ( t ) represents the weighted voltage standard deviation within the time window, s I ( t ) represents the weighted current standard deviation within the time window, wg Indicates time point g The weight of .

[0099] In this embodiment, by assigning different weights to the data points in the time window, the importance of recent data can be highlighted, and the immediate change trend of the chip working state can be more accurately reflected. The weighted third-order central moment of voltage and current is used as the state change feature, which can capture the advanced statistical characteristics of the skewness and peak degree of the data distribution. By subtracting the weighted mean and dividing by the weighted standard deviation, the original data can be mapped to the standard normal distribution space, so that the state change characteristics at different time points have the same scale and distribution characteristics, which is convenient for subsequent threshold comparison and abnormal judgment. Combining the normalized third-order moment of voltage and current into a two-dimensional feature vector can comprehensively consider the changes in the two key parameters of voltage and current, and provide a richer and more comprehensive state description. In summary, this embodiment can effectively capture and quantify the dynamic changes of the chip working state, realize real-time monitoring and abnormality detection, and provide important data support and decision-making basis for the intelligent power management of the chip, which is helpful to improve the accuracy, efficiency and reliability of power conversion optimization, and ensure that the chip always maintains optimal performance and stable operation in a complex and changeable working environment.

[0100] S3. For the operation log, semantic analysis and key information extraction are performed through natural language processing technology, key parameters are determined, and feature engineering is performed on the second working state parameters through a time series analysis algorithm, timing features are determined, and association rules are determined through an association rule mining algorithm, and an optimization control rule set is generated. Based on the optimization control rule set, a pre-set base learner is trained and combined to obtain an integrated optimization model, and the integrated optimization model is merged with the power conversion optimization model. In combination with an active learning mechanism, chip working conditions are selected according to prediction uncertainty and incremental learning is performed to obtain a candidate conversion model and evaluate the optimization effect through a test data set in an expert knowledge base. The evaluation is repeated until the preset maximum number of iterations is reached, and the parameters of the candidate conversion model with the highest optimization effect are fed back as the optimal parameters and power conversion is performed on the current chip to be tested.

[0101] The natural language processing technology is a branch in the field of computer science and artificial intelligence, which aims to enable computers to understand, generate and process natural languages ​​(such as English, Chinese, etc.). The semantic parsing is a process in natural language processing, which aims to convert natural language text into semantic representations that can be understood by computers. The association rule mining algorithm is an algorithm for discovering the relationship between items in a data set. The association rule is a data mining result generated by the association rule mining algorithm, which is used to describe the relationship between items in the data set. The optimization control rule set refers to a set of control rules designed and adjusted in the control system to improve system performance and efficiency. The base learner is a basic building block in the ensemble learning method. Meta refers to a single learner or model trained in an integrated learning model. The integrated optimization model refers to a technology that improves the overall model performance by combining multiple learning models (base learners), and integrates them through different combination strategies (such as weighted average, voting, etc.) to combine the advantages of each model and reduce the bias and variance of individual models. The active learning mechanism refers to the process of machine learning in which the model selects the most informative samples for annotation and training to improve learning efficiency and model performance. The prediction uncertainty refers to the degree of uncertainty in the model's prediction of future events or results in a machine learning or statistical model. The incremental learning is a machine learning method used to process dynamically changing data streams or gradually increasing data.

[0102] In an optional embodiment,

[0103] For the operation log, semantic parsing and key information extraction are performed through natural language processing technology, key parameters are determined, and feature engineering is performed on the second working state parameters through a time series analysis algorithm, time series features are determined, and association rules are determined through an association rule mining algorithm, and an optimization control rule set is generated. A pre-set base learner is trained based on the optimization control rule set and combined to obtain an integrated optimization model, and the integrated optimization model is integrated with the power conversion optimization model. In combination with an active learning mechanism, a chip operating condition is selected according to prediction uncertainty and incremental learning is performed to obtain a candidate conversion model and to evaluate the optimization effect through a test data set in an expert knowledge base, and the evaluation is repeated until a preset maximum number of iterations is reached, and the parameters of the candidate conversion model with the highest optimization effect are fed back as optimal parameters and power conversion is performed on the current chip to be tested, including:

[0104] Preprocess the operation log, perform semantic analysis in combination with natural language processing technology, identify key entities, attributes and relationships, perform semantic association and reasoning on the extracted entities and relationships in combination with domain ontology and knowledge graph, construct a semantic representation of the operation log, extract key parameters and key information from the semantic representation in combination with keyword extraction and text summarization technology, perform structured representation and storage, and obtain structured features of the operation log;

[0105] Performing time series decomposition on the second working state parameter, extracting trend terms, periodic terms and random terms, determining long-term trends, calculating partial autocorrelation coefficients of the second working state parameter in combination with a partial autocorrelation analysis algorithm and determining short-term dependencies, judging the stationarity of the second working state parameter through a stationarity test and selecting a differential order for stationarization processing according to the stationarity, performing feature extraction on the second working state parameter after stationarization processing, constructing time series features corresponding to the second working state parameter in combination with a time delay technology, and obtaining the time series features;

[0106] An expert knowledge base is constructed based on the structural features and the time series features, frequent item sets and association rules between operation log features are determined through an association rule mining algorithm, the association rules are evaluated and screened, strong association rules are selected and semantically interpreted and optimized, optimization control pairs are determined, conflict detection and consistency verification are performed on each optimization control rule, and the optimization control rule set is obtained;

[0107] Based on the optimization control rule set, multiple heterogeneous base learners are trained in combination with the timing features and the structured features, the base learners are combined through an ensemble learning algorithm to obtain a candidate ensemble model, hyperparameter optimization and model selection are performed on the candidate ensemble model to obtain an ensemble optimization model with the best performance, the ensemble optimization model is merged with the power conversion optimization model, and the chip operating conditions are sorted and screened in combination with an active learning mechanism and the prediction uncertainty recorded during the operation of the power conversion optimization model, and the operating conditions with the highest prediction uncertainty are selected as samples to be labeled, and the selected samples are manually labeled to obtain an incremental learning data set, and the incremental learning data set is added to the original training data set, and the fused integrated optimization model and power conversion optimization model are incrementally learned and parameter adjusted to obtain the candidate conversion model;

[0108] For the candidate conversion model, the optimization effect is evaluated through the test data set in the expert knowledge base, the data corresponding to each evaluation indicator is calculated to obtain the evaluation result, and the evaluation result is compared with the performance benchmark in the expert knowledge base. If the evaluation result is greater than the performance benchmark, the current candidate conversion model is used as the optimal conversion model, and the parameters of the current candidate conversion model are used as the optimal parameters. The power conversion of the current chip to be tested is performed through the optimal conversion model, and the optimal parameters are used as feedback information to perform feedback adjustment on the power conversion optimization model.

[0109] The keyword extraction is a technology in information retrieval and text mining, which aims to identify and extract the words or phrases that best represent the text content from the text. The text summarization technology is used to generate a short and concise description of the text content, aiming to extract the most important information from a large amount of text so that users can quickly obtain the core content. The trend term is a component of time series analysis, which is used to describe the long-term change trend of data over time. The periodic term is used in time series analysis to describe the repeated patterns or periodic changes of data in a fixed time interval. The random term is used in time series analysis to describe random noise or unpredictable fluctuations in data. The partial autocorrelation analysis algorithm It is used to measure the conditional correlation of time series at different lag orders, eliminating the influence of intermediate variables. The partial autocorrelation coefficient is used to represent the conditional correlation of time series at a specific lag order, eliminating the influence of other lag orders on the current lag order. The stationarity test is a method in time series analysis, used to determine whether a time series is stationary. The time delay technology is used to analyze the lag effect in time series data, that is, how past time points affect current or future time points. The frequent item set is a concept in association rule mining, which refers to the item set that appears in the data set with a frequency higher than a predetermined threshold. The conflict detection is a process used to identify and resolve conflicts in systems or data.

[0110] Preprocess the operation logs, perform semantic analysis with natural language processing technology, identify key entities, attributes and relationships in the operation logs, perform semantic association and reasoning on the extracted entities and relationships with domain ontology and knowledge graph, build semantic representation of the operation logs, extract key parameters and key information from the semantic representation with keyword extraction and text summarization technology, perform structured representation and storage on the extracted key parameters and key information, and obtain the structured features of the operation logs;

[0111] Decomposing the second working state parameter in time series, extracting trend items, period items and random items, determining the long-term trend of the second working state parameter, calculating the partial autocorrelation coefficient of the second working state parameter in combination with the partial autocorrelation analysis algorithm, and determining the short-term dependency, judging the stability of the second working state parameter through a stability test, selecting the difference order according to the stability to stabilize the second working state parameter, extracting features from the stabilized second working state parameter, constructing the time series features corresponding to the second working state parameter in combination with the time delay technology, and obtaining the time series features;

[0112] An expert knowledge base is constructed based on structured features and time series features. The frequent item sets and association rules between the operation log features are determined through association rule mining algorithms. The association rules are evaluated and screened. Strong association rules are selected. The selected strong association rules are semantically interpreted and optimized. The optimization control rules are determined. Conflict detection and consistency verification are performed on each optimization control rule to obtain an optimization control rule set. Based on the optimization control rule set, multiple heterogeneous base learners are trained in combination with time series features and structured features. The base learners are combined through an integrated learning algorithm to obtain a candidate integrated model. Hyperparameter optimization and model selection are performed on the candidate integrated model to obtain an integrated optimization model with the best performance. The integrated optimization model is integrated with the power conversion optimization model.

[0113] Combined with the active learning mechanism and the prediction uncertainty recorded in the operation of the power conversion optimization model, the chip operating conditions are sorted and screened, and the operating conditions with the highest prediction uncertainty are selected as samples to be labeled. The selected samples are manually labeled to obtain an incremental learning data set, and the incremental learning data set is added to the original training data set. The fused integrated optimization model and the power conversion optimization model are incrementally learned and parameter adjusted to obtain a candidate conversion model. The optimization effect of the candidate conversion model is evaluated through the test data set in the expert knowledge base, and the data corresponding to each evaluation indicator is calculated to obtain the evaluation result. The evaluation result is compared with the performance benchmark in the expert knowledge base. If the evaluation result is greater than the performance benchmark, the current candidate conversion model is used as the optimal conversion model, and the parameters of the current candidate conversion model are used as the optimal parameters. The optimal conversion model is used to perform power conversion on the current chip to be tested, and the optimal parameters are used as feedback information to perform feedback adjustment on the power conversion optimization model.

[0114] In this embodiment, by performing semantic analysis and structured representation on the operation log, extracting key parameters and key information can comprehensively mine the valuable information contained in the data generated during the operation of the chip, and provide rich features and knowledge for subsequent optimization control and decision-making. The expert knowledge base can provide targeted guidance for power conversion optimization, improve the efficiency and accuracy of optimization, and integrate the integrated optimization model with the power conversion optimization model to give full play to the advantages of different models and improve the prediction and control performance of the entire system. By adjusting the parameters of the model through incremental learning, the model can be adapted to the new data distribution and realize continuous optimization and updating of the model. The parameters of the optimal conversion model are used as feedback information to adjust the power conversion optimization model, which can form a closed-loop optimization control system, so that the power conversion optimization model can continuously self-adjust and improve according to the actual operation effect, and improve the system's adaptability and robustness. In summary, this embodiment can fully mine the value of chip operation data, automatically generate the optimal control strategy, and continuously improve the efficiency and reliability of power conversion through continuous learning and optimization, which can not only significantly improve the performance and energy efficiency of the chip, but also extend the service life of the chip, reduce maintenance costs, and provide all-round protection for the stable operation of the chip.

[0115] In an optional embodiment,

[0116] The partial autocorrelation coefficient of the second working state parameter is calculated by combining the partial autocorrelation analysis algorithm as shown in the following formula:

[0117] ;

[0118] in, PACF ( k ) indicates that the lag order is k The partial autocorrelation coefficient under the condition of H represents the total number of samples, j represents the sample index, X t ( j ) is the time series at time j The observed value, which means at the time point t The actual data, Indicates that the lag order is k Under the conditions of X t ( j ), X t-k ( j ) is the time series at time j Lag k The observed value at the time point tk The actual data, Indicates that the lag order isk Under the conditions of X t-k ( j )’s predicted value.

[0119] In this embodiment, by calculating the partial autocorrelation coefficients at different lag orders, the dependence of the time series on different time scales can be determined. By analyzing the size and significance of the partial autocorrelation coefficients, the short-term dependence of the time series can be determined, that is, the data at the current moment and the data at previous moments have a direct correlation. When the partial autocorrelation coefficient becomes insignificant after a certain lag order, it indicates that the data after the lag order has little impact on the current moment, and this order can be selected as the order of the model. The appropriate model order selection can balance the fitting ability and complexity of the model and avoid the problem of overfitting or underfitting. By real-time monitoring of the partial autocorrelation coefficient and prediction error, abnormal points or abnormal patterns in the time series can be discovered in time, providing a basis for fault diagnosis and abnormal processing. In summary, this embodiment can significantly improve the performance of the power conversion optimization system, improve its efficiency, reliability and robustness in practical applications, and provide strong support for the intelligent power management of chips.

[0120] Figure 2 FIG. 1 is a schematic diagram of a power conversion control system for a chip according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0121] The first unit is used to construct a power conversion optimization model based on deep reinforcement learning based on the proximal strategy optimization algorithm, combined with temporal difference learning and multi-step bootstrap return technology, determine the state space, action function and reward value of the power conversion optimization model in combination with the power conversion control characteristics of the chip, randomly generate model training samples and pre-train the power conversion optimization model in combination with the proximal strategy optimization algorithm, generate initial strategy network parameters and initial value network parameters, and transfer the trained parameters to the actual control environment in combination with the transfer learning technology to obtain the trained power conversion optimization model;

[0122] The second unit is used to obtain the working state parameters of the chip under test and the external power supply voltage and add them to the power conversion optimization model, generate an optimal conversion control strategy through forward reasoning, adjust the external power supply voltage through the conversion execution unit based on the optimal conversion control strategy and the intelligent zero-crossing detection algorithm, and adjust the working voltage of the chip under test in combination with the closed-loop feedback control mechanism to generate a second working state parameter, extract the state change characteristics corresponding to the second working state parameter and determine whether a significant change occurs. If so, regenerate the optimal conversion control strategy, otherwise package the second working state parameter and the optimal conversion control strategy to generate a running log;

[0123] The third unit is used to perform semantic analysis and key information extraction on the operation log through natural language processing technology, determine key parameters and perform feature engineering on the second working state parameters through a time series analysis algorithm, determine timing features and determine association rules through an association rule mining algorithm, generate an optimization control rule set, train a pre-set base learner based on the optimization control rule set and combine them to obtain an integrated optimization model, merge the integrated optimization model with the power conversion optimization model, combine with an active learning mechanism, select chip conditions according to prediction uncertainty and perform incremental learning, obtain candidate conversion models, and evaluate the optimization effect through a test data set in an expert knowledge base, repeat the evaluation until a preset maximum number of iterations is reached, and use the parameters of the candidate conversion model with the highest optimization effect as the optimal parameters for feedback and perform power conversion on the current chip to be tested.

[0124] According to a third aspect of the embodiments of the present invention,

[0125] An electronic device is provided, comprising:

[0126] processor;

[0127] a memory for storing processor-executable instructions;

[0128] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0129] According to a fourth aspect of the embodiments of the present invention,

[0130] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0131] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A power conversion control method for a chip, characterized in that: include: Based on the proximal strategy optimization algorithm, combined with the temporal difference learning and multi-step bootstrap return technology, a power conversion optimization model based on deep reinforcement learning is constructed. The state space, action function and reward value of the power conversion optimization model are determined in combination with the power conversion control characteristics of the chip. Model training samples are randomly generated and pre-trained on the power conversion optimization model in combination with the proximal strategy optimization algorithm to generate initial strategy network parameters and initial value network parameters. The parameters obtained by training are transferred to the actual control environment in combination with the transfer learning technology to obtain the trained power conversion optimization model. Obtain the working state parameters of the chip under test and the external power supply voltage and add them to the power conversion optimization model, generate an optimal conversion control strategy through forward reasoning, adjust the external power supply voltage through the conversion execution unit based on the optimal conversion control strategy and the intelligent zero-crossing detection algorithm, and adjust the working voltage of the chip under test in combination with the closed-loop feedback control mechanism to generate a second working state parameter, extract the state change characteristics corresponding to the second working state parameter and determine whether a significant change occurs. If so, regenerate the optimal conversion control strategy, otherwise package the second working state parameter and the optimal conversion control strategy to generate a running log; For the operation log, semantic parsing and key information extraction are performed through natural language processing technology, key parameters are determined, and feature engineering is performed on the second working state parameters through a time series analysis algorithm, the timing characteristics are determined, and the association rules are determined through an association rule mining algorithm, and an optimization control rule set is generated. Based on the optimization control rule set, a pre-set base learner is trained and combined to obtain an integrated optimization model, and the integrated optimization model is merged with the power conversion optimization model. In combination with an active learning mechanism, chip working conditions are selected according to prediction uncertainty and incremental learning is performed to obtain a candidate conversion model and evaluate the optimization effect through a test data set in an expert knowledge base. The evaluation is repeated until a preset maximum number of iterations is reached, and the parameters of the candidate conversion model with the highest optimization effect are fed back as the optimal parameters and power conversion is performed on the current chip to be tested.

2. The method according to claim 1, characterized in that Based on the proximal strategy optimization algorithm, combined with the time difference learning and multi-step bootstrap return technology, a power conversion optimization model based on deep reinforcement learning is constructed. The state space, action function and reward value of the power conversion optimization model are determined in combination with the power conversion control characteristics of the chip. Model training samples are randomly generated and pre-trained on the power conversion optimization model in combination with the proximal strategy optimization algorithm to generate initial strategy network parameters and initial value network parameters. The parameters obtained by training are transferred to the actual control environment in combination with the transfer learning technology. The trained power conversion optimization model includes: Based on the proximal policy optimization algorithm, a power conversion optimization model based on deep reinforcement learning is constructed in combination with temporal difference learning and multi-step bootstrap return technology, and a policy network, a value network and an experience playback buffer are set in the power conversion optimization model; A state space is set based on a state variable of the power system, an action function is defined based on the operation control, a performance indicator is set based on the power conversion requirement, and a reward value is determined based on the performance indicator, wherein the state space includes voltage, current, temperature and load conditions, the operation control includes voltage conversion curve parameters and conversion timing control parameters, and the performance indicator includes chip performance, energy consumption level and chip temperature rise; A power conversion simulation environment is constructed based on chip information, and state parameters, power conversion control parameters and performance indicators corresponding to the chip are collected in the power conversion simulation environment. Initial training samples are randomly generated, and random disturbances are added to the initial training samples through data enhancement technology, and adjacent sample data are interpolated to obtain a constructed model training sample set; Pre-training the power conversion optimization model based on the model training sample set, sampling the model training sample set through the policy network to obtain multiple trajectory data, wherein the trajectory data includes multiple states, actions and reward information, calculating the advantage function value and the importance sampling ratio for the state and action in each trajectory, truncating the importance sampling ratio, calculating the loss function value of the policy network based on the truncated importance sampling ratio and the advantage function value, and obtaining the gradient of the policy network parameters based on the loss function corresponding to the policy network and updating the parameters in combination with the optimizer to obtain the initial policy network parameters; Use the current value network to estimate the value function value corresponding to each state in the model training sample set, combine the temporal difference learning principle, calculate the temporal difference learning target value through the immediate reward of each state and the estimated value of the next state, use the temporal difference learning target value as the learning target of the state value function, solve the gradient of the value network based on the value function value and the temporal difference learning target value, and update the parameters in combination with the optimizer to obtain the initial value network parameters; Based on the initial strategy network parameters and the initial value network parameters, they are loaded into the model of the actual control environment as initial parameters through transfer learning, the power control system is operated in the actual control environment, data is collected and target domain training samples are generated, the strategy network and the value network are trained through the target domain training samples, and the training is repeated until a preset maximum number of training times is reached or the model performance meets the standard, the optimized model parameters are obtained and added to the power conversion optimization model, and a trained power conversion optimization model is obtained.

3. The method according to claim 2, characterized in that The loss function value of the policy network is calculated based on the truncated importance sampling ratio and the advantage function value as shown in the following formula: ； in, L ( θ ) represents the loss function value, θ represents the current parameters of the policy network, N represents the sampled state-action pair, A i Indicates i The advantage function value of a state-action pair, Indicates that in the current strategy π θ Next, given the state s i Take action when a i The probability of Indicates the strategy in the last iteration Next, given the state s i Take action when a i The probability of ε Represents a hyperparameter that controls the cutoff range of the importance sampling ratio.

4. The method according to claim 1, characterized in that: The working state parameters of the chip under test and the external power supply voltage are obtained and added to the power conversion optimization model, and an optimal conversion control strategy is generated through forward reasoning. Based on the optimal conversion control strategy and the intelligent zero-crossing detection algorithm, the external power supply voltage is adjusted through the conversion execution unit, and the working voltage of the chip under test is adjusted in combination with the closed-loop feedback control mechanism to generate a second working state parameter, extract the state change characteristics corresponding to the second working state parameter and determine whether a significant change occurs. If so, the optimal conversion control strategy is regenerated, otherwise the second working state parameter and the optimal conversion control strategy are packaged, and the generated operation log includes: The working state parameters of the chip to be tested and the real-time voltage value of the external power supply are collected in real time through a pre-set data acquisition module, wherein the working state parameters include the voltage, current, temperature and load conditions of the chip, the real-time voltage value is used as the reference voltage for conversion control, the collected working state parameters are feature extracted and normalized, converted into the input format of the power conversion optimization model, and the converted working state parameters and the external power supply voltage are combined into an input vector, which is added to the power conversion optimization model; Based on the input vector, the strategy network in the power conversion optimization model generates an action probability distribution through forward propagation calculation, samples the action probability distribution to generate a conversion control action and uses it as the optimal conversion control strategy in the current state, decodes the optimal conversion control strategy into a voltage conversion curve and a conversion timing control parameter, determines the time and position of the voltage zero-crossing point based on an intelligent zero-crossing detection algorithm, generates a power conversion control instruction sequence in combination with the voltage conversion curve and the conversion timing control parameter, and performs switch switching and parameter adjustment at the voltage zero-crossing point according to the power conversion control instruction sequence. Combined with a closed-loop feedback control mechanism, the chip's operating voltage is monitored in real time and compared with the target voltage, the voltage error is calculated and a feedback control signal is generated, the feedback control signal is merged with the power conversion control instruction sequence, and the control parameters of the conversion execution unit are dynamically adjusted; After the closed-loop feedback control mechanism is completed, the latest working state parameter of the chip is obtained as the second working state parameter and the state change characteristic is extracted, wherein the state change characteristic includes voltage and current skewness, and the state change characteristic is compared with a preset fluctuation threshold. If the state change characteristic is greater than the fluctuation threshold, it is considered that the working state of the chip has changed significantly, and the second working state is added to the power conversion optimization model and forward propagated to generate the optimal conversion control strategy. Otherwise, the current optimal conversion control strategy is retained, and the second working state parameter and the current optimal conversion control strategy are packaged to generate an operation log.

5. The method according to claim 4, characterized in that The latest working state parameter of the chip is obtained as the second working state parameter and the state change characteristics are extracted as shown in the following formula: ； in, t Indicates the current time point. g represents any time point within the time window, M Indicates the time window size, V ( g ) indicates a time point g The voltage value, I ( g ) indicates a time point g The current value, μ V ( t ) represents the weighted voltage mean in the time window, μ I ( t ) represents the weighted current mean value in the time window, σ V ( t ) represents the weighted voltage standard deviation within the time window, σ I ( t ) represents the weighted current standard deviation in the time window, w g Indicates time point g The weight of .

6. The method according to claim 1, characterized in that For the operation log, semantic parsing and key information extraction are performed through natural language processing technology, key parameters are determined, and feature engineering is performed on the second working state parameters through a time series analysis algorithm, time series features are determined, and association rules are determined through an association rule mining algorithm, and an optimization control rule set is generated. A pre-set base learner is trained based on the optimization control rule set and combined to obtain an integrated optimization model, and the integrated optimization model is integrated with the power conversion optimization model. In combination with an active learning mechanism, a chip operating condition is selected according to prediction uncertainty and incremental learning is performed to obtain a candidate conversion model and to evaluate the optimization effect through a test data set in an expert knowledge base, and the evaluation is repeated until a preset maximum number of iterations is reached, and the parameters of the candidate conversion model with the highest optimization effect are fed back as optimal parameters and power conversion is performed on the current chip to be tested, including: Preprocess the operation log, perform semantic analysis in combination with natural language processing technology, identify key entities, attributes and relationships, perform semantic association and reasoning on the extracted entities and relationships in combination with domain ontology and knowledge graph, construct a semantic representation of the operation log, extract key parameters and key information from the semantic representation in combination with keyword extraction and text summarization technology, perform structured representation and storage, and obtain structured features of the operation log; Performing time series decomposition on the second working state parameter, extracting trend terms, periodic terms and random terms, determining long-term trends, calculating partial autocorrelation coefficients of the second working state parameter in combination with a partial autocorrelation analysis algorithm and determining short-term dependencies, judging the stationarity of the second working state parameter through a stationarity test and selecting a differential order for stationarization processing according to the stationarity, performing feature extraction on the second working state parameter after stationarization processing, constructing time series features corresponding to the second working state parameter in combination with a time delay technology, and obtaining the time series features; An expert knowledge base is constructed based on the structural features and the time series features, frequent item sets and association rules between operation log features are determined through an association rule mining algorithm, the association rules are evaluated and screened, strong association rules are selected and semantically interpreted and optimized, optimization control pairs are determined, conflict detection and consistency verification are performed on each optimization control rule, and the optimization control rule set is obtained; Based on the optimization control rule set, multiple heterogeneous base learners are trained in combination with the timing features and the structured features, the base learners are combined through an ensemble learning algorithm to obtain a candidate ensemble model, hyperparameter optimization and model selection are performed on the candidate ensemble model to obtain an ensemble optimization model with the best performance, the ensemble optimization model is merged with the power conversion optimization model, and the chip operating conditions are sorted and screened in combination with an active learning mechanism and the prediction uncertainty recorded during the operation of the power conversion optimization model, and the operating conditions with the highest prediction uncertainty are selected as samples to be labeled, and the selected samples are manually labeled to obtain an incremental learning data set, and the incremental learning data set is added to the original training data set, and the fused integrated optimization model and power conversion optimization model are incrementally learned and parameter adjusted to obtain the candidate conversion model; For the candidate conversion model, the optimization effect is evaluated through the test data set in the expert knowledge base, the data corresponding to each evaluation indicator is calculated to obtain the evaluation result, and the evaluation result is compared with the performance benchmark in the expert knowledge base. If the evaluation result is greater than the performance benchmark, the current candidate conversion model is used as the optimal conversion model, and the parameters of the current candidate conversion model are used as the optimal parameters. The power conversion of the current chip to be tested is performed through the optimal conversion model, and the optimal parameters are used as feedback information to perform feedback adjustment on the power conversion optimization model.

7. The method according to claim 6, characterized in that The partial autocorrelation coefficient of the second working state parameter is calculated by combining the partial autocorrelation analysis algorithm as shown in the following formula: ； in, PACF ( k ) indicates that the lag order is k The partial autocorrelation coefficient under the condition of H represents the total number of samples, j represents the sample index, X t ( j ) is the time series at time j The observed value, which means at the time point t The actual data, Indicates that the lag order is k Under the conditions of X t ( j ), X t-k ( j ) is the time series at time j Lag k The observed value at the time point tk The actual data, Indicates that the lag order is k Under the conditions of X t-k ( j )’s predicted value.

8. A power conversion control system for a chip, used to implement the method according to any one of claims 1 to 7, characterized in that: include: The first unit is used to construct a power conversion optimization model based on deep reinforcement learning based on the proximal strategy optimization algorithm, combined with temporal difference learning and multi-step bootstrap return technology, determine the state space, action function and reward value of the power conversion optimization model in combination with the power conversion control characteristics of the chip, randomly generate model training samples and pre-train the power conversion optimization model in combination with the proximal strategy optimization algorithm, generate initial strategy network parameters and initial value network parameters, and transfer the trained parameters to the actual control environment in combination with the transfer learning technology to obtain the trained power conversion optimization model; The second unit is used to obtain the working state parameters of the chip under test and the external power supply voltage and add them to the power conversion optimization model, generate an optimal conversion control strategy through forward reasoning, adjust the external power supply voltage through the conversion execution unit based on the optimal conversion control strategy and the intelligent zero-crossing detection algorithm, and adjust the working voltage of the chip under test in combination with the closed-loop feedback control mechanism to generate a second working state parameter, extract the state change characteristics corresponding to the second working state parameter and determine whether a significant change occurs. If so, regenerate the optimal conversion control strategy, otherwise package the second working state parameter and the optimal conversion control strategy to generate a running log; The third unit is used to perform semantic analysis and key information extraction on the operation log through natural language processing technology, determine key parameters and perform feature engineering on the second working state parameters through a time series analysis algorithm, determine timing features and determine association rules through an association rule mining algorithm, generate an optimization control rule set, train a pre-set base learner based on the optimization control rule set and combine them to obtain an integrated optimization model, merge the integrated optimization model with the power conversion optimization model, combine with an active learning mechanism, select chip conditions according to prediction uncertainty and perform incremental learning, obtain candidate conversion models, and evaluate the optimization effect through a test data set in an expert knowledge base, repeat the evaluation until a preset maximum number of iterations is reached, and use the parameters of the candidate conversion model with the highest optimization effect as the optimal parameters for feedback and perform power conversion on the current chip to be tested.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images