Circuit control method and device and storage medium

Through the pre-trained magnetic flux density prediction model and agent, the circuit parameters of the transformer core are dynamically adjusted, and the core saturation problem is solved, improving magnetic permeability and cost-effectiveness.

CN118963229AInactive Publication Date: 2024-11-15SHUBANG POWER TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411449078.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-17
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The prior art is difficult to dynamically adjust the magnetic field state of the transformer core to avoid saturation, resulting in a decrease in magnetic permeability efficiency and an increase in material cost.

Method used

Through the pre-trained flux density prediction model and agent, the state parameters and flux density of the transformer core are obtained in real time, the future flux density is predicted, the reward value and action adjustment parameters are determined, and the circuit parameters are dynamically adjusted to avoid core saturation.

Benefits of technology

It realizes that while ensuring magnetic permeability efficiency, dynamically adjusts circuit parameters, avoids core saturation, saves material costs, and improves the accuracy and flexibility of circuit control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118963229A_ABST
    Figure CN118963229A_ABST
Patent Text Reader

Abstract

The invention provides a circuit control method and device and a storage medium. The method comprises the following steps: acquiring a state parameter and a first magnetic flux density of a transformer core in a to-be-controlled circuit at a first moment; based on the state parameter and the first magnetic flux density at the first moment, through a pre-trained magnetic flux density prediction model, determining a second magnetic flux density of the transformer iron core at a future moment; the future moment is later than the first moment; determining a reward value according to the second magnetic flux density of the transformer core at the future moment; based on the state parameter at the first moment, the second magnetic flux density and the reward value, determining an action adjustment parameter at the second moment; adjusting circuit parameters in the to-be-controlled circuit at a second moment according to the action adjusting parameters; the second moment is later than the first moment. According to the real-time condition of the transformer iron core in the circuit to be controlled, the circuit parameters in the circuit to be controlled are adjusted in real time, and the dynamic adjustment capability is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of circuit control technology, and in particular to a circuit control method, device and storage medium. Background Art

[0002] The relationship between the magnetic field strength and flux density of the transformer core is usually nonlinear. In the initial state, the flux density increases linearly with the increase of the magnetic field strength. When the magnetic field strength increases to a certain level, the increase of the flux density will slow down significantly. Finally, after a certain point, the flux density will reach a fixed value, which is the saturation point. After reaching the saturation point, even if the magnetic field strength continues to increase, the increase of the flux density will be very limited, or even remain unchanged. This phenomenon is called core saturation.

[0003] In the related art, in order to avoid the core saturation phenomenon, it is necessary to select a suitable core material and design a reasonable core size and shape. However, this fixed design cannot adapt to real-time changes in the circuit and does not have dynamic adjustment capabilities. Summary of the invention

[0004] The embodiments of the present application provide a circuit control method, device and storage medium, which can adjust the circuit parameters in the circuit to be controlled in real time according to the real-time situation of the transformer core in the circuit to be controlled, and have dynamic adjustment capability.

[0005] The technical solution of the embodiment of the present application is implemented as follows: An embodiment of the present application provides a circuit control method, which includes: obtaining state parameters and a first magnetic flux density of a transformer core in a circuit to be controlled at a first moment; based on the state parameters and the first magnetic flux density at the first moment, determining a second magnetic flux density of the transformer core at a future moment through a pre-trained magnetic flux density prediction model; the future moment is later than the first moment; determining a reward value according to the second magnetic flux density of the transformer core at the future moment; based on the state parameters at the first moment, the second magnetic flux density and the reward value, determining an action adjustment parameter at the second moment; the second moment is later than the first moment; and adjusting the circuit parameters in the circuit to be controlled at the second moment according to the action adjustment parameter.

[0006] An embodiment of the present application provides a circuit control device, including: an acquisition module, used to acquire state parameters and a first magnetic flux density of a transformer core in a circuit to be controlled at a first moment; a determination module, used to determine, based on the state parameters at the first moment and the first magnetic flux density, a second magnetic flux density of the transformer core at a future moment through a pre-trained magnetic flux density prediction model; the future moment is later than the first moment; the determination module is also used to determine a reward value according to the second magnetic flux density of the transformer core at the future moment; the determination module is also used to determine an action adjustment parameter at a second moment based on the state parameters at the first moment, the second magnetic flux density and the reward value; the second moment is later than the first moment; an adjustment module is used to adjust the circuit parameters in the circuit to be controlled at the second moment according to the action adjustment parameters.

[0007] In the above scheme, the determination module is also used to: obtain the inflection point value of the magnetic flux density of the transformer core, the inflection point value of the magnetic flux density is the magnetic flux density value when the curvature in the magnetization curve of the transformer core is zero; normalize the inflection point value of the magnetic flux density to obtain first normalized data; normalize the second magnetic flux density of the transformer core at a future moment to obtain second normalized data; determine the reward value based on the first normalized data and the second normalized data.

[0008] In the above scheme, the determination module is also used to: when the second normalized data is less than the first normalized data, determine the reward value as a positive number through a preset first reward value formula; when the second normalized data is greater than or equal to the first normalized data, determine the reward value as a negative number through a preset second reward value formula.

[0009] In the above scheme, the determination module is also used to: input the state parameters, the second magnetic flux density and the reward value at the first moment into a pre-constructed intelligent agent, and determine the evaluation result through the intelligent agent based on the state parameters, the second magnetic flux density and the reward value; the evaluation result includes multiple action parameters and the expected return value corresponding to each action parameter; the first action parameter in the evaluation result is determined as a candidate action adjustment parameter; the error between the expected return value corresponding to the first action parameter and the magnetic flux density inflection point value is less than a preset error threshold; when there are multiple candidate action adjustment parameters, the candidate action adjustment parameter with the largest expected return value is determined as the action adjustment parameter at the second moment from the multiple candidate action adjustment parameters; when there is one candidate action adjustment parameter, the candidate action adjustment parameter is determined as the action adjustment parameter at the second moment.

[0010] In the above scheme, the determination module is also used to: obtain preset smoothing parameters; call a preset smoothing algorithm to smooth the reward value based on the smoothing parameters to obtain a smoothed reward value; input the state parameters at the first moment, the second magnetic flux density and the smoothed reward value into the pre-constructed intelligent agent to obtain the evaluation result.

[0011] In the above scheme, the determination module is also used to: input the state parameters at the first moment, the second magnetic flux density and the smoothed reward value into the pre-constructed intelligent agent; normalize the state parameters at the first moment through the pre-constructed intelligent agent to obtain third normalized data; and normalize the second magnetic flux density to obtain fourth normalized data, and normalize the smoothed reward value to obtain fifth normalized data; concatenate the third normalized data, the fourth normalized data and the fifth normalized data to obtain a state vector; forward propagate the state vector to obtain the evaluation result.

[0012] In the above scheme, the adjustment module is also used to: obtain an adjustment step corresponding to the circuit parameter in the circuit to be controlled; and adjust the circuit parameter in the circuit to be controlled based on the action adjustment parameter from the second moment according to the adjustment step.

[0013] In the above scheme, the device also includes a training module, which is used to: obtain the state parameters of the transformer core at historical moments; the state parameters at historical moments include state parameters at a third moment and state parameters at a fourth moment, the fourth moment is later than the third moment, and the fourth moment is earlier than the first moment; normalize the state parameters at the third moment to obtain fifth normalized data; normalize the state parameters at the fourth moment to obtain sixth normalized data; train the data prediction model to be trained according to the fifth normalized data and the sixth normalized data to obtain the pre-trained magnetic flux density prediction model; wherein the fifth normalized data is the input data of the data prediction model to be trained, and the sixth normalized data is the expected output data of the data prediction model to be trained.

[0014] An embodiment of the present application provides an electronic device, comprising: a memory for storing computer-executable instructions or computer programs; and a processor for implementing the circuit control method provided in the embodiment of the present application when executing the computer-executable instructions or computer programs stored in the memory.

[0015] An embodiment of the present application provides a computer program product storing computer executable instructions or a computer program, which is used to cause a processor to execute the computer executable instructions or the computer program to implement the circuit control method provided in the embodiment of the present application.

[0016] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions or a computer program, which is used to cause a processor to execute the computer-executable instructions or the computer program to implement the circuit control method provided in the embodiment of the present application.

[0017] The embodiments of the present application have the following beneficial effects: When controlling a circuit to be controlled, first, the state parameters and the first magnetic flux density of the transformer core in the circuit to be controlled at the first moment are obtained; then, based on the state parameters and the first magnetic flux density at the first moment, the second magnetic flux density of the transformer core at a future moment is determined through a pre-trained magnetic flux density prediction model; the future moment is later than the first moment; then, a reward value is determined according to the second magnetic flux density of the transformer core at the future moment; then, based on the state parameters at the first moment, the second magnetic flux density and the reward value, an action adjustment parameter at the second moment is determined; finally, according to the action adjustment parameter, the circuit parameters in the circuit to be controlled are adjusted at the second moment; the second moment is later than the first moment. It can be seen from this that the present application determines the action adjustment parameters at the second moment by combining the predicted second magnetic flux density of the transformer core at the future moment, rather than determining the action adjustment parameters only according to the state parameters at the first moment, so that the action adjustment parameters can be more accurate; and, the action adjustment parameters are determined by the state parameters at the first moment and the second magnetic flux density at the future moment, and the circuit parameters of the control circuit are adjusted in real time according to the action adjustment parameters, with dynamic adjustment capability; there is no need to adjust the structure of the control circuit, thereby saving material costs while ensuring the magnetic conductivity efficiency of the transformer core. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is an optional flow chart of the circuit control method provided in the embodiment of the present application; Figure 2 It is a schematic diagram of the training process of the magnetic flux density prediction model provided in the embodiment of the present application; Figure 3 is a schematic diagram of a process for determining a reward value according to magnetic flux density provided in an embodiment of the present application; Figure 4 It is a flowchart of determining the action adjustment parameter at the second moment according to the state parameter and the reward value at the first moment provided by an embodiment of the present application; Figure 5are magnetization curves of different types of transformer cores provided in the embodiments of the present application; Figure 6 is a schematic diagram of the system structure of the circuit control method provided in an embodiment of the present application; Figure 7 is a structural block diagram of a circuit control device provided in an embodiment of the present application; Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0020] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] If similar descriptions of "first / second" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0022] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0023] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0024] An embodiment of the present application provides a circuit control method. When controlling a circuit to be controlled, first, the state parameters and the first magnetic flux density of a transformer core in the circuit to be controlled at a first moment are obtained; then, based on the state parameters and the first magnetic flux density at the first moment, a second magnetic flux density of the transformer core at a future moment is determined through a pre-trained magnetic flux density prediction model; the future moment is later than the first moment; then, a reward value is determined according to the second magnetic flux density of the transformer core at the future moment; then, based on the state parameters at the first moment, the second magnetic flux density and the reward value, an action adjustment parameter is determined at a second moment; the second moment is later than the first moment; finally, according to the action adjustment parameter, the circuit parameters in the circuit to be controlled are adjusted at the second moment. It can be seen from this that the present application determines the action adjustment parameters at the second moment by combining the predicted second magnetic flux density of the transformer core at the second moment, rather than determining the action adjustment parameters only according to the state parameters at the first moment, thereby making the action adjustment parameters more accurate; and, the action adjustment parameters are determined by the state parameters at the first moment and the second magnetic flux density at the second moment, and the circuit parameters of the control circuit are adjusted in real time according to the action adjustment parameters, with dynamic adjustment capability; there is no need to adjust the structure of the control circuit, thereby saving material costs while ensuring the magnetic conductivity efficiency of the transformer core.

[0025] The circuit control method provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0026] Figure 1 1 is an optional flow chart of the circuit control method provided in the embodiment of the present application. The method can be applied to an electronic device, and the electronic device can be a circuit control device. The following will take the electronic device as an example for exemplary description. Figure 1 As shown, the method includes the following steps S101 to S105: Step S101, obtaining state parameters and a first magnetic flux density of a transformer core in a circuit to be controlled at a first moment.

[0027] The circuit to be controlled refers to a circuit that adjusts and monitors circuit parameters in real time through a circuit control device; the transformer core is one of the core components of the transformer and is used to improve the efficiency of electromagnetic induction; the first moment refers to the current moment; the state parameter is a quantitative description of the state of the transformer core, and the state parameter may include current, voltage, frequency, etc.; the state parameter at the first moment refers to the state parameter adjusted by the action adjustment parameter at the first moment; the action adjustment parameter refers to the specific adjustment value calculated at the moment before the first moment and required to adjust the state variable at the first moment; the first magnetic flux density is the magnetic flux density of the transformer core at the first moment. The magnetic flux density reflects the strength of the core magnetic field and determines the working efficiency of the transformer under different loads. The specific state parameters are not limited in this application. The state parameters can be obtained through sensors and sent to the control device, or they can be directly collected by the control device.

[0028] In some embodiments, a circuit control device refers to a device that can adjust and manage operating parameters in a circuit to be controlled; wherein, the circuit control device may be a chip or microprocessor integrated in the circuit to be controlled, and the state parameters of the transformer core in the circuit to be controlled at the first moment may be directly obtained through the circuit control device; or, the circuit control device is a chip or microprocessor integrated in the circuit to be controlled, but the circuit control device does not have a parameter acquisition function, and it is necessary to acquire the state parameters through sensors deployed in the circuit to be controlled; the circuit control device may also be a controller independent of the circuit to be controlled, and the state parameters of the transformer core in the circuit to be controlled at the first moment may be acquired through sensors deployed in the circuit to be controlled, and the state parameters acquired by the sensor may be sent to the circuit control device through a communication module, and the circuit to be controlled also includes a chip or microprocessor for receiving control instructions or signals sent by the circuit control device and performing corresponding actions.

[0029] Step S102, based on the state parameters and the first magnetic flux density at the first moment, determine the second magnetic flux density of the transformer core at a future moment through a pre-trained magnetic flux density prediction model; the future moment is later than the first moment.

[0030] The magnetic flux density prediction model refers to a machine learning model specifically used to predict the magnetic flux density of the transformer core, such as the long short-term memory network (LSTM). The future moment refers to any moment after the current moment. The magnetic flux density refers to the magnetic flux passing through the transformer core per unit area. The magnetic flux density reflects the strength of the magnetic field in the transformer core. A higher magnetic flux density means a stronger magnetic field in the transformer core. The calculation formula for the magnetic flux density is shown in formula (1): Formula (1) in, represents the magnetic flux density; represents magnetic flux; Represents the area of ​​the transformer core.

[0031] In some embodiments, see Figure 2 , Figure 2 is a schematic diagram of the training process of the magnetic flux density prediction model provided in the embodiment of the present application, Figure 2 It is shown that the training process of the magnetic flux density prediction model can be implemented by the following steps S201 to S204: Step S201, obtaining state parameters of the transformer core at historical moments; the state parameters at historical moments include state parameters at a third moment and state parameters at a fourth moment, the fourth moment is later than the third moment, and the fourth moment is earlier than the first moment.

[0032] The historical moment refers to a time point that has passed and been recorded by the system before the current moment; the third moment and the fourth moment are any two time points in the historical moment, where the time point of the third moment is before the time point of the fourth moment.

[0033] In some embodiments, the state parameters of the transformer core at a historical moment may be historical data collected by a sensor at a historical moment and stored in a control device; or may be historical data directly collected and stored by a control device at a historical moment.

[0034] Step S202, normalizing the state parameters at the third moment to obtain fifth normalized data.

[0035] Normalization is a process of scaling data to a specific range. The main purpose of normalization is to eliminate the impact caused by different data magnitudes. For example, the state parameters at the third moment include the current, voltage, and frequency of the transformer core at the third moment. If directly compared or input into the control device, data with large magnitudes (such as voltage) may have a greater impact on the processing results, resulting in ignoring data with small magnitudes (such as frequency); moreover, normalization can speed up the convergence of the model and improve the performance of the model. Common normalization methods include maximum-minimum normalization, batch normalization, layer normalization, etc. The specific normalization method is not limited in this application.

[0036] In some embodiments, the control device normalizes the state parameters (such as current, voltage, frequency, etc.) at the third moment by a specific normalization method to obtain fifth normalized data.

[0037] Step S203, normalizing the state parameters at the fourth moment to obtain sixth normalized data.

[0038] In some embodiments, the control device normalizes the state parameters at the fourth moment by the normalization method mentioned in step S202 to obtain sixth normalized data.

[0039] Step S204, training the data prediction model to be trained according to the fifth normalized data and the sixth normalized data to obtain a pre-trained magnetic flux density prediction model; wherein the fifth normalized data is the input data of the data prediction model to be trained, and the sixth normalized data is the expected output data of the data prediction model to be trained.

[0040] The data prediction model to be trained refers to an untrained prediction model, which is used to be trained according to known state parameters; the pre-trained magnetic flux density model refers to a data prediction model trained according to known state parameters, which is used to predict the magnetic flux density at a future moment (such as the second moment).

[0041] In some embodiments, the pre-trained magnetic flux density model in step S204 can be trained by the following method: first, randomly initialize the weight data of the data prediction model to be trained; then, input the fifth normalized data into the data prediction model to be trained; then, based on the weight data, perform error calculation according to the prediction result of the data prediction model and the true output (sixth normalized data) to obtain the loss value; then, perform back propagation according to the calculated loss value, update the parameters of the data prediction model, and gradually reduce the loss value; finally, after multiple rounds of iterations, the parameters of the data prediction model gradually stabilize, and the final pre-trained magnetic flux density prediction model is obtained.

[0042] Step S103, determining a reward value according to a second magnetic flux density of the transformer core at a future moment.

[0043] Reward value refers to a feedback signal used to measure the performance of the current model or system during reinforcement learning or model optimization. Reinforcement learning is a machine learning method that learns the optimal strategy through interaction with the environment. Reinforcement learning guides the intelligent agent to learn through reward and punishment signals to maximize long-term benefits.

[0044] In some embodiments, see Figure 3 , Figure 3 is a schematic diagram of a process for determining a reward value according to magnetic flux density provided in an embodiment of the present application; Figure 3 It is shown that the reward value is determined according to the second magnetic flux density of the transformer core at a future moment in step S103, which can be achieved by the following steps S1031 to S1034: Step S1031, obtaining the inflection point value of the magnetic flux density of the transformer core, where the inflection point value of the magnetic flux density is the magnetic flux density value when the curvature of the magnetization curve of the transformer core is zero.

[0045] The magnetization curve shows the relationship between the magnetic flux density and the magnetic field strength of the transformer core under the action of an external magnetic field. The magnetization curve can be used to determine the optimal magnetic flux density range of the transformer core when it is working to avoid over-magnetization; the curvature is used to describe the degree of bending of the curve at a certain point. The larger the curvature, the greater the curvature of the curve. When the curvature is zero, the curve is flat at this point.

[0046] In some embodiments, the control device analyzes the curvature of the magnetization curve of the transformer core to obtain a point in the magnetization curve of the transformer core where the curvature is zero and a magnetic flux density value corresponding to the point, and determines the point as a magnetic flux density inflection point, and determines the magnetic flux density value corresponding to the magnetic flux density inflection point as the magnetic flux density inflection point value.

[0047] Step S1032, normalizing the inflection point value of the magnetic flux density to obtain first normalized data.

[0048] In some embodiments, the control device obtains first normalized data by normalizing the inflection point value of the magnetic flux density; the specific normalization method can be selected according to actual conditions, and this application does not limit it here.

[0049] Step S1033, normalizing the second magnetic flux density of the transformer core at a future moment to obtain second normalized data.

[0050] In some embodiments, the control device normalizes the second magnetic flux density to obtain second normalized data, where the second magnetic flux density is the magnetic flux density of the transformer core at a future moment predicted by a pre-trained magnetic flux density prediction model.

[0051] Step S1034: determining a reward value according to the first normalized data and the second normalized data.

[0052] Here, the reward value is calculated by comparing the magnitude relationship between the first normalized data and the second normalized data.

[0053] In some embodiments, step S1034 can be divided into two cases, specifically: Case 1: When the second normalized data is less than the first normalized data, a positive reward value is determined by using a preset first reward value formula.

[0054] The preset first reward value formula is shown in formula (2): Formula (2) in, Represents the reward value of round t; represents the second normalized data; Represents the first normalized data; n is a value greater than 1, and n can be set to a fixed value (for example, 2, 2.5, or 3), or it can be learned during training through the reward shaping mechanism of reinforcement learning.

[0055] Here, when the second normalized data is less than the first normalized data, it means that the magnetic flux density at the future moment predicted by the magnetic flux density prediction model has not yet reached the magnetic flux density inflection point value. The parameters of the circuit to be controlled can be positively adjusted according to the predicted magnetic flux density, that is, the state parameter value of the circuit to be controlled input at the future moment can be increased.

[0056] Case 2: When the second normalized data is greater than or equal to the first normalized data, a negative reward value is determined by a preset second reward value formula.

[0057] The preset second reward value formula is shown in formula (3): Formula (3) Here, m is a value greater than 1. m can be set to a fixed value (for example, 2, 2.5, or 3), or can be learned during training through the reward shaping mechanism of reinforcement learning. The remaining parameters are consistent with formula (2).

[0058] Here, when the second normalized data is greater than or equal to the first normalized data, it means that the magnetic flux density at the future moment predicted by the magnetic flux density prediction model has reached or exceeded the magnetic flux density inflection point value, and it is necessary to reversely adjust the parameters of the circuit to be controlled, that is, to reduce the state parameter value of the input of the circuit to be controlled at the future moment.

[0059] Step S104, determining the action adjustment parameter at the second moment based on the state parameter at the first moment, the second magnetic flux density and the reward value; the second moment is later than the first moment.

[0060] The second moment refers to any moment after the current moment, and the second moment may be earlier than the future moment, or the second moment may be the future moment; the action adjustment parameter refers to the specific adjustment value that needs to be adjusted for the state variable at the next moment, and the power parameters in the circuit to be controlled can be adjusted in real time through the action adjustment parameter to ensure that the circuit to be controlled can be maintained in the optimal state.

[0061] In some embodiments, see Figure 4 , Figure 4 It is a flowchart of determining the action adjustment parameter at the second moment according to the state parameter and the reward value at the first moment provided by an embodiment of the present application; Figure 4 It is shown that in step S104, based on the state parameter at the first moment, the second magnetic flux density and the reward value, the action adjustment parameter at the second moment is determined, which can be achieved by the following steps S1041 to S1044: Step S1041, input the state parameters, the second magnetic flux density and the reward value at the first moment into the pre-constructed intelligent agent, and determine the evaluation result based on the state parameters, the second magnetic flux density and the reward value by the intelligent agent; the evaluation result includes multiple action parameters and the expected return value corresponding to each action parameter.

[0062] Pre-built agents refer to agents that have been configured in advance; agents are the main body in reinforcement learning. Agents adjust their own behavior according to the environment state (state parameters) and reward values, with the goal of maximizing long-term accumulated rewards; action parameters refer to the specific operations that the agent can perform; expected reward value refers to the total reward that the agent is expected to receive in the future after performing a certain action. The agent selects the optimal action by calculating the expected reward value of each action.

[0063] In some embodiments, step S1041 can be implemented by the following method: first, obtain the preset smoothing parameters; then, call the preset smoothing algorithm to smooth the reward value based on the smoothing parameters to obtain a smoothed reward value; finally, input the state parameters at the first moment, the second magnetic flux density and the smoothed reward value into the pre-constructed intelligent agent to obtain an evaluation result.

[0064] The smoothing parameter is a key parameter used to adjust the smoothing algorithm. The smoothing parameter determines the intensity of the smoothing process. In reinforcement learning, the smoothing parameter can help reduce the noise in the reward value. The smoothing algorithm is an algorithm that smoothes data and is usually used to eliminate random noise and fluctuations in the data. Common smoothing algorithms include: moving average, exponentially weighted average, Gaussian smoothing, etc. The specific smoothing algorithm is not limited in this application. The smoothed reward value refers to the reward value processed by the smoothing algorithm. The noise and fluctuation in the smoothed reward value are greatly reduced compared to the reward value.

[0065] In some examples, the reward value is smoothed using the moving average method. For example, at a certain moment, the reward value sequence is [10, 12, -30, 15, 10]. The smoothing parameter can be a window size, that is, when calculating the moving average, several recent reward values ​​are considered. For example, if the window size is 3, every three consecutive reward values ​​are averaged. Then, the reward value is smoothed by the moving average method: [(10+12+(-30)) / 3, (12+(-30)+15) / 3, (-30+15+10) / 3], and the resulting smoothed reward value is [-2.67, -1, -1.67].

[0066] In some embodiments, the state parameters at the first moment, the second magnetic flux density and the smoothed reward value are input into a pre-constructed intelligent agent to obtain an evaluation result, which can be achieved by the following method: first, the state parameters at the first moment, the second magnetic flux density and the smoothed reward value are input into the pre-constructed intelligent agent; then, the state parameters at the first moment are normalized by the pre-constructed intelligent agent to obtain third normalized data; and, the second magnetic flux density is normalized to obtain fourth normalized data, and the smoothed reward value is normalized to obtain fifth normalized data; then, the third normalized data, the fourth normalized data and the fifth normalized data are concatenated to obtain a state vector; finally, the state vector is forward propagated to obtain an evaluation result.

[0067] In some examples, assuming that the current is 75 mA and the voltage is 220 V at the first moment, the state parameter at the first moment is [75, 220], the second magnetic flux density of the transformer core is 0.85 Tesla, and the calculated smoothed reward value is [-5]; then, the state parameter at the first moment, the second magnetic flux density and the smoothed reward value are normalized to obtain the third normalized data, the fourth normalized data and the fifth normalized data, respectively, such as the third normalized data is [0.75, 0.917], the fourth normalized data is [0.708], and the fifth normalized data is [0.45]; then Then, the third normalized data, the fourth normalized data and the fifth normalized data are spliced ​​to obtain a state vector. For example, the state vector obtained by splicing the third normalized data, the fourth normalized data and the fifth normalized data is [0.75, 0.917, 0.708, 0.45]. Finally, the spliced ​​state vector is forward propagated to obtain an evaluation result, which can be {action 1: voltage -10, expected reward value: flux density +4; action 2: voltage -5, expected reward value: flux density +3; action 3: maintain voltage unchanged, expected reward value: flux density +1}.

[0068] Step S1042, determining the first action parameter in the evaluation result as a candidate action adjustment parameter; the error between the expected reward value corresponding to the first action parameter and the magnetic flux density inflection point value is less than a preset error threshold.

[0069] In some embodiments, assuming that the evaluation result is {action 1: voltage -10, expected reward value: magnetic flux density +4; action 2: voltage -5, expected reward value: magnetic flux density +3; action 3: maintain voltage unchanged, expected reward value: magnetic flux density +1}, wherein the operations corresponding to action 1, action 2 and action 3 are action parameters; assuming that the current magnetic flux density is 5 Tesla, the magnetic flux density inflection point value is 11 Tesla, and the preset error threshold is 5, the error between the expected reward value corresponding to each action and the magnetic flux density inflection point value is calculated according to the expected reward value in the evaluation result. It can be seen that the error between the expected reward value corresponding to action 1, action 2 and action 3 and the magnetic flux density inflection point value is less than the preset error threshold, then action 1, action 2 and action 3 are all used as candidate action adjustment parameters.

[0070] Step S1043: when there are multiple candidate action adjustment parameters, determine the candidate action adjustment parameter with the maximum expected reward value from the multiple candidate action adjustment parameters as the action adjustment parameter at the second moment.

[0071] In some embodiments, when there are multiple candidate action adjustment parameters, the expected reward values ​​corresponding to each candidate action parameter are compared, and the candidate action adjustment parameter with the largest expected reward value is determined as the action adjustment parameter at the second moment. For example, if the expected reward value of action 1 is magnetic flux density + 4, the expected reward value of action 2 is magnetic flux density + 3, and the expected reward value of action 3 is magnetic flux density + 1, then action 1 is used as the action adjustment parameter at the second moment.

[0072] Step S1044: when there is only one candidate action adjustment parameter, determine the candidate action adjustment parameter as the action adjustment parameter at the second moment.

[0073] In some embodiments, assuming that the evaluation result is {action 1: voltage -10, expected reward value: magnetic flux density +4; action 2: voltage -5, expected reward value: magnetic flux density +3; action 3: maintain voltage unchanged, expected reward value: magnetic flux density +1}; when the preset error threshold is 2, there is only one candidate action parameter, namely action 3, and action 3 is determined as the action adjustment parameter at the second moment.

[0074] Step S105 , adjusting the circuit parameters in the circuit to be controlled at the second moment according to the action adjustment parameters.

[0075] After determining the action adjustment parameter, the circuit parameters in the circuit to be controlled are adjusted at the second moment according to the action corresponding to the action adjustment parameter; the circuit parameters refer to the parameters that need to be adjusted in the circuit to be controlled at the second moment.

[0076] Here, the relationship between the first moment, the future moment and the second moment and the prediction scenario are explained: the first moment refers to the current moment, such as 10:00 on XX / XX / XX; the future moment refers to a moment after the first moment, such as 12:00 on XX / XX / XX; the second moment refers to a moment after the first moment, where the second moment can be earlier than the future moment or equal to the future moment.

[0077] In some examples, the first magnetic flux density of the transformer core at 10:00 is 10 Tesla. The first magnetic flux density and state parameters of the transformer core at 10:00 are input into a pre-trained magnetic flux density prediction model, and the second magnetic flux density of the transformer core at 12:00 is predicted to be 20 Tesla. That is to say, the magnetic flux density of the transformer core at 12:00 needs to reach 20 Tesla, or the second magnetic flux density can be maintained at 20 Tesla during the time period from 10:00 to 12:00. The state parameters of the transformer core at 10:00 and the predicted second magnetic flux density and the reward value calculated according to the second magnetic flux density are input into the intelligent agent, and the action adjustment parameters are determined by the intelligent agent. According to the action adjustment parameters, the circuit parameters of the control circuit are adjusted at 10:01, and 10:01 is the second moment.

[0078] In some embodiments, step S105 may be implemented by the following method: first, obtaining an adjustment step corresponding to a circuit parameter in the circuit to be controlled; then, adjusting the circuit parameter in the circuit to be controlled based on the action adjustment parameter from the second moment according to the adjustment step.

[0079] Here, the adjustment step size may be a preset value for controlling the parameter adjustment rate, and the adjustment step size defines the magnitude of the change in the circuit parameter when the control device adjusts the circuit parameter.

[0080] In some examples, assume that the first moment is 10:00 on XX / XX / XX, and the second moment is 11:00 on XX / XX / XX; at 10:00, the current in the circuit to be controlled is 10A, the current adjustment step is 0.5A, and the action adjustment parameter is current+2; then the parameter adjustment value is calculated according to the current adjustment step and the action adjustment parameter, and the parameter adjustment value is: 2×0.5A=1A, and the circuit parameters in the circuit to be controlled at 11:00 are determined according to the current and parameter adjustment value of the circuit to be controlled at 10:00, then the circuit parameters in the circuit to be controlled at 11:00 are: 10A+1A=11A.

[0081] Here, the adjustment step size may also be a preset time interval of the adjustment operation, and the adjustment step size determines the frequency with which the control device adjusts the circuit parameters of the circuit to be controlled.

[0082] In some examples, assuming that the adjustment step is 5 minutes and the action adjustment parameter is a voltage increase of 0.1V, then starting from the second moment, the current voltage value is increased by 0.1V every 5 minutes; for example: the second moment is 10:00 on XX / XX / XX, and the voltage value at this time is 10V, the adjustment step is 5 minutes, and the action adjustment parameter is a voltage increase of 0.1V, then the voltage is adjusted to 10.1V at 10:05, and the voltage is adjusted to 10.2V at 10:10.

[0083] The circuit control method provided in the embodiment of the present application can be applied to at least the following scenarios: Scenario 1: In a smart grid, the magnetic flux density of the transformer core needs to be dynamically adjusted as the power demand changes to ensure the stability and efficient operation of the power grid. The circuit parameters in the power grid can be adjusted according to the circuit control method provided in the embodiment of the present application. The control device can obtain the state parameters of the transformer core in real time, and use the magnetic flux density prediction model to predict the magnetic flux density at future times. When the power load changes, based on the predicted magnetic flux density, the control device can calculate the reward value and determine the optimal action adjustment parameters, thereby adjusting the input voltage, current and other circuit parameters to ensure that the transformer can work efficiently when the load fluctuates and avoid core saturation or excessive loss.

[0084] Scenario 2: In the industrial manufacturing scenario, in certain industrial manufacturing equipment that requires high-precision voltage control, such as electronic component manufacturing or precision processing equipment, the magnetic flux density of the transformer core needs to be maintained within a specific range; in this way, the circuit control method provided in the embodiment of the present application can be adopted; first, the control device obtains the state parameters of the transformer core in real time, and uses the magnetic flux density prediction model to predict the magnetic flux density at future times. When the predicted magnetic flux density exceeds the range that the transformer core needs to maintain, based on the predicted magnetic flux density, the reward value is calculated and the optimal action adjustment parameters are determined, thereby adjusting the input voltage, current and other circuit parameters to ensure that the magnetic flux density of the transformer core is always maintained within a specific range.

[0085] The circuit control method provided in the embodiment of the present application determines the action adjustment parameters at the second moment by combining the predicted second magnetic flux density of the transformer core at the future moment, rather than determining the action adjustment parameters only according to the state parameters at the first moment, thereby making the action adjustment parameters more accurate; and, the action adjustment parameters are determined by the state parameters at the first moment and the second magnetic flux density at the future moment, and the circuit parameters of the control circuit are adjusted in real time according to the action adjustment parameters, thereby having dynamic adjustment capability; there is no need to adjust the structure of the control circuit, thereby saving material costs while ensuring the magnetic conductivity efficiency of the transformer core.

[0086] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0087] In a transformer core, the relationship between flux density and magnetic field strength is usually nonlinear, see Figure 5 , Figure 5 is the magnetization curve of different types of transformer cores provided by the embodiments of the present application. In the initial part of the magnetization curve, the magnetic flux density increases linearly with the increase of the magnetic field strength, and the magnetic permeability of the transformer core is relatively high at this time; however, when the magnetic field strength continues to increase to a certain extent, the increase of the magnetic flux density will slow down significantly and eventually reach a platform, that is, the saturation point; after reaching the saturation point, even if the magnetic field strength continues to increase, the increase of the magnetic flux density is very limited, and this phenomenon is called core saturation.

[0088] When the transformer core saturates, the following problems may occur: 1) Reduced efficiency: After the core is saturated, the magnetic permeability of the transformer core decreases, resulting in a larger current being required to maintain the same magnetic flux, thereby increasing energy loss.

[0089] 2) Overheating: As more current is required, the losses in the transformer core and coil increase, causing the equipment to overheat.

[0090] 3) Output reduction: In a transformer or inductor, core saturation may cause a reduction in output voltage or current.

[0091] 4) Noise and vibration: In motors and generators, core saturation may cause additional noise and vibration.

[0092] In order to avoid core saturation, the present application embodiment proposes a circuit control method, see Figure 6 , Figure 6It is a schematic diagram of the system structure of the circuit control method provided in the embodiment of the present application. The environment 601 includes the environment St+1602 and the LSTM model 603 (the above-mentioned pre-trained magnetic flux density prediction model); the environment St+1 includes the magnetic flux density of the transformer core at the future moment (the above-mentioned second moment) predicted by the LSTM model 603, and the state parameters of the transformer core at the current moment (the above-mentioned first moment); then, the state parameters of the transformer core at the current moment and the magnetic flux density of the transformer core at the future moment are normalized and input into the intelligent agent 604, and at the same time, the reward value is determined according to the magnetic flux density of the transformer core at the future moment, and the reward value is normalized. The normalized data of the reward value is input into the intelligent agent 604. The intelligent agent 604 determines the action adjustment parameters at the future moment according to the normalized data corresponding to the state parameters of the transformer core at the current moment, the normalized data corresponding to the magnetic flux density of the transformer core at the future moment, and the normalized data corresponding to the reward value, and adjusts the circuit parameters of the circuit to be controlled at the future moment according to the action adjustment parameters; collects the state parameters after action adjustment and the magnetic flux density at the current moment, and inputs them into the LSTM model 603 to continue predicting the magnetic flux density at the next moment; and executes the above operations in a loop so that the transformer core always remains in a non-saturated state.

[0093] The circuit control method provided in the embodiment of the present application can predict the future magnetic flux density of the transformer core, which means that the reinforcement learning algorithm can make decisions based on the expected future state rather than just the current state, so that the control device can react in advance to prevent potential problems, such as core saturation, rather than just responding after the problem occurs. The prediction results of the LSTM model are smoother than the actual results. The actual results may have abnormal points (possibly due to the instability of the detection equipment), while the predicted results usually do not have such abnormal points. The prediction results of the LSTM model are input into the reinforcement learning model. The reinforcement learning algorithm can adjust the circuit parameters more smoothly, avoid excessive adjustments due to real-time data fluctuations, help maintain the stability of the system, and reduce energy consumption and equipment wear caused by frequent adjustments.

[0094] Based on the circuit control method described in the above embodiment, Figure 7 A structural block diagram of a circuit control device 100 provided in an embodiment of the present application is shown. The circuit control device may be a device in an electronic device (e.g., a control device). The circuit control device may be implemented in software, which may be software in the form of programs and plug-ins, etc., including the following software modules: an acquisition module 101, a determination module 102, and an adjustment module 103. These modules are logical, and therefore may be arbitrarily combined or further split according to the functions implemented.

[0095] Among them, the acquisition module 101 is used to obtain the state parameters and the first magnetic flux density of the transformer core in the circuit to be controlled at the first moment; the determination module 102 is used to determine the second magnetic flux density of the transformer core at a future moment based on the state parameters and the first magnetic flux density at the first moment through a pre-trained magnetic flux density prediction model; the future moment is later than the first moment; the determination module 102 is also used to determine the reward value according to the second magnetic flux density of the transformer core at the future moment; the determination module 102 is also used to determine the action adjustment parameters at the second moment based on the state parameters at the first moment, the second magnetic flux density and the reward value; the second moment is later than the first moment; the adjustment module 103 is used to adjust the circuit parameters in the circuit to be controlled at the second moment according to the action adjustment parameters.

[0096] In some embodiments, the determination module is also used to: obtain the inflection point value of the magnetic flux density of the transformer core, the inflection point value of the magnetic flux density is the magnetic flux density value when the curvature in the magnetization curve of the transformer core is zero; normalize the inflection point value of the magnetic flux density to obtain first normalized data; normalize the second magnetic flux density of the transformer core at a future moment to obtain second normalized data; determine the reward value based on the first normalized data and the second normalized data.

[0097] In some embodiments, the determination module is also used to: when the second normalized data is less than the first normalized data, determine the reward value as a positive number through a preset first reward value formula; when the second normalized data is greater than or equal to the first normalized data, determine the reward value as a negative number through a preset second reward value formula.

[0098] In some embodiments, the determination module is also used to: input the state parameters, the second magnetic flux density and the reward value at the first moment into a pre-constructed intelligent agent, and determine the evaluation result through the intelligent agent based on the state parameters, the second magnetic flux density and the reward value; the evaluation result includes multiple action parameters and the expected return value corresponding to each action parameter; the first action parameter in the evaluation result is determined as a candidate action adjustment parameter; the error between the expected return value corresponding to the first action parameter and the magnetic flux density inflection point value is less than a preset error threshold; in the case where there are multiple candidate action adjustment parameters, from the multiple candidate action adjustment parameters, the candidate action adjustment parameter with the largest expected return value is determined as the action adjustment parameter at the second moment; in the case where there is one candidate action adjustment parameter, the candidate action adjustment parameter is determined as the action adjustment parameter at the second moment.

[0099] In some embodiments, the determination module is also used to: obtain preset smoothing parameters; call a preset smoothing algorithm to smooth the reward value based on the smoothing parameters to obtain a smoothed reward value; input the state parameters at the first moment, the second magnetic flux density and the smoothed reward value into the pre-constructed intelligent agent to obtain the evaluation result.

[0100] In some embodiments, the determination module is also used to: input the state parameters at the first moment, the second magnetic flux density and the smoothed reward value into the pre-constructed intelligent agent; normalize the state parameters at the first moment through the pre-constructed intelligent agent to obtain third normalized data; and normalize the second magnetic flux density to obtain fourth normalized data, and normalize the smoothed reward value to obtain fifth normalized data; concatenate the third normalized data, the fourth normalized data and the fifth normalized data to obtain a state vector; forward propagate the state vector to obtain the evaluation result.

[0101] In some embodiments, the adjustment module is further used to: obtain an adjustment step corresponding to the circuit parameters in the circuit to be controlled; and adjust the circuit parameters in the circuit to be controlled based on the action adjustment parameters starting from the second moment according to the adjustment step.

[0102] In some embodiments, the device also includes a training module, which is used to: obtain state parameters of the transformer core at historical moments; the state parameters at historical moments include state parameters at a third moment and state parameters at a fourth moment, the fourth moment is later than the third moment, and the fourth moment is earlier than the first moment; normalize the state parameters at the third moment to obtain fifth normalized data; normalize the state parameters at the fourth moment to obtain sixth normalized data; train the data prediction model to be trained according to the fifth normalized data and the sixth normalized data to obtain the pre-trained magnetic flux density prediction model; wherein the fifth normalized data is the input data of the data prediction model to be trained, and the sixth normalized data is the expected output data of the data prediction model to be trained.

[0103] It should be noted that the description of the device of the embodiment of the present application is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment, so it is not repeated. For technical details not disclosed in the embodiment of the device, please refer to the description of the method embodiment of the present application for understanding.

[0104] An embodiment of the present application provides an electronic device, Figure 8Schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 8 As shown, the electronic device 110 includes: at least one processor 111 ( Figure 8 Only one is shown in the figure), a memory 112, and executable instructions 113 stored in the memory 112 and executable on at least one processor 111, when the processor 111 executes the executable instructions 113, the steps in any of the above-mentioned circuit control method embodiments are implemented.

[0105] The electronic device may include but is not limited to a processor 111 and a memory 112. Those skilled in the art may understand that: Figure 8 This is merely an example of the electronic device 110 and does not constitute a limitation on the electronic device 110 . The electronic device 110 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0106] The processor 111 may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPG), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0107] In some embodiments, the memory 112 may be an internal storage unit of the electronic device 110, such as a hard disk or memory of the electronic device 110. In other embodiments, the memory 112 may also be an external storage device of the electronic device 110, such as a plug-in hard disk, a smart memory card (SMC, Smart Media Card), a secure digital (SD, Secure Digital) card, a flash card, etc. equipped on the electronic device 110. Further, the memory 112 may also include both an internal storage unit of the electronic device 110 and an external storage device. The memory 112 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of a computer program, etc. The memory 112 may also be used to temporarily store data that has been output or is to be output.

[0108] The embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions or computer programs. When the computer-executable instructions or computer programs are executed by a processor, the processor will be caused to execute the above-mentioned circuit control method, for example, Figure 1 The circuit control method is shown.

[0109] An embodiment of the present application provides a computer program product, which includes computer-executable instructions or a computer program, and the computer-executable instructions or the computer program are stored in a computer-readable storage medium; wherein, when a processor of an electronic device reads the computer-executable instructions or the computer program from the computer-readable storage medium and executes the computer-executable instructions or the computer program, the above-mentioned circuit control method is implemented.

[0110] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0111] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0112] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0113] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0114] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A circuit control method, characterized in that: The method comprises: Acquire state parameters and a first magnetic flux density of a transformer core in a circuit to be controlled at a first moment; Based on the state parameters at the first moment and the first magnetic flux density, determining a second magnetic flux density of the transformer core at a future moment through a pre-trained magnetic flux density prediction model; the future moment is later than the first moment; Determining a reward value according to a second magnetic flux density of the transformer core at a future time; determining an action adjustment parameter at a second moment based on the state parameter at the first moment, the second magnetic flux density and the reward value; the second moment is later than the first moment; According to the action adjustment parameters, the circuit parameters in the circuit to be controlled are adjusted at the second moment.

2. The method according to claim 1, characterized in that The step of determining the reward value according to the second magnetic flux density of the transformer core at a future moment includes: Obtaining a magnetic flux density inflection point value of the transformer core, wherein the magnetic flux density inflection point value is a magnetic flux density value when the curvature of the magnetization curve of the transformer core is zero; Normalizing the magnetic flux density inflection point value to obtain first normalized data; Normalizing the second magnetic flux density of the transformer core at a future moment to obtain second normalized data; The reward value is determined according to the first normalized data and the second normalized data.

3. The method according to claim 2, characterized in that The determining the reward value according to the first normalized data and the second normalized data includes: When the second normalized data is less than the first normalized data, determining the reward value that is a positive number by using a preset first reward value formula; When the second normalized data is greater than or equal to the first normalized data, the reward value having a negative value is determined by a preset second reward value formula.

4. The method according to claim 2, characterized in that: The determining the action adjustment parameter at the second moment based on the state parameter at the first moment, the second magnetic flux density and the reward value includes: The state parameters at the first moment, the second magnetic flux density and the reward value are input into a pre-constructed intelligent agent, and an evaluation result is determined by the intelligent agent based on the state parameters, the second magnetic flux density and the reward value; the evaluation result includes a plurality of action parameters and an expected return value corresponding to each action parameter; Determine the first action parameter in the evaluation result as a candidate action adjustment parameter; the error between the expected return value corresponding to the first action parameter and the magnetic flux density inflection point value is less than a preset error threshold; In the case where there are multiple candidate action adjustment parameters, determining a candidate action adjustment parameter with a maximum expected reward value from the multiple candidate action adjustment parameters as the action adjustment parameter at the second moment; When there is one candidate action adjustment parameter, the candidate action adjustment parameter is determined as the action adjustment parameter at the second moment.

5. The method according to claim 4, characterized in that The step of inputting the state parameter at the first moment, the second magnetic flux density, and the reward value into a pre-built intelligent agent, and determining an evaluation result based on the state parameter and the reward value by the intelligent agent, comprises: Get the preset smoothing parameters; Calling a preset smoothing algorithm to smooth the reward value based on the smoothing parameter to obtain a smoothed reward value; The state parameters at the first moment, the second magnetic flux density and the smoothed reward value are input into the pre-constructed intelligent agent to obtain the evaluation result.

6. The method according to claim 5, characterized in that The step of inputting the state parameter at the first moment and the smoothed reward value into the pre-built agent to obtain the evaluation result includes: Inputting the state parameter at the first moment, the second magnetic flux density and the smoothed reward value into the pre-constructed intelligent agent; By means of the pre-constructed agent, the state parameters at the first moment are normalized to obtain third normalized data; and the second magnetic flux density is normalized to obtain fourth normalized data, and the smoothed reward value is normalized to obtain fifth normalized data; splicing the third normalized data, the fourth normalized data and the fifth normalized data to obtain a state vector; The state vector is forward propagated to obtain the evaluation result.

7. The method according to claim 1, characterized in that The adjusting the parameters according to the action to adjust the circuit parameters in the circuit to be controlled at the second moment includes: Acquire an adjustment step length corresponding to a circuit parameter in the circuit to be controlled; Starting from the second moment according to the adjustment step, the circuit parameters in the circuit to be controlled are adjusted based on the action adjustment parameters.

8. The method according to claim 1, characterized in that The method further comprises: Acquire state parameters of the transformer core at historical moments; the state parameters at historical moments include state parameters at a third moment and state parameters at a fourth moment, the fourth moment is later than the third moment, and the fourth moment is earlier than the first moment; Normalizing the state parameters at the third moment to obtain fifth normalized data; Normalizing the state parameters at the fourth moment to obtain sixth normalized data; The data prediction model to be trained is trained according to the fifth normalized data and the sixth normalized data to obtain the pre-trained magnetic flux density prediction model; wherein the fifth normalized data is the input data of the data prediction model to be trained, and the sixth normalized data is the expected output data of the data prediction model to be trained.

9. A circuit control device, characterized in that: include: An acquisition module, used for acquiring a state parameter and a first magnetic flux density of a transformer core in a circuit to be controlled at a first moment; A determination module, configured to determine, based on the state parameters at the first moment and the first magnetic flux density, a second magnetic flux density of the transformer core at a future moment through a pre-trained magnetic flux density prediction model; the future moment is later than the first moment; The determination module is further used to determine the reward value according to the second magnetic flux density of the transformer core at a future moment; The determination module is further used to determine the action adjustment parameter at a second moment based on the state parameter at the first moment, the second magnetic flux density and the reward value; the second moment is later than the first moment; An adjustment module is used to adjust parameters according to the action, and adjust circuit parameters in the circuit to be controlled at the second moment.

10. A computer-readable storage medium, characterized in that: Computer executable instructions or computer programs are stored, which are used to cause a processor to execute the computer executable instructions or the computer program to implement the circuit control method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Response information output method and device, electronic equipment and readable storage medium

    CN112084300A

  • Transformer saturation state simulation method and device

    CN115062472A

  • Accurate coal blending method and system based on deep reinforcement learning

    CN118333229A

  • Equipment control method and device, electronic equipment and storage medium

    CN118605263A

  • Machine learning system and magnetizer for motor

    US20170090432A1