Method and system for permutation and combination of detection cases of power load management system

Abstract

The invention provides a method and a system for permutation and combination of detection cases of a power load management system. The method comprises the following steps: performing environment initialization and actor reviewer model initialization according to each detection case of the power load management system; selecting a detection case from the unselected detection cases according to the current strategy and state by adopting an actor reviewer model, and iteratively updating the actor reviewer model according to the selected detection case after the detection case is selected every time until each detection case is selected; obtaining an optimal permutation and combination result of each detection case as a test path; according to the method, the actor reviewer model is adopted to select the optimal detection case permutation and combination, the optimization efficiency of the detection path is improved, meanwhile, through the iterative optimization process of the actor reviewer model, the test path can be gradually optimized, the relevance between the test cases is higher, the test effect is better, the execution of redundant test cases is reduced, and the test efficiency is improved. And the time required by the whole test process is shortened.

Description

Technical Field

[0001] This invention relates to the field of power load management technology, and specifically to a method and system for arranging and combining test cases in a power load management system. Background Technology

[0002] With the continuous evolution of the global energy structure and the deepening of power system reform, the construction of new power load management systems is progressing steadily. This management system not only needs to cope with increasingly complex power networks but also needs to meet users' stringent requirements for power supply efficiency, stability, and reliability. Nevertheless, as a highly intelligent management solution, it still has significant room for improvement in the operation and monitoring areas at the user end and in the control loop. Complex and varied monitoring items and operating procedures directly affect the efficiency and accuracy of on-site monitoring, thus posing challenges to the overall safety and stability of operation.

[0003] User-side and control loop field testing is crucial for ensuring the normal operation of the power system. User energy demand can fluctuate due to seasonality, time, or specific events, making real-time monitoring of changes in user-side electricity demand a complex and critical task. With changing energy consumption patterns and the continuous evolution of energy supply, the status and demand of user-side equipment are also constantly changing, further increasing the demands for the real-time performance and accuracy of field testing. Furthermore, as a core component of the power system, the stability and performance of the control loop directly affect the overall system's operational efficiency and safety. However, the complexity and variability of the control loop make field testing exceptionally tedious and susceptible to various factors such as external environmental factors, equipment failures, or human error. Rapid detection and effective resolution of these influencing factors are essential for ensuring system stability and safe operation.

[0004] Although there is considerable theoretical and practical experience in fault detection for user-side terminals and control loops, existing detection technologies still primarily focus on the system output data for fault detection. Detection cases play a crucial role in evaluating and maintaining power systems, helping to identify potential system problems, guide maintenance decisions, and optimize system performance. Therefore, developing effective methods for arranging and combining detection cases to improve detection efficiency and accuracy is key to enhancing system stability and safe operation. Summary of the Invention

[0005] To address the shortcomings in efficiency and accuracy of existing power load management detection processes, this invention proposes a method for arranging and combining detection test cases in a power load management system, including:

[0006] The environment and actor / critic model are initialized based on the various test cases of the power load management system.

[0007] The actor critic model is used to select test cases from the unselected test cases based on the current strategy and state. After each selection of test cases, the actor critic model is iteratively updated based on the selected test cases until all test cases are selected. The optimal permutation and combination result of each test case is obtained as the test path.

[0008] The actor network of the actor critic model is used to select detection cases based on the current strategy and state, and the critic network of the actor critic model is used to evaluate the benefits based on the selected detection cases, the benefits corresponding to the effectiveness or quality of the test.

[0009] Preferably, the initialization of the environment and the actor / critic model based on each detection case of the power load management system includes:

[0010] One-hot encoding is used to map each detection case of the power load management system to a vector in a multi-dimensional space. Each dimension of the vector represents an attribute of the detection case, and the attribute includes one or more of the following: the function, status or detection purpose of the equipment.

[0011] A first list and a second list are established and initialized as environment initialization. The initial value of the first list is an empty set, which is used to store the selected vectors. The initial value of the second list contains the vectors corresponding to each detection case, which is used to store the vectors that have not yet been selected.

[0012] Randomly initialize the actor network and critic network as the initialization of the actor-critic model.

[0013] Preferably, the step of using the actor critic model to select detection cases from the unselected detection cases based on the current strategy and state, and iteratively updating the actor critic model based on the selected detection cases after each selection, until all detection cases are selected, and obtaining the optimal permutation and combination result of each detection case as the test path, includes:

[0014] The actor network using the actor critic model selects a detection case based on the current strategy and state, wherein the strategy is the probability distribution of selecting each detection case under the current state and strategy parameters, the state includes the first list and the second list; and the strategy parameters are the parameters of the actor network.

[0015] The selected test cases are saved in the first list in the order they were selected, and the selected test cases are deleted from the second list. The reward value after selecting the test cases is calculated.

[0016] The critic network using the actor critic model evaluates the value before and after selecting detection cases, and calculates the value of the advantage function based on the value before and after selecting detection cases and the reward value.

[0017] Update the actor network and critic network based on the value of the advantage function;

[0018] Determine whether all available test cases have been selected. If so, use the first list as the optimal permutation and combination of test cases and the test path, and end the test. Otherwise, jump to the actor network using the actor critic model and select a test case based on the current strategy and state.

[0019] Preferably, the value of the reward is calculated as follows:

[0020]

[0021] In the formula, r represents the reward value, m represents the number of test cases in the test path, and X... i+1 Let X represent the (i+1)th detection case. i This represents the i-th detection case.

[0022] Preferably, the value of the dominance function is calculated as follows:

[0023] A(s,a)=r+γV(s) ′ )-V(s)

[0024] In the formula, A(s,a) represents the value of the advantage function after performing action a in the current state s, γ represents the discount factor, r represents the reward value, and V(s) ′ ) represents the state s after performing action a. ′ The value of V(s) represents the value of the current state s, and the action a represents selecting a test case.

[0025] The preferred formula for updating the actor network is as follows:

[0026]

[0027] In the formula, θ new Let V(s) represent the updated policy parameters of the actor network, α represent the learning rate of the actor network, θ represent the current policy parameters of the actor network, γ represent the discount factor, r represent the reward value, and V(s) represent the policy parameters of the actor network. ′ ) represents the state s after the action is performed. ′ The value of the current state is V(s), and the policy π(a|s,θ) represents the probability distribution of performing action a given the current state s and the policy parameter θ. Let π(a|s,θ) represent the gradient of the policy π with respect to θ, and let action a represent selecting a detection case.

[0028] The preferred formula for updating the critic network is as follows:

[0029] V new (s)←V(s)+β[r+γV(s ′ )-V(s)]

[0030] In the formula, V new V(s) represents the updated value of the critic network with respect to state s, β represents the learning rate of the critic network, γ represents the discount factor, r represents the reward value, and V(s) ′ ) represents the state s after the action is performed. ′ The value of V(s) represents the value of the current state s, and the action to perform represents selecting a test case.

[0031] Based on the same inventive concept, this application also provides a power load management system test case arrangement and combination system, including: an initialization module and an iterative update module;

[0032] The initialization module is used to initialize the environment and the actor / critic model according to the various detection cases of the power load management system.

[0033] The iterative update module is used to select detection cases from the unselected detection cases according to the current strategy and state using the actor critic model, and iteratively update the actor critic model according to the selected detection cases after each selection, until all detection cases are selected, and obtain the optimal permutation and combination result of each detection case as the test path;

[0034] The actor network of the actor critic model is used to select detection cases based on the current strategy and state, and the critic network of the actor critic model is used to evaluate the benefits based on the selected detection cases, the benefits corresponding to the effectiveness or quality of the test.

[0035] Preferably, the initialization module is specifically used for:

[0036] One-hot encoding is used to map each detection case of the power load management system to a vector in a multi-dimensional space. Each dimension of the vector represents an attribute of the detection case, and the attribute includes one or more of the following: the function, status or detection purpose of the equipment.

[0037] A first list and a second list are established and initialized as environment initialization. The initial value of the first list is an empty set, which is used to store the selected vectors. The initial value of the second list contains the vectors corresponding to each detection case, which is used to store the vectors that have not yet been selected.

[0038] Randomly initialize the actor network and critic network as the initialization of the actor-critic model.

[0039] Preferably, the iterative update module is specifically used for:

[0040] The actor network using the actor critic model selects a detection case based on the current strategy and state, wherein the strategy is the probability distribution of selecting each detection case under the current state and strategy parameters, the state includes the first list and the second list; and the strategy parameters are the parameters of the actor network.

[0041] The selected test cases are saved in the first list in the order they were selected, and the selected test cases are deleted from the second list. The reward value after selecting the test cases is calculated.

[0042] The critic network using the actor critic model evaluates the value before and after selecting detection cases, and calculates the value of the advantage function based on the value before and after selecting detection cases and the reward value.

[0043] Update the actor network and critic network based on the value of the advantage function;

[0044] Determine whether all available test cases have been selected. If so, use the first list as the optimal permutation and combination of test cases and the test path, and end the test. Otherwise, jump to the actor network using the actor critic model and select a test case based on the current strategy and state.

[0045] Preferably, the reward value in the iterative update module is calculated using the following formula:

[0046]

[0047] In the formula, r represents the reward value, m represents the number of test cases in the test path, and X... i+1 Let X represent the (i+1)th detection case. i This represents the i-th detection case.

[0048] Preferably, the formula for calculating the value of the advantage function in the iterative update module is as follows:

[0049] A(s,a)=r+γV(s) ′ )-V(s)

[0050] In the formula, A(s,a) represents the value of the advantage function after performing action a in the current state s, γ represents the discount factor, r represents the reward value, and V(s) ′ ) represents the state s after performing action a. ′The value of V(s) represents the value of the current state s, and the action a represents selecting a test case.

[0051] Preferably, the calculation formula for updating the actor network by the iterative update module is as follows:

[0052]

[0053] In the formula, θ new Let V(s) represent the updated policy parameters of the actor network, α represent the learning rate of the actor network, θ represent the current policy parameters of the actor network, γ represent the discount factor, r represent the reward value, and V(s) represent the policy parameters of the actor network. ′ ) represents the state s after the action is performed. ′ The value of the current state is V(s), and the policy π(a|s,θ) represents the probability distribution of performing action a given the current state s and the policy parameter θ. Let π(a|s,θ) represent the gradient of the policy π with respect to θ, and let action a represent selecting a detection case.

[0054] Preferably, the calculation formula for updating the critic network by the iterative update module is as follows:

[0055] V new (s)←V(s)+β[r+γV(s ′ )-V(s)]

[0056] In the formula, V new V(s) represents the updated value of the critic network with respect to state s, β represents the learning rate of the critic network, γ represents the discount factor, r represents the reward value, and V(s) ′ ) represents the state s after the action is performed. ′ The value of V(s) represents the value of the current state s, and the action to perform represents selecting a test case.

[0057] Furthermore, this application also provides a computing device, comprising: at least one processor and a memory;

[0058] The memory is used to store one or more programs;

[0059] When the one or more programs are executed by the one or more processors, a method for arranging and combining test cases for a power load management system as described above is implemented.

[0060] In another aspect, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the above-described method for arranging and combining test cases in a power load management system.

[0061] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0062] This invention provides a method and system for arranging and combining test cases in a power load management system. The method includes: initializing the environment and the actor-commentator model based on the various test cases of the power load management system; using the actor-commentator model to select test cases from unselected test cases based on the current strategy and state, and iteratively updating the actor-commentator model after each selection until all test cases are selected, obtaining the optimal arrangement and combination of test cases as the test path; wherein, the actor network of the actor-commentator model is used to select test cases based on the current strategy and state, and the commentator network of the actor-commentator model is used to evaluate the benefits based on the selected test cases, with the benefits corresponding to the effectiveness or quality of the test; this invention utilizes the reinforcement learning capability of the actor-commentator model to automatically select the optimal arrangement and combination of test cases, greatly improving the optimization efficiency of the test path, reducing manual intervention, and avoiding the limitations of relying on experience judgment in traditional testing methods. Simultaneously, through the iterative optimization process of the actor-commentator model, the test path can be gradually optimized, making the correlation between test cases stronger, the test effect better, and reducing the execution of redundant test cases, shortening the time required for the entire testing process. This helps establish a normalized on-site testing mechanism and system in various regions, ensuring the safe and stable operation of the load management system. Attached Figure Description

[0063] Figure 1 A flowchart of a method for arranging and combining test cases in a power load management system provided by the present invention;

[0064] Figure 2 A flowchart illustrating a specific example of a method for arranging and combining detection test cases in a power load management system provided by the present invention;

[0065] Figure 3 This invention provides a schematic diagram of the system structure for arranging and combining detection test cases in a power load management system.

[0066] Figure 4 This is a schematic diagram of an electronic device structure provided by the present invention. Detailed Implementation

[0067] This invention proposes a method and system for arranging and combining detection test cases in a power load management system. It aims to address the inefficiencies and lack of accuracy in existing power load management detection processes. By introducing an Actor-Critic reinforcement learning algorithm, it provides a method for arranging and combining detection test cases in a power load management system, optimizing the detection process. Its main objective is to improve the automation level of detection, enhance response speed and accuracy, and thus ensure the stability and security of power load management by intelligently selecting the most relevant test cases to the current detection status.

[0068] The overall concept of this invention is:

[0069] 1. Build a test case library:

[0070] Each test case is represented as an n-dimensional vector, and the test case library is constructed into a multi-dimensional representation through vectorization. Each test case is vectorized based on parameters such as the device's function, status, and testing purpose, for subsequent algorithm selection.

[0071] 2. Initialize the Actor-Critic model:

[0072] The Actor (i.e., the actor network) generates detection cases and selects actions based on the current strategy, while the Critic (i.e., the critic network) evaluates the expected benefits of the current path and calculates the immediate reward.

[0073] 3. Path optimization:

[0074] The Actor selects the optimal test case based on the current strategy, generates an action, and the Critic evaluates the value of the current state and provides a reward. Then, the gradient of the improved strategy is calculated using the Advantage function. This process iterates until all test cases have been selected.

[0075] 4. Repeated iterations:

[0076] Through continuous iteration of the Actor-Critic algorithm, the process of permuting and combining test cases will be constantly adjusted, and the parameters of the Actor-Critic network will be continuously optimized until the optimal test path is formed, thus achieving the best permutation and combination of test cases.

[0077] Compared to traditional static detection processes, this invention uses reinforcement learning to dynamically select the optimal detection path, enabling rapid response to changes in complex power load management environments. Utilizing the Actor-Critic algorithm, it automatically filters out the optimal permutation and combination from a large number of test cases, significantly improving the matching accuracy and efficiency of detection test cases. This method greatly reduces human intervention through reinforcement learning, making it suitable for complex and ever-changing detection scenarios and enabling efficient and automated optimization of test case permutations and combinations.

[0078] Example 1:

[0079] A method for arranging and combining test cases in a power load management system, such as... Figure 1 As shown, it includes:

[0080] Step 1: Initialize the environment and the actor / critic model according to the various test cases of the power load management system;

[0081] Step 2: Using the actor critic model, select test cases from the unselected test cases based on the current strategy and state. After each selection of test cases, iterate and update the actor critic model based on the selected test cases until all test cases are selected. The optimal permutation and combination of each test case is obtained as the test path.

[0082] In this model, the actor network of the actor critic model is used to select detection cases based on the current strategy and state, and the critic network of the actor critic model is used to evaluate the benefits based on the selected detection cases. The benefits correspond to the effectiveness or quality of the test.

[0083] Step 1 specifically includes:

[0084] Step 101: Optimize the definition of the test path, as follows: First, use one-hot encoding to define each test case (i.e., detection case) i as a vector in n-dimensional space, as shown in equation (1):

[0085] X i =(x1,x2,…x n ) T (1)

[0086] This process structures the test cases. Where X i Let x1, x2, ..., x represent the i-th test case. n Indicates test case X i It has n attributes, which specifically include the device's function, status, and detection purpose.

[0087] Secondly, a test path Q consisting of m ordered test cases is defined, i.e., the test path is constructed, as shown in equation (2):

[0088] Q = {X1, X2, ..., X} m} (2)

[0089] Finally, path evaluation is defined by introducing weight Y, which is calculated by function f, to evaluate the effectiveness or quality of the path as shown in equation (3). Formally, the optimization problem is expressed as shown in equation (4), and path evaluation is performed.

[0090] Y = f(Q) = f(X1, X2, ... X m (3)

[0091]

[0092] That is, in all X i In the set {X}, find the test path Q that maximizes f(Q).

[0093] Step 101 provides the basic input to the Actor-Critic algorithm for subsequent operations. The core of the Actor-Critic algorithm is to select the optimal action based on the environmental state to maximize the cumulative reward.

[0094] This invention achieves full automation of test path selection by constructing a test case library based on multidimensional vector representation and combining it with the dynamic optimization capabilities of reinforcement learning models.

[0095] Step 102: Initialization of the environment and model. This involves establishing the environment and the Actor-Critic model.

[0096] Specifically, environment initialization includes initializing a vector set X_group containing all test cases, initializing an empty list selected (i.e., the first list) to track selected vectors, and initializing a list remaining (i.e., the second list) containing all vectors and recording vectors that have not yet been selected. Actor-Critic model initialization itself refers to initializing the Actor network used to generate the action probability distribution and the Critic network used to evaluate the value of the current state.

[0097] Step 2 specifically includes:

[0098] Step 201: The Actor network generates an action a (selecting a test case) based on the current policy π(a|s,θ) and the current state s (one-hot encoding of selected and unselected vectors, i.e., the first list and the second list). The policy π(a|s,θ) represents the probability distribution of executing action a given the current state s and policy parameters θ.

[0099] Step 202: Feedback on the new state s after executing action a ‘ And reward r.

[0100] Among them, the new state s ‘ To update the selected list and the remaining list, the selected test cases are stored in the selected list according to the order of selection, and the selected test cases are deleted from the remaining list. The reward r is calculated based on the cumulative example between the selected vectors, as shown in Equation (5).

[0101]

[0102] In the formula, r represents the reward value, m represents the number of test cases in the test path, and X... i+1 Let X represent the (i+1)th detection case. i Let represent the i-th test case. ||·|| represents the vector distance between two test cases. The shorter the distance, the stronger the correlation between the two test cases in the test path, and the higher the efficiency of the path. Therefore, by minimizing the distance between adjacent test cases, the selection of the entire test path can be optimized. That is, the optimization problem of formulas (3) and (4) is transformed into optimizing r.

[0103] Step 203: The Critic network evaluates the value V(s) of the current state and predicts the value V(s) of the new state. ‘ ).

[0104] The value V(s) of a state is the value that the Critic network evaluates for the current detection state, representing the contribution of the selected use cases in this state to the overall detection objective. Specifically, V(s) represents the cumulative benefit of the detection path.

[0105] Step 204: The Critic network calculates the Advantage function.

[0106] A(s,a)=r+γV(s) ′ )-V(s) (6)

[0107] In the formula, A(s,a) represents the value of the advantage function after performing action a in the current state s, γ represents the discount factor used to balance immediate rewards and long-term benefits, r represents the value of the reward, and V(s) ′ ) represents the state s after performing action a. ′ The value of V(s) represents the value of the current state s, and the action a represents selecting a test case.

[0108] Step 205: Update Actor and Critic.

[0109] That is, the Actor network and the Critic network are updated according to Equation (7) and Equation (8) respectively.

[0110]

[0111] Where, θ new Here, θ represents the updated policy parameters of the Actor network, α represents the current policy parameters, α represents the learning rate of the Actor network, and the policy π(a|s,θ) represents the probability distribution of performing action a given the current state s and policy parameters θ. Let π(a|s,θ) represent the gradient of policy π(a|s,θ) with respect to θ, γ represent the discount factor, r represent the reward value, and action a represent selecting a detection case.

[0112] V new (s)←V(s)+β[r+γV(s ′ )-V(s)](8)

[0113] Among them, V new V(s) is the updated Critic network value with respect to state s. ′ ) represents the state s after the action is performed. ′ The value of V(s) represents the value of the current state s, β is the learning rate of the Critic network, γ represents the discount factor, r represents the reward value, and the action to perform represents selecting a detection case.

[0114] Step 206: Determine whether all optional test cases have been selected. If so, use the first list as the optimal permutation and combination result of each test case and the test path and end. Otherwise, jump to step 201.

[0115] The specific process described above is as follows: Figure 2 As shown.

[0116] By implementing this invention, the efficiency and accuracy of power load management and maintenance monitoring can be significantly improved, with the following specific effects:

[0117] 1. Significantly improved detection efficiency: By automatically selecting the optimal combination of detection test cases through reinforcement learning algorithms, the optimization efficiency of the detection path is greatly improved, manual intervention is reduced, and the limitations of relying on experience judgment in traditional detection methods are avoided.

[0118] 2. Automated Detection Path Selection: By constructing a test case library based on multi-dimensional vector representation and combining it with the dynamic optimization capabilities of reinforcement learning models, the test path selection is fully automated.

[0119] 3. Optimize test paths and reduce test time: Through the iterative optimization process of reinforcement learning algorithms, test paths can be gradually optimized, making the correlation between test cases stronger and the test results better. It also reduces the execution of redundant test cases, shortening the overall test process time.

[0120] Example 2:

[0121] Based on the same inventive concept, this invention also provides a power load management system for arranging and combining detection test cases, such as... Figure 3 As shown, it includes:

[0122] Initialization module and iterative update module;

[0123] The initialization module is used to initialize the environment and the actor / critic model according to the various detection cases of the power load management system.

[0124] The iterative update module is used to select detection cases from the unselected detection cases according to the current strategy and state using the actor critic model, and iteratively update the actor critic model according to the selected detection cases after each selection, until all detection cases are selected, and obtain the optimal permutation and combination result of each detection case as the test path;

[0125] The actor network of the actor critic model is used to select detection cases based on the current strategy and state, and the critic network of the actor critic model is used to evaluate the benefits based on the selected detection cases, the benefits corresponding to the effectiveness or quality of the test.

[0126] Preferably, the initialization module is specifically used for:

[0127] One-hot encoding is used to map each detection case of the power load management system to a vector in a multi-dimensional space. Each dimension of the vector represents an attribute of the detection case, and the attribute includes one or more of the following: the function, status or detection purpose of the equipment.

[0128] A first list and a second list are established and initialized as environment initialization. The initial value of the first list is an empty set, which is used to store the selected vectors. The initial value of the second list contains the vectors corresponding to each detection case, which is used to store the vectors that have not yet been selected.

[0129] Randomly initialize the actor network and critic network as the initialization of the actor-critic model.

[0130] Preferably, the iterative update module is specifically used for:

[0131] The actor network using the actor critic model selects a detection case based on the current strategy and state, wherein the strategy is the probability distribution of selecting each detection case under the current state and strategy parameters, the state includes the first list and the second list; and the strategy parameters are the parameters of the actor network.

[0132] The selected test cases are saved in the first list in the order they were selected, and the selected test cases are deleted from the second list. The reward value after selecting the test cases is calculated.

[0133] The critic network using the actor critic model evaluates the value before and after selecting detection cases, and calculates the value of the advantage function based on the value before and after selecting detection cases and the reward value.

[0134] Update the actor network and critic network based on the value of the advantage function;

[0135] Determine whether all available test cases have been selected. If so, use the first list as the optimal permutation and combination of test cases and the test path, and end the test. Otherwise, jump to the actor network using the actor critic model and select a test case based on the current strategy and state.

[0136] Preferably, the reward value in the iterative update module is calculated using the following formula:

[0137]

[0138] In the formula, r represents the reward value, m represents the number of test cases in the test path, and X... i+1 Let X represent the (i+1)th detection case. i This represents the i-th detection case.

[0139] Preferably, the formula for calculating the value of the advantage function in the iterative update module is as follows:

[0140] A(s,a)=r+γV(s) ′ )-V(s)

[0141] In the formula, A(s,a) represents the value of the advantage function after performing action a in the current state s, γ represents the discount factor, r represents the reward value, and V(s) ′ ) represents the state s after performing action a. ′ The value of V(s) represents the value of the current state s, and the action a represents selecting a test case.

[0142] Preferably, the calculation formula for updating the actor network by the iterative update module is as follows:

[0143]

[0144] In the formula, θ new Let V(s) represent the updated policy parameters of the actor network, α represent the learning rate of the actor network, θ represent the current policy parameters of the actor network, γ represent the discount factor, r represent the reward value, and V(s) represent the policy parameters of the actor network. ′) represents the state s after the action is performed. ′ The value of the current state is V(s), and the policy π(a|s,θ) represents the probability distribution of performing action a given the current state s and the policy parameter θ. Let π(a|s,θ) represent the gradient of the policy π with respect to θ, and let action a represent selecting a detection case.

[0145] Preferably, the calculation formula for updating the critic network by the iterative update module is as follows:

[0146] V new (s)←V(s)+β[r+γV(s ′ )-V(s)]

[0147] In the formula, V new V(s) represents the updated value of the critic network with respect to state s, β represents the learning rate of the critic network, γ represents the discount factor, r represents the reward value, and V(s) ′ ) represents the state s after the action is performed. ′ The value of V(s) represents the value of the current state s, and the action to perform represents selecting a test case.

[0148] Example 3

[0149] like Figure 4 As shown, the present invention also provides an electronic device, which may be a computer device, a microcontroller device, a smart mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, processor, and transceiver component are connected via a bus; the memory can be used to store executable programs, and an exemplary executable program may include instructions; the processor is used to execute the instructions stored in the memory. The memory can also be used to store data, which can be accessed and / or modified when instructions are executed.

[0150] The processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, and it is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the storage medium to implement the corresponding method flow or corresponding function, so as to implement the steps of the power load management system detection case arrangement and combination method in the above embodiment.

[0151] Example 4

[0152] Based on the same inventive concept, this invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory). This readable storage medium is a memory device within an electronic device used to store programs and data. It is understood that the storage medium here can include both built-in storage media within the electronic device and extended storage media supported by the electronic device. The storage medium provides storage space, which stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more executable programs (including program code). It should be noted that the storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. Loading and executing one or more instructions stored in the storage medium by the processor can implement the steps of the power load management system detection use case permutation and combination method in the above embodiments.

[0153] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0155] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0156] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0157] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.

Claims

1. A method for arranging and combining test cases in a power load management system, characterized in that, include: The environment and actor / critic model are initialized based on the various test cases of the power load management system. The actor critic model is used to select test cases from the unselected test cases based on the current strategy and state. After each selection of test cases, the actor critic model is iteratively updated based on the selected test cases until all test cases are selected. The optimal permutation and combination result of each test case is obtained as the test path. The actor network of the actor critic model is used to select detection cases based on the current strategy and state, and the critic network of the actor critic model is used to evaluate the benefits based on the selected detection cases, the benefits corresponding to the effectiveness or quality of the test.

2. The method for arranging and combining test cases in a power load management system as described in claim 1, characterized in that, The initialization of the environment and the actor / critic model based on the various detection cases of the power load management system includes: One-hot encoding is used to map each detection case of the power load management system to a vector in a multi-dimensional space. Each dimension of the vector represents an attribute of the detection case, and the attribute includes one or more of the following: the function, status or detection purpose of the equipment. A first list and a second list are established and initialized as environment initialization. The initial value of the first list is an empty set, which is used to store the selected vectors. The initial value of the second list contains the vectors corresponding to each detection case, which is used to store the vectors that have not yet been selected. Randomly initialize the actor network and critic network as the initialization of the actor-critic model.

3. The method for arranging and combining test cases in a power load management system as described in claim 2, characterized in that, The process involves using the actor critic model to select detection cases from the unselected detection cases based on the current strategy and state. After each selection, the actor critic model is iteratively updated based on the selected detection cases until all detection cases are selected. The optimal permutation and combination of detection cases is then obtained as the test path, including: The actor network using the actor critic model selects a detection case based on the current strategy and state, wherein the strategy is the probability distribution of selecting each detection case under the current state and strategy parameters, the state includes the first list and the second list; and the strategy parameters are the parameters of the actor network. The selected test cases are saved in the first list in the order they were selected, and the selected test cases are deleted from the second list. The reward value after selecting the test cases is calculated. The critic network using the actor critic model evaluates the value before and after selecting detection cases, and calculates the value of the advantage function based on the value before and after selecting detection cases and the reward value. Update the actor network and critic network based on the value of the advantage function; Determine whether all available test cases have been selected. If so, use the first list as the optimal permutation and combination of test cases and the test path, and end the test. Otherwise, jump to the actor network using the actor critic model and select a test case based on the current strategy and state.

4. The method for arranging and combining test cases in a power load management system as described in claim 3, characterized in that, The reward value is calculated using the following formula: In the formula, r represents the reward value, m represents the number of test cases in the test path, and X... i+1 Let X represent the (i+1)th detection case. i This represents the i-th detection case.

5. The method for arranging and combining test cases in a power load management system as described in claim 3, characterized in that, The formula for calculating the value of the dominance function is as follows: A(s,a)=r+γV(s ′ )-V(s) In the formula, A(s,a) represents the value of the advantage function after performing action a in the current state s, γ represents the discount factor, r represents the reward value, and V(s) ′ ) represents the state s after performing action a. ′ The value of V(s) represents the value of the current state s, and the action a represents selecting a test case.

6. The method for arranging and combining test cases in a power load management system as described in claim 3, characterized in that, The formula for updating the actor network is as follows: In the formula, θ new Let V(s) represent the updated policy parameters of the actor network, α represent the learning rate of the actor network, θ represent the current policy parameters of the actor network, γ represent the discount factor, r represent the reward value, and V(s) represent the policy parameters of the actor network. ′ ) represents the state s after the action is performed. ′ The value of the current state is V(s), and the policy π(a|s,θ) represents the probability distribution of performing action a given the current state s and the policy parameter θ. Let π(a|s,θ) represent the gradient of the policy π with respect to θ, and let action a represent selecting a detection case.

7. The method for arranging and combining test cases in a power load management system as described in claim 3, characterized in that, The formula for updating the critic network is as follows: 5 new (s)←V(s)+β[r+γV(s ′ )-V(s)] In the formula, V new V(s) represents the updated value of the critic network with respect to state s, β represents the learning rate of the critic network, γ represents the discount factor, r represents the reward value, and V(s) ′ ) represents the state s after the action is performed. ′ The value of V(s) represents the value of the current state s, and the action to perform represents selecting a test case.

8. A test case permutation and combination system for a power load management system, characterized in that, include: Initialization module and iterative update module; The initialization module is used to initialize the environment and the actor / critic model according to the various detection cases of the power load management system. The iterative update module is used to select detection cases from the unselected detection cases according to the current strategy and state using the actor critic model, and iteratively update the actor critic model according to the selected detection cases after each selection, until all detection cases are selected, and obtain the optimal permutation and combination result of each detection case as the test path; The actor network of the actor critic model is used to select detection cases based on the current strategy and state, and the critic network of the actor critic model is used to evaluate the benefits based on the selected detection cases, the benefits corresponding to the effectiveness or quality of the test.

9. The power load management system test case permutation and combination system as described in claim 8, characterized in that, The initialization module is specifically used for: One-hot encoding is used to map each detection case of the power load management system to a vector in a multi-dimensional space. Each dimension of the vector represents an attribute of the detection case, and the attribute includes one or more of the following: the function, status or detection purpose of the equipment. A first list and a second list are established and initialized as environment initialization. The initial value of the first list is an empty set, which is used to store the selected vectors. The initial value of the second list contains the vectors corresponding to each detection case, which is used to store the vectors that have not yet been selected. Randomly initialize the actor network and critic network as the initialization of the actor-critic model.

10. The power load management system test case permutation and combination system as described in claim 9, characterized in that, The iterative update module is specifically used for: The actor network using the actor critic model selects a detection case based on the current strategy and state, wherein the strategy is the probability distribution of selecting each detection case under the current state and strategy parameters, the state includes the first list and the second list; and the strategy parameters are the parameters of the actor network. The selected test cases are saved in the first list in the order they were selected, and the selected test cases are deleted from the second list. The reward value after selecting the test cases is calculated. The critic network using the actor critic model evaluates the value before and after selecting detection cases, and calculates the value of the advantage function based on the value before and after selecting detection cases and the reward value. Update the actor network and critic network based on the value of the advantage function; Determine whether all available test cases have been selected. If so, use the first list as the optimal permutation and combination of test cases and the test path, and end the test. Otherwise, jump to the actor network using the actor critic model and select a test case based on the current strategy and state.

11. The power load management system test case permutation and combination system as described in claim 10, characterized in that, The formula for calculating the reward value in the iterative update module is as follows: In the formula, r represents the reward value, m represents the number of test cases in the test path, and X... i+1 Let X represent the (i+1)th detection case. i This represents the i-th detection case.

12. The power load management system test case permutation and combination system as described in claim 10, characterized in that, The formula for calculating the value of the advantage function in the iterative update module is as follows: A(s,a)=r+γV(s ′ )-V(s) In the formula, A(s,a) represents the value of the advantage function after performing action a in the current state s, γ represents the discount factor, r represents the reward value, and V(s) ′ ) represents the state s after performing action a. ′ The value of V(s) represents the value of the current state s, and the action a represents selecting a test case.

13. The power load management system test case permutation and combination system as described in claim 10, characterized in that, The calculation formula for updating the actor network by the iterative update module is as follows: In the formula, θ new Let V(s) represent the updated policy parameters of the actor network, α represent the learning rate of the actor network, θ represent the current policy parameters of the actor network, γ represent the discount factor, r represent the reward value, and V(s) represent the policy parameters of the actor network. ′ ) represents the state s after the action is performed. ′ The value of the current state is V(s), and the policy π(a|s,θ) represents the probability distribution of performing action a given the current state s and the policy parameter θ. Let π(a|s,θ) represent the gradient of the policy π with respect to θ, and let action a represent selecting a detection case.

14. The power load management system test case permutation and combination system as described in claim 10, characterized in that, The calculation formula for updating the critic network by the iterative update module is as follows: 5 new (s)←V(s)+β[r+γV(s ′ )-V(s)] In the formula, V new V(s) represents the updated value of the critic network with respect to state s, β represents the learning rate of the critic network, γ represents the discount factor, r represents the reward value, and V(s) ′ ) represents the state s after the action is performed. ′ The value of V(s) represents the value of the current state s, and the action to perform represents selecting a test case.

15. A computer device, characterized in that, include: At least one processor and memory; The memory and processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, a method for arranging and combining test cases for a power load management system as described in any one of claims 1 to 7 is implemented.

16. A computer-readable storage medium, characterized in that, It contains an execution program, which, when executed, implements a method for arranging and combining test cases for a power load management system as described in any one of claims 1 to 7.

Images