Retired battery cascade utilization grouping method based on reinforcement learning and local search
By combining reinforcement learning and local search methods with a reinforcement learning environment that rewards intra-group and inter-group consistency, the agent is trained and fine-tuned through local search. This solves the problem of inconsistent performance of battery packs under dynamic operating conditions and achieves efficient and balanced grouping of battery packs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANHUI UNIV
- Filing Date
- 2026-03-05
- Publication Date
- 2026-05-12
AI Technical Summary
Existing battery grouping methods cannot effectively take into account the static and dynamic multidimensional characteristics of batteries, resulting in inconsistent performance of battery packs during charging and discharging, which affects lifespan and safety.
A reinforcement learning and local search approach is adopted. By constructing a reinforcement learning environment that includes intra-group and inter-group consistency rewards, the agent is trained by combining a masked proximal policy optimization algorithm, and fine-tuned using a single-entity swap local search algorithm to achieve battery grouping.
It improves the consistency of voltage and current response of battery packs under dynamic operating conditions, ensures balance within a single pack and balance between all packs, has strong adaptability, high computational efficiency, and meets the requirements for rapid deployment.
Smart Images
Figure CN121786505B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of battery manufacturing and management technology, and in particular relates to a grouping method for the cascade utilization of retired batteries based on reinforcement learning and local search. Background Technology
[0002] Battery pack consistency is a key factor affecting the performance, lifespan, and safety of battery modules. During the battery pack production process, if individual cells connected in series and parallel have differences in parameters such as capacity and internal resistance, it will cause the "barrel effect" during the charging and discharging process. That is, the worst-performing cell limits the capacity of the entire battery pack, and the difference will increase with the number of cycles, eventually leading to premature battery pack failure.
[0003] Currently, traditional battery sorting and grouping methods mainly include: 1. Single-parameter sorting method: This method simply divides batteries into grades based on capacity or voltage. It ignores the influence of dynamic parameters such as internal resistance and polarization characteristics, resulting in poor grouping performance. 2. Multi-parameter static sorting method: This method considers both capacity and internal resistance, using k-means clustering or genetic algorithms for grouping. However, traditional optimization algorithms are prone to getting trapped in local optima when dealing with large-scale battery data, and the computational complexity increases exponentially with the number of batteries, making it difficult to find the global optimum within a limited time. 3. Passive / active balancing technology: Although it can balance differences during operation through hardware circuits, it increases hardware costs and the complexity of the control system, and cannot fundamentally solve the problem of poor individual cell matching.
[0004] Therefore, how to comprehensively consider the multi-dimensional characteristics of batteries (capacity, ohmic internal resistance) and (polarization internal resistance, polarization capacitance) and utilize intelligent algorithms to achieve efficient and highly consistent battery grouping is a problem that urgently needs to be solved. Summary of the Invention
[0005] To address the aforementioned technical problems, this invention provides a grouping method for the tiered utilization of retired batteries based on reinforcement learning and local search, comprising the following steps:
[0006] Collect multidimensional feature parameters of each battery cell in the battery set to be grouped, and perform standardization processing to obtain battery feature data;
[0007] Based on the battery feature data, a reinforcement learning environment is constructed. The state space of the reinforcement learning environment includes mask information reflecting the battery's selectable states and battery features, and a reward function is set.
[0008] Based on the reinforcement learning environment, a masked near-end policy optimization algorithm is used to train the agent, and the initial battery grouping sequence is generated through the trained agent.
[0009] The initial battery grouping sequence is fine-tuned using a local search algorithm based on single-cell exchange to obtain the final battery grouping scheme;
[0010] Based on the final battery grouping scheme, a battery reconfiguration command is output to realize the reconfiguration of retired batteries.
[0011] Optionally, multidimensional feature parameters of each battery cell in the battery set to be grouped are collected and standardized to obtain battery feature data, including:
[0012] For each feature parameter of each battery cell, determine the maximum and minimum values based on the original parameter values of all batteries in the corresponding dimension;
[0013] Based on the maximum and minimum values, the corresponding dimension parameter values are processed using the maximum-minimum normalization method to obtain normalized parameter values, thereby obtaining battery feature data.
[0014] Optionally, the process of constructing the reward function for the reinforcement learning environment includes:
[0015] Based on the agent's action of selecting batteries, when the number of selected batteries meets the preset group size, the intra-group consistency reward of the current group is calculated and accumulated.
[0016] Calculate the inter-group consistency reward for all groups based on the status of all batteries completing grouping;
[0017] The reward function is constructed by summing the intra-group consistency reward and the inter-group consistency reward.
[0018] Optionally, the process of calculating the intra-group consistency reward for the current group includes:
[0019] Calculate the feature mean vector of the current group based on the battery feature data of all batteries in the current group;
[0020] Based on the difference between the feature mean vector and the feature vector of each battery in the group, and combined with the preset divergence feature weights within the group, the weighted Euclidean distance divergence within the current group is calculated.
[0021] The intra-group consistency reward is calculated based on the intra-group weighted Euclidean distance divergence and the preset intra-group consistency weight coefficient.
[0022] Optionally, the process of calculating the inter-group consistency reward across all groups includes:
[0023] Calculate the global center vector based on the feature mean vectors of all groups;
[0024] Based on the difference between the mean vector of each group of features and the global center vector, and combined with the preset inter-group divergence feature weights, the weighted Euclidean distance divergence between groups is calculated.
[0025] The inter-group consistency reward is calculated based on the weighted Euclidean distance divergence between groups and the preset inter-group consistency weight coefficient.
[0026] Optionally, the process of training the agent using a masked proximal policy optimization algorithm based on the reinforcement learning environment includes:
[0027] The current state, including battery features and action mask, is obtained based on the reinforcement learning environment.
[0028] Based on the current state, the agent's policy network outputs the action probability distribution;
[0029] Based on the action probability distribution and the action mask, the index of the next battery is sampled and selected as the action to be executed;
[0030] Based on the executed action, the reward signal and the updated state are obtained from the reinforcement learning loop, and this interaction process is repeated to collect training data.
[0031] Based on the collected training data, the agent's policy network and value network parameters are updated through a proximal policy optimization algorithm to learn battery grouping policies.
[0032] Optionally, during the training of the agent, the reward signal is given using a phased accumulation mechanism, specifically as follows:
[0033] Based on the action of the intelligent agent in selecting batteries, determine whether the current group has been filled with the preset number;
[0034] If the current group is full, calculate the consistency reward within the group based on the characteristic data of the batteries in the current group, and immediately give the agent the corresponding cumulative consistency reward within the group.
[0035] Based on the determination that all batteries have been allocated, if all groups are completed, then the inter-group consistency reward between all groups is calculated and given to the agent.
[0036] Optionally, the process of training the agent also includes intelligent convergence detection, specifically:
[0037] Calculate the average and standard deviation within the sliding time window based on the historical reward values recorded during the training cycle;
[0038] Calculate the average improvement in rewards based on the average value of the current window and the average value of the previous window;
[0039] Based on whether the standard deviation is less than the corresponding threshold and whether the absolute value of the average improvement is less than the corresponding threshold, if this state is maintained continuously for a preset number of cycles, then the training is determined to have converged and training is stopped.
[0040] Optionally, the process of fine-tuning the initial battery grouping sequence using a local search algorithm based on single-unit exchange to obtain the final battery grouping scheme includes:
[0041] Based on the initial battery grouping sequence, the included multiple battery groups are determined;
[0042] Based on a defined number of battery groups, iterate through the battery cell pairs between different battery groups;
[0043] For each pair of individual cells encountered, try swapping them to obtain a new grouping result, and calculate the objective function score after the swap;
[0044] If the objective function score after the swap is higher than the objective function score before the swap, the swap operation is retained to update the grouping scheme; if the objective function score after the swap is lower than or equal to the objective function score before the swap, the swap operation is canceled.
[0045] Repeat the process of traversing, trying swaps, and evaluating comparisons until the preset iteration termination condition is met, and output the optimized final battery grouping scheme.
[0046] On the other hand, the present invention also provides an electronic device including a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the processor implements the method when executing the computing program.
[0047] Compared with the prior art, the present invention has the following advantages and technical effects:
[0048] This invention overcomes the limitations of traditional methods with a single parameter dimension by integrating and standardizing the collection of multi-dimensional static and dynamic characteristic parameters such as battery capacity, ohmic internal resistance, polarization internal resistance and polarization capacitance. It achieves a more comprehensive characterization of battery performance, thereby improving the consistency of voltage and current response of the grouped battery pack under dynamic operating conditions.
[0049] This invention is based on a reinforcement learning environment that includes both intra-group and inter-group consistency rewards, and a near-end policy optimization algorithm with a masking mechanism for sequence decision training. This enables the model to comprehensively consider the long-term planning of dispersion within a single group and balance among multiple groups, thereby finding a better battery grouping scheme in the global scope, ensuring consistency within a single group while taking into account the balance among all groups.
[0050] This invention is based on a flexibly configurable feature weight matrix and reward function coefficients, which enables the core group consistency evaluation criteria to be quickly adjusted according to actual application scenarios, thereby enhancing the adaptability of the method and making it applicable to different types of retired battery reorganization needs.
[0051] This invention is based on an intelligent convergence detection mechanism introduced during training and based on sliding window reward analysis. It can automatically determine the stable state of model learning and terminate training, thereby avoiding invalid calculations, significantly improving the efficiency of algorithm operation, and meeting the requirements of rapid deployment on production lines. Attached Figure Description
[0052] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:
[0053] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;
[0054] Figure 2 This is a block diagram illustrating the data interaction and masking process between the environment domain and the intelligent agent domain in an embodiment of the present invention.
[0055] Figure 3 This is a logic block diagram of a dual-loop local search and single-unit exchange fine-tuning strategy for group optimization according to an embodiment of the present invention.
[0056] Figure 4 This is a schematic diagram of the battery grouping process and discrete reward accumulation logic in an embodiment of the present invention;
[0057] Figure 5 This is a comparison diagram of the voltage consistency of two schemes under the dynamic operating conditions of UDDS in this embodiment of the invention; wherein, (a) is a schematic diagram of random grouping, and (b) is a schematic diagram of algorithm optimization grouping in this embodiment. Detailed Implementation
[0058] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0059] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0060] Example 1
[0061] like Figure 1 As shown, this embodiment provides a grouping method for the tiered utilization of retired batteries based on reinforcement learning and local search, including the following steps:
[0062] Collect multidimensional feature parameters of each battery cell in the battery set to be grouped, and perform standardization processing to obtain battery feature data;
[0063] Based on the battery feature data, a reinforcement learning environment is constructed. The state space of the reinforcement learning environment includes mask information reflecting the battery's selectable states and battery features, and a reward function is set.
[0064] Based on the reinforcement learning environment, a masked near-end policy optimization algorithm is used to train the agent, and the initial battery grouping sequence is generated through the trained agent.
[0065] The initial battery grouping sequence is fine-tuned using a local search algorithm based on single-cell exchange to obtain the final battery grouping scheme;
[0066] Based on the final battery grouping scheme, a battery reconfiguration command is output to realize the reconfiguration of retired batteries.
[0067] The feasible process of acquiring multidimensional feature data includes: acquiring multidimensional characteristic parameters of each battery cell in the battery set to be grouped. The battery set to be grouped includes... Individual battery cells. The collected characteristic parameters include at least four indicators: discharge capacity (Capacity), ohmic internal resistance (Ro), polarization internal resistance (Rp), and polarization capacitance (Cp). The original feature matrix is constructed. , dimension .
[0068] Feasible method: Collect multidimensional feature parameters of each cell in the battery set to be grouped, and perform standardization processing to obtain battery feature data, including:
[0069] For each feature parameter of each battery cell, the maximum and minimum values are determined based on the original parameter values of all batteries in the corresponding dimension. Based on the maximum and minimum values, the parameter values in the corresponding dimension are processed using the max-min normalization method to obtain the normalized parameter values, thereby obtaining the battery feature data.
[0070] As a specific implementation method, the data standardization process includes:
[0071] To eliminate the differences in dimensions and orders of magnitude between different feature parameters, the obtained original feature matrix is... Data standardization is performed. This embodiment uses the Min-Max Normalization method to obtain the normalized feature matrix. The calculation formula is as follows:
[0072] ;
[0073] in, Representing all batteries in the first The original parameter column vector of dimension, The normalized vector, To prevent tiny constants with a denominator of zero.
[0074] The feasible process for constructing the reward function for a reinforcement learning environment includes:
[0075] Based on the agent's action of selecting batteries, when the number of selected batteries meets the preset group size, the intra-group consistency reward of the current group is calculated and accumulated; based on the grouping status of all batteries, the inter-group consistency reward of all groups is calculated; the intra-group consistency reward and the inter-group consistency reward are summed to complete the construction of the reward function.
[0076] Furthermore, the process of calculating the intra-group consistency reward for the current group includes:
[0077] Based on the battery feature data of all batteries in the current group, calculate the feature mean vector of the current group; based on the difference between the feature mean vector and the feature vector of each battery in the group, and combined with the preset intra-group divergence feature weight, calculate the intra-group weighted Euclidean distance divergence of the current group; based on the intra-group weighted Euclidean distance divergence and the preset intra-group consistency weight coefficient, calculate the intra-group consistency reward of the current group.
[0078] Furthermore, the process of calculating the inter-group consistency reward across all groups includes:
[0079] Calculate the global center vector based on the feature mean vectors of all groups; calculate the weighted Euclidean distance divergence between groups based on the difference between the feature mean vectors of each group and the global center vector, combined with the preset inter-group divergence feature weights; calculate the inter-group consistency reward based on the weighted Euclidean distance divergence between groups and the preset inter-group consistency weight coefficients.
[0080] As a specific implementation method, the process of constructing a reinforcement learning environment based on a masking mechanism includes:
[0081] Data interaction and masking processing between the environment domain and the agent domain, such as Figure 2 As shown. This embodiment models the battery grouping problem as a Markov Decision Process (MDP): 1. State: Consists of normalized feature data of all current batteries and action mask vectors (Masks). The mask is a vector of length... 1. A binary vector, where logic 1 indicates that the battery has not yet been selected (optional), and logic 0 indicates that the battery has been selected (not optional). 2. Action: Defined as the index by which the agent selects the next battery from the currently available batteries. 3. Reward Function: Construct a composite reward function that includes "intra-group consistency" and "inter-group balance".
[0082] (1) Within-group weighted Euclidean distance divergence :
[0083] When the number of selected batteries meets the group size M, calculate the feature mean vector of the current group k. :
[0084] ;
[0085] Calculate the weighted divergence:
[0086] ;
[0087] in, Let be the normalized feature vector of the i-th battery. The feature weights are a diagonal matrix. This is the set of battery indices for the current group. The required number of batteries for each group.
[0088] (2) Between-group weighted Euclidean distance divergence :
[0089] Calculate the global center of the mean vector of all battery packs. :
[0090] ;
[0091] Calculate the weighted Euclidean distance divergence between groups:
[0092] ;
[0093] in, For the first The characteristic mean vector of the group The global center vector for all groups. This represents the number of groups after grouping.
[0094] (3) Total reward function:
[0095] ;
[0096] in, The formula for calculating the average intra-group consistency reward is as follows:
[0097] ;
[0098] To achieve balanced rewards across groups, the calculation formula is as follows:
[0099] ;
[0100] in, This is the weighting coefficient for intragroup consistency. This is the inter-group consistency weighting coefficient. Let be the weighted Euclidean distance divergence within the k-th battery pack. The weighted Euclidean distance divergence between groups.
[0101] This embodiment uses a weighted Euclidean distance to construct the reward function. This is achieved by introducing a feature weight matrix. The influence weight of each parameter (such as the importance of capacity being higher than that of internal resistance) on the grouping results can be flexibly adjusted according to the actual application scenario, so that the grouping results are more in line with engineering requirements.
[0102] The feasible process of training an agent using a masked proximal policy optimization algorithm based on the aforementioned reinforcement learning environment includes:
[0103] The current state, including battery features and an action mask, is obtained based on the reinforcement learning environment. Based on the current state, an action probability distribution is output through the agent's policy network. Based on the action probability distribution and the action mask, the index of the next battery is sampled and selected as the action to be executed. Based on the executed action, a reward signal and an updated state are obtained from the reinforcement learning loop, and this interaction process is repeated to collect training data. Based on the collected training data, the parameters of the agent's policy network and value network are updated through a proximal policy optimization algorithm to learn the battery grouping policy.
[0104] This embodiment employs an action masking mechanism, specifically including: generating a Boolean mask vector with a length equal to the total number of batteries; marking the selected battery indices as invalid (False) and the unselected ones as valid (True) in the mask vector; when the agent outputs the action probability distribution, forcing the probability of invalid actions to zero, ensuring that the agent only selects unused batteries, avoiding duplicate selections, and guaranteeing the physical feasibility of the grouping scheme.
[0105] Furthermore, during the training of the agent, the reward signal is given using a phased accumulation mechanism, specifically as follows:
[0106] Based on the agent's action of selecting batteries, determine whether the current group has been filled with the preset number; if the current group has been filled, calculate the intra-group consistency reward based on the characteristic data of the batteries in the current group, and immediately give the agent the corresponding intra-group consistency reward; based on whether all batteries have been allocated, if all groups have been completed, calculate the inter-group consistency reward between all groups and give it to the agent.
[0107] Furthermore, the process of training the agent also includes intelligent convergence detection, specifically:
[0108] Based on the historical reward values recorded during the training cycle, calculate the average and standard deviation within the sliding time window; based on the average value of the current window and the average value of the previous window, calculate the average improvement of the reward; based on whether the standard deviation is less than the corresponding threshold and whether the absolute value of the average improvement is less than the corresponding threshold, if this state is maintained continuously for a preset number of cycles, then the training is determined to have converged and training is stopped.
[0109] As a specific implementation method, the process of intelligent agent interaction training and convergence detection includes:
[0110] This step integrates the training process and intelligent convergence detection mechanism of the Maskable PPO algorithm, such as... Figure 4 As shown, this process demonstrates the logic of batteries moving from the sorting pile into the grouping slot and triggering a tiered reward accumulation, specifically including the following sub-steps:
[0111] Step 1: Environment Initialization and State Observation. The agent observes the current environmental state. This state contains the feature matrix and the current action mask. Initially, the mask is all 1s.
[0112] Step 2: Action Masking. The agent's policy network outputs the raw probabilities (Logits) of all possible actions. A masking layer is then introduced to filter these probabilities. The probability of the action marked as 0 (already used) is forcibly set to 0. Or 0, to ensure sampling is only performed from "unused" batteries.
[0113] Step 3: Action Selection. Based on the probability distribution after masking, the agent samples and selects the index of the next battery as the action to be executed at the current time step. .
[0114] Step 4: Status Update and Reward Feedback. The environment receives the execution action for the current time step. Set the mask at the corresponding position to 0. If the current selection makes a certain battery group (Group) (Satisfies the number of groups) If so, the intra-group consistency reward for that group is immediately calculated and fed back to the agent.
[0115] Step 5: Round Settlement and Looping. Repeat Steps 2 to 4 until all batteries are allocated (masks are all 0). At this point, calculate the inter-group balance reward and total reward, and update the policy network parameters using the PPO algorithm.
[0116] Step 6: Convergence check. Maintain a sliding window (size...). Real-time calculation of the average reward within the window and standard deviation If the conditions are met. And the reward increase Continuous maintenance If the model converges after a certain number of counting cycles, training is stopped and the optimal strategy is saved; otherwise, Step 1 is repeated.
[0117] This embodiment incorporates intelligent convergence detection logic. Compared to a fixed number of training steps, this method can automatically determine the convergence point based on the stability of the reward curve, significantly reducing computation time while ensuring group quality.
[0118] An feasible process for generating the initial battery grouping sequence includes:
[0119] After the training model has converged, switch it to prediction mode and input the battery data to be grouped. Based on the learned optimal policy and under the constraint of action mask, the model sequentially outputs the selection sequence of all batteries, generating an initial battery grouping sequence that satisfies the constraints.
[0120] The feasible process of fine-tuning the initial battery grouping sequence using a local search algorithm based on single-cell exchange to obtain the final battery grouping scheme includes:
[0121] Based on the initial battery grouping sequence, multiple battery groups are determined. Based on the determined multiple battery groups, battery cell pairs between different battery groups are traversed. For each battery cell pair traversed, an attempt is made to swap them to obtain a new grouping result, and the objective function score after the swap is calculated. If the objective function score after the swap is higher than the objective function score before the swap, the swap operation is retained to update the grouping scheme. If the objective function score after the swap is lower than or equal to the objective function score before the swap, the swap operation is canceled. The process of traversing, attempting swaps, and evaluating and comparing is repeated until the preset iteration termination condition is reached, and the optimized final battery grouping scheme is output.
[0122] As a specific implementation method, the process of local search fine-tuning includes:
[0123] To further escape potential local optima, this embodiment uses a local search algorithm to fine-tune the obtained initial sequence. For example... Figure 3 As shown, this step iterates through the individuals within different groups using a double loop, executing the logic of "attempt swap - evaluate - confirm / cancel". The specific steps are as follows:
[0124] Step 1: Determine the components included in the initial grouping scheme One battery pack, marked as .
[0125] Step 2: As Figure 3As shown, traverse different battery packs (e.g. and Try to swap the individual battery cells between them.
[0126] Step 3: For each attempt to swap, use the reward function logic in step S103 to calculate the objective function score after the swap.
[0127] Step 4: Determine the score change. If the score after the swap is higher than the current score, retain the swap operation and update the grouping scheme; if the score after the swap is lower than or equal to the current score, cancel the swap operation.
[0128] Step 5: Repeat Steps 2 to 4 above until all possible combinations of exchanges have been traversed or the objective function score no longer improves in several consecutive iterations.
[0129] Step 6: After the search is complete, output the finely adjusted grouped sequence.
[0130] Step S107: Output the final battery solution. Based on the final grouping index output in step S106, output the corresponding battery physical recombination instruction to complete the group matching of batteries for secondary use.
[0131] On the other hand, this embodiment also provides an electronic device, including a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the processor implements the method when executing the computing program.
[0132] Example 2
[0133] The technical solution of Embodiment 1 will be further illustrated below with a specific application example. This embodiment selects 12 retired 18650 lithium-ion battery cells as experimental subjects. The experimental goal is to reassemble these 12 cells into a 3-parallel 4-series (3P4S) battery module structure, i.e., divided into 4 groups, each containing 3 battery cells. The experiment first uses high-precision testing equipment to collect the original feature data of these 12 batteries before standardization, as shown in Table 1. Regarding the algorithm parameter configuration, considering the balance of the impact of capacity, ohmic internal resistance, polarization internal resistance, and polarization capacitance on battery pack performance, this embodiment uses a feature weight vector... Set as This means that all four indicators are equally important in the difference calculation; at the same time, in order to balance the consistency within a single module and the voltage plateau balance among the four series-connected modules, the intra-group consistency weighting coefficient in the reward function is adjusted. Weighting coefficients for intergroup balance All are set to 0.5.
[0134] Table 1 below shows examples of characteristic parameters for a single battery cell.
[0135] Table 1
[0136]
[0137] To verify the grouping effect, a randomized group (control group) and the algorithm group of this embodiment (experimental group) were set up for comparison. To simulate the actual operation of electric vehicles in complex urban road conditions, the experiment loaded the UDDS (Urban Dynamometer Driving Schedule) urban road cycle condition, and the current sequence of the standard condition was scaled down to the current range adapted to the battery module of this experiment for discharge testing.
[0138] Figure 5 The diagram shows the terminal voltage variation curves of the four series-connected modules under two different schemes during the dynamic discharge process of UDDS. Figure 5 (a) shows the voltage distribution diagram of the random group. Figure 5 (b) shows the voltage distribution diagram optimized using the algorithm of this embodiment.
[0139] As shown in the figure, under random grouping, due to parameter mismatch between modules, the voltage curves of each module exhibit a significant divergence trend under the frequent current fluctuations in UDDS conditions. The voltage drop rate of the lower-performing modules is significantly faster than that of the others, reaching the cutoff voltage first and causing premature termination of battery discharge. However, after using the algorithm of this embodiment, the voltage drop curves of the four modules highly overlap. Even when facing the complex and variable current pulses in UDDS conditions, the dynamic response of each module maintains a very high degree of consistency, and all modules reach the discharge cutoff condition almost synchronously. This indicates that the grouping method effectively suppresses the "weakest link" effect under dynamic conditions, significantly improving the battery pack's group utilization and cycle stability.
[0140] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A grouping method for the tiered utilization of decommissioned batteries based on reinforcement learning and local search, characterized in that, Includes the following steps: Collect multidimensional feature parameters of each battery cell in the battery set to be grouped, and perform standardization processing to obtain battery feature data; Based on the battery feature data, a reinforcement learning environment is constructed. The state space of the reinforcement learning environment includes mask information reflecting the battery's selectable states and battery features, and a reward function is set. Based on the reinforcement learning environment, a masked near-end policy optimization algorithm is used to train the agent, and the initial battery grouping sequence is generated through the trained agent. The initial battery grouping sequence is fine-tuned using a local search algorithm based on single-cell exchange to obtain the final battery grouping scheme; Based on the final battery grouping scheme, a battery reconfiguration command is output to realize the reconfiguration of retired batteries.
2. The method according to claim 1, characterized in that, Multidimensional feature parameters of each cell in the battery set to be grouped are collected and standardized to obtain battery feature data, including: For each feature parameter of each battery cell, determine the maximum and minimum values based on the original parameter values of all batteries in the corresponding dimension; Based on the maximum and minimum values, the corresponding dimension parameter values are processed using the maximum-minimum normalization method to obtain normalized parameter values, thereby obtaining battery feature data.
3. The method according to claim 1, characterized in that, The process of constructing the reward function for a reinforcement learning environment includes: Based on the agent's action of selecting batteries, when the number of selected batteries meets the preset group size, the intra-group consistency reward of the current group is calculated and accumulated. Calculate the inter-group consistency reward for all groups based on the status of all batteries completing grouping; The reward function is constructed by summing the intra-group consistency reward and the inter-group consistency reward.
4. The method according to claim 3, characterized in that, The process of calculating the intra-group consistency reward for the current group includes: Calculate the feature mean vector of the current group based on the battery feature data of all batteries in the current group; Based on the difference between the feature mean vector and the feature vector of each battery in the group, and combined with the preset divergence feature weights within the group, the weighted Euclidean distance divergence within the current group is calculated. The intra-group consistency reward is calculated based on the intra-group weighted Euclidean distance divergence and the preset intra-group consistency weight coefficient.
5. The method according to claim 4, characterized in that, The process of calculating the inter-group consistency reward across all groups includes: Calculate the global center vector based on the feature mean vectors of all groups; Based on the difference between the mean vector of each group of features and the global center vector, and combined with the preset inter-group divergence feature weights, the weighted Euclidean distance divergence between groups is calculated. The inter-group consistency reward is calculated based on the weighted Euclidean distance divergence between groups and the preset inter-group consistency weight coefficient.
6. The method according to claim 1, characterized in that, Based on the aforementioned reinforcement learning environment, the process of training the agent using a masked near-end policy optimization algorithm includes: The current state, including battery features and action mask, is obtained based on the reinforcement learning environment. Based on the current state, the agent's policy network outputs the action probability distribution; Based on the action probability distribution and the action mask, the index of the next battery is sampled and selected as the action to be executed; Based on the executed action, the reward signal and the updated state are obtained from the reinforcement learning loop, and this interaction process is repeated to collect training data. Based on the collected training data, the agent's policy network and value network parameters are updated through a proximal policy optimization algorithm to learn battery grouping policies.
7. The method according to claim 6, characterized in that, During the training of the agent, the reward signal is given using a phased accumulation mechanism, specifically as follows: Based on the action of the intelligent agent in selecting batteries, determine whether the current group has been filled with the preset number; If the current group is full, calculate the consistency reward within the group based on the characteristic data of the batteries in the current group, and immediately give the agent the corresponding cumulative consistency reward within the group. Based on the determination that all batteries have been allocated, if all groups are completed, then the inter-group consistency reward between all groups is calculated and given to the agent.
8. The method according to claim 6, characterized in that, The process of training the agent also includes intelligent convergence detection, specifically: Calculate the average and standard deviation within the sliding time window based on the historical reward values recorded during the training cycle; Calculate the average improvement in rewards based on the average value of the current window and the average value of the previous window; Based on whether the standard deviation is less than the corresponding threshold and whether the absolute value of the average improvement is less than the corresponding threshold, if this state is maintained continuously for a preset number of cycles, then the training is determined to have converged and training is stopped.
9. The method according to claim 1, characterized in that, The process of fine-tuning the initial battery grouping sequence using a local search algorithm based on single-cell exchange to obtain the final battery grouping scheme includes: Based on the initial battery grouping sequence, the included multiple battery groups are determined; Based on a defined number of battery groups, iterate through the battery cell pairs between different battery groups; For each pair of individual cells encountered, try swapping them to obtain a new grouping result, and calculate the objective function score after the swap; If the objective function score after the swap is higher than the objective function score before the swap, the swap operation is retained to update the grouping scheme; if the objective function score after the swap is lower than or equal to the objective function score before the swap, the swap operation is canceled. Repeat the process of traversing, trying swaps, and evaluating comparisons until the preset iteration termination condition is met, and output the optimized final battery grouping scheme.
10. An electronic device comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, characterized in that, When the processor executes the computing program, it implements the method of any one of claims 1-9.