E-Bike Battery Health Monitoring System
By introducing critic neural network and deep reinforcement learning, the problem of overestimation and instability in the model evaluation action value function in the battery health monitoring system of electric bicycles is solved, the prediction accuracy and fitness are improved, and the robustness of the model is achieved and Adaptability to diverse datasets.
Patent Information
- Application Number
- CN202410525075.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-29
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2044-04-29
AI Technical Summary
In the battery health monitoring system of electric bicycles, there is overestimation of the action value function of the model evaluation, instability in training, high impact on parameter updates, low prediction accuracy, and low adaptability to diverse data sets.
Two reviewer neural networks were introduced, using the minimum Q value to avoid overestimation, delayed update method to stabilize Q value, random noise improves model robustness, and design reward functions based on preprocessed dimensional information and historical data through deep reinforcement learning.
Improves the robustness and prediction accuracy of the model, enhances the fitness to diverse data sets, and avoids the influence of error updates and noise.
Smart Images
Figure CN118348440B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of battery monitoring, and in particular to an electric bicycle battery health monitoring system. Background Art
[0002] The electric bicycle battery health monitoring system is a system used to monitor and evaluate the battery status of electric bicycles. Its main purpose is to provide key data and information to ensure the safety, stability and performance of the battery by monitoring the health of the battery in real time. However, the general electric vehicle battery health monitoring model has over-estimation when evaluating the value function of the action; the training is unstable due to incorrect updates during model training; the model has too much influence on noise when updating parameters; the general electric vehicle battery health monitoring model has low prediction accuracy and low adaptability to various data sets. Summary of the invention
[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an electric bicycle battery health monitoring system, which is aimed at the problems that the value function of the action of the general electric vehicle battery health monitoring model is over-estimated; the training is unstable due to erroneous updates during model training; and the model has too much influence on noise during parameter update. This scheme introduces two critic neural networks, selects the smallest Q value to avoid consistent over-estimation; adopts a delayed update method to update the Q value; avoids erroneous updates and stabilizes the Q value; introduces random noise into the parameter update of the critic neural network to improve the robustness of the model; in view of the problems that the general electric vehicle battery health monitoring model has low prediction accuracy and low adaptability to diverse data sets, this scheme introduces deep reinforcement learning into the battery health monitoring model, based on the preprocessed dimensional information as state information; uses data labels as action space; and designs a reward function based on historical data; improves the accuracy of battery health prediction by learning the pattern of historical data.
[0004] The technical solution adopted by the present invention is as follows: the electric bicycle battery health monitoring system provided by the present invention includes a data acquisition module, a data preprocessing module, an electric bicycle battery health monitoring model establishment module and an electric vehicle battery health monitoring module;
[0005] The data acquisition module collects the health history data of the electric bicycle battery;
[0006] The data preprocessing module performs data cleaning, data conversion, standardization processing and data set division on the collected data;
[0007] The electric bicycle battery health monitoring model establishment module introduces an actor neural network and two critic neural networks; defines a target Q value, delays the update of the target Q value, and introduces random noise into the update of the target action; so as to complete the establishment of the electric bicycle battery health monitoring model;
[0008] The electric vehicle battery health monitoring module is based on the established electric bicycle battery health monitoring model; the electric vehicle battery health monitoring is realized based on the model output through the real-time collected data set.
[0009] Furthermore, in the data acquisition module, the electric bicycle battery health history data includes battery information, battery status parameters, environmental information, vehicle information, control information and battery health status.
[0010] Furthermore, in the data preprocessing module, the data cleaning is to process missing values and duplicate values; the data conversion is to convert the data into vector form; the standardization is to standardize the data based on the maximum and minimum normalization method; the data set division is to divide the data into a training set and a test set; and the battery health status is used as a data label.
[0011] Furthermore, the electric bicycle battery health monitoring model establishment module specifically includes the following contents:
[0012] Architecture design, introducing actor neural network and two critic neural networks. The actor neural network is responsible for learning action strategies, and the training goal is to maximize the action value estimated by the critic neural network. The critic neural network is responsible for evaluating the value of actions; it is used to predict the action value rewards obtained by taking different actions in different states; the training goal is to update the network parameters by minimizing the mean square error between the current action value and the target action value.
[0013] Preset, use the preprocessed dimension information as state information; use the data label as action space; design the reward function based on historical data;
[0014] Define the target Q value; use the critic neural network to estimate the action value reward after taking different actions in the next state; the formula used is as follows:
[0015] ;
[0016] Where y is the target Q value of the actor neural network; r t is the immediate reward of the actor neural network at time t; r is the discount factor; is the next state that each critic neural network chooses to predict take The maximum action value reward after the action, and then choose the smallest action value reward between the two;
[0017] Define the loss function, the formula used is as follows:
[0018] ;
[0019] In the formula, is the loss function of the i-th critic neural network; is the network parameter of the i-th critic neural network; E π is the expected value under strategy π; is the prediction value of the i-th critic neural network, s t and a t are the current state and action taken by the critic neural network, respectively;
[0020] Update the gradient and use the delayed update method to update the target Q value. After updating the critic neural network K times, update the actor neural network once. The formula used to update the gradient is as follows:
[0021] ;
[0022] In the formula, is the loss gradient of the i-th critic neural network; y t is the target Q value at time t; is the gradient of the critic neural network output with respect to the network parameters; is the gradient of the actor neural network’s loss function with respect to the parameter θ; is the strategy generated by the actor neural network, i.e., the probability distribution of selecting an action given a state s, and a is the action; is the gradient of the actor’s neural network policy with respect to the parameter θ;
[0023] Correct the parameter update, and set the parameters of the critic neural network and the actor neural network after gradient update to and , the formula used is as follows:
[0024] ;
[0025] In the formula, are the parameters of the modified critic neural network; are the parameters of the modified actor neural network; and They are the parameters of the critic neural network before gradient update and the parameters of the actor neural network before gradient update; is the smoothing factor;
[0026] Update the target action of the actor neural network and introduce random noise into the update of the target action. The formula used is as follows:
[0027] ;
[0028] In the formula, is the updated target action, is the action generated by the actor's neural network, is the next state; is the shear noise that follows a normal distribution; is a shear operation; is the standard deviation; c is the range parameter;
[0029] Model judgment: The electric bicycle battery health monitoring model is trained based on the training set, and an accuracy threshold is set in advance. When the prediction accuracy of the trained electric bicycle battery health monitoring model for the test set is higher than the accuracy threshold, the electric bicycle battery health monitoring model is established.
[0030] Furthermore, the electric vehicle battery health monitoring module is based on an established electric bicycle battery health monitoring model; it collects battery information, battery status parameters, environmental information, vehicle information and control information in real time; and it implements electric vehicle battery health monitoring based on the battery health status output by the model.
[0031] The beneficial effects achieved by the present invention using the above scheme are as follows:
[0032] (1) There is overestimation when evaluating the value function of the action of the general electric vehicle battery health monitoring model; the training is unstable due to erroneous updates during model training; there is a problem of excessive influence of the model on noise during parameter update; this scheme introduces two critic neural networks and selects the smallest Q value to avoid consistent overestimation; a delayed update method is used to update the Q value; to avoid erroneous updates and stabilize the Q value; random noise is introduced into the parameter update of the critic neural network to improve the robustness of the model.
[0033] (2) To address the problems of low prediction accuracy and low adaptability to diverse data sets in general electric vehicle battery health monitoring models, this solution introduces deep reinforcement learning into the battery health monitoring model, using preprocessed dimensional information as state information; using data labels as action space; and designing a reward function based on historical data; improving the accuracy of battery health prediction by learning the patterns of historical data. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 A schematic diagram of an electric bicycle battery health monitoring system provided by the present invention;
[0035] Figure 2 Schematic diagram of the process of building a module for the electric bicycle battery health monitoring model.
[0036] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION
[0037] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0038] In the description of the present invention, it is necessary to understand that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred system or element must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention.
[0039] Example 1, see Figure 1 , the electric bicycle battery health monitoring system provided by the present invention includes a data acquisition module, a data preprocessing module, an electric bicycle battery health monitoring model establishment module, an electric vehicle battery health monitoring module and a vulnerability mining module;
[0040] The data acquisition module collects the health history data of the electric bicycle battery; and sends the data to the data preprocessing module;
[0041] The data preprocessing module receives the data sent by the data acquisition module, cleans the collected data, performs data conversion, standardizes the data, and divides the data set; and sends the data to the electric bicycle battery health monitoring model establishment module;
[0042] The electric bicycle battery health monitoring model establishment module completes the establishment of the electric bicycle battery health monitoring model by introducing an actor neural network and two critic neural networks; and sends the data to the electric vehicle battery health monitoring module;
[0043] The electric vehicle battery health monitoring module receives data sent by the electric bicycle battery health monitoring model establishment module, and implements electric vehicle battery health monitoring based on the model output.
[0044] Example 2, see Figure 1, this embodiment is based on the above embodiment. In the data acquisition module, the electric bicycle battery health history data includes battery information, battery status parameters, environmental information, vehicle information, control information and battery health status.
[0045] Example 3, see Figure 1 This embodiment is based on the above embodiment. In the data preprocessing module, the data conversion is to convert the data into a vector form; the standardization processing is to standardize the data based on the maximum and minimum normalization method; the data set division is to divide the data into a training set and a test set; and the battery health status is used as a data label.
[0046] Example 4, see Figure 1 and Figure 2 Based on the above embodiment, this embodiment includes the following contents:
[0047] Architecture design, introducing actor neural network and two critic neural networks. The actor neural network is responsible for learning action strategies, and the training goal is to maximize the action value estimated by the critic neural network. The critic neural network is responsible for evaluating the value of actions; it is used to predict the action value rewards obtained by taking different actions in different states; the training goal is to update the network parameters by minimizing the mean square error between the current action value and the target action value.
[0048] Preset, use the preprocessed dimension information as state information; use the data label as action space; design the reward function based on historical data;
[0049] Define the target Q value; use the critic neural network to estimate the action value reward after taking different actions in the next state; the formula used is as follows:
[0050] ;
[0051] Where y is the target Q value of the actor neural network; r t is the immediate reward of the actor neural network at time t; r is the discount factor; is the next state that each critic neural network chooses to predict take The maximum action value reward after the action, and then choose the smallest action value reward between the two;
[0052] Define the loss function, the formula used is as follows:
[0053] ;
[0054] In the formula, is the loss function of the i-th critic neural network; is the network parameter of the i-th critic neural network; E π is the expected value under strategy π; is the prediction value of the i-th critic neural network, s t and a t are the current state and action taken by the critic neural network, respectively;
[0055] Update the gradient and use the delayed update method to update the target Q value. After updating the critic neural network K times, update the actor neural network once. The formula used to update the gradient is as follows:
[0056] ;
[0057] In the formula, is the loss gradient of the i-th critic neural network; y t is the target Q value at time t; is the gradient of the critic neural network output with respect to the network parameters; is the gradient of the actor neural network’s loss function with respect to the parameter θ; is the strategy generated by the actor neural network, i.e., the probability distribution of selecting an action given a state s, and a is the action; is the gradient of the actor’s neural network policy with respect to the parameter θ;
[0058] Correct the parameter update, and set the parameters of the critic neural network and the actor neural network after gradient update to and , the formula used is as follows:
[0059] ;
[0060] In the formula, are the parameters of the modified critic neural network; are the parameters of the modified actor neural network; and They are the parameters of the critic neural network before gradient update and the parameters of the actor neural network before gradient update; is the smoothing factor;
[0061] Update the target action of the actor neural network and introduce random noise into the update of the target action. The formula used is as follows:
[0062] ;
[0063] In the formula, is the updated target action, is the action generated by the actor's neural network, is the next state; is the shear noise that follows a normal distribution; is a shear operation; is the standard deviation; c is the range parameter;
[0064] Model judgment: The electric bicycle battery health monitoring model is trained based on the training set, and an accuracy threshold is set in advance. When the prediction accuracy of the trained electric bicycle battery health monitoring model for the test set is higher than the accuracy threshold, the electric bicycle battery health monitoring model is established.
[0065] By performing the above operations, there is over-estimation in the value function of the evaluation action of the general electric vehicle battery health monitoring model; the training is unstable due to erroneous updates during model training; there is a problem of excessive influence of the model on noise during parameter update; this scheme introduces two critic neural networks and selects the smallest Q value to avoid consistent over-estimation; a delayed update method is used to update the Q value; to avoid erroneous updates and stabilize the Q value; random noise is introduced into the parameter update of the critic neural network to improve the robustness of the model.
[0066] Example 5, see Figure 1 This embodiment is based on the above embodiment. The electric vehicle battery health monitoring module is based on the established electric bicycle battery health monitoring model; battery information, battery status parameters, environmental information, vehicle information and control information are collected in real time; and electric vehicle battery health monitoring is realized based on the battery health status output by the model.
[0067] By performing the above operations, this solution introduces deep reinforcement learning into the battery health monitoring model to address the problems of low prediction accuracy and low adaptability to diverse data sets in general electric vehicle battery health monitoring models. It uses preprocessed dimensional information as state information, uses data labels as action space, and designs reward functions based on historical data. It improves the accuracy of battery health prediction by learning the patterns of historical data.
[0068] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0069] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.
[0070] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.
Claims
1. The electric bicycle battery health monitoring system is characterized by: The system includes a data acquisition module, a data preprocessing module, an electric bicycle battery health monitoring model building module and an electric vehicle battery health monitoring module; The data acquisition module collects the health history data of the electric bicycle battery; and sends the data to the data preprocessing module; The data preprocessing module receives the data sent by the data acquisition module, cleans the collected data, performs data conversion, standardizes the data, and divides the data set; and sends the data to the electric bicycle battery health monitoring model establishment module; The electric bicycle battery health monitoring model building module introduces actor neural network and two critic neural networks; By defining the target Q value, delaying the update of the target Q value and introducing random noise into the update of the target action; the electric bicycle battery health monitoring model is established; and the data is sent to the electric vehicle battery health monitoring module; The electric vehicle battery health monitoring module receives data sent by the electric bicycle battery health monitoring model establishment module, based on the established electric bicycle battery health monitoring model; the electric vehicle battery health monitoring is realized based on the model output through the real-time collected data set; The electric bicycle battery health monitoring model establishment module specifically includes the following contents: Architecture design, introducing actor neural network and two critic neural networks. The actor neural network is responsible for learning action strategies, and the training goal is to maximize the action value estimated by the critic neural network. The critic neural network is responsible for evaluating the value of actions; it is used to predict the action value rewards obtained by taking different actions in different states; the training goal is to update the network parameters by minimizing the mean square error between the current action value and the target action value. Preset, use the preprocessed dimension information as state information; use the data label as action space; design the reward function based on historical data; Define the target Q value; use the critic neural network to estimate the action value reward after taking different actions in the next state; the formula used is as follows: ； Where y is the target Q value of the actor neural network; r t is the immediate reward of the actor neural network at time t; r is the discount factor; is the next state that each critic neural network chooses to predict take The maximum action value reward after the action, and then choose the smallest action value reward between the two; Define the loss function, the formula used is as follows: ； In the formula, is the loss function of the i-th critic neural network; is the network parameter of the i-th critic neural network; E π is the expected value under strategy π; is the prediction value of the i-th critic neural network, s t and a t are the current state and action taken by the critic neural network, respectively; Update the gradient and use the delayed update method to update the target Q value. After updating the critic neural network K times, update the actor neural network once. The formula used to update the gradient is as follows: ； In the formula, is the loss gradient of the i-th critic neural network; y t is the target Q value at time t; is the gradient of the critic neural network output with respect to the network parameters; is the gradient of the actor neural network’s loss function with respect to the parameter θ; is the strategy generated by the actor neural network, i.e., the probability distribution of selecting an action given a state s, and a is the action; is the gradient of the actor’s neural network policy with respect to the parameter θ; Correct the parameter update, and set the parameters of the critic neural network and the actor neural network after gradient update to and , the formula used is as follows: ； In the formula, are the parameters of the modified critic neural network; are the parameters of the modified actor neural network; and They are the parameters of the critic neural network before gradient update and the parameters of the actor neural network before gradient update; is the smoothing factor; Update the target action of the actor neural network and introduce random noise into the update of the target action. The formula used is as follows: ； In the formula, is the updated target action, is the action generated by the actor's neural network, is the next state; is the shear noise that follows a normal distribution; is a shear operation; is the standard deviation; c is the range parameter; Model judgment: The electric bicycle battery health monitoring model is trained based on the training set, and an accuracy threshold is set in advance. When the prediction accuracy of the trained electric bicycle battery health monitoring model for the test set is higher than the accuracy threshold, the electric bicycle battery health monitoring model is established.
2. The electric bicycle battery health monitoring system according to claim 1, characterized in that: The electric vehicle battery health monitoring module is based on the established electric bicycle battery health monitoring model; it collects battery information, battery status parameters, environmental information, vehicle information and control information in real time; The battery health status of electric vehicles is monitored based on the battery health status output by the model.
3. The electric bicycle battery health monitoring system according to claim 1, characterized in that: In the data acquisition module, the electric bicycle battery health history data includes battery information, battery status parameters, environmental information, vehicle information, control information and battery health status.
4. The electric bicycle battery health monitoring system according to claim 1, characterized in that: In the data preprocessing module, the data cleaning is to process missing values and duplicate values; the data conversion is to convert the data into vector form; the standardization is to standardize the data based on the maximum and minimum normalization method; The data set division is to divide the data into a training set and a test set; And use the battery health status as the data label.