An online adaptive parameter stabilization optimization method for sensor hysteresis model

Abstract

This invention relates to an online adaptive parameter stabilization optimization method for sensor hysteresis models, belonging to the field of sensor modeling technology. It solves the problem in existing technologies where hysteresis model parameters are difficult to update stably online under drift, noise, and changing operating conditions. The optimization method includes: calculating the predicted output value of the hysteresis model at the current sampling point and the instantaneous error of the current sampling point; calculating the root mean square error of the instantaneous error based on a sliding window of the current sampling point to obtain the root mean square error of the sliding window at the current sampling point; determining whether the current sampling point is an allowable sampling point based on the root mean square error of the sliding window and a preset gating threshold; if the current sampling point is an allowable sampling point, optimizing the current hysteresis model parameters based on the state and action variables of the current sampling point within the evaluation window of the current sampling point. This achieves stable online updating of sensor hysteresis model parameters, improving the real-time measurement accuracy of the sensor.

Description

Technical Field

[0001] This invention relates to the field of sensor modeling technology, and in particular to an online adaptive parameter stabilization optimization method for sensor hysteresis models. Background Technology

[0002] In applications involving pressure, displacement, force, temperature, electromagnetic, piezoelectric, and composite material measurements, sensors often exhibit significant hysteresis characteristics. For the same input value, the sensor may output different values ​​depending on whether the input path is ascending or descending, demonstrating path dependence and memory effects. If the hysteresis model cannot accurately represent these relationships, it will lead to measurement errors, compensation distortion, and a decrease in control accuracy.

[0003] Existing technologies typically first establish a Preisach model, Bouc-Wen model, or other classic hysteresis models based on calibration data, and then use methods such as least squares, genetic algorithms, and particle swarm optimization to obtain a set of fixed parameters offline. While these methods can obtain usable parameters under fixed operating conditions, when the sensor is affected by temperature, humidity, mechanical stress, aging, power supply fluctuations, changes in loading rate, or external disturbances, the offline fixed parameters are difficult to maintain consistently, resulting in significant measurement errors.

[0004] Therefore, there is an urgent need for a new technical solution for stabilizing and optimizing sensor hysteresis models. Summary of the Invention

[0005] Based on the above analysis, the embodiments of the present invention aim to provide an online adaptive parameter stabilization optimization method for sensor hysteresis models, in order to solve the problem that hysteresis model parameters are difficult to update stably online under drift, noise and operating condition changes in the prior art.

[0006] This invention provides an online adaptive parameter stabilization optimization method for a sensor hysteresis model, the optimization method comprising: Obtain the actual sensor input value and actual sensor output value at the current sampling point, and calculate the hysteresis model predicted output value at the current sampling point based on the current hysteresis model parameters; The instantaneous error of the current sampling point is obtained based on the actual output value of the sensor and the predicted output value of the hysteresis model. The root mean square error of the instantaneous error is calculated based on the sliding window of the current sampling point to obtain the root mean square error of the sliding window of the current sampling point. Based on the root mean square error of the sliding window, combined with a preset gating threshold, it is determined whether the current sampling point is a sampling point that is allowed to be updated; If the current sampling point is an updateable sampling point, construct the state variable of the current sampling point, and combine it with the pre-trained decision model to obtain the action variable of the current sampling point; on the evaluation window of the current sampling point, optimize the parameters of the current hysteresis model based on the state variable and action variable of the current sampling point.

[0007] Based on a further improvement to the above optimization method, the step of determining whether the current sampling point is an allowable sampling point for updating includes: Based on the root mean square error of the sliding window and a preset gate threshold, determine whether the current sampling point is in an adaptive update state. If the current sampling point is in an adaptive update state, determine the number of sampling points that remain in the adaptive update state backward from the current sampling point. If the number of sampling points that remain in the adaptive update state is an integer multiple of the control period, then the current sampling point is an allowed sampling point for updating.

[0008] Based on the further improvement of the above optimization method, the preset gating threshold includes a start error threshold, an exit error threshold, an open continuous counting threshold, and a close continuous counting threshold, and the start error threshold is greater than the exit error threshold; Determining whether the current sampling point is in an adaptive update state includes: When the root mean square error of the sliding window is greater than the start error threshold, and there are n consecutive sampling points before the current sampling point whose root mean square error of the sliding window is greater than the start error threshold, if n is greater than or equal to the start consecutive counting threshold, the current sampling point is in an adaptive update state. When the root mean square error of the sliding window is less than the exit error threshold, and there are m consecutive sampling points before the current sampling point whose root mean square error of the sliding window is less than the exit error threshold, if m is greater than or equal to the closing consecutive counting threshold, the current sampling point exits the adaptive update state. Otherwise, the state of the current sampling point remains the same as that of the previous sampling point.

[0009] Based on further improvements to the above optimization method, the state variables of the sampling points include one or more of the following: The actual sensor input value at the current sampling point; The difference between the actual sensor input value at the current sampling point and the previous sampling point; The actual output value of the sensor at the current sampling point; The hysteresis model predicts the output value of the current sampling point; The output error at the current sampling point; where the output error represents the difference between the actual sensor output value at the current sampling point and the hysteresis model predicted output value at the current sampling point; The difference between the output error of the current sampling point and the previous sampling point; The root mean square error of the sliding window at the current sampling point; Hysteresis model parameters at the current sampling point.

[0010] Based on further improvements to the above optimization method, the state variables of the sampling point also include environmental characteristics of the sensor, which may include one or more of the following: temperature; humidity; Vibration frequency; Impact strength; Loading speed; Power supply voltage; Installation stress.

[0011] Based on further improvements to the above optimization method, the decision model adopts the Deep Deterministic Policy Gradient (DDPG) model, and is trained through the following steps: Construct a DDPG model that includes a policy network and a value evaluation network, and initialize the experience replay pool; Based on the simulation data or historical data of the sensor, perform online stability optimization closed loop, generate state transition samples and store them in the experience playback pool; The policy network and value evaluation network are trained based on the state transition samples in the experience replay pool, and the trained policy network is used as the decision model.

[0012] Based on further improvements to the above optimization method, the state transition sample includes the state quantity of the current sampling point, the action quantity of the current sampling point, the reward function value of the current sampling point, and the state quantity of the next sampling point.

[0013] Based on a further improvement to the above optimization method, the step of training the policy network and the value evaluation network using state transition samples from the experience replay pool includes: A predetermined number of state transition samples are randomly sampled from the experience replay pool; The state quantity and action quantity of the current sampling point in each state transition sample are input into the value evaluation network to obtain the actual Q value of the sample; the state quantity of the next sampling point in each state transition sample is input into the policy network and the value evaluation network, and the target Q value of the sample is obtained by combining the reward function value of the current sampling point. The mean squared error loss value of Q is determined based on the actual Q value of the sample and the target Q value of the sample; if the mean squared error loss value of Q is less than the loss threshold, the loop stops and the trained policy network is obtained; if the mean squared error loss value of Q is greater than or equal to the loss threshold, the model parameters of the value evaluation network and the policy network are updated based on the mean squared error loss value of Q. By repeatedly executing the above steps, the training of the value assessment network and the policy network is completed.

[0014] Based on the further improvement of the above optimization method, the sample target Q value is calculated using the following formula: y_i^target = r_i + gamma_d · Q'(s_(i+1),μ'(s_(i+1)|θ'_μ) | θ'_Q); Where y_i^target represents the target Q value of the i-th sample, r_i represents the reward function value of the current sampling point of the i-th sample, gamma_d represents the discount factor parameter, s_(i+1) represents the state quantity of the next sampling point, and Q' and μ' represent the value evaluation network and policy network of this batch. The reward function value for the current sampling point is calculated using the following formula: r_t = α · (E_keep(t) - E_chosen(t)) - β · ||a_t||^2 - γ · ||Δθ_t||^2 - η · I_reject; Where r_t represents the reward function value of the current sampling point, α, β, γ, and η are non-negative weight parameters, E_keep(t) represents the root mean square error of the evaluation window corresponding to the current hysteresis model parameters, E_chosen(t) represents the root mean square error of the evaluation window corresponding to the optimized hysteresis model parameters, a_t represents the action amount of the current sampling point, Δθ_t represents the increment of the hysteresis model parameters corresponding to the action amount of the current sampling point, and I_reject represents the rollback indication.

[0015] Based on a further improvement to the above optimization method, the optimization of the current hysteresis model parameters according to the state and action quantities of the current sampling point includes: The motion quantity at the current sampling point is scaled to obtain the hysteresis model parameter increment, and combined with the current hysteresis model parameter to obtain the candidate hysteresis model parameter. Calculate the root mean square error of the evaluation window corresponding to the current hysteresis model parameters and the candidate hysteresis model parameters within the evaluation window of the current sampling point. If the difference between the root mean square error of the evaluation window of the current hysteresis model parameter and the root mean square error of the evaluation window of the candidate hysteresis model parameter is greater than the safety margin threshold, then the candidate hysteresis model parameter is optimized into the parameter of the hysteresis model; otherwise, the current hysteresis model parameter continues to be used as the parameter of the hysteresis model.

[0016] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects: 1. By using the root mean square error of the sliding window of the sampling points and the throttling of the control cycle, parameter updates are only initiated when the error continues to deteriorate and the update frequency constraint is met, thus avoiding frequent invalid adjustments caused by noise and improving the stability of online operation. 2. When constructing the state variables of sampling points, add environmental features to enable the decision model to perceive changes in operating conditions and further improve the adaptive capability of the sensor hysteresis model. 3. By comparing the root mean square error of the evaluation window of the current hysteresis model parameters with the root mean square error of the evaluation window of the candidate hysteresis model parameters, a safe update judgment and rollback mechanism is introduced. By comparing the error improvement on the evaluation window, only updates that can bring substantial improvement are accepted, effectively suppressing error bounce and oscillation caused by erroneous updates.

[0017] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0018] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0019] Figure 1 This is a flowchart illustrating an online adaptive parameter stabilization optimization method for a sensor hysteresis model provided in an embodiment of the present invention. Detailed Implementation

[0020] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0021] A specific embodiment of the present invention discloses an online adaptive parameter stabilization optimization method for a sensor hysteresis model, such as... Figure 1 As shown, the optimization method includes: Step S1: Obtain the actual sensor input value and actual sensor output value at the current sampling point, and calculate the hysteresis model predicted output value at the current sampling point based on the current hysteresis model parameters; Step S2: Based on the actual output value of the sensor and the predicted output value of the current hysteresis model, obtain the instantaneous error of the current sampling point. Calculate the root mean square error of the instantaneous error based on the sliding window of the current sampling point to obtain the root mean square error of the sliding window of the current sampling point. Step S2: Based on the root mean square error of the sliding window, and combined with the preset gating threshold, determine whether the current sampling point is a sampling point that is allowed to be updated; Step S4: If the current sampling point is an allowable sampling point, construct the state variable of the current sampling point, and combine it with the pre-trained decision model to obtain the action variable of the current sampling point; on the evaluation window of the current sampling point, optimize the parameters of the current hysteresis model based on the state variable and action variable of the current sampling point.

[0022] Specifically, the hysteresis model of a sensor is sensitive to changes in the sensor's operating conditions and parameter drift. When the hysteresis model parameters cannot accurately describe the actual hysteresis characteristics, the prediction error and sliding window error of the hysteresis model will continue to rise.

[0023] It is understood that the optimization method provided in this embodiment of the invention can be used in sensors to optimize the hysteresis model parameters of the sensor and improve the detection accuracy of the sensor.

[0024] Specifically, during the sensor's operation, an optimization method is executed at each sampling point, such as... Figure 1 As shown, in step S1, the actual input signal of the sensor at the current sampling point, such as voltage, pressure or displacement, is acquired, as well as the actual output signal of the sensor, such as the sensor's measured value, thereby obtaining the actual input value and actual output value of the sensor at the current sampling point.

[0025] It is worth noting that different types of sensors can be represented by different hysteresis models according to their physical mechanisms. The optimization provided by this invention can optimize the hysteresis model parameters involved in different hysteresis models online.

[0026] Specifically, the hysteresis model of the sensor is set in advance according to the type of sensor, the number of parameters of the hysteresis model and the feasible domain of each parameter are determined, the hysteresis model is preferably the Preisach model or the Bouc-Wen model, or other models that can characterize path-dependent hysteresis characteristics, and the initial parameters of the hysteresis model are obtained through offline calibration and optimized during online use.

[0027] For example, when the hysteresis model adopts the Preisach model, the hysteresis model parameters can be selected as density functions; when the hysteresis model adopts the Bouc-Wen model, the hysteresis model parameters can be selected as one or more of the following parameters: restoring force amplitude parameter A, hysteresis shape parameter β, hysteresis shape parameter γ, smoothness / sharpness coefficient n, stiffness scaling factor k, and hysteresis output scaling factor c.

[0028] Specifically, the feasible region Θ of each parameter is determined by the upper and lower bounds, sign constraints, and engineering experience of each parameter. For example, if the hysteresis model has m parameters, i.e., the parameter vector is m-dimensional, then box constraints can be used: Θ = { θ | θ_j_min ≤ θ_j ≤ θ_j_max,j = 1, 2, ..., m} Here, θ represents each parameter, and θ_j_min and θ_j_max represent the minimum and maximum value constraints of each parameter. For box constraints, the dimension-wise projection operator can be used.

[0029] Specifically, after the hysteresis model of the sensor is set, it is directly applied in this invention, such as... Figure 1 As shown, the actual input value and actual output value of the sensor at the current sampling point are input into the sensor's hysteresis model, and the predicted output value of the hysteresis model at the current sampling point is calculated under the control of the current hysteresis model parameters.

[0030] Specifically, such as Figure 1 As shown, in step S2, the instantaneous error of the current sampling point is obtained by subtracting the predicted output value of the hysteresis model from the actual output value of the sensor obtained in step S1, that is, the instantaneous error e_t = y_t - y_hat_t, where y_t represents the actual output value of the sensor and y_hat_t represents the predicted output value of the hysteresis model.

[0031] It is worth noting that, in order to reduce the impact of single-point noise on the judgment results, and to smooth out noise and capture short-term trends in error, the W nearest sampling points to each sampling point are combined to obtain a sliding window for each sampling point. The root mean square error of the sliding window is constructed using the W nearest sampling points.

[0032] Where RMSE_w(t) represents the root mean square error of the sliding window at the current sampling point, W represents the width of the sliding window, t represents the current sampling point, and e_i represents the instantaneous error of any sampling point within the sliding window.

[0033] Specifically, the sliding window width W is preferably between 5 and 50 sampling points, which can be adjusted according to the sampling frequency and the dynamic characteristics of the sensor. The smaller W is, the faster the drift response is, and the larger W is, the smoother the noise response is.

[0034] Specifically, the root mean square error of the instantaneous error of the current sampling point on the sliding window is calculated through step S2, and the root mean square error of the sliding window RMSE_w(t) of the current sampling point is obtained.

[0035] Specifically, such as Figure 1 As shown, in step S3, based on the root mean square error RMSE_w(t) of the sliding window of the current sampling point obtained in step S2, and combined with the preset gating threshold, it is determined whether the parameters of the current hysteresis model should be updated and optimized at the current sampling point, that is, whether the current sampling point is a sampling point that is allowed to be updated.

[0036] Specifically, in step S3, a gating trigger logic is set so that when the root mean square error of the sliding window of the sampling point continues to deteriorate, it enters an adaptive state to avoid unnecessary responses to short-term noise and to avoid frequent opening and closing near the threshold.

[0037] Preferably, determining whether the current sampling point is a sampling point that is allowed to be updated includes: Based on the root mean square error of the sliding window and a preset gate threshold, determine whether the current sampling point is in an adaptive update state. If the current sampling point is in an adaptive update state, determine the number of sampling points that remain in the adaptive update state backward from the current sampling point. If the number of sampling points that remain in the adaptive update state is an integer multiple of the control period, then the current sampling point is an allowed sampling point for updating.

[0038] Preferably, the preset gating thresholds include a start error threshold, an exit error threshold, an open continuous counting threshold, and a close continuous counting threshold, and the start error threshold is greater than the exit error threshold; Determining whether the current sampling point is in an adaptive update state includes: When the root mean square error of the sliding window is greater than the start error threshold, and there are n consecutive sampling points before the current sampling point whose root mean square error of the sliding window is greater than the start error threshold, if n is greater than or equal to the start consecutive counting threshold, the current sampling point is in an adaptive update state. When the root mean square error of the sliding window is less than the exit error threshold, and there are m consecutive sampling points before the current sampling point whose root mean square error of the sliding window is less than the exit error threshold, if m is greater than or equal to the closing consecutive counting threshold, the current sampling point exits the adaptive update state. Otherwise, the state of the current sampling point remains the same as that of the previous sampling point.

[0039] Specifically, set the start error threshold eps_th and the exit error threshold eps_good, ensuring that eps_th > eps_good; set the continuous counting enable threshold to count_on and the continuous counting disable threshold to count_good, then determine whether the current sampling point is in an adaptive update state based on the sliding window root mean square error RMSE_w(t) of the current sampling point: If RMSE_w(t) > eps_th, and the number of consecutive sampling points n is greater than onCountTh, then the current sampling point is in an adaptive update state, i.e., adapt_on = 1; If RMSE_w(t) < eps_good and the number of consecutive sampling points m is greater than goodCountTh, then the current sampling point exits the adaptive update state, i.e., adapt_on = 0; If RMSE_w(t) does not fall into the above two cases, then the current sampling point maintains its state, i.e., the same as the state of the previous sampling point.

[0040] It should be noted that the above logic forms a gating structure with hysteresis, which helps to avoid frequent optimization of the hysteresis model parameters near the threshold.

[0041] Specifically, when the state of the current sampling point is the adaptive update state, the hysteresis model parameters are not optimized at the current sampling point. Instead, the optimization frequency is restricted by the control period ctrlPeriod to suppress the high-frequency parameter swings driven by noise. That is, an update is only performed when the number of sampling points with the state continuously being the adaptive update state determined forward from the current sampling point is an integer multiple of the control period, and at this time, the current sampling point is regarded as the sampling point allowed for update. Exemplarily, if the control period ctrlPeriod is set to 3, then when the number of sampling points with the state continuously being the adaptive update state determined forward from the current sampling point is 3, 6, 9..., the current sampling point is the sampling point allowed for update, and the rest with the number of sampling points being 1, 2, 4, 5, 7, 8.. are not the sampling points allowed for update.

[0042] Exemplarily, the control period ctrlPeriod is greater than 1.

[0043] Specifically, it is decided whether to execute step S4 according to the judgment result of step S3. If the result obtained in step S3 is that the current sampling point is not the sampling point allowed for update, then step S4 is not executed at the current sampling point.

[0044] Specifically, as Figure 1 shown, when the judgment result of step S3 is that the current sampling point is the sampling point allowed for update, the current hysteresis model parameters of the sensor's hysteresis model are optimized through step S4.

[0045] Specifically, as Figure 1 shown, in step S4, the state quantity of the current sampling point is constructed, and the action quantity of the current sampling point is obtained by combining the pre-trained decision model.

[0046] Preferably, the state quantity of the sampling point includes one or more of the following: The actual input value of the sensor at the current sampling point; The difference between the actual input values of the sensor at the current sampling point and the previous sampling point; The actual output value of the sensor at the current sampling point; The hysteresis model predicts the output value of the current sampling point; The output error at the current sampling point; where the output error represents the difference between the actual sensor output value at the current sampling point and the hysteresis model predicted output value at the current sampling point; The difference between the output error of the current sampling point and the previous sampling point; The root mean square error of the sliding window at the current sampling point; Hysteresis model parameters at the current sampling point.

[0047] Specifically, when using the optimization method provided in the embodiments of the present invention, the state variables of the sampling points are constructed according to different sensor types and different hysteresis model types.

[0048] Preferably, the state parameters of the sampling point also include environmental characteristics of the sensor, wherein the environmental characteristics include one or more of the following: temperature; humidity; Vibration frequency; Impact strength; Loading speed; Power supply voltage; Installation stress.

[0049] Specifically, environmental features can be reasonably set according to the sensor's application environment. If the sensor's application environment has a significant impact on its measurement accuracy, then one or more suitable environmental features can be selected as state variables of the sampling points to participate in the optimization of the hysteresis model parameters. For example: s_t=[u_t,Δu_t,y_t,y_hat_t,e_t,Δe_t,RMSE_w(t),θ_bar_t, g_t]; Where s_t represents the state vector of the current sampling point, u_t represents the actual sensor input value of the current sampling point, Δu_t represents the difference between the actual sensor input value of the current sampling point and the actual sensor input value of the previous sampling point, Δe_t represents the difference between the output error of the current sampling point and the output error of the previous sampling point, θ_bar_t represents the hysteresis model parameters of the current sampling point, and g_t represents the environmental feature quantity of the current sampling point, which can be one or more.

[0050] Preferably, the decision model adopts the Deep Deterministic Policy Gradient (DDPG) model, and the decision model is trained through the following steps: Construct a DDPG model that includes a policy network and a value evaluation network, and initialize the experience replay pool; Based on the simulation data or historical data of the sensor, perform online stability optimization closed loop, generate state transition samples and store them in the experience playback pool; The policy network and value evaluation network are trained based on the state transition samples in the experience replay pool, and the trained policy network is used as the decision model.

[0051] Specifically, the Deep Deterministic Policy Gradient (DDPG) model is adopted as the continuous action decision-making model. The policy network preferably adopts a multilayer perceptron structure, with two or more hidden layers. The number of neurons in each layer can be determined according to the dimension of the state variables and the task complexity. The output layer preferably adopts the tanh activation function. In the value evaluation network, the concatenated vector of state variables and action variables is preferably used as the input of the value evaluation network, and the value evaluation network outputs a scalar Q value.

[0052] It is worth noting that the policy network and value evaluation network in the optimization method provided in this embodiment of the invention are not limited to a specific number of layers and neurons, as long as continuous action output and value estimation can be achieved.

[0053] Specifically, the experience replay pool is used to store state transition samples and, based on sensor simulation data or historical data, to perform online stability optimization closed-loop, such as... Figure 1 In steps S1 to S4, only state transition samples generated at sampling points that are allowed to be updated are stored in the experience replay pool.

[0054] Specifically, after there are enough state transition samples in the experience replay pool, the policy network and value evaluation network are trained through multiple iterations, and the trained policy network is used as the decision model.

[0055] Preferably, the state transition sample includes the state quantity of the current sampling point, the action quantity of the current sampling point, the reward function value of the current sampling point, and the state quantity of the next sampling point.

[0056] Specifically, the state transition sample is set as (s_t, a_t, r_t, s_(t+1)), where s_t represents the state quantity of the current sampling point, a_t represents the action quantity of the current sampling point, r_t represents the reward function value of the current sampling point, and s_(t+1) represents the state quantity of the next sampling point of the current sampling point.

[0057] Preferably, training the policy network and the value evaluation network based on the state transition samples from the experience replay pool includes: A predetermined number of state transition samples are randomly sampled from the experience replay pool; The state quantity and action quantity of the current sampling point in each state transition sample are input into the value evaluation network to obtain the actual Q value of the sample; the state quantity of the next sampling point in each state transition sample is input into the policy network and the value evaluation network, and the target Q value of the sample is obtained by combining the reward function value of the current sampling point. The mean squared error loss value of Q is determined based on the actual Q value of the sample and the target Q value of the sample; if the mean squared error loss value of Q is less than the loss threshold, the loop stops and the trained policy network is obtained; if the mean squared error loss value of Q is greater than or equal to the loss threshold, the model parameters of the value evaluation network and the policy network are updated based on the mean squared error loss value of Q. By repeatedly executing the above steps, the training of the value assessment network and the policy network is completed.

[0058] Specifically, state transition samples are randomly sampled from the experience replay pool according to a preset batch size. Random sampling helps reduce sequence correlation and improve training stability.

[0059] Specifically, the state quantity and action quantity of the current sampling point in each state transition sample are input into the value evaluation network to obtain the actual Q value of the sample, that is, the actual Q value of each state transition sample Q(s_i, a_i| θ_Q).

[0060] Specifically, the state value of the next sampling point in each state transition sample is input into the policy network and the value evaluation network. The target Q value of the sample is obtained by combining it with the reward function value of the current sampling point. Preferably, the target Q value is calculated using the following formula: y_i^target = r_i + gamma_d · Q'(s_(i+1),μ'(s_(i+1)|θ'_μ) | θ'_Q); Where y_i^target represents the target Q value of the i-th sample, r_i represents the reward function value of the current sampling point of the i-th sample, gamma_d represents the discount factor parameter, s_(i+1) represents the state quantity of the next sampling point, and Q'() and μ'() represent the output functions of the value evaluation network and the policy network of this batch. The reward function value for the current sampling point is calculated using the following formula: r_t = α · (E_keep(t) - E_chosen(t)) - β · ||a_t||^2 - γ · ||Δθ_t||^2 - η · I_reject; Where r_t represents the reward function value of the current sampling point, α, β, γ, and η are non-negative weight parameters, E_keep(t) represents the root mean square error of the evaluation window corresponding to the current hysteresis model parameters, E_chosen(t) represents the root mean square error of the evaluation window corresponding to the optimized hysteresis model parameters, a_t represents the action amount of the current sampling point, Δθ_t represents the increment of the hysteresis model parameters corresponding to the action amount of the current sampling point, and I_reject represents the rollback indication.

[0061] Specifically, after obtaining the actual Q-value and the target Q-value of the sample, the mean squared error loss value of the Q-value is calculated. If the mean squared error loss value of the Q-value is greater than or equal to the loss threshold, the model parameters of the value assessment network and the policy network are updated based on the mean squared error loss value of the Q-value.

[0062] Specifically, after updating the value evaluation network, the policy network parameters are updated based on the deterministic policy gradient, making the policy better in terms of long-term returns. The update direction of the policy network is provided by the gradient of the action from the value evaluation network.

[0063] Specifically, to reduce target drift during training, a soft update method is preferred for iterating the parameters of the policy network and the value evaluation network. θ'_Q ← (1 - tau) · θ'_Q + tau · θ_Q; θ'_μ ← (1 - tau) · θ'_μ + tau · θ_μ; Where tau is the soft update coefficient, θ_Q and θ'_Q represent the parameters of the value evaluation network before and after the update, and θ_μ and θ'_μ represent the parameters of the policy network before and after the update. OU noise or other exploration noise can also be added during the training phase to improve the state-action space exploration capability.

[0064] Specifically, the value evaluation network and the policy network are trained in multiple batches until the Q-value mean squared error loss value is less than the loss threshold during the training of a certain batch. Then the loop stops, and the trained policy network is obtained. The trained policy network is used as a decision model and applied to step S4 of the optimization method provided in this embodiment of the invention.

[0065] Specifically, such as Figure 1 As shown, in step S4, the state quantity of the current sampling point is input into the decision model, that is, into the trained policy network, to obtain the action quantity of the current sampling point.

[0066] Specifically, such as Figure 1 As shown, in the evaluation window of the current sampling point, the parameters of the current hysteresis model are optimized based on the state and action variables of the current sampling point.

[0067] Preferably, optimizing the current hysteresis model parameters based on the state and action quantities at the current sampling point includes: The motion quantity at the current sampling point is scaled to obtain the hysteresis model parameter increment, and combined with the current hysteresis model parameter to obtain the candidate hysteresis model parameter. Calculate the root mean square error of the evaluation window corresponding to the current hysteresis model parameters and the candidate hysteresis model parameters within the evaluation window of the current sampling point. If the difference between the root mean square error of the evaluation window of the current hysteresis model parameter and the root mean square error of the evaluation window of the candidate hysteresis model parameter is greater than the safety margin threshold, then the candidate hysteresis model parameter is optimized into the parameter of the hysteresis model; otherwise, the current hysteresis model parameter continues to be used as the parameter of the hysteresis model.

[0068] It is worth noting that the W_e nearest sampling points to the current sampling point are set as the evaluation window for the current sampling point, and the root mean square error of the evaluation window corresponding to the current hysteresis model parameters is calculated.

[0069] Specifically, the motion quantity s_t at the current sampling point is scaled to obtain the hysteresis model parameter increment Δθ_t. Combined with the current hysteresis model parameter θ_t, the candidate hysteresis model parameter θ_t^cand is obtained. The candidate hysteresis model parameter is then defined as: θ_t^cand = Proj_Θ(θ_t + Δθ_t); Where Proj_Θ(•) is the parameter feasible region projection operator. For box constraints, it is preferred to use the dimension-by-dimensional pruning method to ensure that the parameters of the candidate hysteresis model are always within the parameter feasible region.

[0070] Specifically, the root mean square error of the evaluation window corresponding to the current hysteresis model parameters, E_keep(t), and the root mean square error of the evaluation window corresponding to the candidate hysteresis model parameters, E_cand(t), are calculated as follows:

[0071]

[0072] in, This represents the model prediction output value at any sampling point in the evaluation window under the current hysteresis model parameters. This represents the model prediction output value corresponding to any sampling point on the evaluation window under the candidate hysteresis model parameters.

[0073] Specifically, to reduce the risk of error bounce caused by erroneous updates, this invention compares the root mean square error (RMSE) of the evaluation window corresponding to the current hysteresis model parameters with the RMSE of the evaluation window corresponding to the candidate hysteresis model parameters each time an update is allowed. If the difference between the RMSE of the evaluation window corresponding to the current hysteresis model parameters and the RMSE of the evaluation window corresponding to the candidate hysteresis model parameters is greater than the safe acceptance margin threshold, then the candidate hysteresis model parameters are optimized into the parameters of the hysteresis model; otherwise, the current hysteresis model parameters continue to be used as the parameters of the hysteresis model.

[0074] Preferably, if the difference between the root mean square error of the evaluation window of the current hysteresis model parameter and the root mean square error of the evaluation window of the candidate hysteresis model parameter is less than or equal to the safe acceptance margin threshold, the parameter increment Δθ_t can be further reduced by a ratio of 0-1 and the candidate hysteresis model parameter θ_t^cand can be regenerated and re-evaluated to increase the fineness of the exploration.

[0075] Compared with existing technologies, the online adaptive parameter stabilization optimization method for sensor hysteresis models provided in this invention improves the stability of online operation by using the root mean square error of the sliding window of sampling points and throttling of the control cycle, and only initiating parameter updates when the error continues to deteriorate and the update frequency constraint is met. This avoids frequent invalid adjustments caused by noise. At the same time, when constructing the state variables of sampling points, environmental features are added, enabling the decision model to perceive changes in operating conditions, further improving the adaptive capability of the sensor hysteresis model. Furthermore, by comparing the root mean square error of the evaluation window of the current hysteresis model parameters with the root mean square error of the evaluation window of the candidate hysteresis model parameters, a safe update judgment and rollback mechanism is introduced. By comparing the error improvement on the evaluation window, only updates that can bring substantial improvement are accepted, effectively suppressing error bounce and oscillation caused by erroneous updates.

[0076] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0077] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An online adaptive parameter stabilization optimization method for sensor hysteresis model, characterized in that, The optimization method includes: Obtain the actual sensor input value and actual sensor output value at the current sampling point, and calculate the hysteresis model predicted output value at the current sampling point based on the current hysteresis model parameters; The instantaneous error of the current sampling point is obtained based on the actual output value of the sensor and the predicted output value of the hysteresis model. The root mean square error of the instantaneous error is calculated based on the sliding window of the current sampling point to obtain the root mean square error of the sliding window of the current sampling point. Based on the root mean square error of the sliding window, combined with a preset gating threshold, it is determined whether the current sampling point is a sampling point that is allowed to be updated; If the current sampling point is an updateable sampling point, construct the state variable of the current sampling point, and combine it with the pre-trained decision model to obtain the action variable of the current sampling point; on the evaluation window of the current sampling point, optimize the parameters of the current hysteresis model based on the state variable and action variable of the current sampling point.

2. The optimization method of claim 1, wherein, The step of determining whether the current sampling point is an allowable sampling point for updating includes: Based on the root mean square error of the sliding window and a preset gate threshold, determine whether the current sampling point is in an adaptive update state. If the current sampling point is in an adaptive update state, determine the number of sampling points that remain in the adaptive update state backward from the current sampling point. If the number of sampling points that remain in the adaptive update state is an integer multiple of the control period, then the current sampling point is an allowed sampling point for updating.

3. The optimization method of claim 2, wherein, The preset gating thresholds include a start error threshold, an exit error threshold, an open continuous counting threshold, and a close continuous counting threshold, and the start error threshold is greater than the exit error threshold; Determining whether the current sampling point is in an adaptive update state includes: When the root mean square error of the sliding window is greater than the start error threshold, and there are n consecutive sampling points before the current sampling point whose root mean square error of the sliding window is greater than the start error threshold, if n is greater than or equal to the start consecutive counting threshold, the current sampling point is in an adaptive update state. When the root mean square error of the sliding window is less than the exit error threshold, and there are m consecutive sampling points before the current sampling point whose root mean square error of the sliding window is less than the exit error threshold, if m is greater than or equal to the closing consecutive counting threshold, the current sampling point exits the adaptive update state. Otherwise, the state of the current sampling point remains the same as that of the previous sampling point.

4. The optimization method of claim 1, wherein, The state variables of the sampling points include one or more of the following: The actual sensor input value at the current sampling point; The difference between the actual sensor input value at the current sampling point and the previous sampling point; The actual output value of the sensor at the current sampling point; The hysteresis model predicts the output value of the current sampling point; The output error at the current sampling point; where the output error represents the difference between the actual sensor output value at the current sampling point and the hysteresis model predicted output value at the current sampling point; The difference between the output error of the current sampling point and the previous sampling point; The root mean square error of the sliding window at the current sampling point; Hysteresis model parameters at the current sampling point.

5. The optimization method according to claim 4, characterized in that, The state variables of the sampling point also include environmental characteristics of the sensor, which may include one or more of the following: temperature; humidity; Vibration frequency; Impact strength; Loading speed; Power supply voltage; Installation stress.

6. The optimization method according to any one of claims 1-5, characterized in that, The decision model employs the Deep Deterministic Policy Gradient (DDPG) model, and is trained through the following steps: Construct a DDPG model that includes a policy network and a value evaluation network, and initialize the experience replay pool; Based on the simulation data or historical data of the sensor, perform online stability optimization closed loop, generate state transition samples and store them in the experience playback pool; The policy network and value evaluation network are trained based on the state transition samples in the experience replay pool, and the trained policy network is used as the decision model.

7. The optimization method according to claim 6, characterized in that, The state transition sample includes the state quantity of the current sampling point, the action quantity of the current sampling point, the reward function value of the current sampling point, and the state quantity of the next sampling point.

8. The optimization method according to claim 7, characterized in that, The training of the policy network and value evaluation network based on the state transition samples from the experience replay pool includes: A predetermined number of state transition samples are randomly sampled from the experience replay pool; The state quantity and action quantity of the current sampling point in each state transition sample are input into the value evaluation network to obtain the actual Q value of the sample; the state quantity of the next sampling point in each state transition sample is input into the policy network and the value evaluation network, and the target Q value of the sample is obtained by combining the reward function value of the current sampling point. The mean squared error loss value of Q is determined based on the actual Q value of the sample and the target Q value of the sample; if the mean squared error loss value of Q is less than the loss threshold, the loop stops and the trained policy network is obtained; if the mean squared error loss value of Q is greater than or equal to the loss threshold, the model parameters of the value evaluation network and the policy network are updated based on the mean squared error loss value of Q. By repeatedly executing the above steps, the training of the value assessment network and the policy network is completed.

9. The optimization method according to claim 8, characterized in that, The target Q value for the sample is calculated using the following formula: y_i^target = r_i + gamma_d · Q'(s_(i+1),μ'(s_(i+1)|θ'_μ) | θ'_Q); Where y_i^target represents the target Q value of the i-th sample, r_i represents the reward function value of the current sampling point of the i-th sample, gamma_d represents the discount factor parameter, s_(i+1) represents the state quantity of the next sampling point, and Q' and μ' represent the value evaluation network and policy network of this batch. The reward function value for the current sampling point is calculated using the following formula: r_t = α · (E_keep(t) - E_chosen(t)) - β · ||a_t||^2 - γ · ||Δθ_t||^2- η · I_reject; Where r_t represents the reward function value of the current sampling point, α, β, γ, and η are non-negative weight parameters, E_keep(t) represents the root mean square error of the evaluation window corresponding to the current hysteresis model parameters, E_chosen(t) represents the root mean square error of the evaluation window corresponding to the optimized hysteresis model parameters, a_t represents the action amount of the current sampling point, Δθ_t represents the increment of the hysteresis model parameters corresponding to the action amount of the current sampling point, and I_reject represents the rollback indication.

10. The optimization method according to claim 6, characterized in that, The optimization of the current hysteresis model parameters based on the state and action quantities at the current sampling point includes: The motion quantity at the current sampling point is scaled to obtain the hysteresis model parameter increment, and combined with the current hysteresis model parameter to obtain the candidate hysteresis model parameter. Calculate the root mean square error of the evaluation window corresponding to the current hysteresis model parameters and the candidate hysteresis model parameters within the evaluation window of the current sampling point. If the difference between the root mean square error of the evaluation window of the current hysteresis model parameter and the root mean square error of the evaluation window of the candidate hysteresis model parameter is greater than the safety margin threshold, then the candidate hysteresis model parameter is optimized into the parameter of the hysteresis model; otherwise, the current hysteresis model parameter continues to be used as the parameter of the hysteresis model.

Images