Tool change strategy dynamic search method
By constructing a tooling time-series feature extraction model and a replacement strategy dynamic search model, the problem of inaccurate tooling life prediction was solved, intelligent decision-making for tooling replacement was realized, and production efficiency and safety were improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGDONG UNIV OF TECH
- Filing Date
- 2023-03-20
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies make it difficult to accurately predict tooling lifespan, leading to untimely or excessively frequent tooling replacements, which affects production quality and safety, and increases resource waste and costs.
We construct a tooling time-series feature extraction model and a tooling replacement strategy dynamic search model. By training and optimizing the model, and combining historical and real-time tooling data for dynamic search, we can achieve intelligent decision-making on tooling replacement strategies.
It improved tooling utilization and equipment production efficiency, reduced defective products and production line downtime, and enhanced overall production efficiency.
Smart Images
Figure CN116433219B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of tooling change, and more particularly to a dynamic search method for tooling change strategies. Background Technology
[0002] Product manufacturing requires precise tooling and reliable production processes. To ensure product quality and workplace safety, tooling must be replaced promptly when its remaining service life reaches the replacement threshold. In reality, tooling lifespan is influenced by multiple factors, including the working environment, product shape, materials, surface roughness, production process flow, and the type, shape, material, operating temperature, and operating pressure of the tooling itself. Furthermore, the lack of readily available on-site data makes tooling lifespan difficult to predict. To avoid excessively frequent tooling replacements that waste resources and increase production costs, as well as the impact on product quality and reliability caused by delayed tooling replacement, which increases rework, defective, and scrap products and raises safety hazards at the assembly site, it is always desirable to issue early warnings or alerts before tooling fails, guiding on-site personnel to replace it. Summary of the Invention
[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a dynamic search method for tooling change strategy. This method achieves dynamic, accurate, and intelligent dynamic search function for tooling change decision-making, while reducing the number of defective products produced due to untimely or unreasonable tooling changes and the downtime for tooling changes on the production line, thereby improving tooling utilization, equipment production efficiency, and overall production line efficiency.
[0004] To achieve the above objectives, the technical solution provided by this invention is as follows:
[0005] A dynamic search method for tooling change strategies includes:
[0006] Construct a tooling time-series feature extraction model;
[0007] Construct a dynamic search model for tooling replacement strategies that includes a tooling operation state space and a replacement strategy space;
[0008] The tooling time-series feature extraction model and the tooling change strategy dynamic search model were trained and optimized.
[0009] Dynamic search for tooling replacement strategies is performed by combining an optimized tooling time-series feature extraction model and a tooling replacement strategy dynamic search model.
[0010] Furthermore, the constructed tooling time-series feature extraction model is as follows:
[0011]
[0012] ft =Λ(W f ·[as t-1 x t ]+b f (2)
[0013] i t =Λ(W i ·[as t-1 x t ]+b i (3)
[0014]
[0015] o t =Λ(W o ·[as t-1 x t ]+b o (5)
[0016] as t =o t ×tanh(C t (6)
[0017] Among them, C t f represents the tooling label data at time t. t The feature storage vector for the tooling time-series feature extraction model is used to represent C. t-1 Which features will be used to calculate C? t i t o t These are the data input node, data output node, and output node o. t Used from C t Extract tooling time series features as t W f W i W C Wo is the weight matrix; b f b i b C b o It is the bias vector; The cell state update value uses the hyperbolic tangent function as its activation function; Λ is the normalization function to ensure that the parameter values of the input node, output node, and feature storage vector are between [0, 1]; as t-1 The tooling timing characteristics at time t-1; as t The tooling timing characteristics at time t.
[0018] Furthermore, a dynamic search model for tooling change strategies is constructed, including:
[0019] Define the tooling life prediction decision state space s t Tooling timing characteristics as t and the real-time operating status characteristics of the tooling rs t Together they constitute:
[0020] s t ={as t rs t} (7)
[0021] Among them, as t For tooling timing characteristics, rs t This refers to the real-time operating status characteristics of the tooling;
[0022] The online policy network of the tooling change strategy dynamic search model outputs tooling change decisions based on the decision state space. To enable the model to handle multiple devices and multiple tooling, an action space 'a' is defined to output a tooling change strategy at each step. t :
[0023] a t ={c, α1, α2, ..., α n}(8)
[0024] Where n is the number of tools; c is the cycle length factor of this tool replacement cycle; α n The remaining time factor of the remaining useful life of tooling number n after normalization by the normalized exponential function.
[0025] The online evaluation network uses an estimated value function to quantify the value of the tooling replacement decision, and the online policy network adjusts its own parameters based on the evaluation results of the current tooling replacement decision.
[0026] The objective of the reward function is defined as follows: the closer the predicted value is to the actual value, the greater the reward, and vice versa.
[0027] The reward function r that comprehensively considers both tooling value and product value t As shown in the following formula:
[0028]
[0029] Where ω1 and ω2 are the weighting coefficients for the two scenarios of changing tooling in advance and not changing tooling in time, F i For the value of the tooling currently loaded on device i, FT i P represents the actual service life of the current tooling for equipment i. i PT represents the current value of the products produced by equipment i. i The production cycle time of the current product being produced by equipment i.
[0030] Furthermore, in a training cycle, the tooling change strategy dynamic search model first uses the tooling temporal feature extraction model to extract the tooling temporal features as... t Features of real-time operating status of tooling rs t Constituting state s t The results are used as input; then, based on the tooling replacement strategy results and evaluation results, the parameters of the tooling replacement strategy dynamic search model and the tooling temporal feature extraction model are updated through the action loss function of reward weight adjustment, the minimum mean square error loss of reward weight adjustment, and the deterministic strategy gradient; finally, when the number of iterations reaches the preset number or the model output tends to stabilize, the trained tooling temporal feature extraction model and tooling replacement strategy dynamic search model are output.
[0031] Furthermore, the specific training optimization process includes:
[0032] B1. Input the historical observation dataset of the tooling X = {x1, x2, ..., x...} t};
[0033] B2. Initialize the tooling temporal feature extraction model and output node parameters W. o b o Other parameters θ of the tooling time series feature extraction model L Initialize the online evaluation network Q(s, a|θ) Q Online policy network μ(s|θ) μ Network parameters θ Q and θ μ Initialize the parameters of the target evaluation and target policy networks Q′ and μ′: θ Q′ ←θ Q and θ μ′ ←θ μ Initialize the action space A and the experience pool R; initialize the episode parameter to 0; set the number of samples N, discount rate γ, soft update coefficient τ, learning rate α, parameter update interval C, and weight coefficients ω1 and ω2 for batch gradient descent;
[0034] B3. Initialize the tooling state space s t Initialize parameter t = 0;
[0035] B4. If t = 0, then set the tooling timing characteristic as t =0, otherwise the tooling time series feature extraction model uses the tooling historical time series data x at time t. t The tooling label data C saved at time t-1 t-1 The time series feature data saved at time t-1 is as t-1 Extract and save the tooling label data C at time t. t and tooling timing characteristics ast Real-time multi-source production data of tooling x t The real-time operating status characteristics of the tooling were obtained after data preprocessing. t Tooling timing characteristics as t Features of the real-time operating status of the tooling at time t t State s at time t t ={as t rs t};
[0036] B5. The online strategy network of the tooling change strategy dynamic search model is based on the current state of the tooling. t Based on the set search strategy and the noise level during exploration, select the current tooling replacement decision action a. t =μ({as t rs t}|θ μ )+N t ;
[0037] B6. Online evaluation network execution action a of the dynamic search model for tooling change strategy t Receive reward r t ; Determine the reward r t Is it greater than the reward threshold r? Value If it is greater than s, then record it in the experience record <s t a t r t > Add high reward weight ω H If it is less than, then in the experience record <s t a t r t > Add low reward weight ω L ;
[0038] B7. Calculate the action loss function adjusted for reward weights. Where k = {H, L}; through the formula Update the output node parameters of the tooling time series feature extraction model. t The output node parameter o is obtained. t+1 ;
[0039] B8. The tooling time-series feature extraction model extracts the tooling time-series features (as) from the tooling historical time-series data at time t+1. t+1 The real-time operating status characteristics of the tooling at time t+1 are rs t+1 State s at time t+1 t+1 ={as t+1 rs t+1}, save experience records <s t a t r t, ω > φ;
[0040] B9. Add the high-reward-weight experience record <s t , a t , r t , ω H , s t+1 > to the experience pool R H in. Add the low-reward-weight experience record <s t , a t , r t , ω L , s t+1 > to the experience pool R L in;
[0041] B10. Until the number of experience records stored in the experience pools R H and R L reaches the set quantity, proceed to step B11. Otherwise, set t = t + 1 and return to step B4;
[0042] B11. Randomly sample N experience records <s H from the experience pools R L and R t , a t , r t , ω, s t+1 > as the mini-batch training data for the online evaluation network Q and the online policy network μ;
[0043] B12. Update the parameters θ μ of the online policy network μ using deterministic policy gradients. The calculation formula is as follows:
[0044]
[0045] B13. Update the parameters θ of the online evaluation network Q and the parameters θ Q of the tooling time-series feature extraction model using the minimized mean squared error loss adjusted by the reward weight, where k = {H, L}, y L = r i + Q′(s i , μ′(s i+1 |θ i+1 )|θ μ′ ); Q′ );
[0046] B14. Update the parameters θ Q′ and θ μ′ of the target evaluation network Q′ and the target policy network μ′ by means of exponential moving average every C steps. The formula is as follows: [[ID=八十六]]
[0047] θ Q′ ← τθQ +(1-τ)θ Q′
[0048] θ μ′ ←τθ μ +(1-τ)θ μ′
[0049] B15. Determine whether the system has reached the termination state. If it has reached the termination state, proceed to step B16. If it has not reached the termination state, set t = t + 1 and return to step B4.
[0050] B16. Determine whether episode has reached the set value. If it has, output the current optimal tooling timing feature extraction model and tooling replacement strategy dynamic search model. Otherwise, set episode = episode + 1 and return to step B3.
[0051] Furthermore, the tooling label data C at time t is extracted using a tooling time-series feature extraction model. t and tooling timing characteristics as t The process is as follows:
[0052] A1. Obtain the tooling status data x from time t. t Tooling label data C at time t-1 t-1 The timing characteristics of the tooling at time t-1 before the time step t-1 t-1 Input into the tooling time-series feature extraction model;
[0053] A2. The tooling time-series feature extraction model performs the following extraction process:
[0054] 1) Based on the tooling status data x at time t before t The timing characteristics of the tooling at time t-1 before the time step t-1 t-1 Combined with equation (2), the feature storage vector f of the tooling time-series feature extraction model is obtained. t ;
[0055] 2) Based on the tooling status data x at time t before t The timing characteristics of the tooling at time t-1 before the time step t-1 t-1 Combine equations (3) and (5) to obtain the data input node i respectively. t and data output node o t ;
[0056] 3) Based on the tooling status data x at time t before t The timing characteristics of the tooling at time t-1 before the time step t-1 t-1 Combine equation (4) to obtain the cell status update value.
[0057] 4) Feature storage vector f based on tooling time-series feature extraction modelt Tooling label data C at time t-1 t-1 Data input node i t Cell status update value Combine equation (1) to obtain the tooling label data C at time t. t ;
[0058] 5) Based on data output node o t And the tooling label data C at time t t Combining equation (6), we obtain the implicit data at time t, namely the tooling timing characteristics as at time t. t .
[0059] Furthermore, a dynamic search for tooling replacement strategies is performed by combining the optimized tooling time-series feature extraction model and the tooling replacement strategy dynamic search model, including:
[0060] C1. Input the historical time series data of the tooling X1 and the real-time multi-source production data of the tooling X2;
[0061] C2. Load the tooling time-series feature extraction model and the tooling replacement strategy dynamic search model;
[0062] C3. The tooling time series feature extraction model uses the tooling historical time series data X1 and the tooling label data C saved at time t-1. t-1 The time series feature data saved at time t-1 is as t-1 Extract and save the tooling label data C at time t. t and tooling timing characteristics as t The real-time multi-source production data of the tooling (X2) is preprocessed to obtain the real-time operating status characteristics of the tooling (rs). t , as t with rs t The state s that constitutes the dynamic search model of tooling change strategy t ={as t rs t} and enter;
[0063] C4. The target strategy network of the dynamic search model for tooling replacement strategy selects the current tooling replacement decision action a. out =μ′({as t rs t}|θ μ′ ), and output;
[0064] C5. The online strategy network of the tooling change strategy dynamic search model selects the current tooling change decision action a. t =μ({as t rs t}|θμ Online evaluation of network execution actions a t Receive reward r t ; Determine the reward r t Is it greater than the reward threshold r? Value If it is greater than s, then record it in the experience record <s t a t r t > Add high reward weight ω H Add to experience pool R H If it is less than, then in the experience record <s t a t r t > Add low reward weight ω L Add to experience pool R L middle;
[0065] C6, until it is stored in the experience pool R H and R L Once the number of experience entries reaches the set quantity, proceed to step (7); otherwise, end the tooling change strategy search process.
[0066] C7, in the experience pool R H With R L Randomly collect N experience records for {<s t a t r t ω t >, <s t+1 a t+1 r t+1 ω t+1 Reconstruct the data to generate N experience records <s. t a t r t ω t s t+1 > Mini-batch training data for the online evaluation network Q and the online policy network μ.
[0067] C8. Update the parameters θ of the online policy network μ using a deterministic policy gradient. μ The calculation formula is as follows:
[0068]
[0069] C9. Minimize the mean squared error using reward-weighted adjustments. Update the parameter θ of the online evaluation network Q. Q Where k = {H, L}, y i =r i +Q′(s i+1 ,μ′(s i+1 |θ μ′)|θ Q′ );
[0070] C10. Every C steps, update the parameters θ of the target evaluation network Q′ and the target policy network μ′ using a moving average method. Q′ and θ μ′ The formula is as follows:
[0071] θ Q′ ←τθ Q +(1-τ)θ Q′
[0072] θ μ′ ←τθ μ +(1-τ)θ μ′
[0073] C11. End the tooling change strategy search process.
[0074] Compared with existing technologies, the principles and advantages of this solution are as follows:
[0075] This solution defines historical tooling health data, current tooling status data, and tooling replacement plans. It proposes a tooling operation status space and a replacement strategy space, standardizing the dynamic search function for tooling replacement strategies. This allows for rapid and accurate application to the assembly line, providing decision support for tooling replacement and significantly improving the efficiency of tooling replacement strategy search. While achieving dynamic, accurate, and intelligent tooling replacement decision-making search, it reduces the number of defective products produced due to untimely or unreasonable tooling replacements, as well as production line downtime for tooling replacements, thereby improving tooling utilization, equipment production efficiency, and overall production line effectiveness. Attached Figure Description
[0076] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the services required in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0077] Figure 1 This is a flowchart illustrating the principle of a dynamic search method for tooling change strategy according to the present invention.
[0078] Figure 2 This is a flowchart illustrating the training process of the tooling time-series feature extraction model and the tooling replacement strategy dynamic search model in this invention.
[0079] Figure 3 This is a flowchart illustrating the dynamic search for tooling replacement strategies in this invention, which combines an optimized tooling time-series feature extraction model with a tooling replacement strategy dynamic search model. Detailed Implementation
[0080] The present invention will be further described below with reference to specific embodiments:
[0081] like Figure 1 As shown in this embodiment, a dynamic search method for tooling change strategies includes the following steps:
[0082] S1. Construct a tooling time-series feature extraction model, as follows:
[0083]
[0084] f t =Λ(W f ·[as t-1 x t ]+b f (2)
[0085] i t =Λ(W i ·[as t-1 x t ]+b i (3)
[0086]
[0087] o t =Λ(W o ·[as t-1 x t ]+b o (5)
[0088] as t =o t ×tanh(C t (6)
[0089] Among them, C t f represents the tooling label data at time t. t The feature storage vector for the tooling time-series feature extraction model is used to represent C. t-1 Which features will be used to calculate C? t i t o t These are the data input node, data output node, and output node o. t Used from C t Extract tooling time series features as t W f W i W C W o b is the weight matrix; f bi b C b o It is the bias vector; The cell state update value uses the hyperbolic tangent function as its activation function; Λ is the normalization function to ensure that the parameter values of the input node, output node, and feature storage vector are between [0, 1]; as t-1 The tooling timing characteristics at time t-1; as t The tooling timing characteristics at time t.
[0090] S2. Construct a dynamic search model for tooling replacement strategies, including a tooling operation state space and a replacement strategy space, comprising:
[0091] In the dynamic search model for tooling replacement strategies, to enable the model to comprehensively consider the impact of historical time-series data and real-time multi-source production data on the formulation of tooling replacement strategies, the tooling life prediction decision state space s is defined. t Tooling timing characteristics as t and the real-time operating status characteristics of the tooling rs t Together they constitute:
[0092] s t ={as t rs t} (7)
[0093] Among them, as t For tooling timing characteristics, rs t This refers to the real-time operating status characteristics of the tooling;
[0094] The online policy network of the tooling change strategy dynamic search model outputs tooling change decisions based on the decision state space. To enable the model to handle multiple devices and multiple tooling, an action space 'a' is defined to output a tooling change strategy at each step. t :
[0095] a t ={c, α1, α2, ..., α n} (8)
[0096] Where n is the number of tools; c is the cycle length factor of this tool replacement cycle; α n The remaining time factor of the remaining useful life of tooling number n after normalization by the normalized exponential function.
[0097] The online evaluation network uses an estimated value function to quantify the value of the tooling replacement decision, and the online policy network adjusts its own parameters based on the evaluation results of the current tooling replacement decision.
[0098] To improve the accuracy of tooling life prediction, we aim to prevent both excessive tooling replacement (which wastes resources and increases production costs) and delays in tooling replacement (which negatively impacts product quality and reliability, increases rework, defective, and scrap, and raises safety hazards at the assembly site). Therefore, the reward function objective is defined as follows: the closer the predicted value is to the actual value, the greater the reward; conversely, the closer the predicted value is to the actual value, the smaller the reward.
[0099] The reward function r that comprehensively considers both tooling value and product value t As shown in the following formula:
[0100]
[0101] Where ω1 and ω2 are the weighting coefficients for the two scenarios of changing tooling in advance and not changing tooling in time, F i For the value of the tooling currently loaded on device i, FT i P represents the actual service life of the current tooling for equipment i. i PT represents the current value of the products produced by equipment i. i The production cycle time of the current product being produced by equipment i.
[0102] S3. Train and optimize the tooling time-series feature extraction model and the tooling replacement strategy dynamic search model;
[0103] like Figure 2 As shown, the specific training optimization process includes:
[0104] B1. Input the historical observation dataset of the tooling X = {x1, x2, ..., x...} t};
[0105] B2. Initialize the tooling temporal feature extraction model and output node parameters W. o b o Other parameters θ of the tooling time series feature extraction model L Initialize the online evaluation network Q(s, a|θ) Q Online policy network μ(s|θ) μ Network parameters θ Q and θ μ Initialize the parameters of the target evaluation and target policy networks Q′ and μ′: θ Q′ ←θ Q and θ μ′ ←θ u Initialize the action space A and the experience pool R; initialize the episode parameter to 0; set the number of samples N, discount rate γ, soft update coefficient τ, learning rate α, parameter update interval C, and weight coefficients ω1 and ω2 for batch gradient descent;
[0106] B3. Initialize the tooling state space st Initialize parameter t = 0;
[0107] B4. If t = 0, then set the tooling timing characteristic as t =0, otherwise the tooling time series feature extraction model uses the tooling historical time series data x at time t. t The tooling label data C saved at time t-1 t-1 The time series feature data saved at time t-1 is as t-1 Extract and save the tooling label data C at time t. t and tooling timing characteristics as t Real-time multi-source production data of tooling x t The real-time operating status characteristics of the tooling were obtained after data preprocessing. t Tooling timing characteristics as t Features of the real-time operating status of the tooling at time t t State s at time t t ={as t rs t};
[0108] B5. The online strategy network of the tooling change strategy dynamic search model is based on the current state of the tooling. t Based on the set search strategy and the noise level during exploration, select the current tooling replacement decision action a. t =μ({as t rs t}|θ μ )+N t ;
[0109] B6. Online evaluation network execution action a of the dynamic search model for tooling change strategy t Receive reward r t ; Determine the reward r t Is it greater than the reward threshold r? Value If it is greater than s, then record it in the experience record <s t a t r t > Add high reward weight ω H If it is less than, then in the experience record <s t a t r t > Add low reward weight ω L ;
[0110] B7. Calculate the action loss function adjusted for reward weights. Where k = {H, L}; through the formula Update the output node parameters of the tooling time series feature extraction model. t The output node parameters O are obtained.t+1 ;
[0111] B8. The tooling time-series feature extraction model extracts the tooling time-series features from the tooling historical time-series data at time t+1 as t+1 , and the real-time operation state feature rs of the tooling at time t+1 t+1 constitute the state s at time t+1 t+1 ={as t+1 , rs t+1}, and store it in the experience record <s t , a t , r t , ω>;
[0112] B9. Add the high-reward-weight experience record <s t , a t , r t , ω H , s t+1 > to the experience pool R H ; add the low-reward-weight experience record <s t , a t , r t , ω L , s t+1 > to the experience pool R L ;
[0113] B10. Until the number of experience entries in the experience pool R H and R L reaches the set number, proceed to step B11; otherwise, set t = t+1 and return to step B4;
[0114] B11. Randomly sample N experience records <s H in R L and R t , a t , r t , ω, s t+1 > as the mini-batch training data for the online evaluation network Q and the online policy network μ;
[0115] B12. Use deterministic policy gradient to update the parameter θ μ of the online policy network μ, and the calculation formula is as follows:
[0116]
[0117] B13. Use the minimized mean square error loss adjusted by the reward weight to update the parameter θ Q of the online evaluation network Q and the parameter θ L of the tooling time-series feature extraction model, where k = {H, L}, y i=r i +Q′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ );
[0118] B14. Every C steps, update the parameters θ of the target evaluation network Q′ and the target policy network μ′ using a moving average method. Q′ and θ μ′ The formula is as follows:
[0119] θ Q′ ←τθ Q +(1-τ)θ Q′
[0120] θ μ′ ←τθ μ +(1-τ)θ μ′
[0121] B15. Determine whether the system has reached the termination state. If it has reached the termination state, proceed to step B16. If it has not reached the termination state, set t = t + 1 and return to step B4.
[0122] B16. Determine whether episode has reached the set value. If it has, output the current optimal tooling timing feature extraction model and tooling replacement strategy dynamic search model. Otherwise, set episode = episode + 1 and return to step B3.
[0123] In the above, the tooling label data C at time t is extracted using the tooling time-series feature extraction model. t and tooling timing characteristics as t The process is as follows:
[0124] A1. Obtain the tooling status data x from time t. t Tooling label data C at time t-1 t-1 The timing characteristics of the tooling at time t-1 before the time step t-1 t-1 Input into the tooling time-series feature extraction model;
[0125] A2. The tooling time-series feature extraction model performs the following extraction process:
[0126] Based on the tooling status data x at time t before t The timing characteristics of the tooling at time t-1 before the time step t-1 t-1 Combined with equation (2), the feature storage vector f of the tooling time-series feature extraction model is obtained. t ;
[0127] Based on the tooling status data x at time t before t The timing characteristics of the tooling at time t-1 before the time step t-1 t-1Combine equations (3) and (5) to obtain the data input node i respectively. t and data output node o t ;
[0128] Based on the tooling status data x at time t before t The timing characteristics of the tooling at time t-1 before the time step t-1 t-1 Combine equation (4) to obtain the cell status update value.
[0129] Feature storage vector f based on tooling time-series feature extraction model t Tooling label data C at time t-1 t-1 Data input node i t Cell status update value Combine equation (1) to obtain the tooling label data C at time t. t ;
[0130] Based on data output node o t And the tooling label data C at time t t Combining equation (6), we obtain the implicit data at time t, namely the tooling timing characteristics as at time t. t .
[0131] S4. Dynamically search for tooling replacement strategies by combining the optimized tooling time-series feature extraction model and the tooling replacement strategy dynamic search model, such as... Figure 3 As shown, the process includes:
[0132] C1. Input the historical time series data of the tooling X1 and the real-time multi-source production data of the tooling X2;
[0133] C2. Load the tooling time-series feature extraction model and the tooling replacement strategy dynamic search model;
[0134] C3. The tooling time series feature extraction model uses the tooling historical time series data X1 and the tooling label data C saved at time t-1. t-1 The timing characteristics of the tooling saved at time t-1 are as t-1 Extract and save the tooling label data C at time t. t and tooling timing characteristics as t The real-time multi-source production data of the tooling (X2) is preprocessed to obtain the real-time operating status characteristics of the tooling (rs). t , as t with rs t The state s that constitutes the dynamic search model of tooling change strategy t ={as t rs t} and enter;
[0135] C4. The target strategy network of the dynamic search model for tooling replacement strategy selects the current tooling replacement decision action a. out =μ′({as t rs t}|θ μ′ ), and output;
[0136] C5. The online strategy network of the tooling change strategy dynamic search model selects the current tooling change decision action a. t =μ({as t rs t}|θ μ Online evaluation of network execution actions a t Receive reward r t ; Determine the reward r t Is it greater than the reward threshold r? Value If it is greater than s, then record it in the experience record <s t a t r t > Add high reward weight ω H Add to experience pool R H If it is less than, then in the experience record <s t a t r t > Add low reward weight ω L Add to experience pool R L middle;
[0137] C6, until it is stored in the experience pool R H and R L Once the number of experience entries reaches the set quantity, proceed to step (7); otherwise, end the tooling change strategy search process.
[0138] C7, in the experience pool R H With R L Randomly collect N experience records for {<s t a t r t ω t >, <s t+1 a t+1 r t+1 ω t+1 Reconstruct the data to generate N experience records <s. t a t r t ω t s t+1 > Mini-batch training data for the online evaluation network Q and the online policy network μ;
[0139] C8. Update the parameters θ of the online policy network μ using a deterministic policy gradient.u The calculation formula is as follows:
[0140]
[0141] C9. Minimize the mean squared error using reward-weighted adjustments. Update the parameter θ of the online evaluation network Q. Q Where k = {H, L}, y i =r i +Q′(s i+1 ,μ′(s i+1 |θ μ′ )|θ Q′ );
[0142] C10. Every C steps, update the parameters θ of the target evaluation network Q′ and the target policy network μ′ using a moving average method. Q′ and θ μ′ The formula is as follows:
[0143] θ Q′ ←τθ Q +(1-τ)θ Q′
[0144] θ μ′ ←τθ μ +(1-τ)θ μ′
[0145] C11. End the tooling change strategy search process.
[0146] In the training process of the dynamic search model for tooling replacement strategy in this embodiment, the online policy network is used to dynamically learn the features in the tooling life prediction dataset, and the online evaluation network is used to dynamically evaluate the learning effect of the online policy network. To prevent the model from learning too much noise, the parameters in the online policy network and the online evaluation network are loaded into the target policy network and the target evaluation network using a moving average method, ultimately outputting the optimal dynamic search model for tooling replacement strategy. To prevent the model from learning too many features of the normal tooling state, resulting in insensitivity to the features of the abnormal tooling state, this model introduces a high-reward replay experience pool and a low-reward replay experience pool, storing high-reward and low-reward replay experience data respectively, and assigning high-reward weights and low-reward weights to these two types of data, enabling the model to learn more feature values of the abnormal tooling state and become more sensitive to the abnormal tooling state.
[0147] The tooling time-series feature extraction model is used to extract the time-series features of the tooling. During the training process of the tooling time-series feature extraction model, in order to fully utilize the time-series features of the tooling and meet the requirement of improving the prediction accuracy of the remaining useful life of the tooling, the parameters of the tooling time-series feature extraction model are optimized using a minimum mean square error loss optimizer based on the prediction and evaluation results of the remaining useful life of the tooling from the online policy network and online evaluation network. Finally, the optimal tooling time-series feature extraction model is output.
[0148] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Therefore, any changes made in accordance with the shape and principle of the present invention should be covered within the protection scope of the present invention.
Claims
1. A dynamic search method for tooling change strategies, characterized in that, include: Construct a tooling time-series feature extraction model; Construct a dynamic search model for tooling replacement strategies that includes a tooling operation state space and a replacement strategy space; The tooling time-series feature extraction model and the tooling change strategy dynamic search model were trained and optimized. Dynamic search for tooling replacement strategies is performed by combining an optimized tooling time-series feature extraction model and a tooling replacement strategy dynamic search model. The constructed tooling time-series feature extraction model is as follows: ； in, This refers to the tooling label data at time t. The feature storage vector for the tooling time-series feature extraction model is used to represent Which features will be used to calculate? ; , These are the data input node, data output node, and output node, respectively. Used from Extract tooling timing features ; , , This is the weight matrix; It is the bias vector; The cell state is updated using the hyperbolic tangent function as its activation function; This is a normalization function to ensure that the parameter values of the input node, output node, and feature storage vector are between [0,1]. The tooling timing characteristics at time t-1; The tooling timing characteristics at time t; Construct a dynamic search model for tooling change strategies, including: Define the tooling life prediction decision state space Tooling timing characteristics and the real-time operating status characteristics of the tooling Together they constitute: ； in, For tooling timing characteristics, This refers to the real-time operating status characteristics of the tooling; The online policy network of the tooling change strategy dynamic search model outputs tooling change decisions based on the decision state space. To enable the model to handle multiple devices and multiple tooling, an action space is defined that outputs a tooling change strategy at each step. : ； Where n is the number of tooling fixtures; c is the cycle duration factor of the tooling replacement cycle in this round; The remaining time factor of the remaining useful life of tooling number n after normalization by the normalized exponential function. The online evaluation network uses an estimated value function to quantify the value of the tooling replacement decision, and the online policy network adjusts its own parameters based on the evaluation results of the current tooling replacement decision. The objective of the reward function is defined as follows: the closer the predicted value is to the actual value, the greater the reward, and vice versa. A reward function that comprehensively considers both tooling value and product value. As shown in the following formula: ； in, The weighting coefficients for the scenarios of replacing tooling in advance and not replacing tooling in a timely manner. The value of the tooling currently loaded on device i. The actual service life of the current tooling of equipment i. The value of the products currently produced by equipment i. The production cycle time of the current product being produced by equipment i.
2. The tooling change strategy dynamic search method according to claim 1, characterized in that, During a training cycle, the tooling change strategy dynamic search model first extracts the tooling time-series features from the tooling time-series feature extraction model. Real-time operating status characteristics of tooling Constituent state The results are used as input; then, based on the tooling replacement strategy results and evaluation results, the parameters of the tooling replacement strategy dynamic search model and the tooling temporal feature extraction model are updated through the action loss function of reward weight adjustment, the minimum mean square error loss of reward weight adjustment, and the deterministic strategy gradient; finally, when the number of iterations reaches the preset number or the model output tends to stabilize, the trained tooling temporal feature extraction model and tooling replacement strategy dynamic search model are output.
3. The tooling change strategy dynamic search method according to claim 2, characterized in that, The specific training optimization process includes: B1. Input the historical observation dataset of the tooling. ; B2. Initialize the tooling time-series feature extraction model and output node parameters. , Other parameters of the tooling time series feature extraction model Initialize the online evaluation network Online policy network Network parameters and Initialize the target evaluation and target policy network. and Parameters: and Initialize action space Experience Pool Initialize the episode parameter to 0; Set the number of samples for batch gradient descent. Discount rate Soft update coefficient Learning rate Parameter update interval Weighting coefficients ; B3. Initialize the tooling state space Initialize parameter t=0; B4. If t=0, then set the tooling timing characteristics. Otherwise, the tooling time series feature extraction model uses the tooling historical time series data at time t. Tooling label data saved at time t-1 Time series feature data saved at time t-1 Extract and save the tooling label data at time t. Tooling timing characteristics Real-time multi-source production data from tooling Real-time operating status characteristics of the tooling obtained through data preprocessing Tooling timing characteristics Features of the tooling's real-time operating status at time t State at time t ; B5. The online strategy network of the tooling change strategy dynamic search model is based on the current state of the tooling. Based on the set search strategy and the noise level during exploration, the current tooling replacement decision action is selected. ; B6. Online evaluation network execution actions of the dynamic search model for tooling change strategies. Receive rewards ; Determine the reward Is it greater than the reward threshold? If it is greater than, then record it in the experience log. Add high reward weights to If it is less than, then record it in the experience log. Add low reward weights ; B7. Calculate the action loss function adjusted for reward weights. ,in ; through formula , Update the output node parameters of the tooling time series feature extraction model To obtain the output node parameters ; B8. Tooling Time-Series Feature Extraction Model: Extracts tooling time-series features from historical time-series data of the tooling at time t+1. Features of the tooling's real-time operating status at time t+1 State at time t+1 Save experience records middle; B9. Record high-reward weighted experience. Add to experience pool In the middle, low-reward weighted experience records will be recorded. Add to experience pool middle; B10, until it is stored in the experience pool. and Once the number of experience points reaches the set quantity, proceed to step B11; otherwise, set t=t+1 and return to step B4. B11, in the experience pool and Randomly collect N experience records As an online rating network Online policy network Mini-batch training data; B12. Using deterministic policy gradient updates for online policy networks. parameters The calculation formula is as follows: ； B13. Minimize the mean squared error using reward-weighted adjustments. Update online review network parameters Tooling time series feature extraction model parameters ,in , ; B14. Update the target evaluation network every C steps using a moving average method. and target policy network parameters and The formula is as follows: ； ； B15. Determine whether the system has reached the termination state. If it has reached the termination state, proceed to step B16. If it has not reached the termination state, set t=t+1 and return to step B4. B16. Determine whether episode has reached the set value. If it has, output the current optimal tooling timing feature extraction model and tooling replacement strategy dynamic search model. Otherwise, set episode = episode + 1 and return to step B3.
4. The tooling change strategy dynamic search method according to claim 3, characterized in that, Tooling label data at time t is extracted using a tooling time-series feature extraction model. Tooling timing characteristics The process is as follows: A1. Obtain the tooling status data from time t. Tooling label data at time t-1 Tooling timing characteristics at time t-1 Input into the tooling time-series feature extraction model; A2. The tooling time-series feature extraction model performs the following extraction process: Based on the tooling status data at time t Tooling timing characteristics at time t-1 Combined with equation (2), the feature storage vector of the tooling time-series feature extraction model is obtained. ; Based on the tooling status data at time t Tooling timing characteristics at time t-1 Combine equations (3) and (5) to obtain the data input nodes respectively. and data output node ; Based on the tooling status data at time t Tooling timing characteristics at time t-1 Combined with equation (4), the cell status update value is obtained. ; Feature storage vector based on tooling time-series feature extraction model Tooling label data at time t-1 Data input node Cell status update value Combine equation (1) to obtain the tooling label data at time t. ; Based on data output node and the tooling label data at time t Combining equation (6), we obtain the implicit data at time t, that is, the tooling timing characteristics at time t. .
5. The tooling change strategy dynamic search method according to claim 4, characterized in that, Dynamic search for tooling replacement strategies is performed by combining an optimized tooling time-series feature extraction model and a dynamic tooling replacement strategy search model, including: C1. Input historical time series data of tooling Real-time multi-source production data of tooling ; C2. Load the tooling time-series feature extraction model and the tooling replacement strategy dynamic search model; C3. Tooling Time Series Feature Extraction Model extracts historical time series data from tooling. Tooling label data saved at time t-1 The timing characteristics of the tooling saved at time t-1 Extract and save the tooling label data at time t. Tooling timing characteristics Real-time multi-source production data from tooling Real-time operating status characteristics of the tooling obtained through data preprocessing , and The state constituting the dynamic search model of tooling change strategy And enter; C4. The target policy network of the dynamic search model for tooling replacement strategy selects the current tooling replacement decision action. and output; C5. The online strategy network of the tooling change strategy dynamic search model selects the current tooling change decision action. Online evaluation of network execution actions Receive rewards ; Determine the reward Is it greater than the reward threshold? If it is greater than, then record it in the experience log. Add high reward weights to Add to experience pool If it is less than, then it is recorded in the experience record. Add low reward weights Add to experience pool middle; C6. Until it is stored in the experience pool and Once the number of experience entries reaches the set quantity, proceed to step (7); otherwise, end the tooling replacement strategy search process. C7. In the experience pool and Randomly collect N experience records to pair Refactor to generate N experience records. As an online rating network Online policy network Mini-batch training data; C8. Using deterministic policy gradient updates for online policy networks. parameters The calculation formula is as follows: ； C9. Minimize the mean squared error using reward-weighted adjustments. Update online review network parameters ,in , ; C10. Update the target evaluation network every C steps using a moving average method. and target policy network parameters and The formula is as follows: ； ； C11. End the tooling change strategy search process.