A profile composite threading system and threading control method

Abstract

The present application relates to the technical field of thermal insulation profile processing, and particularly discloses a profile composite threading system and a threading control method, wherein the method comprises the following steps: S1, model construction and initialization; S2, real-time data acquisition and state construction; S3, forward-looking prediction and state extension; S4, deep reinforcement learning control decision; and S5, execution and path tracking. The present application constructs a complete closed loop integrating intelligent perception, forward-looking prediction and dynamic cooperative control, realizes the intelligentization, stabilization and high reliability of the threading process through algorithm deep fusion and step logic closed loop, and effectively improves the production efficiency and product quality.

Description

Technical Field

[0001] This invention relates to the field of thermal insulation profile processing technology, and in particular to a profile composite strip insertion system and strip insertion control method. Background Technology

[0002] In the manufacturing of building doors and windows, curtain walls, and industrial frames, profile insertion is a crucial process. It involves inserting flexible strips such as thermal break strips and sealing strips into the pre-designed cavities of aluminum alloy or plastic profiles to achieve structural reinforcement, sealing, and thermal insulation. The efficiency and quality of this process directly affect the performance and manufacturing cost of the final product. Currently, the commonly used strip insertion control methods in the industry mainly rely on two types of technologies: one is simple control based on preset programs, which sets fixed insertion speed curves for different profile types based on experience. This method is completely unable to adapt to local geometric changes in the same profile cavity caused by processing errors, deformation, or foreign objects; the other is PID control combined with sensor feedback. Although it can fine-tune the speed based on real-time resistance, it is essentially a passive and lagging response mechanism. Because the geometry of the profile cavity is often complex and variable, with sudden necking, bending, or bifurcation, and the inserted strips have significant flexible deformation characteristics, the resistance dynamics during the insertion process exhibit a high degree of nonlinearity, time-varying nature, and uncertainty. The aforementioned traditional methods all lack the ability to predict the working conditions ahead, and can only react after the resistance has abnormally increased (even to the point of strip buckling or equipment overload alarms). This can easily lead to strip jamming, strip surface damage or permanent deformation, and even equipment shutdown. This not only causes production interruptions and material waste, but also seriously restricts the development of strip threading technology towards higher speeds, more complex profiles, and full automation. Summary of the Invention

[0003] To address the problem of insufficient predictive capability for profile threading in existing technologies, this invention provides a composite profile threading system and threading control method.

[0004] The technical solution adopted in this invention is:

[0005] A method for controlling the insertion of composite strips in profiles includes the following steps:

[0006] S1. Establish a digital twin model of the strip-threading process, which includes a strip-threading resistance prediction sub-model based on a long short-term memory network and a strip-threading controller model based on a deep deterministic policy gradient network; collect historical strip-threading process data to train the strip-threading resistance prediction sub-model, and use a simulation environment to train the strip-threading controller model; set the initial strip-threading target speed. and system control cycle ;

[0007] S2, in a single control cycle Within the system, data from the current threading stage is collected in real time, and the data includes at least:

[0008] The preset distance in front of the profile's inner cavity is collected by a micro-laser sensor located at the front end of the strip. Contour point cloud data ,in, , and Points The coordinates of the x, y, and z axes in the coordinate system of the micro-laser sensor itself;

[0009] The current actual threading speed is obtained through the encoder of the threading machine spindle. ;

[0010] The current measured threading resistance is obtained from the strain sensor on the thrust shaft of the threading machine. ;

[0011] The contour point cloud data is denoised and feature extracted to obtain a feature vector representing the geometric complexity of the inner cavity of the front profile. Construct the current state vector , ,in, The elastic modulus of the strip. The coefficient of friction between the strip and the profile. The length of the strip that has been inserted;

[0012] S3, the feature vector Current actual threading speed Material parameters of the strip and Input the trained strip resistance prediction sub-model to predict the future. Predicted value of the piercing force after one control cycle Extend the state vector to ;

[0013] S4. Expand the state vector The trained strip-threading controller model is input, and its Actor network directly outputs the optimal speed adjustment action for the current state. The output action of the strip-threading controller model As the speed adjustment amount of the servo motor of the threading machine;

[0014] S5, drive the threading machine in the next control cycle Inside, according to speed Run; in the next control cycle Initially, the deviation angle of the actual insertion path of the bar is obtained by an attitude sensor located at the tail of the bar. ;If the deviation angle Exceeding the preset tolerance Then according to the deviation angle and the feature vector The scanning focus center plane of the macro laser sensor is dynamically corrected, and the process returns to step S2; if the deviation angle... Not exceeding the preset tolerance If so, steps S2 to S4 will continue to be executed in the next control cycle until the threading process is completed.

[0015] Preferably, in step S1, the strip resistance prediction sub-model is a long short-term memory network model, and its training process includes:

[0016] S11. Arrange the historical striping process data according to time steps. The data is divided into samples, and the input features of each sample are... For: at time eigenvectors Threading speed , bar elastic modulus coefficient of friction Output tags For: moment Measured resistance to threading ;

[0017] S12, for the first Long Short-Term Memory Network at time The calculation is as follows:

[0018] Input gate ;

[0019] Forgotten Gate ;

[0020] Candidate cell status ;

[0021] Output gate ;

[0022] Current cell state ;

[0023] Current hidden state ;

[0024] in, Enter the current time. This is the hidden state from the previous moment. This represents the cell state at the previous moment. It is the sigmoid activation function. The hyperbolic tangent activation function is used. For Hadama accumulation, , , , , , , , These are the input weight matrix and the cyclic weight matrix for the corresponding gate, respectively. , , , These are the bias vectors for the corresponding gates;

[0025] S13. Using mean squared error as the loss function The network weights are adjusted using the time backpropagation algorithm and optimizer until the loss converges.

[0026] Preferably, in step S1, the strip-threading controller model is trained using a deep deterministic policy gradient algorithm framework, including an Actor network. and Critic Network ,in, For the Actor policy function, based on the current state Output Action Right now , The extended state vector constructed in step S2 , Here are the weight matrices and bias vectors for all layers of the Actor network. The action value function evaluates the value of an action in a given state. Next, execute the action. The expected long-term cumulative rewards that can be obtained afterward. This is the set of weight parameters for the Critic neural network, and its offline training process is performed in a simulation environment, including:

[0027] S14. Establish a simulation environment for the profile threading process based on a physics engine, capable of receiving speed adjustment actions. And calculate the state at the next moment. ,award And the marker indicating whether the process has ended (done);

[0028] S15. Explore the environment and collect the transfer samples generated by each interaction. Store in the experience replay buffer , For a moment Speed ​​adjustment action;

[0029] S16, from the buffer Randomly sample a small batch of samples and update the network according to the following steps:

[0030] a. Calculation target value ,in, As a discount factor, and These are the target Actor and Critic networks, respectively; the loss function is minimized. To update the current Critic network parameters ;

[0031] b. Using the policy gradient theorem, maximize the expected return. The Actor network is updated using an approximation, and the formula is: ;

[0032] c. Use , Perform a soft update on the target network, whereby... For soft update parameters;

[0033] S17. Repeat steps S15 and S16 until the policy converges, and obtain the trained Actor network as the strip controller model.

[0034] Preferably, in step S2, the feature vector The extraction method is as follows: for a single control cycle Contour point cloud data acquired internally Calculate its surface area in three-dimensional space ,volume Central Moment and the principal direction vector of the point cloud. ;Will Constructing an 8-dimensional feature vector ,in, , and These are the principal direction vectors. Components on the three coordinate axes.

[0035] Preferably, in the simulation environment of step S15, a reward function is designed. The formula is ,in, , , and These are the weighting coefficients for each reward or punishment. For the target speed, As a current obstacle, For the safety resistance threshold, This represents the currently estimated axial compressive deformation of the strip. For ideal deformation variables.

[0036] Preferably, in step S2, the axial compressive deformation of the strip is estimated. The formula is ,in, Let be the cross-sectional area of ​​the strip.

[0037] Preferably, in step S5, the step of determining the deviation angle... and the feature vector The method for dynamically correcting the scanning focus center plane of a macro laser sensor is as follows: Let the normal vector of the original focus center plane of the sensor be... The forward direction vector of the strip theory is Actual path deviation angle The forward direction vector of the strip theory With respect to the actual direction vector of the strip The included angle; the corrected normal vector of the focal center plane. From the formula ,in, For rotation around the axis Rotation Rotation matrix of angle, rotation axis , For vector normalization, For feature vectors The projection vector of the predicted centroid of the front cavity region in the sensor coordinate system. Let be the center vector of the sensor's field of view. This is the gain coefficient.

[0038] Preferably, both the Actor network and the Critic network of the strip controller model are fully connected feedforward neural networks. The output layer of the Actor network uses the tanh activation function to restrict the action to the range of [−1, 1], and converts it into the actual speed adjustment range through a linear mapping. ,in, and These represent the maximum allowable negative and maximum allowable positive speed adjustment values ​​of the servo motor of the threading machine within a single control cycle.

[0039] A composite strip threading system for profiles includes a data acquisition module, a state construction and prediction module, a deep reinforcement learning control decision module, an execution driving module, and a path perception and correction module;

[0040] The data acquisition module includes:

[0041] A macro laser sensor, positioned at the front end of the strip, is used to collect data at a preset distance from the front of the profile's inner cavity. Contour point cloud data ;

[0042] An encoder, connected to the main shaft of the threading machine, is used to obtain the current actual threading speed. ;

[0043] A strain sensor, mounted on the thrust shaft of the threading machine, is used to obtain the current measured threading resistance. ;

[0044] An attitude sensor, located at the tail of the bar, is used to obtain the deviation angle of the actual insertion path of the bar. ;

[0045] The state construction and prediction module includes:

[0046] The feature extraction unit, communicatively connected to the macro laser sensor, is used to perform noise reduction and feature extraction on the contour point cloud data, and output a feature vector characterizing the geometric complexity of the inner cavity of the front profile. ;

[0047] The state construction unit, which is communicatively connected to the feature extraction unit, encoder, and strain sensor, is used to receive the feature vector. Current actual threading speed Current measured resistance to threading Obtain the elastic modulus of the strip Obtain the coefficient of friction between the strip and the profile. and get the length of the bar that has been inserted Construct the current state vector ;

[0048] The resistance prediction unit incorporates a sub-model for predicting the resistance of the threaded bar, which is a model trained based on a long short-term memory network. The resistance prediction unit is communicatively connected to the state construction unit and is used to receive the feature vector. Current actual threading speed Current measured resistance to threading , bar elastic modulus coefficient of friction between the strip and the profile and output the future Predicted resistance value after one control cycle The state construction unit is also used to convert the predicted resistance value. The state vector is incorporated to form an extended state vector. ;

[0049] The deep reinforcement learning control decision module includes:

[0050] The controller has a built-in Actor network trained based on a deep deterministic policy gradient algorithm. The controller is communicatively connected to the state construction and prediction module and is used to receive the extended state vector. The Actor network directly outputs the optimal speed adjustment action. ;

[0051] The execution driver module includes:

[0052] A servo driver is electrically connected to the controller and the servo motor of the threading machine; the servo driver is used to receive the optimal speed adjustment action. And drive the servo motor in the next control cycle Inside, according to speed run;

[0053] The path awareness and correction module includes:

[0054] The judgment and calculation unit is communicatively connected to the attitude sensor and the feature extraction unit, and is used to receive the deviation angle. and the feature vector And in determining the deviation angle Exceeding the preset tolerance At that time, according to the deviation angle and the feature vector The correction command for the scanning focusing center plane of the micro-laser sensor is calculated;

[0055] The execution unit, connected to the judgment and calculation unit and the macro laser sensor control, is used to execute the correction command and dynamically adjust the scanning focus center plane of the macro laser sensor.

[0056] The beneficial effects of this invention are:

[0057] 1. By establishing a digital twin model of the strip threading process and training it based on historical data, the system is able to learn the mapping relationship of complex working conditions from the data, which significantly enhances its adaptability and generalization ability when facing different profiles and strips, and gets rid of the dependence on fixed parameters and human experience.

[0058] 2. By collecting the profile of the front profile cavity, current resistance and speed in real time, and using the model to predict future resistance, the system has achieved a shift from passive response to active adaptation. It can predict the upward trend of resistance in advance and adjust the speed, thereby reducing the risk of strip jamming and equipment overload.

[0059] 3. Through the speed-deformation coordinated control steps, while adjusting the speed, the axial compressive deformation of the strip is estimated and constrained in real time, which effectively prevents the strip buckling or plastic deformation caused by pursuing speed, and ensures the structural integrity of the product.

[0060] 4. Through path tracking and dynamic correction, the system can compensate for trajectory deviations during the threading process in real time, ensuring the accuracy of the threading path and improving the success rate of threading operations.

[0061] 5. A complete closed loop integrating intelligent sensing, forward prediction and dynamic collaborative control was constructed. Through deep algorithm integration and step logic closed loop, the intelligent, stable and highly reliable strip-threading process was achieved, effectively improving production efficiency and product quality. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating the composite strip control method for profiles in an embodiment of the present invention;

[0063] Figure 2 This is a schematic diagram of the module structure of the profile composite strip threading control system in an embodiment of the present invention;

[0064] Reference numerals: 10, Data acquisition module; 11, Micro-laser sensor; 12, Encoder; 13, Strain sensor; 14, Attitude sensor; 20, State construction and prediction module; 21, Feature extraction unit; 22, State construction unit; 23, Resistance prediction unit; 30, Deep reinforcement learning control decision module; 31, Controller; 40, Execution drive module; 41, Servo driver; 50, Path perception and correction module; 51, Judgment and calculation unit; 52, Execution unit; 60, Servo motor. Detailed Implementation

[0065] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0066] Example 1:

[0067] A method for controlling the insertion of composite strips in profiles includes the following steps:

[0068] S1. Model Construction and Initialization: Establish a digital twin model of the strip-threading process, which includes a strip-threading resistance prediction sub-model based on a long short-term memory network and a strip-threading controller model based on a deep deterministic policy gradient network; collect historical strip-threading process data to train the strip-threading resistance prediction sub-model, and use a simulation environment to train the strip-threading controller model; set the initial strip-threading target speed. and system control cycle Among them, the system control cycle The value can range from 10ms to 100ms;

[0069] S2, Real-time Data Acquisition and Status Construction: Within a single control cycle Within the system, data from the current threading stage is collected in real time, and the data includes at least:

[0070] The preset distance in front of the profile's inner cavity is collected by a micro-laser sensor located at the front end of the strip. Contour point cloud data ,in, , and Points The coordinates of the x, y, and z axes in the coordinate system of the micro-laser sensor itself;

[0071] The current actual threading speed is obtained through the encoder of the threading machine spindle. ;

[0072] The current measured threading resistance is obtained from the strain sensor on the thrust shaft of the threading machine. ;

[0073] The contour point cloud data is denoised and feature extracted to obtain a feature vector representing the geometric complexity of the inner cavity of the front profile. Construct the current state vector , ,in, The elastic modulus of the strip. The coefficient of friction between the strip and the profile. The length of the strip that has been inserted;

[0074] S3. Forward Prediction and State Extension: The feature vector... Current actual threading speed Material parameters of the strip and Input the trained strip resistance prediction sub-model to predict the future. Predicted value of the piercing force after one control cycle Among them, the number of forward prediction steps The value range of can be 3 to 10; the state vector is extended to ;

[0075] S4. Deep Reinforcement Learning Control Decision: The expanded state vector... The trained strip-threading controller model is input, and its Actor network directly outputs the optimal speed adjustment action for the current state. The output action of the strip-threading controller model As the speed adjustment amount of the servo motor of the threading machine;

[0076] S5, Execution and Path Tracking: Drives the threading machine in the next control cycle. Inside, according to speed Run; in the next control cycle Initially, the deviation angle of the actual insertion path of the bar is obtained by an attitude sensor located at the tail of the bar. ;If the deviation angle Exceeding the preset tolerance Then according to the deviation angle and the feature vector The scanning focus center plane of the macro laser sensor is dynamically corrected, and the process returns to step S2; if the deviation angle... Not exceeding the preset tolerance If so, steps S2 to S4 will continue to be executed in the next control cycle until the threading process is completed.

[0077] Existing control methods mostly rely on preset rules or traditional PID, which are difficult to adaptively handle the complex nonlinear and time-varying dynamic characteristics during the threading process.

[0078] This method replaces the traditional controller with a deep reinforcement learning controller, enabling it to learn the optimal control strategy by interacting with the environment. Based on the multi-dimensional state, including prediction information, it outputs continuous and precise speed adjustment actions, achieving better dynamic performance.

[0079] This method constructs a novel closed loop of perception-prediction-intelligent decision-making-execution. Steps S1-S3 provide rich state information, including real-time data and prediction information. Step S4 is the core innovation, using an Actor network of deep reinforcement learning models such as deep deterministic policy gradients as the controller. This network, through extensive trial and error during the training phase, has learned to process data from high-dimensional continuous states. To continuous action The optimal mapping strategy; during the deployment phase, it can evaluate the state in real time, including the current speed, resistance, forward geometry, and predicted future resistance, and instantly output a control action that fully considers efficiency, safety, and smoothness. Compared with PID, which requires manual parameter tuning, and fuzzy control, which relies on rule bases, deep reinforcement learning controllers are data-driven and autonomously optimized. They can automatically handle the complex coupling relationships between various state variables, achieve superior adaptive control and dynamic optimization, and are particularly good at handling nonlinear, partially observable dynamic processes such as threading.

[0080] Example 2:

[0081] Simple linear models or empirical formulas cannot handle the highly nonlinear spatiotemporal relationship between the complex geometry of the profile cavity and the resistance. To enable the control system to accurately predict future strip resistance, this embodiment, based on Embodiment 1, uses a Long Short-Term Memory (LSTM) network model in step S1. Its training process includes:

[0082] S11. Construct the training dataset: Organize the historical striping process data according to time steps. The data is divided into samples, and the input features of each sample are... For: at time eigenvectors Threading speed , bar elastic modulus coefficient of friction Output tags For: moment Measured resistance to threading ;

[0083] S12. Define the forward propagation process of the Long Short-Term Memory (LSTM) network model: For the... Long Short-Term Memory Network at time The calculation is as follows:

[0084] Input gate ;

[0085] Forgotten Gate ;

[0086] Candidate cell status ;

[0087] Output gate ;

[0088] Current cell state ;

[0089] Current hidden state ;

[0090] in, Enter the current time. This is the hidden state from the previous moment. This represents the cell state at the previous moment. It is the sigmoid activation function. The hyperbolic tangent activation function is used. For Hadama accumulation, , , , , , , , These are the input weight matrix and the cyclic weight matrix for the corresponding gate, respectively. , , , These are the bias vectors for the corresponding gates;

[0091] S13. Model Training: Using mean squared error as the loss function. The network weights are adjusted using the time backpropagation algorithm and optimizer until the loss converges.

[0092] Long Short-Term Memory (LSTM) network models are particularly suitable for processing time-series data and their long-term dependencies. In step S11, the strip-wearing process is serialized, with multi-source data such as contour features, velocity, and material parameters as inputs and future resistance as output, training the network to learn the complex mappings. In step S12, the core computational process of the LTM network model is defined in detail, through the input gate... Controlling the ingestion of new information, the forgetting gate Controlling the discarding of old memories, output gate Controls the generation of the current output, cell state. Bearing historical information, these gating mechanisms enable the model to effectively memorize the long-term impact of the complexity of the preceding contour on resistance, such as a long, complex cavity structure. After training, the model can output predictions for the future based on the current real-time perceived conditions. Accurate prediction of resistance after a step provides crucial future state information for subsequent intelligent control decisions.

[0093] In step S11, data augmentation processing can be performed on the historical strip-threading process data. The processing includes: adding random Gaussian noise to the profile inner cavity contour sequence to simulate measurement error, and randomly scaling the strip-threading resistance sequence within a preset scale to simulate different friction states, thereby generating an expanded training sample set.

[0094] Example 3:

[0095] To obtain an intelligent controller capable of replacing traditional controllers, handling continuous motion spaces, and automatically learning complex control strategies, the basic embodiment, based on embodiment 2, in step S1, trains the strip controller model using a deep deterministic policy gradient algorithm framework, including an Actor network. and Critic Network ,in, Let Actor be the policy function, which is a neural network that determines the policy based on the current state. Output a defined, continuous action. That is, the speed adjustment amount output by the Actor network. , The extended state vector containing prediction information constructed in step S2 , This is the set of weight parameters for the Actor neural network, containing the weight matrices and bias vectors from all layers of the Actor network. The goal of training is to optimize these parameters. This makes the strategy It can output actions that maximize long-term cumulative rewards. Let be the action value function, and let be a neural network that evaluates the action value in a given state. Next, execute the action. After that, the expected long-term cumulative reward that can be obtained is value, This is the set of weight parameters for the Critic neural network; the goal of training is to optimize these parameters. Actor Network It's a controller; it adjusts based on the state. Make a decision and output an action Critic Network It is the evaluator, which evaluates the decisions made by the Actor (in the state). Next action The quality of output (Value), the offline training process is carried out in a simulation environment, including:

[0096] S14. Simulation Environment Construction: A simulation environment for the profile threading process is established based on a physics engine, capable of receiving speed adjustment actions. And calculate the state at the next moment. ,award And the marker indicating whether the process has ended (done);

[0097] S15, Experience Replay: The agent explores the environment and records the transition samples generated by each interaction. Store in the experience replay buffer , For a moment Speed ​​adjustment action;

[0098] S16, Network Training: From Buffer Randomly sample a small batch of samples and update the network according to the following steps:

[0099] a. Critic network update: Calculation target value ,in, This is a discount factor, typically ranging from (0,1). It determines the degree to which future rewards diminish in value at the current moment and is used to calculate long-term cumulative rewards. For target Actor network Its network structure is similar to the main Actor network. Same, but parameters It updates slowly and is used to calculate stable target values ​​during the training process. For the target Critic network Its network structure is similar to the main Critic network. Same, but parameters Slower updates, used to provide stable targets Value; by minimizing the loss function To update the current Critic network parameters ;

[0100] b. Actor Network Update: Using the policy gradient theorem, by maximizing the expected return. The Actor network is updated using an approximation, and the formula is: ,in, The gradient operator represents taking the derivative of a function to obtain the direction of the fastest change of the function in each dimension of its parameter space, i.e., the gradient direction. For the objective function Relative to Actor network parameters The gradient;

[0101] c. Target network soft update: using , Perform a soft update on the target network, whereby... For soft update parameters;

[0102] S17. Repeat steps S15 and S16 until the policy converges, and obtain the trained Actor network as the strip controller model.

[0103] The strip control problem is modeled as a Markov decision process, and a neural network is used to approximate the optimal policy and state-action value function. Experience replay breaks down temporal correlations between data points, improving data utilization efficiency and training stability. The Critic network learns to accurately evaluate the long-term value of performing an action in a given state. The Actor network, on the other hand, operates based on the gradient direction provided by the Critic network. The Critic network updates its own parameters to ensure that its output actions yield higher long-term rewards. The target network and soft updates stabilize the training process; through continuous interactive training, the Critic network guides the Actor network to update its parameters. Ultimately, this enables the Actor network to learn to make decisions that yield the highest evaluation, i.e., the greatest long-term reward, and the target network. and It is an auxiliary network used to stabilize learning during training. After training is completed, the Actor network... It is itself a highly efficient nonlinear controller, capable of handling complex states. Directly mapped to continuous, fine-grained optimal control actions. The above steps achieve fully data-driven and adaptive optimal control, eliminating the need for manual design of control rules or parameter tuning. It can automatically discover and execute complex control strategies that balance multiple objectives such as efficiency, safety, and energy consumption, resulting in control performance superior to traditional methods.

[0104] Example 4:

[0105] To transform the raw, high-dimensional, and noisy point cloud data of the profile cavity into low-dimensional, stable features that can effectively characterize its geometric complexity and are suitable for prediction model input, this embodiment, based on embodiment 3, in step S2, the feature vector... The extraction method is as follows: for a single control cycle Contour point cloud data acquired internally Calculate its surface area in three-dimensional space ,volume Central Moment and the principal direction vector of the point cloud. ;Will Constructing an 8-dimensional feature vector ,in, , and These are the principal direction vectors. Components on the three coordinate axes, central moment These are the second moments about the x, y, and z axes, respectively.

[0106] The raw point cloud data is large and contains noise; directly inputting it into the model is inefficient and produces poor results. The steps described above define a carefully selected set of geometric features, including surface area. and volume Reflects the macroscopic dimensions of the internal cavity, center distance It describes the dispersion of point cloud distribution in its centroid coordinate system, and can capture the irregularity of the internal cavity cross section, principal direction vector. These eight features, reflecting the main orientation of the cavity in space, together constitute a compact and information-rich descriptor that can comprehensively characterize the ease or difficulty of passing through the cavity of the front profile, such as narrowness, curvature, and irregularity. As a key component of the state vector, they are input into the prediction model and the controller model. This feature engineering transforms the perceived data into indicators with clear physical and geometric meanings, improving the accuracy and interpretability of subsequent models.

[0107] Example 5:

[0108] In reinforcement learning training, to ensure that the reward function accurately guides the agent to learn control strategies that meet engineering requirements, such as efficiency, safety, and stability, this embodiment, based on embodiment 4, designs a reward function in the simulation environment of step S15. The formula is ,in, , , and These are the weighting coefficients for each reward or punishment. For the target speed, As a current obstacle, For the safety resistance threshold, This represents the currently estimated axial compressive deformation of the strip. For ideal deformation variables.

[0109] It's an efficiency reward; the higher the speed, the closer you are to the target speed, and the greater the reward. It is a safety penalty, which is only imposed when the resistance exceeds the safety threshold, and the more the resistance exceeds the threshold, the greater the penalty, forcing the agent to learn to avoid overload; It is a quality penalty that encourages controlling the compression deformation of the strip within an ideal range that avoids buckling, thus ensuring the mechanical properties of the product after the strip is threaded. It's a smoothness penalty, encouraging gentle control actions, reducing mechanical shock, and extending equipment life. This is achieved by adjusting the weighting coefficients. , , and This allows for a balance of priorities among different objectives. The reward function forces the agent (Actor network) to simultaneously optimize speed, safety, bar deformation, and control stability during exploration. Consequently, the learned strategy is naturally a multi-objective optimized, intelligently collaborative control strategy, outperforming any manually designed controller with a single objective.

[0110] Example 6:

[0111] During the insertion process, the strip undergoes elastic compression under axial thrust. Excessive compression deformation may lead to local buckling (instability) of the strip or springback after insertion, resulting in loose assembly. Therefore, real-time estimation of the deformation is necessary to input the state information to the controller. Thus, based on Example 5, this embodiment estimates the axial compression deformation of the strip in step S2. The formula is ,in, Let be the cross-sectional area of ​​the strip.

[0112] An estimation is performed based on a simplified model of Hooke's Law in mechanics of materials, treating the penetrated portion of the strip as bearing the thrust. The axial compressive deformation of the rod. With thrust and the length already inserted Proportional to the axial stiffness of the strip The deformation is inversely proportional to the deformation. This formula has a clear physical meaning, is simple to calculate, and can reflect the cumulative deformation trend in real time. Although it is a simplified model, it provides a crucial estimate of the deformation level. In reinforcement learning, this estimate... Included in the state vector And through the penalty of deformation The Critic network learns and evaluates, ultimately enabling the Actor network to learn to regulate output actions. When the deformation of the strip is automatically considered, the optimal control under safety constraints is achieved without the need for additional optimization steps.

[0113] Example 7:

[0114] To address the issue that when a flexible strip traverses a complex cavity, its actual path may deviate from the theoretical straight line, causing the scanning center of the front sensor (assuming a fixed installation) to fail to align with the cavity area it is about to enter, thus affecting the sensing quality, a dynamic front sensor is needed to continuously focus on the critical area. This embodiment, based on embodiment 6, in step S5, the step of calculating the deviation angle... and the feature vector The method for dynamically correcting the scanning focus center plane of a macro laser sensor is as follows: Let the normal vector of the original focus center plane of the sensor be... The forward direction vector of the strip theory is Actual path deviation angle The forward direction vector of the strip theory With respect to the actual direction vector of the strip The included angle; the corrected normal vector of the focal center plane. From the formula ,in, For rotation around the axis Rotation Rotation matrix of angle, rotation axis , For vector normalization, For feature vectors The projection vector of the predicted centroid of the front cavity region in the sensor coordinate system. Let be the center vector of the sensor's field of view. It is a gain coefficient less than 1, used to control the correction magnitude.

[0115] The correction amount consists of two parts:

[0116] 1. Path deviation compensation section Based on the path deviation angle that has occurred Rotate the sensor's focusing plane in the opposite direction by half an angle. This is a predictive compensation that anticipates the strip will continue to move a portion of the way it deviates, causing the sensor to look in the direction of deviation in advance.

[0117] 2. Forward-looking feature guidance section Based on the feature vector of the front cavity Implicit geometric centroid information The focus center is fine-tuned to be more biased toward potentially more complex or critical areas within the cavity.

[0118] The two are combined in a weighted manner, with the latter being... By mitigating its impact and avoiding abrupt changes, the sensor's attention can follow both the macroscopic changes in the body's posture and be drawn to the details of the terrain ahead. This significantly enhances the system's ability to continuously acquire high-quality information about the environment ahead in complex cavities such as bends and bifurcations, providing a more reliable perception foundation for state construction and intelligent decision-making.

[0119] Example 8:

[0120] To clarify the specific structure of the core neural network of the deep reinforcement learning controller and ensure that its output action, i.e., the speed adjustment amount, is within a reasonable physical range, thus guaranteeing the feasibility and safety of control, this embodiment, based on Embodiment 7, uses fully connected feedforward neural networks for both the Actor network and the Critic network of the bar controller model. The Actor network output layer uses the tanh activation function to restrict the action to the range [−1, 1], and converts it to the actual speed adjustment range through a linear mapping. ,in, and These represent the maximum allowable negative and maximum allowable positive speed adjustment values ​​of the servo motor of the threading machine within a single control cycle. It is a positive constant, if This means that the speed can increase by a maximum of 0.2 m / s compared to the previous cycle. This means that the speed can decrease by a maximum of 0.2 m / s compared to the previous cycle.

[0121] The above steps utilize a fully connected feedforward neural network, which is suitable for handling nonlinear mappings from state to action due to its simple structure, stable training, and suitability. The key design element is the use of the tanh activation function in the Actor network's output layer. The tanh function's range (−1, 1) naturally restricts the network's original output to a normal range, preventing output value explosion or instability. Subsequently, a simple linear transformation is applied... This allows the (−1,1) gauge output to be mapped to the velocity adjustment range allowed by the actual physical system. ,in, and These are the scaling and translation parameters, respectively. For example, if the maximum adjustment is 0.2 m / s, then it can be set as follows: , The above design separates policy learning and action scaling, resulting in more stable network training, preventing gradient vanishing outside the tanh saturation region, and easily adapting to different device actuators; only the mapping parameters need to be adjusted. and This enhances the method's versatility and engineering applicability.

[0122] Example 9:

[0123] A composite strip threading system for profiles includes a data acquisition module 10, a state construction and prediction module 20, a deep reinforcement learning control decision module 30, an execution driving module 40, and a path perception and correction module 50.

[0124] The data acquisition module includes:

[0125] The macro laser sensor 11 is located at the front end of the strip and is used to collect data at a preset distance from the front of the profile's inner cavity. Contour point cloud data ;

[0126] Encoder 12, connected to the main shaft of the threading machine, is used to obtain the current actual threading speed. ;

[0127] Strain sensor 13 is installed on the thrust shaft of the threading machine to obtain the current measured threading resistance. ;

[0128] Attitude sensor 14, located at the tail of the bar, is used to obtain the deviation angle of the actual insertion path of the bar. ;

[0129] The state construction and prediction module includes:

[0130] Feature extraction unit 21, communicatively connected to the macro laser sensor, is used to perform noise reduction and feature extraction on the contour point cloud data, and output a feature vector characterizing the geometric complexity of the inner cavity of the front profile. ;

[0131] The state construction unit 22 is communicatively connected to the feature extraction unit, encoder, and strain sensor, and is used to receive the feature vector. Current actual threading speed Current measured resistance to threading Obtain the elastic modulus of the strip Obtain the coefficient of friction between the strip and the profile. and get the length of the bar that has been inserted Construct the current state vector ;

[0132] The resistance prediction unit 23 has a built-in sub-model for predicting resistance during threading, which is a model trained based on a long short-term memory network. The resistance prediction unit is communicatively connected to the state construction unit and is used to receive the feature vector. Current actual threading speed Current measured resistance to threading , bar elastic modulus coefficient of friction between the strip and the profile and output the future Predicted resistance value after one control cycle The state construction unit is also used to convert the predicted resistance value. The state vector is incorporated to form an extended state vector. ;

[0133] The deep reinforcement learning control decision module includes:

[0134] Controller 31 has a built-in Actor network trained based on a deep deterministic policy gradient algorithm. The controller is communicatively connected to the state construction and prediction module and is used to receive the extended state vector. The Actor network directly outputs the optimal speed adjustment action. ;

[0135] The execution driver module includes:

[0136] Servo driver 41 is electrically connected to the controller and the servo motor 60 of the threading machine; the servo driver is used to receive the optimal speed adjustment action. And drive the servo motor in the next control cycle Inside, according to speed run;

[0137] The path awareness and correction module includes:

[0138] The judgment and calculation unit 51 is communicatively connected to the attitude sensor and feature extraction unit, and is used to receive the deviation angle. and the feature vector And in determining the deviation angle Exceeding the preset tolerance At that time, according to the deviation angle and the feature vector The correction command for the scanning focusing center plane of the micro-laser sensor is calculated;

[0139] The execution unit 52 is connected to the judgment and calculation unit and the macro laser sensor control, and is used to execute the correction command to dynamically adjust the scanning focus center plane of the macro laser sensor.

[0140] Each module unit in this system implements the corresponding method steps in Examples 1 to 8, and its principles and effects will not be elaborated here.

[0141] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for controlling the insertion of composite strips into profiles, characterized in that, Includes the following steps: S1. Establish a digital twin model of the strip-threading process, which includes a strip-threading resistance prediction sub-model based on a long short-term memory network and a strip-threading controller model based on a deep deterministic policy gradient network; collect historical strip-threading process data to train the strip-threading resistance prediction sub-model, and use a simulation environment to train the strip-threading controller model. Set the initial target speed for the strip. and system control cycle ; S2, in a single control cycle Within the system, data from the current threading stage is collected in real time, and the data includes at least: The preset distance in front of the profile's inner cavity is collected by a micro-laser sensor located at the front end of the strip. Contour point cloud data ,in, , and Points The coordinates of the x, y, and z axes in the coordinate system of the micro-laser sensor itself; The current actual threading speed is obtained through the encoder of the threading machine spindle. ; The current measured threading resistance is obtained from the strain sensor on the thrust shaft of the threading machine. ; The contour point cloud data is denoised and feature extracted to obtain a feature vector representing the geometric complexity of the inner cavity of the front profile. Construct the current state vector , ,in, The elastic modulus of the strip. The coefficient of friction between the strip and the profile. The length of the strip that has been inserted; S3, the feature vector Current actual threading speed Strip material parameters and Input the trained strip resistance prediction sub-model to predict the future. Predicted resistance value after one control cycle Expand the state vector to ; S4. Expand the state vector The trained strip-threading controller model is input, and its Actor network directly outputs the optimal speed adjustment action for the current state. The output action of the strip-threading controller model As the speed adjustment amount of the servo motor of the threading machine; S5, drive the threading machine in the next control cycle Inside, according to speed Run; in the next control cycle Initially, the deviation angle of the actual insertion path of the bar is obtained by an attitude sensor located at the tail of the bar. ;If the deviation angle Exceeding the preset tolerance Then according to the deviation angle and the feature vector The scanning focus center plane of the macro laser sensor is dynamically corrected, and the process returns to step S2; if the deviation angle... Not exceeding the preset tolerance If so, steps S2 to S4 will continue to be executed in the next control cycle until the threading process is completed.

2. The profile composite strip control method according to claim 1, characterized in that, In step S1, the strip resistance prediction sub-model is a long short-term memory network model, and its training process includes: S11. Arrange the historical striping process data according to time steps. The data is divided into samples, and the input features of each sample are... For: at time eigenvectors Threading speed , bar elastic modulus coefficient of friction Output tags For: moment Measured resistance to threading ; S12, for the first Long Short-Term Memory Network at time The calculation is as follows: Input gate ; Forgotten Gate ; Candidate cell status ; Output gate ; Current cell state ; Current hidden state ; in, Enter the current time. This is the hidden state from the previous moment. This represents the cell state at the previous moment. It is the sigmoid activation function. The hyperbolic tangent activation function is used. For Hadama accumulation, , , , , , , , These are the input weight matrix and the cyclic weight matrix for the corresponding gate, respectively. , , , These are the bias vectors for the corresponding gates; S13. Using mean squared error as the loss function The network weights are adjusted using the time backpropagation algorithm and optimizer until the loss converges.

3. The profile composite strip control method according to claim 2, characterized in that, In step S1, the strip-threading controller model is trained using a deep deterministic policy gradient algorithm framework, including an Actor network. and Critic Network ,in, For the Actor policy function, based on the current state Output Action Right now , The extended state vector constructed in step S2 , Here are the weight matrices and bias vectors for all layers of the Actor network. The action value function evaluates the value of an action in a given state. Next, execute the action. The expected long-term cumulative rewards that can be obtained afterward. This is the set of weight parameters for the Critic neural network, and its offline training process is performed in a simulation environment, including: S14. Establish a simulation environment for the profile threading process based on a physics engine, capable of receiving speed adjustment actions. And calculate the state at the next moment. ,award And the marker indicating whether the process has ended (done); S15. Explore the environment and collect the transfer samples generated by each interaction. Store in the experience replay buffer , For a moment Speed ​​adjustment action; S16, from the buffer Randomly sample a small batch of samples and update the network according to the following steps: a. Calculation target value ,in, As a discount factor, and These are the target Actor and Critic networks, respectively; the loss function is minimized. To update the current Critic network parameters ; b. Using the policy gradient theorem, maximize the expected return. The Actor network is updated using an approximation, and the formula is: ; c. Use , Perform a soft update on the target network, whereby... For soft update parameters; S17. Repeat steps S15 and S16 until the policy converges, and obtain the trained Actor network as the strip controller model.

4. The profile composite strip control method according to claim 3, characterized in that, In step S2, the feature vector The extraction method is as follows: for a single control cycle Contour point cloud data acquired internally Calculate its surface area in three-dimensional space ,volume Central Moment and the principal direction vector of the point cloud. ;Will Constructing an 8-dimensional feature vector ,in, , and These are the principal direction vectors. Components on the three coordinate axes.

5. The profile composite strip control method according to claim 4, characterized in that, In the simulation environment of step S15, design the reward function. The formula is ,in, , , and These are the weighting coefficients for each reward or punishment. For the target speed, As a current obstacle, For the safety resistance threshold, This represents the currently estimated axial compressive deformation of the strip. For ideal deformation variables.

6. The profile composite strip control method according to claim 5, characterized in that, In step S2, the axial compressive deformation of the strip is estimated. The formula is ,in, Let be the cross-sectional area of ​​the strip.

7. The profile composite strip control method according to claim 6, characterized in that, In step S5, the step of determining the deviation angle... and the feature vector The method for dynamically correcting the scanning focus center plane of a macro laser sensor is as follows: Let the normal vector of the original focus center plane of the sensor be... The forward direction vector of the strip theory is Actual path deviation angle The forward direction vector of the strip theory With respect to the actual direction vector of the strip The included angle; the corrected normal vector of the focal center plane. From the formula ,in, For rotation around the axis Rotation Rotation matrix of angle, rotation axis , For vector normalization, For feature vectors The projection vector of the predicted centroid of the front cavity region in the sensor coordinate system. Let be the center vector of the sensor's field of view. This is the gain coefficient.

8. The profile composite strip control method according to claim 7, characterized in that, The Actor network and Critic network of the strip controller model are both fully connected feedforward neural networks. The output layer of the Actor network uses the tanh activation function to restrict the action to the range of [−1, 1], and converts it into the actual speed adjustment range through a linear mapping. ,in, and These represent the maximum allowable negative and maximum allowable positive speed adjustment values ​​of the servo motor of the threading machine within a single control cycle.

9. A composite strip-threading system for profiles, characterized in that, It includes a data acquisition module, a state construction and prediction module, a deep reinforcement learning control and decision module, an execution-driven module, and a path awareness and correction module; The data acquisition module includes: A macro laser sensor, positioned at the front end of the strip, is used to collect data at a preset distance from the front of the profile's inner cavity. Contour point cloud data ; An encoder, connected to the main shaft of the threading machine, is used to obtain the current actual threading speed. ; A strain sensor, mounted on the thrust shaft of the threading machine, is used to obtain the current measured threading resistance. ; An attitude sensor, located at the tail of the bar, is used to obtain the deviation angle of the actual insertion path of the bar. ; The state construction and prediction module includes: The feature extraction unit, communicatively connected to the macro laser sensor, is used to perform noise reduction and feature extraction on the contour point cloud data, and output a feature vector characterizing the geometric complexity of the inner cavity of the front profile. ; The state construction unit, which is communicatively connected to the feature extraction unit, encoder, and strain sensor, is used to receive the feature vector. Current actual threading speed Current measured resistance to threading Obtain the elastic modulus of the strip Obtain the coefficient of friction between the strip and the profile. and get the length of the bar that has been inserted Construct the current state vector ; The resistance prediction unit incorporates a sub-model for predicting the resistance of the threaded bar, which is a model trained based on a long short-term memory network. The resistance prediction unit is communicatively connected to the state construction unit and is used to receive the feature vector. Current actual threading speed , bar elastic modulus coefficient of friction between the strip and the profile and output the future Predicted resistance value after one control cycle The state construction unit is also used to convert the predicted resistance value. The state vector is incorporated to form an extended state vector. ; The deep reinforcement learning control decision module includes: The controller has a built-in Actor network trained based on a deep deterministic policy gradient algorithm. The controller is communicatively connected to the state construction and prediction module and is used to receive the extended state vector. The Actor network directly outputs the optimal speed adjustment action. ; The execution driver module includes: A servo driver is electrically connected to the controller and the servo motor of the threading machine; the servo driver is used to receive the optimal speed adjustment action. And drive the servo motor in the next control cycle Inside, according to speed run; The path awareness and correction module includes: The judgment and calculation unit is communicatively connected to the attitude sensor and the feature extraction unit, and is used to receive the deviation angle. and the feature vector And in determining the deviation angle Exceeding the preset tolerance At that time, according to the deviation angle and the feature vector The correction command for the scanning focusing center plane of the micro-laser sensor is calculated; The execution unit, connected to the judgment and calculation unit and the macro laser sensor control, is used to execute the correction command and dynamically adjust the scanning focus center plane of the macro laser sensor.

Images