An Active Vibration Control Method for Thin-Walled Part Milling Based on Reinforcement Learning

Abstract

This invention proposes a method and device for active vibration control during thin-walled part milling based on reinforcement learning. Belonging to the field of machining technology, this invention relates to an active vibration control method for thin-walled part structures. The vibration suppression method is as follows: During thin-walled part milling, an accelerometer is used to acquire the vibration signal during milling. The acquired vibration signal is amplified by a charge amplifier, then input to a host computer via a digital-to-analog converter. After processing by a Q-learning controller, a driving signal is generated. This signal is amplified by a voltage amplifier and applied to a piezoelectric fiber sheet, generating an ideal driving force. This method can achieve real-time vibration control during thin-walled part milling with high precision; using the piezoelectric fiber sheet as an actuator provides excellent driving functionality; the Q-learning controller can adjust the output force of the piezoelectric fiber sheet in real time, adapting to the time-varying and complex nature of the workpiece-tool machining process; and the Q-learning algorithm exhibits good stability and fast computation speed.

Description

Technical Field

[0001] This invention belongs to the field of machining technology, and specifically relates to an active control method for vibration during milling of thin-walled parts based on reinforcement learning. Background Technology

[0002] The manufacturing of thin-walled parts represents the advanced stage of manufacturing and is widely used in important national industrial sectors such as aerospace, automotive, and military. Therefore, the performance requirements for thin-walled parts are extremely high; lightweighting and high performance have become essential goals in their processing. The processing of thin-walled parts has the following characteristics: First, the parts are large in size and need to be completed as a whole, resulting in significant material removal during processing. This necessitates high processing efficiency to ensure sufficient processing time, leading to larger cutting depths and rotational speeds. Second, thin-walled parts are mostly made of difficult-to-machine high-temperature alloys such as titanium and nickel alloys, resulting in high strength and hardness, thus generating significant cutting forces and heat during processing. Third, thin-walled parts are inherently weakly rigid components, and are often cantilevered without additional measures during processing, resulting in poor system rigidity during the process. In the manufacturing process of thin-walled aerospace parts, difficult-to-machine materials are often used. Based on the above characteristics, it can be concluded that thin-walled parts are very prone to vibration during processing. When the processing conditions are poor and the workpiece processing parameters do not meet the specific requirements, vibration may lead to poor surface quality, tool wear and other negative effects. Therefore, vibration should be avoided as much as possible during the processing.

[0003] Currently, vibration suppression methods are mainly divided into two categories: passive control and active / semi-active control. Passive control, also known as passive-free control, is a control method that does not require external energy input. It suppresses vibration by optimizing the system's machining parameters (such as spindle speed, cutting thickness, and feed rate) or by adding damping devices to the system structure. Active / semi-active control, also known as active control, is a control method that relies on external energy input. It involves placing actuators on the workpiece or machine tool, designing control algorithms to make the actuators generate output force to change the system's structural response, and can be adjusted in real time according to the workpiece's real-time state.

[0004] In 2019, Zhang Dinghua et al. proposed a device and method for suppressing vibration in thin-walled workpiece processing based on bending actuators in CN 110153781 A. By changing the voltage across the actuator, a reverse force is applied to the workpiece to suppress the vibration of the thin-walled workpiece. However, the control algorithm used in the control process is relatively classic and cannot be applied to the complexity and time-varying nature of current thin-walled workpiece processing.

[0005] None of the above studies mentioned a method for real-time adjustment of the driving force of piezoelectric fiber sheets using Q-learning control algorithm. However, Q-learning control algorithm has the characteristics of fast operation speed and strong adaptability. Based on this, this invention proposes an active control method for milling vibration of thin-walled parts based on reinforcement learning. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies in the design, characterization, and control capabilities of controllers for time-varying systems. By using a controller to adjust the inverse piezoelectric effect of piezoelectric fiber sheets, a vibration suppression method for thin-walled components based on piezoelectric fiber sheets is invented. This method uses a thin-walled plate as the controlled object and a piezoelectric fiber sheet as the actuator. By changing the voltage applied to the piezoelectric fiber sheet, different magnitudes of reverse forces are applied to the workpiece, thereby suppressing the vibration of the thin-walled plate. A controller based on a Q-learning algorithm is employed. The controller first initializes its parameters, setting all Q values ​​to 0. Then, it randomly selects initial states and actions. Subsequently, it iterates through the Bellman equation, continuously updating the value functions in the table until the algorithm converges. The final Q values ​​represent the true values ​​of each state and action, thus achieving the optimal vibration control effect. This invention uses piezoelectric fiber sheets as actuators, which have good machinability, high sensitivity, convenient arrangement, and can be applied to both planar and curved surface structures.

[0007] The technical solution adopted in this invention is:

[0008] An active vibration control method for thin-walled part milling based on reinforcement learning is disclosed. An accelerometer is placed on the workpiece surface to monitor vibration signals during the milling process. The detected vibration signals are amplified by a charge amplifier and then converted into digital signals by a digital-to-analog converter (DAC). The digital signals are transmitted to a computer as input to a Q-learning controller. The Q-learning controller calculates the acceleration signals in real time and outputs the results. The digital signals output by the computer are converted into analog signals by the DAC. The analog signals are amplified by a voltage amplifier to the required driving voltage for the piezoelectric fiber sheet, applying a corresponding vibration damping force to the thin-walled part. The specific steps are as follows:

[0009] The first step is to establish a dynamic model of the thin-walled component driven by piezoelectric fiber sheets.

[0010] When the thin-walled plate is subjected to an external milling force, the workpiece 8 can be regarded as an Euler-Bernoulli beam, and its vibration equation under the action of the milling force is:

[0011]

[0012] Where EI represents the bending strength of the workpiece, and F1 represents the milling force on the workpiece 8.

[0013] When a voltage is applied to a piezoelectric fiber sheet, it will produce a total voltage in the x and y directions. Unconstrained strain, due to exist and If they are in the same direction, then The expression is:

[0014]

[0015] In the formula, The voltage applied in the polarization direction, The thickness of the piezoelectric fiber sheet, is the strain constant of the piezoelectric fiber sheet.

[0016] When the thickness and mass of the piezoelectric fiber sheet are much smaller than those of the thin-walled component, and it is completely bonded to the thin-walled component, the inertial and end effects of the piezoelectric fiber sheet can be ignored. Assuming that the internal torque of the piezoelectric fiber sheet in the x and y directions only occurs within the bonding area of ​​the piezoelectric sheet, the expression is:

[0017]

[0018] In the formula, and For the angular coordinates of the piezoelectric fiber sheet, , The material's geometric parameters are expressed as follows:

[0019]

[0020] In the formula, For thin-walled plate thickness, The thickness is the piezoelectric fiber sheet.

[0021] Here Let be the unit step function, defined as:

[0022]

[0023] Substituting the torques into the equations of motion for the thin-walled component, solving the differential operator, and moving the piezoelectric term to the right side of the equation, the final expression for the dynamic equations of the thin-walled component is:

[0024]

[0025] In the formula, For the Laplace operator, For Dirac The derivative of the function with respect to the phase angle.

[0026] Simplified to:

[0027]

[0028] In the above formula, Indicates the application of voltage.

[0029]

[0030] The second step is to design a Q-learning controller for milling vibrations in thin-walled parts.

[0031] The central idea of ​​this Q-learning algorithm is to obtain a corresponding immediate reward value through learning at each step. The state-action pair function is approximated by iteratively calculating the immediate reward value and the Q value of the next state. This allows us to utilize complete Q-function information, meaning the Q-table can select the action 'a' with the highest Q-value in state 's'. The iterative formula for the Q-function is:

[0032]

[0033] in In the state Select control action The total reward value that can be obtained when the target state is reached. For state Select control action at time Then reach the next state The immediate reward received In the state Select control action at time The total reward value that can be obtained when the target state is reached. For the learning rate parameter, This is the decay coefficient for the value of subsequent states. Parameter The size of the learning process affects its stability and speed. The learning process becomes more stable when the value is reduced, but the Q-function updates become slower. The magnitude of the parameter determines the degree of importance attached to the size of the subsequent reward. When it gets bigger, it means that the subsequent rewards are valued more.

[0034] When using a Q-learning controller to control the vibration of a thin-walled plate, the first step is to define the three basic elements of reinforcement learning: state, action, and reward function.

[0035] (1) State definition: displacement-velocity pair Simultaneously, the position of the object at time t and whether the object is rising or falling at that time are determined, thus allowing for a unique representation of the state. Therefore, this study defines the state space as follows: .

[0036] (2) Action Definition: Since this study uses a piezoelectric element as the actuator and the control output is the control voltage u, the control action a is defined as the control voltage u, and the action space is...

[0037] (3) Definition of reward function: For vibration control, if the disturbance load is a pulse load, it is desirable that the vibration displacement decays rapidly over time, i.e., the displacement is as small as possible during vibration. For periodic disturbance loads, it is desirable that the vibration amplitude is as small as possible, i.e., the velocity is as small as possible during vibration. Based on the above control objectives, the reward function is set as follows:

[0038]

[0039] in, , This is the scaling factor. , Let these represent displacement and velocity during vibration. In reinforcement learning, the goal is to obtain the largest possible reward value, while in vibration control, the goal is... , The smaller the value, the better; therefore, a negative sign is used before the proportionality coefficient.

[0040] Since the Q-learning algorithm requires that the elements contained in the state space and action space must be finite, the equal division method is used to establish a discrete space of variables to discretize displacement, velocity, and control voltage in order to reduce the spatial dimension contained in the control algorithm.

[0041] The displacement physical quantity is discretized by selecting the maximum value during the vibration process. and minimum value And divide the maximum and minimum values ​​equally. If the displacement discrete space contains [parts], then [the space] includes [parts]. There are elements, respectively ,in .

[0042] The physical quantities of velocity are respectively ,in .

[0043] The physical quantities of the control voltage are respectively ,in .

[0044] The state space S consists of two physical quantities, displacement and velocity; therefore, the state space contains... They are respectively:

[0045]

[0046] Since the action space is the same as the voltage space, the action space is:

[0047] .

[0048] The third step is to determine the control method of the Q-learning controller for milling vibration of thin-walled parts.

[0049] When the thin-walled part interacts with the cutting tool, the clamping end of the thin-walled part is completely fixed. A piezoelectric fiber sheet located on the non-machined surface of the thin-walled part provides the control force, and an accelerometer located at the top of the thin-walled part detects the displacement change at that point. The entire control loop process is as follows: when the thin-walled part contacts the cutting tool at the machining position, the accelerometer detects the magnitude and speed of the displacement change at the machining point and submits this information to the Q-learning controller. The Q-learning controller calculates the control quantity, which is then amplified and transmitted to the piezoelectric actuation layer to initiate the corresponding control action.

[0050] The fourth step is to build the hardware required for the Q-learning control system of the milling vibration of thin-walled parts.

[0051] The Q-learning control system for vibration during thin-walled part milling includes a fixture, voltage amplifier, computer, charge amplifier, digital-to-analog converter, accelerometer, machine tool spindle, milling cutter, workpiece, and piezoelectric fiber sheet.

[0052] The accelerometer is connected to the thin-walled part to detect the vibration signal of the workpiece during the milling process. The charge amplifier amplifies the acquired voltage signal, and the digital-to-analog converter converts the acquired analog signal into a digital signal. The digital signal is transmitted to the computer, which obtains the real-time changing drive signal through the Q learning controller. The drive signal is converted into an analog signal by the digital-to-analog converter, and the analog signal is amplified by the voltage amplifier to drive the piezoelectric fiber sheet. The driving force of the piezoelectric fiber sheet is changed by changing the driving voltage.

[0053] A further technical solution of the present invention is: the clamp fixes the thin-walled workpiece in a side-milling state, and the piezoelectric fiber sheet can be fixed to the thin-walled workpiece near the clamping end by adhesive, so that the free end of the thin-walled workpiece can generate the maximum vibration amplitude.

[0054] A further technical solution of the present invention is that the acceleration sensor is fixed on the unprocessed surface of the thin-walled plate.

[0055] The most prominent feature and significant beneficial effect of this invention are:

[0056] (1) The present invention uses piezoelectric fiber sheet as actuator, which has good mechanical processing performance, high sensitivity, fast response speed and large strain energy per unit volume, and is suitable for processing space of thin-walled parts with complex shape.

[0057] (2) The piezoelectric fiber sheet of the present invention has low power consumption, low price and low cost of use.

[0058] (3) The piezoelectric fiber sheet of the present invention can be fixed with glue, which is convenient to install and suitable for actual processing without changing the overall clamping method of the workpiece.

[0059] (4) The present invention adopts a Q-learning controller in the control strategy, which discretizes the state space and action space of the thin-walled plate, analyzes the influence of the changes in the learning efficiency parameter and the decay parameter of the subsequent state value function on the control effect, and has high accuracy in control, especially suitable for the complexity and time-varying nature of the thin-walled plate processing. Attached Figure Description

[0060] Figure 1 The process of learning the Q controller.

[0061] Figure 2 This is a flowchart of the present invention.

[0062] Figure 3 This is a schematic diagram of the piezoelectric fiber sheet control system.

[0063] Figure 4 The vibration curves of the workpiece are shown below; (a) is the vibration curve of the workpiece without control, and (b) is the vibration curve of the workpiece with the controller applied.

[0064] Among them, 1-clamp, 2-voltage amplifier, 3-computer, 4-charge amplifier, 5-accelerometer, 6-machine tool spindle, 7-milling cutter, 8-workpiece, 9-piezoelectric fiber sheet. Detailed Implementation

[0065] The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the invention, and should not be construed as limiting the invention.

[0066] Taking a 5mm thick titanium alloy thin-walled plate as an example, this invention uses a five-axis CNC machine tool as the processing equipment. With a spindle speed of 3000r / min, a feed per tooth of 0.15mm, an axial depth of cut of 0.4mm, and a radial depth of cut of 0.8mm, a piezoelectric fiber sheet is used as the actuator and an accelerometer is used as the acceleration detection unit. A reinforcement learning-based active control method for milling vibration of thin-walled parts is designed.

[0067] The specific steps of the reinforcement learning-based active control method for vibration in thin-walled part milling are as follows:

[0068] The first step is the dynamic modeling of the piezoelectric fiber sheet control system.

[0069] The vibration of workpiece 8 during machining is caused by the interaction between workpiece 8 and milling cutter 7. When the thin-walled plate is subjected to an external milling force, workpiece 8 can be regarded as an Euler-Bernoulli beam, and its vibration equation under the action of the milling force is:

[0070]

[0071] in, Indicates the bending strength of the workpiece. This indicates the milling force acting on workpiece 8.

[0072] When a voltage is applied to the piezoelectric fiber sheet, it will... and Total amount generated in direction Unconstrained strain, due to exist and If they are in the same direction, then The expression is:

[0073]

[0074] In the formula, The voltage applied in the polarization direction, The thickness of the piezoelectric fiber sheet, is the strain constant of the piezoelectric fiber sheet.

[0075] When the thickness and mass of the piezoelectric fiber sheet are much smaller than those of the thin-walled part, and it is completely adhered to the thin-walled plate, the inertial effect and end effect of the piezoelectric fiber sheet can be ignored. Assuming the piezoelectric fiber sheet... and The internal torques in the direction only appear within the piezoelectric sheet bonding area, and these torques are all equal, expressed as:

[0076]

[0077] In the formula, and For the angular coordinates of the piezoelectric fiber sheet, .

[0078] The material's geometric parameters are expressed as follows:

[0079]

[0080] In the formula, For thin-walled plate thickness, The thickness is the piezoelectric fiber sheet.

[0081] Here Let be the unit step function, defined as:

[0082]

[0083] Substituting the torques into the two-dimensional thin-plate motion equations, solving the differential operators, and moving the piezoelectric terms to the right side of the equations, the final expression for the dynamic equations of the thin-walled component is:

[0084]

[0085] In the formula, For the Laplace operator, For Dirac The derivative of the function with respect to the phase angle.

[0086] The final dynamic model expression is:

[0087]

[0088] In the formula, Indicates the application of voltage.

[0089]

[0090] This leads to a new dynamic equation.

[0091] The second step is to design a Q-learning controller for milling vibrations in thin-walled parts.

[0092] The state-space equation of the vibration system during thin-walled part milling is derived from the formula:

[0093]

[0094] Then, set various parameters for reinforcement learning, including setting the state space, action space, greedy algorithm parameters, value decay coefficient, number of training iterations, learning rate parameters, and Q-table initialization.

[0095] Training is performed using the Q-learning algorithm. The first training iteration checks if the target value has been reached. If it has, training ends; otherwise, the control object is initialized. The state is discretized to begin training. The Q-table is used to calculate the control voltage with the maximum Q value based on the current state, and the current control voltage is determined according to a greedy strategy. The current disturbance load is applied, and the new state of the controlled object is calculated according to the state-space equation. The immediate reward is calculated by discretizing the state and calculating the maximum control action for the next Q value based on the current Q value table and the new state.

[0096]

[0097] By updating the Q-value table according to the above formula, updating the state and action in the Q algorithm, and updating the current state of the controlled object, the real-time control voltage can be continuously obtained.

[0098] The third step is to determine the control method of the Q-learning controller for milling vibration of thin-walled parts.

[0099] When the thin-walled part interacts with the cutting tool, the clamping end of the thin-walled part is completely fixed. A piezoelectric fiber sheet located on the non-machined surface of the thin-walled part provides the control force, and an accelerometer located at the top of the thin-walled part detects the displacement change at that point. The entire control loop process is as follows: when the thin-walled part contacts the cutting tool at the machining position, the accelerometer detects the magnitude and speed of the displacement change at the machining point and submits this information to the Q-learning controller. The Q-learning controller calculates the control quantity, which is then amplified and transmitted to the piezoelectric actuation layer to initiate the corresponding control action.

[0100] The fourth step is to build the hardware required for the Q-learning control system of the milling vibration of thin-walled parts.

[0101] The Q-learning control system for milling vibration of thin-walled parts consists of 1-fixture, 2-voltage amplifier, 3-computer, 4-charge amplifier, 5-accelerometer, 6-machine tool spindle, 7-milling cutter, 8-workpiece, and 9-piezoelectric fiber sheet.

[0102] First, the titanium alloy thin-walled workpiece 8, to be machined, is mounted on the worktable of a five-axis CNC machine tool using fixture 1, with calibration, tool setting, and installation completed in advance. Then, piezoelectric fiber sheet 9 is glued to the non-cutting side of the thin-walled workpiece to ensure that the force applied by the actuator during machining maximizes the vibration amplitude at the free end. Accelerometer 5 is installed on the non-cutting side of the thin-walled workpiece to detect the vibration signal generated by the workpiece during milling. The detected vibration signal is transmitted to computer 3 via charge amplifier. Computer 3 calculates the drive signal for the piezoelectric fiber sheet using an algorithm. The drive signal is transmitted to piezoelectric fiber sheet 9 via charge amplifier 4, causing the actuator to bend and deform, applying a reverse damping force to the workpiece, thus actively controlling the vibration generated during the machining of the thin-walled workpiece.

[0103] from Figure 4 As can be seen, taking a 5mm thick titanium alloy thin-walled part as an example, using a five-axis CNC machine tool as the machining equipment, with a spindle speed of 3000 r / min, a feed per tooth of 0.15mm, an axial depth of cut of 0.4mm, and a radial depth of cut of 0.8mm, the maximum vibration acceleration of the thin-walled part is 54.27 m / s². 2 Reduced to 43.41 m / s 2 The vibration suppression rate is 20.01%.

[0104] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A method for active vibration control in thin-walled part milling based on reinforcement learning, characterized in that, The method includes the following steps: Step 1: Establish a dynamic model of the thin-walled component driven by piezoelectric fiber sheets to obtain the state control space equations of the system; Step 2: Design a Q-learning controller for milling vibration of thin-walled parts, defining the three basic elements of reinforcement learning: state, action, and reward function; obtain the corresponding immediate reward value through each learning step. By iteratively calculating the immediate reward value and the Q-value of the next state, a state-action pair function Q(s,a) that approximates the true state is calculated. This allows the use of complete Q-function information, i.e., the Q-table, to select the action a with the highest Q-value in state s. The specific process is as follows: The iterative formula for the Q function is: in In the state Select control action The total reward value that can be obtained when the target state is reached. For state Select control action Then reach the next state The immediate reward received In the state Select control action The total reward value that can be obtained when the target state is reached; The displacement physical quantity is discretized by selecting the maximum value during the vibration process. and minimum value And divide equally between the maximum and minimum values. If the displacement discrete space contains [parts], then [the space] includes [parts]. There are elements, respectively ,in ; The physical quantities of velocity are respectively ,in ; The physical quantities of the control voltage are respectively ,in ; state space The state space consists of two physical quantities: displacement and velocity. There are elements, namely: Since the action space is the same as the voltage space, the action space is: ； Step 3: Establish the control method of the Q-learning controller for milling vibration of thin-walled parts; Step 4: Build the hardware required for the Q-learning control system of milling vibration for thin-walled parts.

2. A method for active control of milling vibration of thin-walled parts based on reinforcement learning according to claim 1, characterized in that: The specific process for establishing the dynamic model of the thin-walled component driven by the piezoelectric fiber sheet in step one is as follows: When a thin-walled part is subjected to an external milling force, the vibration equation is expressed as follows: in, Indicates the bending strength of the workpiece. This indicates the milling force acting on the workpiece; When a voltage is applied to a piezoelectric fiber sheet, a total voltage will be generated in the x and y directions. The unconstrained strain is expressed as: In the formula, The voltage applied in the polarization direction, The thickness of the piezoelectric fiber sheet, is the strain constant of the piezoelectric fiber sheet; Assuming that the internal torque of the piezoelectric fiber sheet in the x and y directions only occurs within the piezoelectric sheet bonding area, the expression is: In the formula, and For the angular coordinates of the piezoelectric fiber sheet, , The material's geometric parameters are expressed as follows: In the formula, For thin-walled plate thickness, The thickness of the piezoelectric fiber sheet; Here Let be the unit step function, defined as: Substituting the torques into the dynamic equations of the thin-walled component, solving the differential operator, and moving the piezoelectric term to the right side of the equation, the final expression for the dynamic equations of the thin-walled component is: In the formula, For the Laplace operator, For Dirac The derivative of the function with respect to the phase angle; Simplified to: In the formula, Indicates the application of voltage. 。 3. A method for active control of milling vibration of thin-walled parts based on reinforcement learning according to claim 1, characterized in that: The control process of the Q-learning controller for the milling vibration of thin-walled parts in step three is as follows: When the cutting tool comes into contact with the machining position of the thin-walled part, the accelerometer detects the magnitude and speed of the displacement change at the machining position of the thin-walled part and submits it to the Q-learning controller. After the Q-learning controller calculates the control quantity, it sends the control quantity to the piezoelectric actuator layer through the power amplifier so that the piezoelectric actuator layer can make the corresponding control action.

4. A method for active control of milling vibration of thin-walled parts based on reinforcement learning according to claim 1, characterized in that: The hardware equipment for building the Q-learning control system for milling vibration of thin-walled parts in step four mainly includes a fixture, a voltage amplifier, a computer, a charge amplifier, a digital-to-analog converter, an accelerometer, a machine tool spindle, a milling cutter, a workpiece, and a piezoelectric fiber sheet. The accelerometer is connected to the thin-walled part to detect the vibration signal of the workpiece during the milling process. The charge amplifier amplifies the acquired voltage signal, and the digital-to-analog converter converts the acquired analog signal into a digital signal. The digital signal is transmitted to the computer, which obtains the real-time changing drive signal through the Q learning controller. The drive signal is converted into an analog signal by the digital-to-analog converter, and the analog signal is amplified by the voltage amplifier to drive the piezoelectric fiber sheet. The driving force of the piezoelectric fiber sheet is changed by changing the driving voltage.

Images