Robot welding seam polishing control method, system and storage medium based on DDPG reinforcement learning algorithm

Abstract

The application discloses a kind of based on DDPG reinforcement learning algorithm's robot weld seam polishing control method, system and storage medium, method includes the following steps: S1, transport workpiece to be polished and fix it;S2, cutting arc plate;S3, collection weld seam point cloud data;S4, extract weld seam feature, feature is compared with the data in process database, if there is same feature weld seam related data, then directly call corresponding polishing parameter in process database, otherwise the feature extracted is input pre-trained DDPG reinforcement learning model and generates polishing parameter;S5, polishing is carried out to weld seam based on polishing parameter by robot arm, then weld seam surface is scanned again, judge whether to polish again, if yes, return step S3, otherwise enter step S6;S6, relevant parameters are recorded to process database;S7, remove workpiece fixation and transport it to discharge frame;The application can be self-adapting different specifications weld seam and generate corresponding polishing parameter.

Description

Technical Field

[0001] This invention relates to robotic weld grinding, and more particularly to a robotic weld grinding control method, system, and storage medium based on the DDPG reinforcement learning algorithm. Background Technology

[0002] For grinding the straight weld seams on the outer surface of roller wheel rims with diameters of 1100mm to 1800mm, traditional grinding methods mainly rely on manual operation. The grinding site produces a lot of dust particles and is noisy, which has many disadvantages, including the serious harm to the human body caused by prolonged exposure to dust; the complex machining of curved workpieces makes it difficult to maintain precision by manual operation; workers are easily affected by environmental factors, making it difficult to maintain high-precision work quality for a long time, resulting in poor efficiency and difficulty in ensuring consistent grinding quality.

[0003] Industrial robots are superior to human labor in terms of operational precision and anti-interference ability. The use of robots to replace manual grinding has achieved certain results, but there are still many shortcomings. The main shortcomings are the lack of real-time quality feedback and parameter iteration mechanism, reliance on manual input of grinding parameters, inability to autonomously adapt to new specifications of workpieces, and difficulty in adapting fixed parameters to grinding multiple specifications of workpieces due to the complex and diverse characteristics of weld seams. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a robotic weld grinding control method, system, and storage medium based on the DDPG reinforcement learning algorithm that can adaptively generate corresponding grinding parameters for welds of different specifications.

[0005] Technical solution: The robot weld grinding control method based on DDPG reinforcement learning algorithm of the present invention includes the following steps:

[0006] S1. Transport the workpiece to be ground to the support platform and fix it in place using an RGV transfer vehicle;

[0007] S2. The position of the arc-starting plates at both ends of the weld seam of the workpiece to be ground is confirmed by scanning with the vision sensor at the front end of the robotic arm and then cut. After cutting, the cutting tool is replaced with a grinding tool.

[0008] S3. Obtain point cloud data by scanning the weld seam of the workpiece to be ground using a vision sensor and perform noise reduction processing.

[0009] S4. The PointNet++ network is used to extract multiple features of the weld from the denoised point cloud data. The extracted features are compared with the data in the process database. If there is weld-related data with the same features, the corresponding grinding parameters in the process database are directly called. Otherwise, the extracted features are input into the pre-trained DDPG reinforcement learning model to generate grinding parameters.

[0010] S5. The robotic arm grinds the weld seam based on the grinding parameters. After grinding, the surface of the weld seam is scanned by a vision sensor. The system determines whether to grind again according to the preset quality assessment standard. If so, it returns to step S3; otherwise, it proceeds to step S6.

[0011] S6. Record the current workpiece parameters and corresponding grinding parameters to the process database;

[0012] S7. Release the workpiece from its fixed position and transport it to the unloading rack using an RGV transfer vehicle.

[0013] By extracting features using the PointNet++ network, it can directly process disordered and unstructured weld point cloud data. Through its hierarchical feature learning and local-global feature fusion, it accurately extracts multi-dimensional geometric features of the weld, such as width, depth, curvature, and weld height shape. Its interpolation strategy restores local features of missing regions, ensuring high discriminative power and robustness of feature representation. Its symmetric function reduces computation and improves feature extraction speed. The extracted features are compared with existing data in the process database. If weld grinding data with the same features exists, the corresponding grinding parameters can be directly called for rapid grinding. For weld types not found in the process database, the extracted features can be input into a pre-trained DDPG reinforcement learning model to generate corresponding grinding parameters. The robotic arm grinds the weld seam according to the grinding parameters, and then scans the weld seam surface through a vision sensor at its front end to evaluate the grinding quality in real time. If the quality is not up to standard, the above steps are repeated until the quality is up to standard. All relevant parameters of the entire grinding process are stored in the process database to provide a basis for subsequent grinding. This method can store a large amount of grinding data through the process database, enabling rapid data retrieval for subsequent grinding and improving work efficiency. Moreover, it can generate new grinding parameters based on the pre-trained DDPG reinforcement learning model, which can adapt to workpieces of various specifications. There is no need for manual input of grinding parameters, making it widely applicable and greatly improving work efficiency. In addition, the vision sensor can provide real-time feedback on grinding quality to ensure the final grinding effect and avoid the occurrence of unqualified grinding.

[0014] Preferably, the training process of the pre-trained DDPG reinforcement learning model in step S4 includes:

[0015] S101. Collect weld point cloud data of workpieces of various specifications under various environments and perform noise reduction;

[0016] S102. The PointNet++ network is used to extract multiple features of the weld from the denoised point cloud data as the state of the weld.

[0017] S103. Initialize the Actor network and Critic network. The Actor network generates the corresponding grinding action based on the state of all welds.

[0018] S104. Execute all polishing actions in the simulation environment and calculate their rewards, and store all the obtained interactive data in the experience replay buffer.

[0019] S105. Randomly sample multiple sets of interactive data from the experience replay buffer, update the Critic network parameters through the backpropagation algorithm, use the Q-value gradient output by the Critic network to calculate the policy gradient to update the Actor network parameters, and use a soft update strategy to synchronize the target network. Repeat this step until the model converges and training is completed.

[0020] By collecting weld point cloud data of workpieces of various specifications under different environments, the applicability of the pre-trained DDPG reinforcement learning model can be improved. The PointNet++ network is used to extract features. Through its multi-level feature fusion capability, the local details of the weld are combined with the global state, providing the DDPG model with feature vectors containing complete geometric and spatial information of the weld. This enhances the model's ability to learn the action strategies corresponding to grinding complex welds, thereby improving the generalization and decision accuracy of the pre-trained DDPG reinforcement learning model.

[0021] Preferably, in step S5, during the robotic arm grinding process, the changes in weld seam dimensions are acquired in real time using a vision sensor, and the cumulative cutting amount is calculated accordingly. When the cumulative cutting amount reaches a threshold, the grinding tool is replaced, and the cumulative cutting amount is recalculated. The formula for calculating the cumulative cutting amount is as follows:

[0022] V=ΔW×ΔT×L

[0023] Where V is the cumulative cutting amount, ΔW is the change in the width dimension of the weld, ΔT is the change in the thickness dimension of the weld, and L is the ground length.

[0024] The cumulative cutting amount is monitored and calculated in real time by a vision sensor. When the cumulative cutting amount reaches a threshold, it is determined that the tool wear is relatively serious. Continuing to use the tool for grinding is not only inefficient but also results in poor grinding effect. Replacing the tool accordingly can ensure the final grinding quality.

[0025] Preferably, the preset quality evaluation criteria in step S5 include roughness criteria and indentation depth criteria. When both criteria meet the corresponding thresholds, no further polishing is required; otherwise, further polishing is required.

[0026] Roughness and indentation depth can reflect the flatness of the polished surface. Only when both meet the corresponding threshold requirements can the polished surface be considered to be flat enough. Using these two parameters as preset quality assessment standards can not only reasonably assess the polishing quality, but also easily obtain these two parameters through vision sensors, reducing the complexity of assessment and improving work efficiency.

[0027] The robot weld grinding control system based on the DDPG reinforcement learning algorithm described in this invention includes:

[0028] Workpiece pretreatment module: Used to transport the workpiece to be ground to the support platform and fix it in place via an RGV transfer vehicle; the position of the arc-starting plates at both ends of the weld seam of the workpiece to be ground is confirmed by the vision sensor at the front end of the robotic arm and then cut; after cutting, the cutting tool is replaced with a grinding tool.

[0029] Data acquisition and preprocessing module: used to acquire point cloud data by scanning the weld seam of the workpiece to be ground using a vision sensor and to perform noise reduction processing;

[0030] Grinding parameter generation module: It is used to extract multiple features of the weld from the denoised point cloud data using the PointNet++ network, compare the extracted features with the data in the process database, and if there is weld-related data with the same features, it directly calls the corresponding grinding parameters in the process database; otherwise, it inputs the extracted features into the pre-trained DDPG reinforcement learning model to generate grinding parameters.

[0031] Evaluation module: Used to grind the weld seam using a robotic arm based on grinding parameters. After grinding, the weld seam surface is scanned by a vision sensor. The system determines whether to grind again based on preset quality evaluation standards. If so, the system returns to the data acquisition and preprocessing module; otherwise, it enters the recording module.

[0032] Recording module: Used to record the current workpiece parameters and corresponding grinding parameters to the process database;

[0033] Unloading module: Used to release the workpiece from its fixed position and transport it to the unloading rack via an RGV transfer vehicle.

[0034] The computer-readable storage medium for storing one or more programs according to the present invention includes one or more programs comprising instructions that, when executed by a computing device, cause the computing device to perform any of the methods described above.

[0035] Beneficial effects: Extracting weld features using the PointNet++ network efficiently parses disordered point cloud data, adapts to complex weld structures, and accurately captures key geometric attributes such as weld width, curvature, and weld height, generating highly discriminative and interpretable features. This achieves end-to-end automation and meets real-time requirements. The extracted features are input into a pre-trained DDPG reinforcement learning model to generate grinding parameters that control the robotic arm for grinding. This method is adaptable to workpieces of various specifications, has wide applicability, eliminates the need for manual input of grinding parameters, and improves work efficiency. Furthermore, all grinding data is stored in a process database. After feature extraction, it can be compared with the process database. If welds with the same features exist, the corresponding grinding parameters can be directly called. As the number of grinding operations increases, the process database will gradually expand and improve, further enhancing work efficiency. Additionally, real-time monitoring and feedback of grinding quality through visual sensors ensures the smoothness of the final ground surface. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the overall structure of the robot system of the present invention;

[0037] Figure 2 This is a three-dimensional structural schematic diagram of the roller frame of the present invention;

[0038] Figure 3 This is a schematic diagram of the pneumatic clamping device in the roller frame of the present invention;

[0039] Figure 4 This is a schematic diagram of the structure of the robotic arm of the present invention;

[0040] Figure 5 This is a schematic diagram of the replaceable tool and its support structure, in which... Figure 5 a is a schematic diagram of the polishing tool and its support structure. Figure 5 b is a schematic diagram of the cutting tool and support structure;

[0041] Figure 6 This is a schematic diagram of the enclosed tool magazine 4 structure;

[0042] Figure 7 This is a hierarchical structure diagram of the DDPG reinforcement learning model;

[0043] Figure 8 This is a schematic diagram of the pre-training process for the DDPG reinforcement learning model.

[0044] Figure 9 This is a schematic diagram of the execution flow of the present invention;

[0045] Figure 10 This is a flowchart of the visual recognition process of the present invention;

[0046] Figure 11This is a flowchart of the process database data retrieval mechanism of the present invention. Detailed Implementation

[0047] As shown in the figure, the robot weld grinding control method based on the DDPG reinforcement learning algorithm described in this invention uses a robot system, specifically including a base, on which are provided an RGV transfer vehicle 1, a support platform 2, a robotic arm 3, a closed tool magazine 4, a robot control cabinet 5, an electrical control device 6, and a spiral duct 7.

[0048] The support platform 2 includes a collection box 2-1, a workbench 2-2, a support column 2-3, a fixed seat 2-4, roller cylinders 2-5, a worm gear reducer motor 2-6, a linear guide rail 2-7, and a pneumatic clamping device 2-8. The collection box 2-1 is installed on the workbench 2-2. After the arc-drawing plate is cut, it falls into the collection box 2-1. The RGV transfer car 1 is equipped with a platform with a protruding part for placing workpieces. The top of the RGV transfer car 1 is equipped with a retractable support column 2-3. The top of the support column 2-3 is connected to the platform for carrying workpieces. The workpieces are loaded and unloaded by raising and lowering the platform. The fixed seat 2-4 is installed on both sides of the workbench 2-2, and a platform is provided above it. The platform is equipped with a linear guide rail 2-7, and roller cylinders 2-5 are installed on the linear guide rail 2-7. Each fixed seat 2-4 on each side is equipped with two roller cylinders 2-5. The four roller cylinders 2-5 on both sides of the workbench 2-2 work together to clamp the workpieces. The workpiece is held in place; the internal transmission device moves the roller cylinder 2-5 along the linear guide rail 2-7 to achieve fine adjustment of the wheelbase (that is, the adjustment of the distance between the two roller cylinders 2-5). For large workpieces, the distance is increased to prevent instability or slippage during grinding due to a high center of gravity and too close distance; for small workpieces, the distance is reduced to increase the height of the workpiece, adapting to the grinding height range (800~1850mm) of the robotic arm 3; the worm gear reducer motor 2-6 is connected to the roller 2-5. The rotation of the motor drives the roller cylinder 2-5 to rotate, and the rotation of the roller cylinder 2-5 can also drive the straight weld seam of the workpiece to the upward position; the pneumatic clamping device 2-8 is installed in the middle of the worktable 2-2. After the workpiece is positioned, it extends downward and presses against the inner surface of the workpiece, while the roller cylinder 2-5 contacts the outer surface of the workpiece to form support. The roller cylinder 2-5 and the pneumatic clamping device 2-8 work together to fix the workpiece.

[0049] The grinding system includes a robotic arm 3 and a closed tool magazine 4. The robotic arm 3 is mounted on a base. The robot is equipped with a vision sensor, which scans the workpiece through joint movement. A quick-change male connector is installed at the end of the robot. The quick-change male connector connects to a quick-change female connector installed on a force-controlled belt sander 8 (grinding tool) or a force-controlled cutting machine 9 (cutting tool) to achieve fully automatic tool changing. The force-controlled cutting machine 9 is used to cut the arc-starting plate, and the force-controlled belt sander 8 performs burr removal, end face grinding, and chamfering grinding on the cutting area, followed by weld grinding. A support frame 10 is installed near the robotic arm 3. Two support frames 10 can be set up to place tools during grinding operations, making it easy for the robot to quickly pick them up and improving the system's operating efficiency. In other words, when the robotic arm 3 changes between two tools, the idle tools can be placed on the support frame 10.

[0050] The enclosed tool magazine 4 includes a cutting disc changing station 4-1 and an automatic belt changing station 4-2. The robot control cabinet 5 is equipped with a constant force floating monitoring module and a belt wear assessment module. The constant force floating monitoring module is communicatively connected to the force position compensator installed on the robot force-controlled belt sander. The constant force floating monitoring module is used to maintain a constant force state in real time during the grinding process. Under constant force conditions, the belt wear assessment module uses a vision sensor to obtain the workpiece width dimension change ΔW, thickness dimension change ΔT, and grinding length L during the grinding process, and approximates the cutting amount V. The calculation formula is as follows:

[0051] V=ΔW×ΔT×L

[0052] The system presets a cumulative cutting amount threshold V. th When V = V th When the abrasive belt reaches its maximum wear limit, the system immediately generates a control command to pause grinding. The robotic arm 3 then sends the force-controlled belt sander 8 into the enclosed tool magazine 4 and the automatic belt changing station 4-2 to replace the belt. The cutting quantity calculation parameter is reset to zero, and the calculation restarts before grinding resumes until the cutting quantity V accumulates back to the threshold. Similarly, when the cutting disc of the force-controlled cutting machine 9 needs to be replaced, the robotic arm 3 can also send the machine into the cutting disc changing station 4-1 to replace the disc.

[0053] In addition, when not performing grinding operations, the tool can be stored in a dual-station configuration. The dual-station configuration includes a sensor that can detect the approach of the robotic arm and send a signal to a sensor switch to open the storage door.

[0054] The support platform 2 is suitable for supporting and placing workpieces of various specifications, with a diameter range of 1100-1800mm and a length range of 1200-2500mm. The electrical control device 6 establishes a communication connection with each piece of equipment to automatically control and monitor the operation of the equipment. During system operation, the spiral duct 7 collects and processes the chips, grinding shavings, and dust generated during equipment operation. The robot control cabinet 5 is equipped with a controller for controlling the operation of each component of the robot system, including the movement of each component mentioned above (the robot systems used in this invention are all existing devices, so their specific control principles are not detailed), as well as collecting weld data and generating corresponding grinding data, which is the control method and core content described in this invention.

[0055] The control method of the present invention includes the following steps:

[0056] S1. The workpiece to be ground is transported to the support platform 2 and fixed using the RGV transfer vehicle 1;

[0057] Specifically, the workpiece to be ground is transported to the support platform 2 by the RGV transfer vehicle 1. The support column 2-3 drives the platform to descend, and the workpiece falls onto the roller cylinder 2-5. The roller cylinder 2-5 adjusts the wheelbase along the linear guide 2-7 according to the size of the workpiece, driven by the internal transmission device, and rotates the workpiece to the position with the weld seam facing upward. The pneumatic clamping device 2-8 extends downward and presses on the inner surface of the workpiece to fix the workpiece.

[0058] S2. The position of the arc-starting plates at both ends of the weld seam of the workpiece to be ground is confirmed by the vision sensor at the front end of the robotic arm 3 and then cut. After cutting, the cutting tool is replaced with a grinding tool.

[0059] The specific grinding robot 3 scans both ends of the workpiece weld seam using a vision sensor, obtains the position of the arc-starting plate, and then uses a force-controlled cutting machine 9 to cut it. The arc-starting plate automatically falls into the collection box 2-1 for recycling. After both ends are cut, the force-controlled belt sander 8 is replaced to perform end face grinding, chamfering grinding, and burr removal.

[0060] S3. Obtain point cloud data by scanning the weld seam of the workpiece to be ground using a vision sensor and perform noise reduction processing.

[0061] S4. The PointNet++ network is used to extract multiple features of the weld from the denoised point cloud data. The extracted features are compared with the data in the process database. If there is weld-related data with the same features, the corresponding grinding parameters in the process database are directly called. Otherwise, the extracted features are input into the pre-trained DDPG reinforcement learning model to generate grinding parameters.

[0062] Specifically, after extracting the weld features, the following steps are performed:

[0063] Step a: The system compares the obtained weld size information (including thickness, width, etc.) with the existing grinding data in the process database. First, it determines the range of the obtained weld size information, and then searches for the grinding data of the corresponding range in the grinding process database.

[0064] Step b: If the process database contains weld-related data with the same characteristics, i.e., welds of workpieces of the same specifications that have been ground before, then directly retrieve the corresponding grinding parameters from the database for this grinding operation; otherwise, proceed to step c.

[0065] Step c: If there is no weld-related data with the same characteristics in the database, that is, the weld of the workpiece of this specification has not been ground, the system calls the model, inputs the extracted weld features, and generates grinding parameters suitable for this operation.

[0066] The pre-training of the DDPG reinforcement learning model includes the following process:

[0067] S101. Collect weld point cloud data of workpieces of various specifications under various environments and perform statistical filtering to remove noise;

[0068] S102. The PointNet++ network is used to extract multiple features of the weld from the denoised point cloud data as the state of the weld. The multiple features include the width, depth, height, curvature, shape of the weld reinforcement, edge slope, circumferential position and radial position.

[0069] S103. Initialize the Actor network and Critic network. The Actor network generates corresponding grinding actions based on the state of all welds. The grinding actions are grinding parameters, including grinding pressure, grinding time, grinding posture, spindle speed, and end TCP point, etc.

[0070] S104. Execute all grinding actions in the simulation environment and calculate their rewards. The rewards are calculated using a reward function designed based on the grinding effect. Store all the obtained interactive data in the experience replay buffer. The interactive data includes the current state of the weld, the grinding action, the reward corresponding to the grinding action, and the next state of the weld. The next state of the weld is derived from the current state and the grinding action.

[0071] S105. Randomly sample multiple sets of interactive data from the experience replay buffer, update the Critic network parameters through the backpropagation algorithm, use the Q-value gradient output by the Critic network to calculate the policy gradient to update the Actor network parameters, and use a soft update strategy to synchronize the target network. Repeat this step until the model converges and training is completed.

[0072] Updating the Critic network parameters involves the following process:

[0073] The target Q value is calculated using the following formula.

[0074] y = r + γ·Q target (s',μ target (s'))

[0075] Where y is the target Q value, r is the current reward, γ is the discount factor, s' is the next state, and μ target It is the polishing action generated by the target Actor network, Q target The predicted value of the target Critic network;

[0076] The loss function L minimizes the mean squared error between the predicted Q-value and the target Q-value. The formula for calculating L is:

[0077]

[0078] Where Q(s,a) is the Q value predicted by the Critic network, s is the current state of the weld, a is the grinding action generated by the Actor network based on s, and N is the number of sets of interactive data sampled from the experience playback buffer.

[0079] The policy gradient is calculated according to the following formula:

[0080]

[0081] Where J is the objective function, i.e., the expected long-term cumulative reward; N is the number of sets of interaction data sampled from the experience replay buffer. The gradient of the Q value with respect to action a is predicted by the Critic network, representing the influence of a on the Q value under the current state s of the weld. a is the grinding action generated by the Actor network based on the current state s of the weld. The polishing action μ(s) generated by the Actor network with respect to its own parameters θ μ The gradient represents how adjusting the Actor parameters changes the generated action μ(s).

[0082] S5 and robotic arm 3 grind the weld seam based on grinding parameters. After the grinding parameter format is converted, it is transmitted to the robotic arm control system via Ethernet and TCP / IP protocol. After parsing, instructions are generated to drive each joint to move and execute grinding. During the process, the constant force floating monitoring module realizes constant force compensation, and the sanding belt wear assessment module monitors the sanding belt wear. If the wear is excessive, grinding is stopped and the belt is replaced.

[0083] After grinding, the weld surface is scanned by a vision sensor. The system determines whether to grind again based on the preset quality assessment standard. If so, the process returns to step S3; otherwise, it proceeds to step S6.

[0084] The preset quality assessment standards include roughness standards and indentation depth standards. When both meet the corresponding thresholds, no further polishing is required; otherwise, further polishing is required.

[0085] Roughness is calculated by measuring the arithmetic mean of the absolute values ​​of the distances from each point on the surface profile to the baseline using a vision sensor.

[0086] The depth of the depression is detected by constructing a three-dimensional model of the weld, identifying the depression area, and calculating the maximum depth.

[0087] The roughness standard range can be set to 6.3 μm ≤ R a The indentation depth standard can be set to no more than 0.4 mm, and other reasonable thresholds can be set according to the actual situation. If any preset quality assessment standard deviates far from its threshold, it is considered to have a serious defect. The system will immediately alarm and suspend grinding, waiting for manual intervention.

[0088] S6. Record the current workpiece parameters and corresponding grinding parameters to the process database; workpiece parameters include relevant information such as workpiece specifications and weld characteristics.

[0089] S7. Release the workpiece from the fixation and transport the workpiece to the unloading rack via RGV transfer vehicle 1.

[0090] Specifically, the pneumatic clamping device 2-8 retracts to release the clamp, the RGV transfer car 1 lifts its platform to support the workpiece, moves along its guide rail, and automatically transports the workpiece to the unloading rack. The workpiece is then lifted away to begin the next cycle.

[0091] The robot weld grinding control system based on the DDPG reinforcement learning algorithm of the present invention includes:

[0092] Workpiece pretreatment module: Used to transport the workpiece to be ground to the support platform and fix it in place via an RGV transfer vehicle; the position of the arc-starting plates at both ends of the weld seam of the workpiece to be ground is confirmed by the vision sensor at the front end of the robotic arm and then cut; after cutting, the cutting tool is replaced with a grinding tool.

[0093] Data acquisition and preprocessing module: used to acquire point cloud data by scanning the weld seam of the workpiece to be ground using a vision sensor and to perform noise reduction processing;

[0094] Grinding parameter generation module: It is used to extract multiple features of the weld from the denoised point cloud data using the PointNet++ network, compare the extracted features with the data in the process database, and if there is weld-related data with the same features, it directly calls the corresponding grinding parameters in the process database; otherwise, it inputs the extracted features into the pre-trained DDPG reinforcement learning model to generate grinding parameters.

[0095] Evaluation module: Used to grind the weld seam using a robotic arm based on grinding parameters. After grinding, the weld seam surface is scanned by a vision sensor. The system determines whether to grind again based on preset quality evaluation standards. If so, the system returns to the data acquisition and preprocessing module; otherwise, it enters the recording module.

[0096] Recording module: Used to record the current workpiece parameters and corresponding grinding parameters to the process database;

[0097] Unloading module: Used to release the workpiece from its fixed position and transport it to the unloading rack via an RGV transfer vehicle.

[0098] The computer-readable storage medium for storing one or more programs according to the present invention includes one or more programs comprising instructions that, when executed by a computing device, cause the computing device to perform any of the methods described above.

Claims

1. A robot weld grinding control method based on DDPG reinforcement learning algorithm, characterized in that, Includes the following steps: S1. Transport the workpiece to be polished to the support platform (2) and fix it in place using an RGV transfer vehicle (1); S2. The position of the arc-starting plates at both ends of the weld seam of the workpiece to be ground is confirmed by scanning the vision sensor at the front end of the robotic arm (3) and then cut. After cutting, the cutting tool is replaced with a grinding tool. S3. Obtain point cloud data by scanning the weld seam of the workpiece to be ground using a vision sensor and perform noise reduction processing. S4. The PointNet++ network is used to extract multiple features of the weld from the denoised point cloud data. The extracted features are compared with the data in the process database. If there is weld-related data with the same features, the corresponding grinding parameters in the process database are directly called. Otherwise, the extracted features are input into the pre-trained DDPG reinforcement learning model to generate grinding parameters. S5. The robotic arm (3) grinds the weld seam based on the grinding parameters. After grinding, it scans the weld seam surface through a vision sensor and determines whether to grind again according to the preset quality assessment standard. If so, it returns to step S3; otherwise, it proceeds to step S6. During the grinding process, the robotic arm (3) uses a vision sensor to obtain real-time changes in weld dimensions and calculates the cumulative cutting amount accordingly. When the cumulative cutting amount reaches a threshold, the grinding tool is replaced and the cumulative cutting amount is recalculated. The formula for calculating the cumulative cutting amount is as follows: ,in, This is the cumulative cutting amount. This represents the change in the width dimension of the weld. L represents the thickness variation of the weld, and L represents the length that has been ground. The preset quality assessment criteria include roughness criteria and indentation depth criteria. When both criteria meet the corresponding thresholds, no further grinding is required; otherwise, further grinding is required. S6. Record the current workpiece parameters and corresponding grinding parameters to the process database; S7. Release the workpiece from the fixation and transport the workpiece to the unloading rack via the RGV transfer vehicle (1).

2. The method according to claim 1, characterized in that: The training process of the pre-trained DDPG reinforcement learning model in step S4 includes: S101. Collect weld point cloud data of workpieces of various specifications under various environments and perform noise reduction; S102. The PointNet++ network is used to extract multiple features of the weld from the denoised point cloud data as the state of the weld. S103. Initialize the Actor network and Critic network. The Actor network generates the corresponding grinding action based on the state of all welds. S104. Execute all polishing actions in the simulation environment and calculate their rewards, and store all the obtained interactive data in the experience replay buffer. S105. Randomly sample multiple sets of interactive data from the experience replay buffer, update the Critic network parameters through the backpropagation algorithm, use the Q-value gradient output by the Critic network to calculate the policy gradient to update the Actor network parameters, and use a soft update strategy to synchronize the target network. Repeat this step until the model converges and training is completed.

3. The method according to claim 2, characterized in that: The multiple features of the weld extracted in step S102 include the weld width, depth, height, curvature, reinforcement shape, edge slope, circumferential position, and radial position.

4. The method according to claim 2, characterized in that: The interactive data in step S104 includes the current state of the weld, the grinding action, the reward corresponding to the grinding action, and the next state of the weld; the next state of the weld is derived from the current state and the grinding action.

5. The method according to claim 2, characterized in that: The process of updating the Critic network parameters in step S105 includes the following steps: The target Q value is calculated using the following formula. ， in, For the target Q value, For the current reward, It is a discount factor. The next state is... It is the polishing action generated by the target Actor network. The predicted value of the target Critic network; The loss function L minimizes the mean squared error between the predicted Q-value and the target Q-value. The formula for calculating L is: ， in, The Critic network predicts the Q-value, s represents the current state of the weld, and a represents the grinding action generated by the Actor network based on s. The number of sets of interactive data sampled from the experience playback buffer.

6. The method according to claim 2, characterized in that: In step S105, the policy gradient is calculated according to the following formula. ， in, Let be the objective function. The number of sets of interactive data sampled from the experience playback buffer. The gradient of the Q-value with respect to action a is predicted by the Critic network, where a is the grinding action generated by the Actor network based on the current state s of the weld. Polishing actions generated for the Actor network For its own parameters The gradient.

7. A robot weld grinding control system based on the DDPG reinforcement learning algorithm, characterized in that, The system includes: Workpiece pretreatment module: Used to transport the workpiece to be ground to the support platform and fix it in place via an RGV transfer vehicle; the position of the arc-starting plates at both ends of the weld seam of the workpiece to be ground is confirmed by the vision sensor at the front end of the robotic arm and then cut; after cutting, the cutting tool is replaced with a grinding tool. Data acquisition and preprocessing module: used to acquire point cloud data by scanning the weld seam of the workpiece to be ground using a vision sensor and to perform noise reduction processing; Grinding parameter generation module: It is used to extract multiple features of the weld from the denoised point cloud data using the PointNet++ network, compare the extracted features with the data in the process database, and if there is weld-related data with the same features, it directly calls the corresponding grinding parameters in the process database; otherwise, it inputs the extracted features into the pre-trained DDPG reinforcement learning model to generate grinding parameters. Evaluation module: Used to grind the weld seam using a robotic arm based on grinding parameters. After grinding, the weld seam surface is scanned by a vision sensor. The system determines whether to grind again based on preset quality evaluation standards. If so, the system returns to the data acquisition and preprocessing module; otherwise, it enters the recording module. During the robotic arm grinding process, vision sensors are used to acquire real-time data on weld seam size changes and calculate the cumulative cutting amount accordingly. When the cumulative cutting amount reaches a threshold, the grinding tool is replaced, and the cumulative cutting amount is recalculated. The formula for calculating the cumulative cutting amount is as follows: ,in, This is the cumulative cutting amount. This represents the change in the width dimension of the weld. L represents the thickness variation of the weld, and L represents the length that has been ground. The preset quality assessment criteria include roughness criteria and indentation depth criteria. When both criteria meet the corresponding thresholds, no further grinding is required; otherwise, further grinding is required. Recording module: Used to record the current workpiece parameters and corresponding grinding parameters to the process database; Unloading module: Used to release the workpiece from its fixed position and transport it to the unloading rack via an RGV transfer vehicle.

8. A computer-readable storage medium for storing one or more programs, characterized in that: The program includes one or more instructions that, when executed by a computing device, cause the computing device to perform any of the methods according to claims 1 to 6.

Images