Defective filter stick sorting method for reinforcement learning self-adaptive grabbing

Abstract

The invention provides a filter stick defective product sorting method for reinforcement learning self-adaptive grabbing, and relates to the technical field of filter stick sorting. The defects of the filter stick are recognized through a multi-view visual module, image coordinates are compensated in real time in combination with the speed of a conveying belt, and the image coordinates are accurately mapped to a three-dimensional working space of a mechanical arm; then, defect poses, production line states and mechanical arm parameters are input to a reinforcement learning module, so that the optimal grabbing action and clamping pressure are output, and efficient sorting of the driving mechanical arm and the soft clamping jaw is achieved; and meanwhile, the feedback evaluation unit acquires a grabbing result through a sensor, generates a reward signal optimization strategy network, and realizes continuous online learning. The technical problems that in a high-speed flexible production line, a traditional sorting system is insufficient in grabbing precision and high in false grabbing rate due to the fact that visual recognition and a mechanical grabbing strategy are disjointed are solved. According to the invention, the collaborative precision of defect identification and grabbing and the sorting efficiency in the filter stick production line are obviously improved.

Description

Technical Field

[0001] This invention relates to the field of filter rod sorting technology, specifically to a reinforcement learning adaptive grasping method for sorting defective filter rods. Background Technology

[0002] Traditional defect sorting methods on filter rod production lines mainly include manual visual inspection and fixed-track teach-pen robotic arm sorting. Manual visual inspection is labor-intensive, inefficient, and time-consuming; while teach-pen robotic arms improve automation, they lack flexibility, requiring frequent re-teaching and parameter adjustments when production batches, conveying speeds, or defect locations change, failing to meet the demands of modern high-speed, flexible production. Although the application of deep learning vision inspection technology has improved the accuracy of online defect identification, existing systems still suffer from a lack of effective coordination between visual recognition and robotic arm grasping actions, resulting in limited grasping accuracy, a high false grasping rate, and an inability to form true closed-loop adaptive control. Summary of the Invention

[0003] Therefore, this invention provides a reinforcement learning-based adaptive gripping method for sorting defective filter rods, addressing the technical problem that existing systems still suffer from limited gripping accuracy, high false-grip rates, and the inability to form true closed-loop adaptive control due to the lack of effective coordination between visual recognition and robotic arm gripping actions. This method achieves high-speed, high-precision, and flexible sorting of defective filter rods by accurately visually detecting the location of filter rod defects and using reinforcement learning to optimize the robotic arm's gripping strategy in real time, significantly reducing false-grip rates and maintenance costs.

[0004] The present invention provides a reinforcement learning-based adaptive grasping method for sorting defective filter rods, which mainly includes the following steps:

[0005] Step S1: Use the improved YOLOv8-LB model to identify filter rod defects in real time, and output the defect type, pixel coordinates and detection confidence.

[0006] Step S2: Combining the extrinsic calibration of the industrial camera and the real-time speed of the conveyor belt, the pixel coordinates of the visual recognition are mapped to the workspace coordinates of the robotic arm, and motion compensation is performed on the filter rod to predict the real-time three-dimensional pose of the defective filter rod.

[0007] Step S3: Based on the SAC framework, with the three-dimensional pose of the defective filter rod, the conveyor belt speed, the real-time status of the robotic arm, and the pressure at the end of the gripper as input states, output the incremental pose and gripping pressure of the optimal gripping action.

[0008] Step S4: The robotic arm adjusts its gripping posture and gripper pressure in real time according to the instructions output by the reinforcement learning unit, accurately gripping the defective filter rod and placing it in the defective product buffer area.

[0009] Step S5: Monitor the gripping effect of the robotic arm in real time using photoelectric sensors and vacuum pressure sensors, and generate feedback signals to update the reinforcement learning algorithm parameters to achieve closed-loop online learning optimization.

[0010] Further, in step S1, the camera images are first color-normalized using a mean-standard deviation standardization method to reduce the differences in image distribution under variations in white balance, exposure, and illumination from different cameras, thereby enhancing the consistency and robustness of the image input data. Secondly, to address potential distortion issues in industrial cameras, a joint radial and tangential distortion correction algorithm based on camera intrinsic parameters is applied to geometrically restore the image, ensuring that the filter rod's shape and defect areas are not distorted. Subsequently, a multi-scale image pyramid is used to enhance the filter rod's detailed texture, improving the saliency of small defect areas in the YOLOv8-LB model. Online augmentation techniques are then combined with image rotation, translation, blurring, and contrast perturbation to expand sample diversity, thereby enhancing the generalization ability and stability of the detection model.

[0011] Further, in step S2, firstly, based on the extrinsic calibration parameters of the industrial camera and the real-time acquired conveyor belt speed, a projection inverse transformation based on the PINHOLE camera model is used to map the pixel coordinates into three-dimensional coordinates, and then further transformed to the Cartesian working coordinate system of the robotic arm. To compensate for the position delay of the filter rod during high-speed movement, a temporal extrapolation prediction algorithm is used to calculate the target position of the filter rod after a certain period of time and predict its displacement.

[0012] The system dynamically corrects the position by combining the real-time conveyor belt speed with a system delay compensation factor, ensuring spatial consistency of the 3D pose at the moment of robotic arm grasping. Finally, the recognition results are smoothed by multi-frame timestamp fusion and Kalman filtering to reduce pose drift of the filter rod between consecutive frames caused by jitter or detection fluctuations, thereby improving the stability of trajectory prediction and grasping hit rate.

[0013] Further, in step S3, firstly, multi-source information such as the three-dimensional pose of the defect filter rod, the conveyor belt speed, the current joint angle and speed of the robotic arm, the real-time pressure of the end effector gripper, and the target confidence are fused to construct a state vector S, which is then input into a reinforcement learning policy network based on the Soft-Actor-Critic (SAC) algorithm. The policy network outputs a continuous action vector A containing the grasping posture increment and the gripping force F of the gripper. Its physical rationality is constrained by an action restriction mechanism, and the grasping speed and path are dynamically adjusted in conjunction with working parameters. Subsequently, a mechanism based on priority experience replay is used to store the state-action-reward results of each grasping action, and the policy network is updated online to form a continuously iterative and adaptively convergent grasping policy. Virtual simulation and Shadow-Domain-Randomization techniques are introduced during the training phase to improve the policy generalization ability, and safety limiting and soft-start mechanisms are adopted in actual deployment to avoid the risk of overfitting of the reinforcement learning policy.

[0014] Further, in step S4, the robotic arm, based on the motion commands output by the reinforcement learning module, first calculates the joint solution of the target end-effector pose using an inverse kinematics solver, and dynamically plans the optimal path to minimize path length and obstacle avoidance risk. Subsequently, the execution system drives the 6-DOF robotic arm to complete pose adjustment, while simultaneously controlling the end-effector soft gripper to compliantly close at a set pressure F. The vacuum pressure and micro-tactile sensors embedded in the gripper collect contact information in real time. If the gripping force deviates from a preset threshold, a fine-tuning closed-loop correction is immediately performed to prevent excessive compression or slippage of the filter rod. Furthermore, when the gripper approaches the target within 10mm, a secondary camera or visual fine-tuning module performs high-precision positioning of the filter rod defects and sub-millimeter-level fine-tuning of the end-effector pose, improving gripping and positioning accuracy and reducing the risk of secondary damage.

[0015] Further, in step S5, a photoelectric sensor is first used to detect whether the filter rod has been successfully grasped and placed into the defective product buffer area, obtaining a binary feedback signal indicating whether the grasp was successful. Simultaneously, a vacuum pressure sensor collects changes in the adsorption or clamping pressure within the grippers to determine whether the clamping state is stable or if there are any abnormal events such as misgrabbing or dropping. Based on the sensor results, the system constructs a reward function R: a +1 reward for successful grasping and placement, a -α penalty for failed grasping, and a -γ penalty for misgrabbing good products or damaging the filter rod. The higher the production cycle time, the greater the reward, dynamically coupling quality and efficiency factors to form a multi-objective optimization function. Finally, this reward signal is used for the online update of the policy network in the reinforcement learning decision unit, achieving closed-loop adaptive optimization of the sorting strategy and significantly improving the system's long-term grasping success rate and robustness.

[0016] The present invention has the following advantages over the prior art:

[0017] 1. The present invention provides a reinforcement learning adaptive grasping method for sorting defective filter rods, which integrates an improved YOLOv8-LB visual detection model, a conveyor belt motion compensation mechanism, a grasping decision algorithm based on off-policy deep reinforcement learning (Soft-Actor-Critic, SAC), and a multi-sensor feedback control module to achieve high-precision identification, high-speed and stable sorting, and compliant and safe grasping of defective filter rods.

[0018] 2. This invention provides a reinforcement learning-based adaptive grasping method for sorting defective filter rods. First, a multi-view vision module identifies filter rod defects, and the image coordinates are compensated in real-time by combining the conveyor belt speed, accurately mapping them to the three-dimensional workspace of a robotic arm. Then, the defect pose, production line status, and robotic arm parameters are input into the reinforcement learning module to output the optimal grasping action and clamping pressure, achieving efficient sorting by driving the robotic arm and soft grippers. Simultaneously, a feedback evaluation unit collects grasping results through sensors, generates reward signals to optimize the strategy network, and achieves continuous online learning. This system significantly improves the collaborative accuracy and sorting efficiency of defect identification and grasping in filter rod production lines, providing key technical support for the application of industrial robots in intelligent quality inspection, flexible manufacturing, and high-speed automated sorting. Attached Figure Description

[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the process for sorting defective filter rods based on visual defect recognition and reinforcement learning adaptive grasping according to the present invention.

[0021] Figure 2 This is a graph showing the experimental results comparing the performance of the filter rod grasping strategy before and after reinforcement learning in this invention.

[0022] Figure 3 This is a schematic diagram of the visual recognition and coordinate mapping process of the present invention;

[0023] Figure 4 This is a schematic diagram of the feedback and learning strategy optimization process of the present invention. Detailed Implementation

[0024] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Example 1

[0026] Reference Figures 1 to 4 This invention provides a reinforcement learning-based adaptive grasping method for sorting defective filter rods. The invention mainly includes the following steps: Step S1: A defect identification unit uses an improved YOLOv8-LB model to identify filter rod defects in real time, outputting the defect type, pixel coordinates, and detection confidence level.

[0027] Step S2: The coordinate mapping and trajectory prediction unit, combined with the external parameter calibration of the industrial camera and the real-time speed of the conveyor belt, maps the pixel coordinates of the visual recognition to the workspace coordinates of the robotic arm, and performs motion compensation on the filter rod to predict the real-time three-dimensional pose of the defective filter rod.

[0028] Step S3: The reinforcement learning decision unit, based on the SAC framework, takes the three-dimensional pose of the defect filter rod, the conveyor belt speed, the real-time state of the robotic arm, and the end pressure of the gripper as input states, and outputs the incremental pose and gripping pressure of the optimal gripping action.

[0029] Step S4: The robotic arm execution unit adjusts the gripping posture and gripper pressure in real time according to the instructions output by the reinforcement learning unit, accurately gripping the defective filter rod and placing it in the defective product buffer area.

[0030] Step S5: Feedback evaluation unit monitors the robotic arm's grasping effect in real time through photoelectric sensors and vacuum pressure sensors, and generates feedback signals to update the reinforcement learning algorithm parameters, thereby achieving closed-loop online learning optimization.

[0031] In step S1, the camera image data is first normalized using mean-standard deviation normalization:

[0032]

[0033] in, For the original image at pixel location The Channel value, These are the normalized image values. and These are the mean and standard deviation of each RGB channel, often using the standard values ​​defined in the pre-trained model of the ImageNet dataset.

[0034] Then, distortion correction is performed on the camera image, which involves remapping the image coordinates using the camera's intrinsic parameter matrix and distortion parameters, calculated as follows:

[0035]

[0036]

[0037] in, To normalize image coordinates, These are the coordinates after distortion correction. The square of the radial distance. The radial distortion coefficient is... The intrinsic parameter matrix used for the tangential distortion coefficients is:

[0038]

[0039] in, For the camera's focal length, The coordinates of the principal point (optical center) are then used. Next, the camera image is scaled using an affine transformation matrix, as shown below:

[0040]

[0041] in, The coordinates of the original image are... The transformed target coordinates, These are the scaling factors in the X and Y directions. The translation offset is used to resize the image to a uniform standard for subsequent multi-model fusion. Then, a multi-scale image pyramid construction strategy is employed, performing Gaussian blur and downsampling operations on the image to construct the image pyramid. ,in:

[0042]

[0043] in, For the first Layer image, For Gaussian kernel function, The sampling locations are defined in the original image. Finally, using online data augmentation techniques, random processing is applied to the training images, including rotation, translation, blurring, brightness perturbation, and contrast enhancement, to construct a sample perturbation space.

[0044]

[0045] in, rotate,shift,blur,contrast,noise This represents a random combination function for image enhancement operations, designed to improve the robustness and generalization ability of the YOLOv8-Plus model to defect diversity.

[0046] In step S2, firstly, based on the external parameter calibration parameters of the industrial camera and the real-time acquired conveyor belt speed, the pixel coordinates are mapped to three-dimensional world coordinates using an inverse projection transformation based on the PINHOLE camera model.

[0047]

[0048] in, These are the pixel coordinates in the image. For the estimated depth value, A point in the camera coordinate system. A three-dimensional point in the world coordinate system. For the camera intrinsic parameter matrix, , These are the camera's rotation matrix and translation vector, derived from the extrinsic calibration results. Next, temporal extrapolation is used to predict its... The target position after the specified time is calculated using the following formula:

[0049]

[0050] in, For the real-time speed of the conveyor belt, The time delay from system recognition to execution, This represents the predicted 3D position of the filter rod after compensation. Then, multi-frame temporal fusion and Kalman filtering are used to smooth the recognition results.

[0051]

[0052] in, The estimated pose of the filter rod in the current frame. This represents the observation position of the current detection frame. This is the Kalman gain at the current moment.

[0053] Defined as:

[0054]

[0055] in, It is the prediction error covariance. To observe the noise covariance.

[0056] Kalman filtering is used to smooth the filter rod trajectory on the time axis, reducing pose fluctuations caused by recognition jitter, noise, and object vibration.

[0057] In step S3, the state vector S is first constructed by fusing multi-source information such as the three-dimensional pose of the defect filter rod, the conveyor belt speed, the current joint angle and speed of the robotic arm, the real-time pressure of the end effector gripper, and the target confidence level. The form is as follows:

[0058]

[0059] in, The three-dimensional pose of the filter rod at time t, including its position and orientation; : Conveyor belt speed; , The current angles and angular velocities of each joint of the robotic arm; : End gripper pressure sensor reading; The detection model scores the confidence level of defects. This state vector is input into a policy network built on a Soft-Actor-Critic (SAC) architecture, which outputs a continuous action vector.

[0060]

[0061] in: Spatial adjustment of the capture position. End attitude adjustment angle, The required gripper force setting is used to constrain the motion output through a constraint function to ensure the physical rationality of the action.

[0062]

[0063] in, These are the upper and lower safety limits for each action dimension.

[0064] During reinforcement learning training, each capture result The data is written into the priority experience replay pool and used for network updates according to TD error priority sampling.

[0065]

[0066] in: It is the current action value function. It is the action of the policy network in the next state. As a discount factor, The reward value for the current step is defined as follows:

[0067]

[0068] Furthermore, to enhance the generalization ability of the strategy, a Shadow-Domain-Randomization virtual simulation perturbation mechanism is introduced during the training phase to change environmental parameters such as background lighting, filter material texture, and belt speed jitter, thereby constructing a dynamic domain.

[0069]

[0070] in, It is used to simulate real-world changing conditions by providing diverse environmental distributions.

[0071] In actual deployment, a safety limiting function is used to ensure system stability:

[0072]

[0073] in, To mitigate the initial impact of actions, the system initially employs low-intensity actions to prevent overfitting of the reinforcement learning strategy.

[0074] In step S4, the robotic arm, based on the motion commands output by the reinforcement learning module, first calculates the joint angle solutions for the target's end-effector pose using an inverse kinematics solver:

[0075]

[0076] in: The solution obtained Joint angles of a degree-of-freedom robotic arm ( or ); The target end-effector pose matrix, constructed from the output of the policy network, includes translation and rotation components. This is then combined with the current joint state of the robotic arm. We use dynamic path planning based on cost functions to minimize path costs:

[0077]

[0078] Where: the first term is the path smoothness cost, and the second term is the obstacle avoidance risk cost. For weight parameters, To plan the total duration. After reaching the target point at the end, control the grippers to perform a compliant closing operation, and set the target gripping pressure. And establish a closed-loop force control model:

[0079]

[0080] in: The vacuum pressure / tactile sensor data measured in real time by the gripper. Force control gain parameter, if The system will automatically trigger a closed-loop fine-tuning process to avoid overpressure or disengagement.

[0081] When the gripper approaches the target area of ​​the filter rod, it is smaller than At this time, the secondary camera activates its high-precision fine-tuning mode to perform visual fine-tuning:

[0082]

[0083] in: The target pose after refinement by the secondary camera. : indicates incremental attitude superposition operation (can be SE(3) pose perturbation superposition). : The final execution is to capture the pose.

[0084] In step S5, the photoelectric sensor is first used to determine whether the filter rod has been successfully grasped and placed, and a binary feedback signal indicating successful grasp is obtained:

[0085]

[0086] Meanwhile, the vacuum pressure sensor inside the gripper collects the pressure change values ​​before and after gripping:

[0087]

[0088] in, and These represent the gripper vacuum / gripping pressure values ​​before and after the gripping action. If the condition is not met, it is determined that the clamping has failed or the object has slipped. A reward function is constructed based on the multimodal sensor information described above. The format is as follows:

[0089]

[0090] in: Capture failure penalty factors. Mistakenly seizing good products or damaging items will result in severe punishment.

[0091] , Number of items successfully sorted per unit time (production cycle time). Beat weights encourage high-speed and stable grasping. The reward function couples accuracy and beat weights, serving as a composite objective for optimizing the reinforcement learning policy. This reward signal is linked to the corresponding state-action transition pair. Write it into the experience pool together:

[0092]

[0093] It is also used for online parameter updates of the policy network to achieve closed-loop adaptive optimization of the crawling policy:

[0094]

[0095] in, For the policy network learning rate, For action value functions, For policy networks, reinforcement learning improves long-term crawling efficiency and stability through continuous sampling and updating.

[0096] Example 2

[0097] This embodiment was conducted on a computer equipped with an Intel i9-14900KF processor and an Nvidia RTX4090 graphics processor (64GB of RAM). The deep learning framework used was PyTorch (CUDA 12.1), the development and debugging platform was PyCharm, and the programming language was Python 3.8. All comparison algorithms were performed in the same environment. Evaluation metrics included average reward and crawling success rate. Process logs and model weights were recorded and saved in the same directory.

[0098] from Figure 2 The average return trend curve in the upper half shows that the average return value of the traditional control method remained stable at around 950 for a long time, while after introducing the reinforcement learning algorithm, the average return value of the system rapidly increased and stabilized at around 2700 after about 200,000 steps. This indicates that the reinforcement learning decision unit of this invention can continuously optimize the grasping strategy through interactive learning, thereby significantly improving the overall system yield.

[0099] Figure 2 The trend curve of the success rate of grasping in the lower half of the image shows that the traditional method has a low grasping success rate and fluctuates significantly (approximately 25%) over long-term operation. However, after adopting reinforcement learning, the success rate gradually increases with the training process and eventually stabilizes at approximately 90%. This indicates that the reinforcement learning module can effectively combine visual recognition and multi-sensor feedback signals to autonomously adjust the grasping posture and clamping force, achieving stable convergence of the sorting action.

[0100] Experimental results fully demonstrate that the method of the present invention can achieve adaptive optimization and online learning of the grasping strategy under complex dynamic working conditions, significantly improving the sorting accuracy of filter rods and the robustness of the system.

[0101] This embodiment verifies the effectiveness of the method of the present invention through comparative experiments. Under the same visual recognition module and robotic arm hardware conditions, the experiments used both the traditional fixed trajectory control method and the method incorporating reinforcement learning optimization strategies for training and evaluation of the grasping task. The experimental results are as follows: Figure 2 As shown, the horizontal axis represents the number of training steps, and the vertical axes represent the average reward value and the capture success rate, respectively.

[0102] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for sorting defective filter rods using reinforcement learning-based adaptive grasping, characterized in that, Includes the following steps: S1: Real-time identification of filter rod defects is performed using the improved YOLOv8-LB model, and the defect type, pixel coordinates and detection confidence are output. S2: Combining the external parameter calibration of the industrial camera and the real-time speed of the conveyor belt, the pixel coordinates of the visual recognition are mapped to the workspace coordinates of the robotic arm, and motion compensation is performed on the filter rod to predict the real-time three-dimensional pose of the defective filter rod. S3: Based on the Soft-Actor-Critic framework, it takes the three-dimensional pose of the defect filter rod, the conveyor belt speed, the real-time status of the robotic arm, and the pressure at the end of the gripper as input states, and outputs the incremental pose and gripping pressure of the optimal gripping action. S4: The robotic arm adjusts its gripping posture and gripper pressure in real time according to the instructions output by the reinforcement learning unit, accurately grabbing the defective filter rod and placing it in the defective product buffer area. S5: Real-time monitoring of the robotic arm's grasping effect via photoelectric sensors and vacuum pressure sensors, generating feedback signals to update reinforcement learning algorithm parameters, achieving closed-loop online learning optimization.

2. The reinforcement learning adaptive grasping method for sorting defective filter rods according to claim 1, characterized in that, In step S1, the camera images are first subjected to color normalization processing. The mean-standard deviation normalization method is used to reduce the differences in image distribution between different cameras under changes in white balance, exposure and illumination, thereby enhancing the consistency and robustness of the image input data.

3. The reinforcement learning adaptive grasping method for sorting defective filter rods according to claim 1, characterized in that, In step S1, a joint correction algorithm for radial and tangential distortion based on camera intrinsic parameters is applied to geometrically restore the image. Multi-scale image pyramids are used to enhance the detail texture of the filter rods to improve the saliency of small defect regions in the YOLOv8-LB model. Online augmentation techniques are combined to expand sample diversity by rotating, translating, blurring, and perturbing the image with contrast.

4. The reinforcement learning adaptive grasping method for sorting defective filter rods according to claim 3, characterized in that, In step S2, firstly, based on the external parameter calibration parameters of the industrial camera and the real-time acquired conveyor belt speed, the pixel coordinates are mapped to three-dimensional coordinates using the projection inverse transformation based on the PINHOLE camera model, and further transformed to the Cartesian working coordinate system of the robotic arm; the time-series extrapolation prediction algorithm is used to calculate the target position of the filter rod after a period of time and predict the displacement.

5. The reinforcement learning adaptive grasping method for sorting defective filter rods according to claim 4, characterized in that, In step S2, dynamic correction is performed by combining the real-time speed of the conveyor belt and the system delay compensation factor to ensure that the three-dimensional pose still has spatial consistency at the moment of grasping by the robotic arm; the recognition results are smoothed by multi-frame timestamp fusion and Kalman filtering.

6. The reinforcement learning adaptive grasping method for sorting defective filter rods according to claim 5, characterized in that, In step S3, the state vector S is first constructed by fusing multi-source information such as the three-dimensional pose of the defect filter rod, the speed of the conveyor belt, the current joint angle and speed of the robotic arm, the real-time pressure of the end gripper, and the target confidence. This state vector S is then input into the reinforcement learning policy network constructed based on the Soft-Actor-Critic algorithm.

7. The reinforcement learning adaptive grasping method for sorting defective filter rods according to claim 6, characterized in that, The policy network outputs a continuous motion vector A containing the grasping posture increment and the gripper gripping force F. The physical rationality of the vector is constrained by the motion restriction mechanism, and the grasping speed and path are dynamically adjusted in combination with the working condition parameters. The state-action-reward result of each grasp is stored using a mechanism based on priority experience replay, and the policy network is updated online to form a continuously iterative and adaptively convergent grasping strategy. Virtual simulation and Shadow-Domain-Randomization techniques are introduced during the training phase to improve the generalization ability of the policy, and safety limiting and slow start mechanisms are adopted in actual deployment to avoid the risk of overfitting of the reinforcement learning policy.

8. The reinforcement learning adaptive grasping method for sorting defective filter rods according to claim 7, characterized in that, In step S4, the robotic arm calculates the joint solution of the target end pose using an inverse kinematics solver based on the motion instructions output by the reinforcement learning module, and dynamically plans the optimal path to minimize the path length and obstacle avoidance risk. Subsequently, the execution system drives the 6-DOF robotic arm to complete the pose adjustment, while controlling the end soft gripper to compliantly close at a set pressure F. The vacuum pressure and micro tactile sensors embedded in the gripper collect contact information in real time. If the gripping force deviates from the preset threshold, a fine-tuning closed-loop correction is immediately performed to prevent the filter rod from being over-compressed or slipping.

9. The reinforcement learning adaptive grasping method for sorting defective filter rods according to claim 8, characterized in that, In step S5, a photoelectric sensor is first used to detect whether the filter rod has been successfully grabbed and placed into the defective product buffer area, and a binary feedback signal indicating whether the grabbing was successful is obtained. At the same time, a vacuum pressure sensor collects the changes in adsorption or clamping pressure in the gripper to determine whether the clamping state is stable or whether there are abnormal events such as mis-grabbing or dropping.

10. The reinforcement learning adaptive grasping method for sorting defective filter rods according to claim 9, characterized in that, A reward function R is constructed based on the sensor results: a +1 reward is given for successful grabbing and placement, a -α penalty is given for failed grabbing, and a -γ penalty is given for misgrabbing good products or damaging the filter rod. The higher the production cycle, the greater the reward. The dynamic coupling of quality and efficiency factors forms a multi-objective optimization function. Finally, this reward signal is used for the online update of the policy network of the reinforcement learning decision unit to achieve closed-loop adaptive optimization of the sorting strategy.

Images