Material sorting control method and system based on PLC control

Abstract

The invention discloses a material sorting control method and system based on PLC control, and relates to the technical field of industrial robot control, and the method comprises the steps: periodically obtaining a sorting area material image, and synchronously latching the real-time operation parameters of a conveying mechanism; outputting material pose information and a grabability evaluation result by a recognition model based on the image; generating a grabbing control instruction by combining the pose, the evaluation result and the operation parameters, and issuing and executing the grabbing control instruction; acquiring execution feedback data; when the feedback meets a preset condition, the feedback and the corresponding image form a training sample to perform incremental learning updating on the recognition model; and the updated model is used for circulating operation in subsequent sorting. According to the method, sensing, compensation and learning closed loop is achieved through PLC deterministic scheduling, continuous production is guaranteed through a controlled starting and rollback mechanism, the sorting efficiency and stability are continuously improved under the condition that the material category, illumination and the tail end state change, and the empty grabbing cost, the mistaken grabbing cost, the mistaken throwing cost and the shutdown maintenance cost are reduced. The comprehensive benefits are obvious.

Description

Technical Field

[0001] This invention relates to the field of industrial robot control technology, specifically to a material sorting control method and system based on PLC control. Background Technology

[0002] With the development of industrial automation and intelligent manufacturing, material sorting has gradually evolved from traditional manual sorting to a collaborative operation mode of "PLC cycle control + conveying actuator + robot grasping". In recent years, the maturity of technologies such as industrial cameras, 3D imaging, industrial Ethernet and edge computing has enabled the widespread application of machine vision-based target localization and posture estimation on the production line. At the same time, the robustness of deep learning recognition models in complex appearance, weak texture, occlusion and lighting fluctuation scenarios has been continuously improved, driving the sorting system to expand from simple "recognition and localization" to "operable decision-making oriented towards grasping execution". The supporting control system is also gradually showing a trend of integration of "real-time performance, security and data": PLC is responsible for deterministic scheduling and safety interlocking, and computing unit is responsible for inference calculation and strategy output, forming an online sorting control link oriented towards cycle constraints.

[0003] However, existing technologies still generally suffer from bottlenecks that hinder long-term stable operation under complex working conditions. Firstly, traditional vision sorting relies heavily on static image inference. While it can output target location, it fails to adequately consider the coupling of dynamic factors such as conveyor speed fluctuations, start-stop, and acceleration / deceleration. This leads to accumulated deviations in grasping timing compensation and target prediction, resulting in false grasps, empty grasps, or landing point offsets. Secondly, most systems have limited collection and semantic understanding of execution-side feedback. Grasping success / failure often relies on a single signal or manual statistics, lacking a task-level correlation mechanism that corresponds one-to-one with "image—recognition output—control command." This makes it difficult to attribute anomalies, trace samples, and form a closed-loop data chain for analysis and optimization. Thirdly, recognition models typically rely on offline retraining and manual annotation. When faced with the introduction of new materials, changes in packaging appearance, light drift, and wear of end effectors, the deviation between model output and actual graspability gradually widens. Without an incremental learning mechanism based on on-site execution results, the system can only maintain availability by downtime parameter tuning or model version replacement, making it difficult to achieve continuous adaptive optimization driven by "execution results." Therefore, existing technologies struggle to achieve traceable, attributable, and sustainably evolving intelligent sorting capabilities without adding dedicated training processes or disrupting production line cycle time and safety constraints. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by this invention is that existing PLC-based vision sorting control methods suffer from insufficient synchronization between images and conveyor motion states, inaccurate dynamic compensation, imperfect correlation between execution feedback acquisition and task level, difficulty in traceable attribution of abnormal results, reliance on offline retraining of recognition models, difficulty in coping with performance degradation caused by on-site distribution drift, and the problem of how to use execution results to drive incremental updates of recognition models and continuously improve sorting stability without affecting production line cycle time.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, embodiments of the present invention provide a material sorting control method based on PLC control, characterized by comprising the following steps: S1: Periodically acquire material images of the sorting area and obtain real-time operating parameters of the conveyor mechanism; S2: Based on the material image, the material is processed by a recognition model to obtain the material's pose information and its graspability evaluation results; S3: Combining the pose information, graspability assessment results, and real-time operating parameters of the conveying mechanism, generate a robotic arm grasping control command and issue it to execute the sorting operation; S4: Obtain the execution feedback data of the sorting operation; S5: When the execution feedback data meets the preset conditions, the execution feedback data and the corresponding material image are used as training samples to incrementally learn and update the recognition model. S6: In subsequent sorting, the updated recognition model is used, and steps S1 to S5 are repeated.

[0007] As a preferred embodiment of the material sorting control method based on PLC control described in this invention, the periodic acquisition of material images in the sorting area includes: the PLC triggering an imaging device to acquire material images according to a preset sampling period, or the PLC triggering acquisition based on an event signal related to the arrival of materials; and the PLC performing gating control on the acquisition based on the entry or exit of materials from the sorting area; wherein the acquired material images are associated with sampling sequence numbers or time identifiers for subsequent data pairing.

[0008] As a preferred embodiment of the material sorting control method based on PLC control described in this invention, the method of obtaining the real-time operating parameters of the conveying mechanism includes: the PLC obtaining the real-time speed and / or running direction of the conveying mechanism through industrial communication interface and / or encoder feedback; and the PLC latching the real-time operating parameters within the same control cycle when the imaging device is triggered to acquire material images, so that the real-time operating parameters are paired with the corresponding material images according to the sampling sequence number or time identifier.

[0009] As a preferred embodiment of the material sorting control method based on PLC control described in this invention, the recognition model is configured to analyze and process the material image and simultaneously output a recognition result including at least the material pose information and the graspability evaluation result; the recognition model is triggered to execute by the PLC or its output result is called by the PLC.

[0010] As a preferred embodiment of the material sorting control method based on PLC control described in this invention, the method of obtaining the pose information of the material and its graspability evaluation results includes: locating the material target in the material image to obtain target area information; Based on the target area information, the position and orientation of the material target relative to the preset reference coordinate system are determined as the pose information; Based on the pose information and the material image, a graspability evaluation result is generated to characterize the feasibility or reliability of at least one candidate grasping scheme.

[0011] As a preferred embodiment of the PLC-based material sorting control method of the present invention, the generation of robotic arm gripping control instructions includes: Based on the pose information and the graspability evaluation result, at least one candidate grasping scheme is generated, wherein the candidate grasping scheme includes at least a candidate grasping pose and an end effector motion mode. Based on the crawlability assessment results, a target crawling scheme is determined from the candidate crawling schemes; Based on the real-time operating parameters of the conveying mechanism, at least one of the grasping pose and grasping timing corresponding to the target grasping scheme is dynamically corrected; Based on the dynamically corrected results, a grasping control command is generated that includes the target grasping pose, end effector motion parameters, and deployment position. The PLC sends the gripping control command to the robot controller to execute the sorting operation.

[0012] As a preferred embodiment of the material sorting control method based on PLC control described in this invention, the execution feedback data includes at least one or more of the following: the grasping result, execution deviation, and execution process stability. Among them, the feedback data used to characterize the grasping result includes status feedback indicating grasping success or grasping failure; the feedback data used to characterize the execution deviation includes deviation feedback indicating grasping pose deviation and / or drop point deviation; and the feedback data used to characterize the stability of the execution process includes process feedback indicating adsorption or clamping establishment status, load change or motion following error. The PLC then associates and binds the feedback data with the task identifier and / or target identifier corresponding to the sorting operation to form traceable structured execution feedback data for subsequent sample screening or model update judgment.

[0013] As a preferred embodiment of the material sorting control method based on PLC control described in this invention, the incremental learning and updating of the recognition model includes: The PLC associates the execution feedback data with the corresponding material image based on the task identifier and / or target identifier, and determines whether the execution feedback data meets preset conditions; wherein, the preset conditions include at least one of the following: the execution feedback data indicates grasping failure or abnormal placement, execution deviation exceeds a preset threshold, process is unstable, the confidence level of the pose reliability or graspability assessment output by the recognition model is lower than the threshold, and the failure rate or deviation rate increases within a preset statistical window; When the preset conditions are met, the PLC pairs the material image with the execution feedback data based on the task identifier and / or target identifier to generate training samples, and the supervision label of the training samples is generated by the execution feedback data; The recognition model is then incrementally learned and updated using the training samples.

[0014] Secondly, embodiments of the present invention provide a material sorting control system based on PLC control, comprising: Image and conveying parameter acquisition module: periodically acquires material images of the sorting area and obtains real-time operating parameters of the conveying mechanism; Recognition and graspability assessment module: Based on the material image, the module processes the image using a recognition model to obtain the material's pose information and its graspability assessment results. Grasping control command generation and issuance module: Combining the pose information, graspability assessment results and real-time operating parameters of the conveying mechanism, it generates grasping control commands for the robotic arm and issues them to execute sorting operations; Execution feedback data acquisition module: Acquires the execution feedback data of the sorting operation; Training sample construction and incremental learning update module: When the execution feedback data meets the preset conditions, the execution feedback data and the corresponding material image are used as training samples to perform incremental learning and update of the recognition model; Model version management and closed-loop operation module: In subsequent sorting, the updated recognition model is used, and steps S1 to S5 are repeated.

[0015] The beneficial effects of this invention are as follows: By using a PLC as the deterministic scheduling core, a closed-loop link is constructed for "image acquisition—motion parameter synchronization—recognition and reasoning—dynamic compensation control—execution feedback acquisition—incremental learning and updating," achieving high-precision dynamic grasping under continuous conveyor belt operation conditions, significantly reducing the rate of missed grasps and false grasps; it endows the system with online incremental learning capabilities, enabling it to automatically adapt to new materials and changes in working conditions, greatly reducing manual debugging and downtime; ultimately, it forms an intelligent control method that can continuously self-optimize along with production and continuously improve sorting efficiency and stability. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 The first embodiment of the present invention provides an overall flowchart of a material sorting control method based on PLC control; Figure 2 The diagram below shows the module connection of a PLC-based material sorting control system, which is provided as a third embodiment of the present invention. Detailed Implementation

[0017] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0018] Example 1, referring to Figure 1 As an embodiment of the present invention, a material sorting control method based on PLC control is provided.

[0019] S1: Periodically acquire material images of the sorting area and obtain real-time operating parameters of the conveyor mechanism.

[0020] Step S1 is used to obtain visual information of the materials to be sorted in the sorting area under a preset sampling cycle, and simultaneously obtain the real-time operating parameters of the conveying mechanism. This provides the same-time basic input for pose estimation, graspability assessment and robot gripping control command generation in subsequent steps, so that the material state reflected by the image information is corresponding to the conveying motion state, and reduces the gripping timing and position deviation caused by fluctuations in conveying speed, changes in start and stop or uneven material spacing.

[0021] The "periodic acquisition" can be triggered by the PLC control system at fixed time intervals or by event-based timing. When triggered by fixed time intervals, the PLC drives the imaging device to continuously acquire data according to a preset sampling period, suitable for scenarios with relatively stable conveyor speeds. When triggered by event-based timing, the PLC can combine the material presence signal from upstream detection sensors, the status of the stop mechanism, or changes in the conveyor encoder displacement to trigger acquisition, establishing a correlation between the image acquisition time and the time when the material enters the sorting area, suitable for scenarios with fluctuating conveyor speeds or uneven material arrival. Furthermore, the PLC can be configured with gating logic to start acquisition when material is detected entering the sorting area, and stop acquisition or reduce the acquisition frequency when the material leaves the sorting area or exceeds a preset effective window, thereby reducing invalid images and improving the effectiveness of subsequent processing. Even further, the sampling cycle can be configured based on the maximum operating speed of the conveyor mechanism and target tracking requirements, ensuring that the displacement change of the material between adjacent acquisitions remains within a stable range for identification and tracking, thus reducing the probability of missed detections and target loss.

[0022] It should be noted that the "material image" refers to image data obtained by imaging the sorting area using an imaging device. The imaging device can be a two-dimensional industrial camera, an imaging device with depth information, or a combination thereof. The imaging device is preferably fixedly installed above or to the side of the sorting area, so that the field of view covers the path of the material and the gripping workspace accessible to the robotic arm. Furthermore, the effective image area can be set via software to limit the range of the sorting area within the field of view; alternatively, the field of view can be physically limited using structures such as light shields or guide plates to reduce background interference. Even further, supplementary lighting or a ring light source can be configured, and a fixed exposure or adaptive exposure strategy can be adopted to reduce the impact of reflections, shadows, and changes in ambient light on image quality, thereby improving the stability of subsequent pose estimation and graspability assessment.

[0023] It should also be noted that the "real-time operating parameters of the conveying mechanism" are used to characterize the motion background of materials within the sorting area. These parameters include at least one or more of conveying speed and direction of travel, and preferably one or more of start / stop status, acceleration / deceleration status, encoder displacement, or cycle time. These operating parameters can be fed back to the PLC by a frequency converter, servo drive, or motor controller via an industrial communication interface. Alternatively, they can be output pulse signals from an encoder mounted on the drive wheel or roller for the PLC to read, or status signals provided by photoelectric sensors, proximity switches, etc., in the conveying section for the PLC to acquire. Furthermore, to ensure consistency between the operating parameters and the image acquisition time, the PLC can sample and latch the conveying mechanism's operating parameters simultaneously with triggering image acquisition, or record the corresponding sampling sequence number and synchronously record the operating parameters at that time when receiving image data, so that subsequent steps can pair the image with the operating parameters. Even further, to address communication jitter or sensor noise, the PLC can perform filtering, amplitude limiting, or consistency verification on the speed or displacement feedback to improve the reliability of the operating parameters and reduce control errors introduced by parameter fluctuations.

[0024] The output of step S1 includes at least the material image data corresponding to the sorting area and the real-time operating parameters of the conveying mechanism corresponding to the image. The above output provides the basic input for subsequent steps: on the one hand, the image data is used to extract the appearance and posture features of the material to obtain pose information and form a graspability assessment result; on the other hand, the real-time operating parameters of the conveying mechanism are used to determine the grasping timing and the relative motion relationship of the target when generating grasping control commands, so that the robot can grasp and place the target more accurately under the conveying motion conditions, thereby improving the stability and feasibility of the sorting process.

[0025] S2: Based on the material image, the material pose information and its graspability evaluation results are obtained through the recognition model.

[0026] Step S2 is used to process the material image obtained in step S1 to output two key results required for subsequent gripping planning and sorting control: the first is the material's pose information, which is used to characterize the material's position and orientation in the sorting area; the second is the gripping feasibility assessment result, which is used to characterize the feasibility of the robot arm to perform gripping under the current material appearance and posture conditions and to select the optimal gripping scheme, so that the system output changes from "only identifying the target" to "operable decision basis for gripping execution".

[0027] The recognition model is a model or combination of models capable of extracting features and performing inference calculations on material images, and outputting pose information and graspability assessment results. It can be deployed in the computing unit within a PLC, an industrial computer communicating with the PLC, a vision controller, or an edge computing unit, and is invoked or triggered by the PLC in each sampling cycle. Furthermore, in engineering practice, it is preferable that a vision processing unit with image processing and inference capabilities performs the inference calculations, while the PLC is used to complete task triggering, task identifier allocation, result reception, and consistency management, thereby reducing dependence on PLC computing power and ensuring the determinism of the control system. Even further, the recognition model can include sub-models for target detection and localization, sub-models for pose estimation, and sub-models for graspability assessment; it can also be an end-to-end integrated model that directly outputs pose information and graspability assessment results from images. Furthermore, to adapt to different material types such as express parcels and industrial parts, the recognition model can be pre-set with multiple model versions or parameter sets, and can select the corresponding model version or parameter set to perform inference based on sorting tasks, material sources, or historical recognition results; among them, evaluation parameters oriented towards planar suction or flexible gripping can be given priority for express parcels, while evaluation parameters oriented towards edge gripping, hole gripping, or posture-sensitive grasping can be given priority for industrial parts.

[0028] It should be noted that in one feasible processing flow, the recognition model can first preprocess the material image to improve feature stability and inference consistency. The preprocessing includes at least image cropping to define the sorting area, and may optionally include one or more of distortion correction, brightness normalization, noise suppression, and background suppression to reduce the impact of ambient light variations, conveyor belt texture, reflections, or shadows on the recognition results. Furthermore, the recognition model can locate the material targets in the image and output target area information. When multiple materials exist in the image, it can output the area information corresponding to each target separately and assign a target identifier to each target. This allows subsequent pose estimation and graspability assessment to be performed independently according to the target dimension, facilitating subsequent steps to generate grasping control commands or determine grasping priorities according to the target dimension and avoiding crosstalk between targets.

[0029] It should also be noted that the pose information is used to describe the position and orientation of the material relative to a preset reference coordinate system. This reference coordinate system can be one of a camera coordinate system, a conveyor belt coordinate system, or a robot arm base coordinate system, and conversion between different coordinate systems can be achieved through pre-calibrated coordinate relationships. Furthermore, the pose information includes at least position and orientation parameters: for materials that can be approximated as planar objects or whose main posture changes are concentrated on the sorting plane, the position and orientation within the sorting plane can be output; for materials with significant height differences or sensitive grasping posture, the position including a height component and the corresponding spatial orientation can be output. Even further, the pose information can simultaneously output pose reliability characterization information, used to reflect the credibility of the pose estimation results, and used for subsequent steps such as grasping strategy selection, re-inspection triggering, or conservative control.

[0030] Furthermore, pose information can be obtained using any of the following methods or a combination thereof: When the imaging device contains depth information, the recognition model can extract depth or point cloud data within the target area, estimate the spatial location of the target by combining the target contour or key points, and obtain the target orientation by fitting the main direction of the point cloud, feature plane or key geometric structure. When the imaging device is a two-dimensional camera, the recognition model can output the key points, boundaries or pose features of the target, and combine the pre-calibrated camera parameters and the target's prior morphological information to solve the target pose, or directly output the target orientation by model regression. When the target is a regular industrial part, a stable attitude reference can be output based on its stable structural features (such as holes, edges, and end faces), thereby improving the consistency of repeatable positioning.

[0031] It should be noted that, in order to enable the pose information to be directly used for control calculations in the robot coordinate system, the system can store the calibration relationship between the camera coordinate system and the robot's base coordinate system, and transform the target's spatial points in the camera coordinate system to the robot's base coordinate system. For example, a homogeneous transformation relationship can be used for coordinate transformation: ; in, This represents the homogeneous coordinates of the target point in the camera coordinate system. This represents the homogeneous coordinates of the target point in the robot's base coordinate system. This is the coordinate transformation matrix obtained through pre-calibration. It should also be noted that the above is merely an illustrative example to illustrate the feasibility of coordinate transformation; any equivalent coordinate mapping method can be used for the actual implementation.

[0032] The graspability assessment result is a comprehensive evaluation output for robotic arm grasping execution, used to characterize whether the current material is suitable for grasping in the current posture and the recommended grasping method. Further, the graspability assessment result may include one or more of the following: grasping feasibility judgment result, grasping confidence or score, candidate grasping point set, candidate grasping posture set, recommended end effector action mode, and grasping-related risk warnings. Furthermore, to improve feasibility, the graspability assessment result can be output in structured fields, including at least: target identifier, recommended grasping scheme identifier, recommended end effector mode, recommended grasping pose, and the score or confidence level corresponding to the recommended scheme; when multiple candidate schemes exist, a candidate scheme list and its ranking information can also be output, allowing subsequent steps to select or downgrade among the candidate schemes.

[0033] Furthermore, gripability assessment can comprehensively consider one or more of the following: target boundary integrity, degree of obstruction, distance between the target and the conveyor belt boundary, minimum distance between the target and adjacent materials, accessible area of ​​the target surface, and estimated location of the target's center of gravity or main stress area. This reduces the risk of gripping failure caused by gripping interference, insufficient adsorption coverage, or post-grip instability. For express parcels, gripability assessment can prioritize suitable surface areas for suction or gripping, and provide risk warnings to avoid gripping seals, labels, transparent tape, or highly reflective areas. For industrial parts, gripability assessment can prioritize suitable feature edges, holes, or planar areas for clamping, and provide assessment results to reduce suction risk or change clamping methods when oil stains, perforations, or highly reflective materials are present.

[0034] It should also be noted that, in one feasible approach, the crawlability assessment score can be formed by combining multiple sub-indicators to reflect crawling stability and executability. For example, the crawling score can be represented as a weighted combination of several evaluation items: ; Among them, the overall crawlability score To provide a comprehensive evaluation of candidate crawling solutions, As a dimensionless scalar, a larger value indicates that the candidate crawling scheme is more suitable for crawling under the current operating conditions. Furthermore, It can be obtained by weighted synthesis of multiple component scores, among which to Configurable weighting coefficients are used to reflect the importance of each evaluation dimension under different material types, end effector types, and field strategy preferences. Preferably, the weighting coefficients satisfy non-negativity constraints and can be normalized as needed to maintain a stable scoring scale and facilitate cross-task comparisons. It should also be noted that the scores for each component... , , , and Preferably, a dimensionless normalized quantity is used to ensure that it can be linearly combined on the same scale. This is used to characterize the degree to which the end effector establishes effective contact with the target surface in a candidate grasping posture. It can correspond to factors such as the coverage ratio of the effective contact area, the degree of contact normal matching, and whether the contact area falls within unfavorable contact areas such as seals, labels, holes, or chamfers. Used to characterize the ability to resist slippage, flipping, or detachment after grasping, it can correspond to factors such as the moment margin of the candidate grasping point relative to the estimated center of gravity, contact friction conditions or gripping / adsorption margin, and robustness to pose disturbances; This component is used to characterize the clearance margin between the gripper and adjacent materials, conveying mechanisms, or fixed structural components during the gripping process. It can be evaluated based on the minimum safe clearance along the approach, gripping / adsorption, lifting, and withdrawal paths, and its score is reduced when interference is predicted. Used to characterize the feasibility and motion margin of a robotic arm reaching a candidate grasping pose under time constraints, it can correspond to factors such as the existence of feasible joint solutions, joint limit margins, reachability within the allowable time, and the degree of avoidance of singular configurations; The punitive risk metric characterizes risk factors that may lead to unreliable identification or uncontrollable grasping. A higher value indicates a higher risk. These risk factors may include pose instability caused by reflections, shadows, or occlusion; unstable adsorption due to soft bag deformation or transparent / mirror materials; and misgrabbing or being pulled out due to stacking, adhesion, or entanglement. Therefore, by uniformly quantifying and weighting candidate grasping schemes across dimensions such as contact effectiveness, grasping stability, gap interference, kinematic reachability, and risk factors, the system can perform comparable ranking and screening of different candidate grasping points and postures. This provides a more consistent and feasible decision-making basis for generating subsequent grasping control commands.

[0035] Furthermore, to enhance the stability of the evaluation results, when the same target obtains multiple pose and graspability evaluation outputs within multiple consecutive sampling periods, the system can perform consistency verification or smoothing on the ranking results of candidate schemes or recommended grasping poses. For example, when the change in recommended grasping pose between two adjacent periods exceeds a preset threshold and the pose reliability decreases at the same time, a re-inspection mark can be triggered or a conservative scheme can be adopted to reduce the fluctuation of grasping points caused by instantaneous imaging noise.

[0036] Furthermore, when the graspability assessment result is lower than a preset threshold, the pose reliability is lower than a preset threshold, or there are uncertain characteristics, the graspability assessment result may include a re-inspection or secondary processing mark. This mark is used to trigger a conservative grasping strategy, delay grasping, re-imaging, or import the material into an abnormal processing channel in subsequent steps to avoid dropping, missorting, or abnormal shutdown of the robot arm caused by forced grasping under uncertain conditions. Even further, the re-inspection or secondary processing mark can be associated with and stored with the target identifier and task identifier, so that subsequent execution feedback can correspond to the target "previously marked as uncertain," thereby forming more valuable training samples in step S5.

[0037] It should also be noted that, in order to ensure the traceability and time consistency of the output of step S2 with the subsequent control link, the PLC can assign a task identifier to each image acquisition task and transmit the task identifier and image sequence number together when triggering the recognition model inference; the pose information and graspability evaluation results output by the recognition model can be returned to the PLC with the same task identifier and target identifier, so that step S3 can form a one-to-one grasping planning input for different targets, and provide a consistent correlation basis for the feedback acquisition of step S4 and the sample construction of step S5.

[0038] It should be noted that the output of step S2 includes at least the material's pose information and graspability assessment results. This output will serve as the direct input for generating the robotic arm grasping control command in step S3, enabling the robotic arm control to dynamically adjust the grasping point and grasping posture according to the material's posture and graspability. This will improve the grasping success rate and reduce the probability of false grasping and empty grasping under conditions of frequent changes in material type, random posture, occlusion, and uncertain spacing. At the same time, it will provide a traceable recognition output basis for the incremental learning and updating in the subsequent step S5.

[0039] S3: Combining the pose information, graspability assessment results, and real-time operating parameters of the conveying mechanism, generate a robotic arm grasping control command and issue it to execute the sorting operation.

[0040] Step S3, based on the pose information and graspability assessment results output in step S2, and combined with the real-time operating parameters of the conveyor obtained in step S1, generates grasping control commands that can directly drive the robot to complete grasping and sorting. This ensures that the robot can reach the appropriate position at the appropriate time and perform grasping in the appropriate manner even when the conveyor is running continuously and the material is in relative motion, avoiding timing and landing point deviations caused by generating commands solely based on static images. Furthermore, the core of step S3 lies in converting the "recognition output" into an "executable action" and ensuring that this action is consistent with the conveyor cycle time, the relative motion of the target, and on-site safety constraints.

[0041] The generation of the grasping control command includes at least the determination of the grasping target, the grasping scheme, and the grasping timing. Specifically, the system determines the position and orientation of the grasping target in a preset reference coordinate system based on pose information, and determines the target grasping scheme based on the graspability assessment results. The target grasping scheme includes at least a preferred grasping point, a preferred grasping posture, and a grasping mode. Furthermore, the grasping mode can be one or more of suction, clamping, or a combination thereof; for express parcel materials, suction or flexible clamping is preferred on the accessible surface indicated by the graspability assessment results; for industrial parts materials, clamping or suction methods are preferred based on their characteristic edges, holes, or planar areas, and alternative modes corresponding to risk warnings can be triggered for oil stains, hollowed-out surfaces, or highly reflective surfaces.

[0042] Furthermore, when the feasibility assessment results provide multiple candidate grasping schemes, the system can rank the candidate schemes based on scores, feasibility judgments, or risk warnings, and select the scheme that meets preset conditions as the target grasping scheme. When a scheme with a higher score has interference risk, unreachability risk, or other execution risk, the system can automatically select the suboptimal scheme or trigger a conservative strategy. It should also be noted that the conservative strategy includes at least one or more of the following: adopting a grasping posture with greater tolerance, reducing the grasping speed, increasing secondary confirmation before grasping, delaying grasping and waiting for a more suitable window, or marking the material as abnormal and importing it into the abnormal handling channel, in order to reduce the probability of false grasping and abnormal downtime.

[0043] It should be noted that the real-time operating parameters of the conveying mechanism are used to dynamically compensate for the relative motion of the grasped target, ensuring that the grasping control command is consistent with the target's motion. Furthermore, the system can estimate the expected position of the target at the moment of grasping based on parameters such as conveying speed, running direction, and encoder displacement, and accordingly determine the timing of the robot's entry into the grasping window and the corresponding target grasping posture. It should also be noted that the grasping window can be understood as the effective range within which the target is located in the reachable space of the robot and meets the conditions for safe contact by the end effector. This effective range can be characterized by both a time range and a spatial range; the time range is jointly determined by the conveying speed, the target's position in the field of view, and the time required for the robot to reach the target posture, while the spatial range is jointly defined by the robot's workspace boundary, the conveyor belt boundary, and the safety distance.

[0044] It should also be noted that there is usually an inherent delay in the process from the generation and issuance of the grasping control command to the actual grasping posture of the robot end effector, caused by communication, control processing, and robot movement. Therefore, the estimation of the expected position mentioned above can be used to perform feedforward compensation for the grasping posture or grasping timing, thereby improving the spatiotemporal alignment consistency between the robot end effector and the moving material. Furthermore, the inherent delay can be composed of the communication cycle, PLC processing cycle, robot controller response time, end effector action setup time, and the shortest arrival time of the robot under the current speed constraint; at least a portion of the delay can be obtained through calibration during the equipment commissioning phase and used as a parameter during operation. Furthermore, when a change in conveying speed, a start / stop state switch, or an acceleration / deceleration process is detected, the system can update, replan, or adjust the timing of grasping control commands that have not yet been executed; for example, before the grasping has entered the irreversible action stage, the grasping target posture, the grasping trigger time, or the start and end points of the path into the grasping window can be recalculated to reduce the grasping deviation caused by sudden changes in the conveying state.

[0045] Furthermore, when multiple target materials exist within the same sampling period, the system can establish a task queue management mechanism to generate candidate instructions for each target and perform executability screening based on grasping window constraints and robot cycle time capability. It should also be noted that executability screening includes at least one or more of the following criteria: whether the target will still enter the grasping window, whether the robot can reach the target pose within the window, whether the end effector has sufficient motion setup time, and whether there are path conflicts or cycle time conflicts between adjacent tasks. When the screening results show that the requirements are not met, the target can be marked as delayed grasping, transferred grasping, or handled abnormally, thereby avoiding blindly issuing commands that result in missed grasps or misplacements.

[0046] It should also be noted that the grasping control command can be a structured instruction set, which may include one or more of the following: target grasping pose information, grasping window related time control information, robot arm motion constraint information, end effector motion parameters, and target placement position command. Furthermore, to enhance engineering feasibility, the structured instruction set may at least include key action segment information corresponding to "approach, grasp, lift, transport, place, and withdraw"; wherein the approach segment is used to enter the pre-grasping pose from a safe distance, the grasping segment is used to trigger the end effector action at the target grasping pose, the lifting segment is used to remove the target from the conveyor belt surface or adjacent material interference zone, and the placement segment is used to complete the release at the predetermined placement position and exit the placement area. Furthermore, the end effector motion parameters may include action type, action triggering condition, holding condition, and release condition to match the end effector action with the robot arm's positioning state; for example, for the suction mode, it may include vacuum establishment trigger, vacuum establishment confirmation, and vacuum release condition; for the clamping mode, it may include the gripper opening and closing width, clamping closure confirmation, and necessary secondary clamping conditions.

[0047] Furthermore, the target delivery location can be determined by sorting task rules, which can be associated with material category, destination, or sorting strategy to achieve targeted delivery and sorting of different materials. It should also be noted that when there is uncertainty in the graspability assessment results or identification output, the sorting task rules can prioritize delivery paths or abnormal channels with lower risk to avoid the spread of misdelivery caused by uncertain grasping.

[0048] Furthermore, to ensure execution reliability and on-site safety, preset safety and execution constraints can be introduced when generating grasping control commands. The safety constraints include at least the robot's workspace boundaries, safe distances from conveying mechanisms or surrounding structures, avoidance requirements with adjacent materials or tooling fixtures, and conservative handling rules for low-confidence or abnormally marked materials. The execution constraints include at least the robot's cycle time capability, load capacity, end effector response capability, and communication delay tolerance. Furthermore, the system can perform constraint arbitration on candidate grasping schemes; that is, if a candidate scheme has a high graspability score, the scheme that satisfies both safety and execution constraints is prioritized. When all candidate schemes fail to meet the constraints, an alternative command or anomaly handling command is output, instead of issuing a high-risk grasping command.

[0049] The phrase "issuing the gripping control command to execute the sorting operation" refers to the PLC sending the gripping control command to the robot controller, triggering the robot to complete the gripping and delivery actions according to the gripping control command. Furthermore, the PLC can attach a task identifier or time identifier when sending the gripping control command, for the robot controller to manage the sequence and confirm execution. The PLC can also receive status information such as receipt confirmation, execution start, and execution completion returned by the robot controller, and combine this information with the conveyor mechanism's operating parameters to manage the consistency of the execution process. This allows for timely triggering of update, cancellation, degradation, or anomaly handling strategies when execution delays exceed limits, conveying speed changes abruptly, or task window mismatches are detected, providing task association basis for subsequent steps to obtain execution feedback data.

[0050] It should be noted that by combining the pose information, graspability assessment results, and real-time operating parameters of the conveying mechanism in step S3 to generate grasping control commands, the robotic arm can compensate for the target motion under dynamic conveying conditions and select the more reliable and constraint-satisfying solution from multiple grasping schemes based on the graspability assessment. At the same time, by using the grasping window, task queue, and constraint arbitration mechanism, the system can improve the stability of grasping and delivery under typical working conditions such as random posture, varying spacing, and speed fluctuations of materials such as express parcels and industrial parts, and reduce the risks of misgrabbing, empty grasping, and misdelivery. It also provides clear task identification and action basis for obtaining execution feedback data in step S4.

[0051] S4: Obtain the execution feedback data of the sorting operation.

[0052] Step S4 is used to obtain execution feedback data that objectively characterizes the execution result and quality of the sorting operation after the grabbing and sorting are completed in step S3. This allows subsequent steps to determine whether the sorting operation was successful and whether there are any abnormal deviations, and provides a basis for training, filtering, and labeling samples for incremental learning and updating in step S5. This enables the system to form a closed-loop iterative execution result-driven optimization mechanism. Furthermore, the execution feedback data is preferably output in a structured record format, including at least a task identifier, a target identifier, a grabbing result label, and key feedback fields associated with the label, to ensure one-to-one correspondence and traceability during subsequent sample construction.

[0053] The execution feedback data includes at least status feedback characterizing the grasping result, which can be one or both of grasping success and grasping failure. Further, the grasping success status can be determined by one or more of the following: the robot controller's task completion signal, the end effector's positioning signal, the adsorption establishment signal, or the clamping closure confirmation signal; the grasping failure status can be determined by one or more of the following: vacuum establishment failure, clamping not in place, material loss after grasping, load change during lifting, or abnormal alarm signal. Furthermore, to improve the reliability of the judgment, the PLC can adopt a multi-source signal consistency judgment method to cross-verify the key states from the robot controller and the end effector; for example, grasping success is determined when the adsorption establishment signal and the task completion signal are simultaneously satisfied, grasping failure is determined when the vacuum establishment failure and the task abnormal alarm are simultaneously satisfied, or grasping success is determined when at least a preset number of success criteria are satisfied, and grasping failure is determined when at least a preset number of failure criteria are satisfied, thereby reducing feedback deviation caused by misjudgment of a single sensor signal. It should also be noted that when success and failure criteria appear simultaneously or cannot reach a consistent conclusion, the task can be marked as an uncertain sample and a re-examination or buffering process can be triggered to avoid directly using conflicting feedback for subsequent learning and introducing noise.

[0054] It should be noted that the execution feedback data may also include deviation-type feedback to characterize execution deviations, describing the difference between the actual execution result and the target execution result. Further, deviation-type feedback may include one or more of the following: robot end-effector positioning deviation, gripping point deviation, placement landing point deviation, or placement target slot deviation. This deviation can be obtained by comparing the actual pose fed back by the robot controller with the target pose, or by verification imaging devices in the sorting or placement area, or indirectly verified by sensors, weighing devices, or counting devices at the placement port. Furthermore, when using a verification imaging device, the changes in material state before and after gripping can be compared to confirm whether the material has been successfully removed from its original position, or the target landing point can be identified after placement to determine if there is misplacement or landing point deviation. The verification results can serve as a high-confidence source for deviation-type feedback, thereby enhancing the guiding value of the "deviation label" for subsequent model updates. Furthermore, to avoid the deviation calculation being affected by time alignment errors, the PLC can preferably trigger deviation sampling at the moment when the grasping action enters a critical node or when the delivery action is completed, and bind and store the sampling time with the task identifier, thereby ensuring that the deviation feedback has a consistent correspondence with the target task.

[0055] Furthermore, the execution feedback data may also include process-related feedback to characterize the stability of the execution process, reflecting whether there are risks such as slippage, jitter, or abnormal force during the grasping process. This process-related feedback may include one or more of the following: changes in end-effector vacuum pressure, clamping force, motor current, joint load, or speed following error. Through process-related feedback, the system can distinguish grasping situations that appear successful but are unstable, such as situations where vacuum decay, clamping force fluctuations, or abnormal increases in joint load occur during the lifting phase, even after successful deployment. This provides more valuable sample labels and risk warnings for subsequent learning and updates. Furthermore, process-related feedback can be used to generate process stability labels or risk level labels, which, together with status-related feedback, form composite labels. This allows subsequent steps to correct "successful but high-risk" grasping schemes, rather than simply performing coarse-grained updates based on "success / failure."

[0056] Furthermore, the execution feedback data can be acquired by the PLC from the robot controller, the end effector controller, and sorting-related sensors or verification devices. The robot controller provides information such as task execution status, actual pose, and alarm codes; the end effector provides establishment and positioning information related to adsorption or gripping; and the verification device provides independent verification information for the grasping and placement results. Further, to ensure that the execution feedback data can be accurately associated with the material image in step S1 and the intermediate results in steps S2 and S3, the PLC can assign a task identifier or time identifier to each sorting task, and when issuing the grasping control command, associate and store the identifier with the corresponding material image record, target identifier, and the target grasping scheme identifier. When acquiring execution feedback data, the PLC binds various types of feedback data with the same identifier, thereby forming a traceable data association link. Furthermore, when there are multiple materials in the image or multiple grasping tasks are generated within the same sampling period, the PLC can assign a sub-task identifier to each target and bind its target area information, grasping control instructions and feedback data respectively, so as to avoid feedback crosstalk in multi-target scenarios and ensure that the subsequent step S5 can form training samples with the correct material image and corresponding execution results.

[0057] It should also be noted that, to enhance the feasibility of collecting execution feedback data, the PLC can set a feedback collection timing window based on the grasping process, allowing different types of feedback to be collected at corresponding action stages: for example, collecting the adsorption establishment or clamping positioning signal at the grasping trigger stage, collecting load change and slippage trend signals at the lifting stage, and collecting the placement confirmation signal or landing point verification signal at the placement stage. This ensures that the feedback data corresponds to the action stage and is interpretable. Furthermore, when a task execution timeout, missing key confirmation signal, or alarm code is detected, the PLC can record the anomaly category and trigger safety or anomaly handling strategies, while simultaneously marking the task as a high-priority sample candidate for subsequent steps to perform incremental learning or rule correction based on the anomaly pattern.

[0058] The PLC can perform preliminary processing on the collected raw feedback data to generate more semantic feedback conclusions. For example, it can classify and mark states such as successful capture, failed capture, abnormal delivery, unstable process, or uncertain samples, and generate corresponding label fields. The label fields can be used to determine whether preset update conditions are met, thereby reducing the complexity of subsequent model update triggering logic and improving the consistency of sample selection.

[0059] It should be noted that by acquiring and structuring the execution feedback data through step S4, the system can quantify the sorting operation results and form a correlation link between images, decisions, instructions and results. On the one hand, this enables subsequent steps to perform incremental learning and updates based on verifiable execution results. On the other hand, it also enables the system to identify and attribute abnormalities, misdelivery, and process instability, thereby providing data support for subsequent conservative strategies, re-inspection, or abnormal handling, and improving the feasibility and long-term stability of overall sorting control.

[0060] S5: When the execution feedback data meets the preset conditions, the execution feedback data and the corresponding material image are used as training samples to incrementally learn and update the recognition model.

[0061] Step S5, after obtaining the execution feedback data in step S4, filters out tasks suitable for learning based on preset conditions, and constructs training samples from the material images and execution feedback data corresponding to the tasks. This is used to incrementally learn and update the recognition model described in step S2, enabling the model to continuously adapt to feature drift caused by factors such as new material forms, packaging changes, changes in ambient lighting, and changes in the state of end effectors. This achieves adaptive improvement in sorting capacity without stopping the production line or affecting real-time control as much as possible. It should also be noted that the incremental learning update in this step is preferably scheduled and version-managed by the PLC, and the training or parameter update calculation is completed by a computing unit that collaborates with the PLC to ensure that the real-time control link is not occupied by the training process.

[0062] The phrase "execution feedback data meets preset conditions" is used to limit the timing and sample quality for triggering incremental learning updates, thus avoiding the direct introduction of noisy feedback or occasional anomalies into training and causing model instability. Further, the preset conditions preferably satisfy at least one or more of the "attributability conditions" and "availability conditions": Attributability conditions include that the execution feedback data, corresponding material images, corresponding recognition outputs, and corresponding grasping control commands correspond one-to-one in the task identification dimension, and there is no break in the association caused by multi-target crosstalk, command cancellation, or task reassignment; availability conditions include that the image quality meets the availability threshold, the verification information or key state signals are complete, and the execution result labels can form a consistent conclusion. Further, based on this, triggering conditions may include one or more of the following situations: the execution result is judged as grasping failure or delivery anomaly; the execution deviation exceeds the allowable range; process feedback indicates that although grasping is completed, there is an instability risk; the pose reliability or graspability confidence of the recognition model output is lower than the threshold; similar anomalies occur continuously or cumulatively within a set time window; or the failure rate or deviation rate increases significantly relative to the benchmark within a set statistical window. Furthermore, regarding "new category or new mode triggering", to avoid the uncertainty brought about by conceptual description, it can be described as: when the failure mode corresponding to the same appearance feature or the same posture feature appears repeatedly within the set window, and the verification device or multi-source consistency judgment supports that the failure mode has stable reproducibility, learning is triggered.

[0063] It should be noted that using execution feedback data and corresponding material images as training samples includes at least pairing material images with the execution result labels of the task to form sample records for incremental learning. Furthermore, to improve sample usability, in addition to material images and execution result labels, training samples may also include recognition output summary information associated with the task, the identification of the capture scheme used, and feedback fields reflecting process quality, so that samples can not only express "success or failure" but also "why it failed / why it was unstable." Furthermore, the PLC can perform sample quality screening on candidate samples, which includes at least one or more of the following: removing frames without materials, severely blurred frames, or frames that are severely occluded and cannot be verified; marking conflicting label samples and adding them to the confirmation queue; directly excluding non-attributable samples from the training set; and using low-confidence label samples only as auxiliary samples in the buffer pool without directly participating in updates. It should also be noted that when the execution feedback data cannot directly form a high-confidence label, the sample can be marked as a sample to be confirmed and put into a buffer queue. During the system's idle period, it can be included in the training after review sampling or a small amount of manual confirmation, so as to reduce the adverse effect of weakly supervised samples on the update effect.

[0064] It should also be noted that the incremental learning update is a controlled iterative process. To enhance industrial feasibility, the incremental learning update can be limited to updating only some parameters or sub-modules in the recognition model, rather than performing an unconstrained update of the entire model. Further, optional update methods include: fine-tuning only the output layer or sorting strategy parameters related to the graspability assessment; locally updating only the parameter set related to a specific material category or a specific end effector mode; or updating the threshold and post-processing rules without changing the main structure of the model, making the model output more closely match the on-site execution results. Furthermore, the incremental learning update can be scheduled during equipment idle periods, low-load periods, or preset maintenance windows. During the update, the PLC still calls the currently running version of the model to complete real-time sorting, and delegates the update calculation to the background computing unit. The updated model is only activated after the update is completed and verified. Furthermore, the update training can employ a sample caching and selective replay mechanism, that is, while introducing new samples, a certain number of historical representative samples are retained to participate in the update, in order to balance old and new knowledge and reduce the risk of a decline in the ability to recognize existing materials.

[0065] Furthermore, to ensure the controllability of model updates and production line stability, the PLC can perform pre-launch verification on the updated recognition model. This verification includes at least one or more of the following: inference consistency checks based on preset verification samples; comparison checks of key output indicators and thresholds; and conservative checks for low-confidence or abnormal scenarios, to ensure that the update does not introduce significant risky behavior. It should also be noted that, to reduce launch risks, the updated model can first participate in inference in a "shadow run" mode, that is, outputting comparison results without affecting online control decisions. It is only switched to the running version after meeting preset stability conditions within a set observation window. Furthermore, when verification fails or abnormal fluctuations occur after launch, the PLC continues to use the original model or reverts to the previous stable version, and includes the task samples causing the fluctuations in the subsequent learning sample set to form a self-stabilizing mechanism with closed-loop correction.

[0066] Furthermore, the PLC can record the sample source range, update time, version information, activation condition fulfillment, and key operational indicator changes corresponding to each model update, making the model update process traceable and facilitating subsequent review and continuous optimization. Through this method, step S5 directly aligns the optimization goals of the recognition model with the actual grasping effect. The model not only corrects static recognition results but also continuously refines the graspability assessment and grasping scheme output based on verifiable results such as grasping stability, grasping success rate, and placement accuracy. This reduces the reliance on manual retraining and downtime debugging in industrial settings with frequent material updates and fluctuating operating conditions, and provides an implementable, controllable, and traceable closed-loop update foundation for the continuous adaptive operation in step S6.

[0067] S6: In subsequent sorting, the updated recognition model is used, and steps S1 to S5 are repeated.

[0068] Step S6 involves updating the recognition model through incremental learning in step S5, then putting the updated model into subsequent sorting cycles. Steps S1 through S5 are repeatedly executed to create an adaptive closed loop throughout the entire operating cycle. This allows for continuous tracking and optimization of changes in material type, packaging appearance, lighting, and equipment status, preventing the system's capabilities from remaining at the fixed level of initial deployment. It should also be noted that the core of this step is not frequent model replacement, but rather ensuring that model updates remain effective without disrupting production line rhythm and safety through version management, controlled switching, and runtime monitoring.

[0069] The phrase "adopting the updated recognition model" means that in subsequent sorting tasks, the recognition model called in step S2 is the model version updated and verified in step S5. Furthermore, the PLC can manage the recognition model's version, creating distinguishable candidate versions for each update and registering the currently used online model version as the running version. This distinction between candidate and running versions allows the system to maintain stable operation after an update, and then switch when activation conditions are met. Even further, the PLC can switch to the running version when preset activation conditions are met. These preset activation conditions include at least an update completion flag and a pre-launch verification pass flag, and optionally a running window condition fulfillment flag, thereby avoiding putting the model online before the update is complete or verification is complete, which would affect sorting stability.

[0070] It should be noted that, to ensure the feasibility of the aforementioned operating window conditions, they can be limited to a state window where the production line is allowed to switch. Examples include a state where no grabbing instruction is currently being executed, the robot is in standby or has returned to a preset safety position, the conveyor is in stable operation and the current target has been successfully placed, or the task queue is empty and the next task has not yet locked a grabbing scheme. Furthermore, the activation of the model version can be set to be performed at a preset switching point, such as when a sorting task cycle ends or the system enters a state window where switching is allowed, to ensure that the impact of the switching process on continuous production is controllable.

[0071] It should also be noted that, to reduce the risk of output drift after the candidate version is launched, the PLC can perform a pre-run consistency check on the candidate version before switching. This consistency check is used to confirm that the candidate version does not exhibit systematic or unexpected significant deviations from the current running version in key outputs. Furthermore, the consistency check can be performed by comparing the candidate version with the current running version based on the same batch of cached images or online input within the same time window. For example, it can check the consistency of pose output offset trends, graspability score distribution, or candidate grasp point sorting, as well as changes in the proportion of re-examination or secondary processing markers. When the comparison results show that the candidate version output has abnormal drift or a significant change in the proportion of abnormal markers, the consistency check can be determined to have failed, thereby delaying its activation and incorporating the relevant samples into the subsequent retraining or re-verification process.

[0072] Furthermore, the repeated execution in step S6 includes at least the following closed-loop relationship: Step S1 continuously provides new material images and conveying operation parameters; Step S2 outputs pose information and graspability assessment results based on the current operating version of the recognition model; Step S3 generates and issues grasping control commands based on the above output to complete sorting; Step S4 obtains execution feedback data associated with the task identifier; Step S5 constructs training samples from the feedback and images according to preset conditions and incrementally learns and updates the recognition model; Step S6 then uses the updated recognition model in subsequent step S2. Through the above closed loop, the system can use the actual grasping and delivery results as verifiable evidence to continuously correct the accuracy of the recognition model's graspability assessment and grasping scheme output, and gradually solidify recurring difficult postures or abnormal patterns into recognizable and avoidable model capabilities.

[0073] To ensure the feasibility of this closed loop under continuous production conditions, step S6 can adopt a separate operation mode for online inference and offline updates. That is, during subsequent sorting processes, the current running version model is continuously used for real-time inference, while newly generated training samples are cached and filtered in the background. Incremental updates are completed at preset times before switching to the updated model. It should also be noted that this separation method can be manifested in that training computation is completed by a computing unit that collaborates with the PLC, while the PLC is only responsible for sample scheduling, version registration, and activation control. This isolates the real-time control link from the training link, avoiding the training process from consuming real-time control resources. Furthermore, the PLC can also use a shadow operation mode to evaluate candidate versions. That is, without changing the generation of online grasping control instructions, candidate versions are evaluated for the same input / output pose information and graspability, and the comparison data is recorded. When a candidate version shows stable key outputs and passes the consistency check within a preset observation window, it is then switched to the running version.

[0074] Furthermore, the PLC can be configured with a learning frequency control mechanism, allowing the triggering and activation of model updates to be controlled by preset constraints. These constraints may include one or more of the following: minimum time interval, minimum sample size threshold, recurrence threshold of similar abnormal patterns, or performance change threshold, to avoid system instability caused by frequent updates and switching. It should be noted that the performance change threshold can be set based on one or more of the following: capture success rate, deployment accuracy rate, deviation exceeding limit rate, abnormal downtime rate, or abnormal channel occupancy rate. For example, if the capture success rate of a candidate version in a preset verification sample or shadow running observation window is less than the preset increase or even decreases, it can be determined that the performance change threshold has not been met, thus delaying its activation. Alternatively, if a candidate version improves the capture success rate but simultaneously causes a significant increase in the deviation exceeding limit rate or abnormal channel occupancy rate, it can be determined that it does not meet the comprehensive performance threshold, and the current running version remains unchanged.

[0075] Furthermore, to avoid performance fluctuations or bias towards a single recent material due to model updates, the PLC can constrain model activation based on the version verification results of step S5, and continuously monitor key operating indicators after activation. These key operating indicators may include one or more of the following: capture success rate, delivery accuracy rate, deviation exceeding limit rate, or abnormal downtime rate. Furthermore, key operating indicators can be statistically analyzed according to a preset number of tasks or a preset time window and compared with benchmark indicators. Only when a candidate version meets stability requirements within the observation window will it remain as the operating version for continued use. Even further, when a deterioration or abnormal fluctuation in key operating indicators is detected, the PLC can switch the identification model back to the previous stable version and mark the task samples that triggered the deterioration as high-priority samples for inclusion in the subsequent learning sample set, thus forming a self-stabilizing mechanism of monitoring, rollback, and relearning. After rollback, an activation freeze period can be set, during which only sample caching and filtering are performed without activating a new version to avoid repeated switching under the same disturbance conditions.

[0076] It should be noted that by adopting the updated recognition model in subsequent sorting through step S6 and repeating steps S1 to S5, the system can continuously transform verifiable execution results in the sorting process into a basis for model improvement, achieving continuous improvement without adding a dedicated training process. At the same time, by using version management, runtime window switching control, pre-run consistency check, shadow run evaluation, learning frequency constraints, and key indicator monitoring and rollback mechanisms, the controllability of the model self-optimization process and the stability of production line operation are ensured. This allows the method to operate stably for a long time and continuously improve sorting performance in real-world scenarios with multiple categories and changes, such as express parcels and industrial parts.

[0077] Example 2 is the second embodiment of the present invention, which differs from the previous embodiment in that: If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art or the current technical solution, can be embodied in the form of a software product. This current computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0078] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0079] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0080] Example 3, referring to Figure 2 As an embodiment of the present invention, a material sorting control system based on PLC control is provided, which includes an image and conveying parameter acquisition module, an identification and graspability assessment module, a grasping control command generation and issuance module, an execution feedback data acquisition module, a training sample construction and incremental learning update module, and a model version management and closed-loop operation module. Image and conveying parameter acquisition module: periodically acquires material images of the sorting area and obtains real-time operating parameters of the conveying mechanism; Recognition and graspability assessment module: Based on the material image, the module processes the image using a recognition model to obtain the material's pose information and its graspability assessment results. Grasping control command generation and issuance module: Combining the pose information, graspability assessment results and real-time operating parameters of the conveying mechanism, it generates grasping control commands for the robotic arm and issues them to execute sorting operations; Execution feedback data acquisition module: Acquires the execution feedback data of the sorting operation; Training sample construction and incremental learning update module: When the execution feedback data meets the preset conditions, the execution feedback data and the corresponding material image are used as training samples to perform incremental learning and update of the recognition model; Model version management and closed-loop operation module: In subsequent sorting, the updated recognition model is used, and steps S1 to S5 are repeated.

[0081] Example 4 is an embodiment of the present invention, which provides a material sorting control method based on PLC control. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation / comparative experiments.

[0082] This embodiment selects a continuously operating conveyor sorting station as the verification scenario. The conveyor belt runs within the range of 0.58–0.74 m / s, with periodic short-term acceleration / deceleration and start / stop disturbances. The sorting objects are mixed incoming materials, including soft-bag express parcels (film bags, woven bags, laminated bags), cardboard boxes, and industrial parts (perforated parts, edged parts, locally highly reflective metal parts). The material loading adopts random spacing and random posture to simulate the working condition of "incoming material distribution drifting over time" in a real sorting center. To construct reproducible long-term changes, three types of disturbances are set within an 8-hour shift: first, illumination drift (slightly randomized changes in supplementary lighting brightness, with localized shading introduced at certain times); second, end-state changes (slight wear on the edge of the vacuum adsorption end and fluctuations in clamping friction conditions, leading to a scenario of "surface success but unstable process"); and third, new material impact (newly laminated parcels with stronger reflectivity are mixed in proportionally in the middle of the shift as a representative subset of unknown appearance distribution).

[0083] The experiment was strictly executed according to the closed loop formed by S1 to S6. In S1, the PLC adopted a combination of "preset sampling period + material arrival event trigger": when the material arrival event was valid, the acquisition gating was activated; when the material left the sorting area or exceeded the valid window, the acquisition frequency was reduced to reduce invalid frames; and a sampling sequence number / time identifier was bound to each frame image. At the same time, the real-time operating parameters of the conveyor mechanism (including at least speed and direction) were latched in the same control cycle for timing alignment in subsequent compensation calculations. In S2, the recognition model was triggered in each sampling cycle to analyze and process the image, outputting the target area, pose information, and graspability assessment results; the graspability assessment provided at least one candidate grasping scheme with its quantitative indicators and risk warnings in a structured manner for subsequent "optimal decision-making". In S3, the PLC combined the pose information, graspability assessment results, and latched conveyor speed information to generate grasping control instructions and dynamically corrected the grasping pose and trigger timing, so that the robot arm could complete the action sequence of approaching-grabbing-lifting-deploying-withdrawing when the target entered the grasping window. In stage S4, execution feedback data is collected according to the task identifier. The feedback should at least cover the grasping result (success / failure), execution deviation (grasping pose deviation / deployment landing point deviation), and process stability (adsorption / gripping establishment status, load changes, motion following errors, etc.), forming a traceable structured record. In stage S5, incremental learning updates are triggered only when preset conditions are met (e.g., grasping failure, deviation exceeding limits, process instability, concentrated occurrence of low confidence, etc.). The PLC retrieves the corresponding image and feedback based on the task identifier to generate training samples, and the background computing unit performs controlled fine-tuning updates. After the update, shadow evaluation and consistency checks are performed first, and then the running version is switched in the allowed switching window. In stage S6, the updated model is put into the subsequent sorting cycle and continues to repeat S1 to S5 to form a continuous adaptive closed loop.

[0084] To demonstrate the marginal contribution of each technology, seven experimental groups were set up to run on the same line and with the same statistical caliber, recording 8-hour shift data: Control A was a fixed model without gating or latching (representing traditional extensive integration); Control B was a fixed model with gating and latching enabled (verifying time consistency contribution); Control C was a fixed model with latching and speed compensation added (verifying dynamic compensation but without learning); Invention D had closed-loop enabled but no version update occurred during the shift (verifying the contribution of the closed-loop framework and crawlability optimization decision); Invention E / F / G corresponded to the running status after 1, 3, and 5 incremental updates, respectively (verifying the performance convergence trend brought about by continuous adaptation and controlled online deployment). Key data are recorded as shown in Table 1: Table 1: Experimental Data Recording Table

[0085] The comparison results from control A to control B / control C, and then to the present invention D / E / F / G, show that the improvement in sorting performance mainly comes from the synergistic optimization of three key aspects: "time consistency, dynamic compensation decision-making, and feedback-driven updates".

[0086] First, time consistency is a fundamental prerequisite for the reliability of dynamic sorting. In Control A, the lack of gating and same-cycle latching resulted in an "image-velocity synchronization error" of 27.9 ms. At a conveying speed of approximately 0.63 m / s, this time mismatch directly translates into centimeter-level motion displacement deviations, amplifying the timing errors in pose estimation and grasping triggering. This ultimately manifests as a pose error RMSE of 7.6 mm, a grasping success rate of only 86.5%, and a cumulative abnormal downtime of 18.2 min / 8 h. In Control B, the introduction of gating and latching significantly reduced the synchronization error to 3.9 ms, while the pose error decreased to 5.4 mm and the grasping success rate increased to 90.6%. This demonstrates that the combination of "gating acquisition + sampling identifier binding + same-cycle latching" in S1 effectively suppresses timing mismatches in dynamic scenarios. This is a key point that many solutions relying solely on fixed-cycle acquisition or post-processing speed parameter reading struggle to achieve stably.

[0087] Secondly, while motion compensation can improve the alignment accuracy of dynamic grasping, it is insufficient to cope with complex disturbances when used alone. Control C, by adding velocity compensation to latching, further reduced the pose error to 5.0 mm, increased the grasping success rate to 91.7%, and reduced the misplacement rate to 1.05%, demonstrating that dynamic compensation has a clear benefit for systematic deviations caused by transport. However, Control C still faces bottlenecks in "process instability rate (3.07%)", "re-inspection / abnormal channel rate (5.21%)", and "grasping success rate of the new reflective material subset (84.9%)", reflecting that kinematic compensation alone is insufficient to cover the uncertainties and risks brought about by light drift, reflective materials, soft bag deformation, and changes in end-point condition.

[0088] Furthermore, the key difference of this invention lies in incorporating "scraping feasibility assessment and optimal decision-making" and "feedback-driven controlled incremental learning updates" into the same closed-loop system, thereby forming a sustainable and convergent performance improvement path. Even with the closed-loop enabled but no version update, this invention D already outperforms control C: the scraping success rate increases from 91.7% to 92.3%, the percentage of process instability decreases from 3.07% to 2.86%, and the percentage of re-inspection / abnormal channels decreases to 4.98%. This demonstrates that the control strategy of "candidate solution + risk warning + optimal decision-making" can reduce high-risk blind scraping and bring interpretable stability gains. More importantly, after completing 1, 3, and 5 incremental updates, the success rate of grasping in this invention E / F / G further improved to 95.1%, 96.2%, and 96.8%, respectively. The pose error converged to 3.4 mm, the mis-projection rate decreased to 0.41%, the proportion of process instability decreased to 1.58%, and the cumulative duration of abnormal downtime decreased to 2.5 min / 8h. At the same time, the success rate of grasping the new reflective material subset increased from 86.1% in D to 95.5% in G, showing a typical closed-loop adaptation curve of "facing distribution drift - sampling accumulation - incremental update - performance recovery and surpassing".

[0089] Finally, the "shadow evaluation pass rate" increased from 93.4% for E to 96.7% for G, accompanied by continuous improvement in key operational indicators. This demonstrates that the present invention is not an unconstrained update, but rather achieves controlled deployment through shadow evaluation, consistency checks, and switching window control, thereby reducing the probability of systemic risks caused by model drift. Compared with the traditional paths of "fixed model + rule extraction" or "offline retraining + shutdown deployment," the present invention transforms on-site execution results into traceable training samples and incrementally solidifies them into model capabilities in a controlled manner. This enables the system to continuously improve core sorting indicators (success rate, accuracy, stability, and downtime loss) without interrupting continuous production.

[0090] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A material sorting control method based on PLC control, characterized by, The method is executed by a PLC control system, and comprises: S1: periodically acquiring material images of a sorting area, and acquiring real-time running parameters of a conveying mechanism; S2: processing the material images based on an identification model to obtain pose information of the materials and an assessable result of the materials; S3: combining the pose information, the assessable result, and the real-time running parameters of the conveying mechanism to generate a robot grasping control instruction, and issuing the instruction to execute a sorting operation; S4: acquiring execution feedback data of the sorting operation; S5: when the execution feedback data meets a preset condition, using the execution feedback data and a corresponding material image as training samples to perform incremental learning and updating on the identification model; S6: using the updated identification model in subsequent sorting, and repeatedly executing steps S1 to S5.

2. The PLC control-based material sorting control method according to claim 1, wherein, The periodic acquisition of the material images of the sorting area comprises: triggering an imaging device to collect material images by the PLC at a preset sampling period, or triggering collection according to an event signal related to the arrival of the materials; and the PLC controls the collection based on the entry or exit of the materials into or from the sorting area; wherein the collected material images are associated with a sampling serial number or a time identifier, which are used for subsequent data pairing.

3. The PLC control-based material sorting control method according to claim 2, wherein The acquisition of the real-time running parameters of the conveying mechanism comprises: acquiring real-time speed and / or running direction of the conveying mechanism by the PLC through an industrial communication interface and / or encoder feedback; and the PLC latches the real-time running parameters within the same control period of triggering the imaging device to collect the material images, so that the real-time running parameters and the corresponding material images are paired one by one according to the sampling serial number or the time identifier.

4. The PLC control-based material sorting control method according to claim 3, wherein The identification model is configured to analyze and process the material images, and simultaneously output an identification result including at least the pose information of the materials and the assessable result; the identification model is triggered and executed by the PLC or its output result is called by the PLC.

5. The PLC control-based material sorting control method according to claim 4, wherein The obtaining of the pose information of the materials and the assessable result thereof comprises: positioning a material target in the material images to obtain target region information; determining a position and an orientation of the material target relative to a preset reference coordinate system based on the target region information as the pose information; and generating, based on the pose information and the material images, an assessable result for representing the feasibility or reliability of at least one candidate grasping scheme.

6. The PLC control-based material sorting control method according to claim 5, wherein The generation of the robot grasping control instruction comprises: generating at least one candidate grasping scheme based on the pose information and the assessable result, wherein the candidate grasping scheme at least includes a candidate grasping pose and an end effector action mode; determining a target grasping scheme from the candidate grasping scheme based on the assessable result; dynamically correcting at least one of a grasping pose and a grasping timing corresponding to the target grasping scheme according to the real-time running parameters of the conveying mechanism; generating a grasping control instruction including the target grasping pose, end effector action parameters, and a deposit position based on the dynamically corrected result; issuing, by the PLC, the grasping control instruction to a robot controller to execute a sorting operation.

7. The PLC control-based material sorting control method according to claim 6, wherein The execution feedback data at least includes one or more of: a result of the grabbing, an execution deviation, and a stability of an execution process; The feedback data for representing the result of the grabbing includes state feedback for indicating a success or a failure of the grabbing, the feedback data for representing the execution deviation includes deviation feedback for indicating a deviation of a grabbing pose and / or a deviation of a drop point, and the feedback data for representing the stability of the execution process includes process feedback for indicating a state of establishment of the suction or the clamping, a load change, or an action following error; The feedback data is associated and bound by the PLC with a task identifier and / or a target identifier corresponding to the sorting operation, to form structured execution feedback data that can be traced back, for subsequent sample screening or model updating.

8. The PLC control-based material sorting control method according to claim 7, wherein, The incremental learning and updating of the recognition model includes: The PLC associates the execution feedback data with a corresponding material image based on the task identifier and / or the target identifier, and determines whether the execution feedback data meets a preset condition; the preset condition at least includes at least one of: the execution feedback data indicates a failure of the grabbing or an abnormality of the dropping, an execution deviation exceeds a preset threshold, a process is unstable, a confidence of a pose reliability or a grabbability evaluation output by the recognition model is lower than a threshold, and a failure rate or a deviation rate rises within a preset statistical window; When the preset condition is met, the PLC pairs the material image with the execution feedback data to generate a training sample, a supervision label of the training sample is generated by the execution feedback data; and the recognition model is updated by incremental learning using the training sample. The incremental learning and updating of the recognition model includes:

9. A PLC control-based material sorting control system for implementing the PLC control-based material sorting control method according to any one of claims 1 to 8, characterized by An image and conveying parameter acquisition module: periodically acquires a material image of a sorting area, and acquires real-time running parameters of a conveying mechanism; An identification and grabbability evaluation module: processes the material image based on the material image by a recognition model to obtain pose information of the material and a grabbability evaluation result of the material; A grabbing control instruction generation and issuing module: generates a mechanical hand grabbing control instruction in combination with the pose information, the grabbability evaluation result, and the real-time running parameters of the conveying mechanism, and issues the grabbing control instruction to execute a sorting operation; An execution feedback data acquisition module: acquires execution feedback data of the sorting operation; A training sample construction and incremental learning and updating module: when the execution feedback data meets a preset condition, uses the execution feedback data and a corresponding material image as a training sample to update the recognition model by incremental learning; A model version management and closed-loop operation module: uses the updated recognition model in subsequent sorting to repeatedly execute steps S1 to S5. ​

Images