Parallel robot sorting control system and method based on fusion of vision and force sensation

Abstract

The disclosure provides a parallel robot sorting control system and method based on fusion of vision and force sense, which comprises: a visual recognition module for identifying the category and three-dimensional position information of the material based on the image data of the material to be sorted; a motion control module for moving the parallel robot body to the grabbing position in response to the three-dimensional position information sent by the visual recognition module, grabbing the material with the initial clamping force according to the material category, and driving the parallel robot body to move the material to the target position; a clamping force control module for receiving the first force sense information sent by the force sense acquisition module and extracting the actual clamping force during the grabbing process, performing stability analysis on the actual clamping force, generating a clamping force adjustment instruction when it is determined that there is a grabbing abnormality, and dynamically adjusting the clamping force of the clamping jaw to stabilize the actual clamping force within the preset interval. Through multi-module collaborative control, the accuracy of sorting recognition and the stability of material clamping are effectively improved.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent robot control, specifically to a parallel robot sorting control system and method based on the fusion of vision and force perception. Background Technology

[0002] Existing robotic arm sorting systems mostly rely on preset fixed parameters to control the gripping action. For example, patent document CN17680382A discloses an automatic sorting system based on visual recognition and multi-stage pneumatic pushers. A camera is installed diagonally above the material feeding direction of the conveyor belt to capture real-time images of the electrolyte and carbon block mixture on the conveyor belt, and the captured image data is sent to an algorithm server. The algorithm server deploys the YOLOv5 target detection algorithm, which can identify the residual carbon blocks on the conveyor belt and calculate their physical positions. The actuator of this device consists of pushers connected by multiple cylinders. Each pneumatic pusher has a baffle connected to its lower end. When the residual carbon block is transported to the pusher position, the cylinders actuate, driving the pusher and baffle to move perpendicular to the belt direction, pushing the residual carbon block off the belt and into the sorting box, completing the sorting process.

[0003] The aforementioned existing technologies mainly rely on visual positioning or single force threshold judgment, lacking a multi-module collaborative mechanism, and have the following drawbacks in practical applications:

[0004] 1. Insufficient gripping stability. Judging solely by the static gripping force threshold cannot analyze the trend of force fluctuations. When slippage occurs while gripping smooth materials, the system cannot adjust in time, leading to sorting failure; at the same time, excessive gripping force will also affect the service life of the machine.

[0005] 2. Low accuracy in foreign object sorting. Single-vision recognition is insufficient to distinguish foreign objects with similar shapes but different weights or hardness (such as anode carbon blocks and metal blocks), resulting in a missorting rate of >15%.

[0006] 3. High reliance on manual labor. In various material sorting scenarios, manual adjustment of clamping force parameters is required every time materials are switched (e.g., adjustment time exceeds 2 hours when switching materials), resulting in low efficiency.

[0007] 4. Poor adaptability to dynamic environments. When the material position shifts (such as the shift >3mm caused by the vibration of the sorting table, or the position of the charcoal block changes during transportation), the system lacks the collaborative logic of "force feedback-obstacle avoidance-stop", and the end effector of the robotic arm is prone to collision with obstacles, causing equipment damage.

[0008] Therefore, there is an urgent need in this field for an improved parallel robot sorting control method and system to systematically solve the above problems, thereby improving the stability of the sorting process, system safety and identification accuracy. Summary of the Invention

[0009] In response, this disclosure provides a parallel robot sorting control system and method based on the fusion of vision and force perception.

[0010] In a first aspect, this disclosure provides a parallel robot sorting control system based on vision and force sensing fusion. The control system is communicatively connected to a vision acquisition module, a force sensing acquisition module, and the parallel robot body. The system includes:

[0011] The visual recognition module is used to identify the category and three-dimensional position information of the material in response to the image data of the material to be sorted sent by the visual acquisition module.

[0012] The motion control module is used to respond to the three-dimensional position information sent by the vision recognition module, control the parallel robot body to move to the gripping position, grip the material with an initial gripping force according to the material type, and drive the parallel robot body to move the material to the target position.

[0013] The gripping force control module is used to receive the first force information sent by the force sensing acquisition module and extract the actual gripping force during the gripping process. It performs stability analysis on the actual gripping force and generates a gripping force adjustment command when a gripping abnormality is detected. The gripping force of the gripper is dynamically adjusted so that the actual gripping force is stabilized within a preset range.

[0014] In some embodiments, the force sensing acquisition module is disposed on the gripper at the end of the parallel robot's main arm, and is used to collect first force sensing information of the interaction between the gripper and the material during the grasping process in real time.

[0015] In some embodiments, the clamping force control module includes:

[0016] The clamping force determination unit is configured as follows:

[0017] During the grasping process, the first force information collected by the force sensing module is acquired, and the actual gripping force is extracted based on the force components in the X-axis or Y-axis direction. ;

[0018] Perform static interval verification to determine Is it within the preset material clamping force range? Inside;

[0019] Perform dynamic trend analysis on multiple actual clamping forces sampled continuously. Perform fluctuation amplitude calculation and trend slope calculation;

[0020] When static interval verification or dynamic trend analysis triggers abnormal conditions, a clamping force adjustment demand signal is generated.

[0021] In some embodiments, the clamping force control module further includes:

[0022] The PID control unit is configured as follows:

[0023] Receive the clamping force adjustment request signal sent by the clamping force judgment unit;

[0024] Based on the actual clamping force Fluctuation range The clamping force is iteratively adjusted according to the preset clamping force range using a PID control algorithm, and the adjusted clamping force is calculated in each iteration. The gripping force of the grippers is dynamically adjusted until the actual gripping force F is reached. 实 It remains stable within the preset range.

[0025] In some embodiments, the adjusted clamping force is calculated in each iteration cycle:

[0026]

[0027] in: This represents the current actual clamping force; The current fluctuation amplitude is calculated in the fluctuation amplitude judgment. , , These are the proportional coefficient, integral coefficient, and differential coefficient; the adjusted clamping force. Limited to a preset clamping force range Inside.

[0028] In some embodiments, the system further includes:

[0029] The threshold adaptive adjustment module is used to obtain the state characteristics of the current sorting process when sorting is completed, and to build a closed-loop optimization model based on reinforcement learning to adjust the preset clamping force range. Perform autonomous iterative optimization.

[0030] In some embodiments, the system further includes:

[0031] The foreign object identification module is used to acquire the first force information collected by the force sensing module when the parallel robot body completes the grasping and starts to lift the material. It extracts the actual load force based on the first force information of the material, and determines whether the current material is a foreign object based on the comparison between the actual load force and the theoretical load force. Based on the judgment result, it generates the corresponding sorting instruction.

[0032] In some embodiments, extracting the actual load force based on the first force information of the material includes: extracting the force component in the Z-axis direction of the first force information as the actual load force. .

[0033] In some embodiments, the visual recognition module is configured to identify the material category based on image data of the material to be sorted using a deep learning algorithm.

[0034] Secondly, this disclosure provides a sorting control method for parallel robots based on the fusion of vision and force perception, applied to the aforementioned sorting control system for parallel robots based on the fusion of vision and force perception, comprising the following steps:

[0035] S101, the visual recognition module responds to the image data of the material to be sorted sent by the visual acquisition module, and identifies the category and three-dimensional position information of the material;

[0036] S102, the motion control module responds to the three-dimensional position information sent by the vision recognition module, controls the parallel robot body to move to the gripping position, grips the material with an initial gripping force according to the material type, and drives the parallel robot body to move the material to the target position;

[0037] S103, the gripping force control module, receives the first force information sent by the force sensing acquisition module and extracts the actual gripping force during the gripping process. It performs stability analysis on the actual gripping force and generates a gripping force adjustment command when it determines that there is a gripping abnormality. It dynamically adjusts the gripping force of the gripper to make the actual gripping force stable within a preset range.

[0038] The beneficial effects of this disclosure are that, compared with the prior art, this disclosure has the following advantages:

[0039] 1. Improved material gripping stability. A dual force-trend judgment mechanism analyzes material stability in real time, effectively suppressing slippage and loosening during the gripping process. Practical applications show that the material slippage rate has decreased from 10% in existing technologies to below 5%; the machine's service life has increased from replacing it every 30 tons of material processed to replacing it every 70 tons, significantly reducing equipment maintenance costs.

[0040] 2. Improved material sorting accuracy. Through visual-force feedback fusion recognition technology, foreign objects with similar shapes but different weights or hardnesses are accurately identified, effectively improving material identification accuracy. Actual test data shows that the foreign object sorting accuracy increased from 85% to over 90%, while the missorting rate decreased to below 2%, effectively ensuring sorting quality.

[0041] 3. Improved efficiency in adjusting gripping force parameters. Employing reinforcement learning-based AI threshold adaptive control technology, the gripping force parameters are autonomously and iteratively optimized using historical sorting data, replacing the traditional manual, repetitive adjustment method and significantly reducing human intervention. Real-world testing shows that parameter adjustment time has been reduced from 2 hours / product to 30 minutes / product, labor costs have decreased by 80%, and production efficiency in multi-product material sorting scenarios has been significantly improved.

[0042] 4. Enhanced System Safety. A force feedback and emergency stop collaborative control mechanism is adopted, prioritizing active obstacle avoidance while retaining a fallback safety measure of emergency stop. Regardless of whether the system is in grasping, transferring, or standby mode, once the detected external force exceeds the threshold, the system immediately interrupts the current task and executes reverse obstacle avoidance. Combined with the timeout stop guarantee mechanism, this ensures that the duration of abnormal force application is controlled within a preset range (e.g., within 0.3 seconds), effectively preventing sensor over-range damage and long-term force-induced damage to the robotic arm, achieving a balance between safety protection and operational continuity. Actual test data shows that the risk of robotic arm damage is reduced by more than 80%, and the overall system safety is improved by 70%. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0044] Figure 1 A schematic diagram of a parallel robot sorting control system based on vision and force fusion provided in an embodiment of this disclosure;

[0045] Figure 2 A schematic diagram of a parallel robot sorting device based on vision and force fusion provided in an embodiment of this disclosure;

[0046] Figure 3 A threshold adaptive adjustment learning curve diagram provided in this embodiment of the disclosure;

[0047] Figure 4 A schematic flowchart of a parallel robot sorting control method based on vision and force fusion provided in an embodiment of this disclosure;

[0048] Figure 5 This is a schematic flowchart of an obstacle avoidance-emergency stop collaborative control method provided in an embodiment of the present disclosure.

[0049] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0050] The present disclosure will be further described below with reference to the accompanying drawings. The following embodiments are only used to illustrate the technical solutions of the present disclosure more clearly, and should not be used to limit the scope of protection of the present disclosure.

[0051] Example 1

[0052] like Figure 1As shown, this embodiment provides a parallel robot sorting control system 12 based on vision and force sensing fusion, suitable for controlling the parallel robot body 6 to automatically grasp and sort materials (such as anode carbon blocks, mixed materials of multiple varieties, etc.) on a conveyor belt. The control system 12 is communicatively connected to the vision acquisition module 8, the force sensing acquisition module 5, and the parallel robot body 6. The system 12 specifically includes the following modules:

[0053] The visual recognition module 13 is used to identify the category and three-dimensional position information of the material in response to the image data of the material to be sorted sent by the visual acquisition module 8.

[0054] Specifically, the visual recognition module 13 is configured to: identify the material category (such as anode carbon block, electrolyte or unknown category) based on the image data of the material to be sorted using a deep learning algorithm (such as YOLO); perform three-dimensional reconstruction on the image containing depth information to generate point cloud data of the material surface, and segment and fit the point cloud data to a three-dimensional bounding box, calculate the geometric center coordinates (X0, Y0, Z0) of the material as well as its length, width and height dimensions, and estimate the material volume V accordingly.

[0055] The motion control module 14 is used to respond to the three-dimensional position information sent by the vision recognition module 13, control the parallel robot body 6 to move to the gripping position, grip the material with an initial gripping force according to the material type, and drive the parallel robot body 6 to transfer the material to the target position.

[0056] The gripping force control module 15 is used to receive the first force information sent by the force sensing acquisition module 5 and extract the actual gripping force during the gripping process, perform stability analysis on the actual gripping force, and generate a gripping force adjustment command when a gripping abnormality is determined to exist, so as to dynamically adjust the gripping force of the gripper 9 to make the actual gripping force stable within a preset range; wherein, the force sensing acquisition module 5 is set on the gripper at the end of the mechanical arm of the parallel robot body 6, and is used to collect the first force information of the interaction between the gripper and the material in real time during the gripping process.

[0057] In this embodiment, the control system 12 acquires the material's category, pose, and geometric features through the vision recognition module 13, providing precise initial guidance for grasping. The motion control module 14 drives the parallel robot body 6 to sequentially execute the complete process of positioning, pre-opening, grasping, dynamic adjustment, and transfer sorting. Simultaneously, the gripping force control module 15 senses the interaction force between the gripper 9 and the material in real time, performs stability analysis on the actual gripping force, and dynamically adjusts the gripping force when an anomaly is detected. Through multi-module collaborative control, the accuracy of sorting and recognition, as well as the stability of material gripping, are effectively improved.

[0058] like Figure 2As shown, the material sorting device includes a ramp chute 2, a conveyor belt 4, a force sensing acquisition module 5, a parallel robot body 6, a supplementary light 7, grippers 9, a vision acquisition module 8, and a control system 12.

[0059] The vision acquisition module 8 is located above the sorting area and is used to acquire image data of the materials to be sorted and send it to the control system 12.

[0060] Specifically, the vision acquisition module 8 is an industrial 3D camera. This module acquires scene images of materials on the conveyor belt at a fixed frequency, which can cover the complete surface contour of the materials.

[0061] The force sensing acquisition module 5 is installed on the gripper 9 at the end of the robotic arm of the parallel robot body 6, and is used to collect force sensing information of the interaction between the gripper 9 and the material during the grasping process in real time.

[0062] Specifically, the force acquisition module 5 uses a multi-dimensional force sensor (such as the LA77 type, with a range of 0-60N and an accuracy of ±0.1N), which can output force components (Fx, Fy, Fz) in the three directions of X-axis, Y-axis and Z-axis in real time, respectively meeting the detection requirements of the horizontal direction (clamping force) and the vertical direction (load force).

[0063] Example 2

[0064] Based on Embodiment 1, the clamping force control module 15 includes:

[0065] The clamping force determination unit is configured as follows:

[0066] During the grasping process, the first force information collected by the force sensing module 5 is acquired, and the actual gripping force is extracted based on the force components in the X-axis or Y-axis direction. ;

[0067] Perform static interval verification to determine Is it within the preset material clamping force range? Inside;

[0068] Perform dynamic trend analysis on multiple actual clamping forces sampled continuously. Perform fluctuation amplitude calculation and trend slope calculation;

[0069] When static interval verification or dynamic trend analysis triggers abnormal conditions, a clamping force adjustment demand signal is generated.

[0070] The PID control unit is configured as follows:

[0071] Receive the clamping force adjustment request signal sent by the clamping force judgment unit;

[0072] Based on the actual clamping force Fluctuation range The clamping force is iteratively adjusted according to the preset clamping force range using a PID control algorithm, and the adjusted clamping force is calculated in each iteration. The gripping force of the gripper 9 is dynamically adjusted until the actual gripping force F is reached. 实 It remains stable within the preset range.

[0073] In practical applications, when the clamping force determination unit performs dynamic trend analysis, it is configured as follows:

[0074] Clamping force range determination: Preset clamping force ranges for different materials. (e.g., 14-16N anode carbon block), real-time detection of actual clamping force F 实 If F 实 Not in the range If it falls within the specified range, it is considered abnormal;

[0075] Fluctuation amplitude judgment: Calculate the average value of the actual clamping force value of n consecutive samples (e.g., n=10, sampling frequency 100Hz). And the actual clamping force value of each sample relative to fluctuation range ,when When the fluctuation exceeds the preset fluctuation threshold (e.g., ΔF>|±2N|), it is determined that there is a risk of material slippage.

[0076] Trend slope determination: Perform linear fitting on the actual clamping force values ​​sampled n times consecutively, and calculate the slope of force change. ,when When the value is negative and its absolute value exceeds the preset descent threshold (e.g.) The value >|-0.5N| / cycle indicates a continuous decrease in force, suggesting a risk of the clamp loosening.

[0077] The PID control unit is configured to iteratively adjust according to the following formula:

[0078] In each adjustment cycle, calculate the adjusted clamping force:

[0079]

[0080] in: This represents the current actual clamping force; The current fluctuation amplitude is calculated in the fluctuation amplitude judgment. , , These are the proportional coefficient, integral coefficient, and differential coefficient; the adjusted clamping force. Limited to a preset clamping force range Inside.

[0081] For example, Kp=0.3, Ki=0.1, Kd=0.05, and the adjustment range is controlled within 0.3-0.5N / time.

[0082] The PID control unit repeats the above iterative process, monitoring the actual clamping force in real time. The adjustment will stop when the following conditions are met simultaneously:

[0083] 1) Actual clamping force Stable within the preset range Inside;

[0084] 2) Fluctuation range Preset fluctuation threshold;

[0085] 3) Trend slope The preset threshold for the decrease.

[0086] It is understood that the clamping force adjustment in this embodiment can occur either during the initial gripping stage or during the subsequent material lifting and movement process.

[0087] Example 3

[0088] Based on Embodiment 2, the control system 12 further includes:

[0089] The threshold adaptive adjustment module 16 is used to obtain the state characteristics of the current sorting condition when sorting is completed, and to construct a closed-loop optimization model based on reinforcement learning to adjust the preset clamping force range. Perform autonomous iterative optimization.

[0090] In one optional implementation, the threshold adaptive adjustment module 16 includes:

[0091] The state space construction unit is used to obtain the state characteristics of the current sorting process when sorting is completed. The state characteristics include at least: material category characteristics and steady-state gripping force. Fluctuation range and recent sorting success rate ;

[0092] Among them, steady-state clamping force This is the actual gripping force value at the end of this sorting process, with fluctuation range. This refers to the force fluctuation statistic calculated based on the most recent n consecutive samples at the end of this sorting process; recent sorting success rate. Based on the most recent Sliding window statistics of the sorting results;

[0093] Action space definition unit, used to define the adjustment amount of the clamping force threshold. , The value range is the preset adjustment step size interval;

[0094] The reward function calculation unit is used to calculate the immediate reward based on the result of each sorting operation. The reward function is:

[0095]

[0096] in, This represents the recent sorting success rate, indicating the rate based on the most recent... Sliding window statistics of the sorting results; This is the equipment wear coefficient for this sorting operation; This is the current clamping force threshold. Preset interval The central value; This is the optimal clamping force estimate for the current material category; , , These are the preset weighting coefficients; for example, the weighting coefficients are set to a=1.0, b=0.8, and c=0.5.

[0097] Strategy update unit, used to update based on immediate rewards The model parameters are updated using a reinforcement learning algorithm to adjust the gripping force threshold to obtain higher rewards under the same or similar state features. ;

[0098] Threshold iteration unit, used to adjust the output. Update the clamping force threshold and send it to the clamping force control module 15:

[0099]

[0100] For example, Adjust the step size interval to .

[0101] Preferably, module 16 further includes:

[0102] The optimal estimation update unit is used to update the clamping force threshold used in each successful sorting operation. Add the successful sample queue and update the optimal gripping force estimate based on the force values ​​in the successful sample queue. ;

[0103] Preferably, module 16 further includes:

[0104] Convergence determination unit, used when continuous Success rate of secondary sorting The preset success rate threshold, and the fluctuation range at the end of this sorting process. When the preset fluctuation threshold is reached, stop parameter iteration and lock the current clamping force threshold. and set the current clamping force threshold. Send to the clamping force control module 15 to update the preset range. .

[0105] For example, when the success rate S ≥ 98% for 100 consecutive sorting operations and the fluctuation range ΔF ≤ |±1N|, the parameter iteration is stopped and the current clamping force threshold is locked. .

[0106] like Figure 3 As shown, this is a learning curve for threshold adaptive adjustment. The horizontal axis represents the number of sorting operations (0-1000 times), and the vertical axis represents the threshold adjustment amount corresponding to different sorting operations during the convergence process of gradually optimizing the clamping force threshold from the initial 15N to 14.5N.

[0107] Preferably, in the optimal estimation update unit, the successful sample queue adopts a sliding window mechanism, retaining only the most recent samples. The gripping force threshold for a single successful sorting operation, and the optimal gripping force estimate. Updated to the arithmetic mean of all clamping force thresholds within the sliding window.

[0108] Preferably, the policy update unit uses the policy gradient algorithm to update the neural network parameters, and the update formula is:

[0109]

[0110] in For neural network parameters, For the policy function, For learning rate, To accumulate discount rewards, This is the current state.

[0111] Example 4

[0112] Based on Embodiments 1-3, the control system 12 further includes:

[0113] The foreign object identification module 17 is used to acquire the first force information collected by the force acquisition module 5 when the parallel robot body 6 completes the grasping and starts to lift the material, and extract the actual load force based on the first force information of the material. Based on the comparison result of the actual load force and the theoretical load force, it determines whether the current material is a foreign object; and generates the corresponding sorting instruction based on the judgment result.

[0114] Understandably, at the moment of lifting after gripping (when the gripper is in full contact with the material), the force sensing module 5 detects the force signal (unit: N) in real time. This signal is directly related to the weight of the material. Therefore, the component corresponding to the weight of the material (F_weight ≈ mg, m is the mass of the material) can be extracted from the force signal at this time. By comparing it with the theoretical weight value of the current type of material, it can be identified whether the current material is a foreign object.

[0115] In one alternative implementation, the foreign object detection module 17 is configured as follows:

[0116] During the stage when the parallel robot body 6 completes the grasping and begins to lift the material, the first force information collected by the force sensing module 5 is acquired, and the force component in the Z-axis direction of the first force information is extracted as the actual load force. ;

[0117] Obtain the material's category label and volume from the visual acquisition module 8. The corresponding density parameter is called based on the category label. Calculate the theoretical load capacity ,in =9.8 m / s² is the acceleration due to gravity. The preset clamping load compensation coefficient is used to compensate for the additional load caused by dynamic impact and vibration between the material and the gripper during the gripping process. Preferably, f=1.2.

[0118] Calculate the deviation rate between the actual load force and the theoretical load force. ;

[0119] According to the deviation rate The comparison result with the preset threshold determines whether the current material is a foreign object and the type of foreign object. Based on the determination result, a corresponding sorting instruction is generated. The sorting instruction includes: if it is identified as normal material, the motion control unit continues to drive the parallel robot body 6 to move the material to the target position; if it is identified as a foreign object, the parallel robot body 6 is controlled to perform a safe discarding action.

[0120] Specifically, the theoretical force signal is calculated as follows: based on the category identified by visual recognition, the corresponding density parameter is used to calculate the baseline value of the theoretical force signal.

[0121] If it is an anode carbon block: ρ1 = 1.6 g / cm³, if it is an electrolyte: ρ2 = 2.4 g / cm³.

[0122] Foreign object determination rules:

[0123] If | | ≤ 10%: judged as "normal crawling", and the category is consistent with visual recognition (e.g. near 1, then it is confirmed to be an anode carbon block);

[0124] If | | > 10%: Triggering an anomaly check, further analyzing the cause of the deviation;

[0125] Foreign object differentiation logic:

[0126] like ≈2× (For example, if the weight of the foreign object is twice that of the normal material): Combined with the "unknown category" label of visual recognition, it is determined to be a "heavy foreign object" (such as a high-density metal block mixed in).

[0127] like Much larger (But the weight did not double): It may be a "high-hardness foreign object" (such as a stone), because the high hardness causes an abnormal peak value of the force signal at the moment of contact;

[0128] like much smaller This could be due to a "lightweight foreign object" (such as a piece of plastic) or a failed grasp (incomplete gripping).

[0129] When the material is identified as normal, the motion control unit drives the parallel robot body 6 to transfer the material to the chute.

[0130] When a foreign object is identified, heavy foreign objects may slip during the process of grabbing and lifting the material. In this case, the position of the material can be recorded and handled by other mechanisms. If it is another type of foreign object, the "safe disposal" action will be performed.

[0131] Preferably, when a foreign object is identified, the characteristics of the foreign object (force signal curve, visual contour) are recorded and fed back to the system to optimize the YOLO deep learning model in the vision module (increasing the number of foreign object training samples), thereby continuously improving the accuracy of material identification.

[0132] Example 5

[0133] Based on Embodiments 1-4, the control system 12 further includes:

[0134] The obstacle avoidance and collaborative control module 18 is used to receive the second force information collected by the force acquisition module 5 during the sorting process, and calculate the resultant external force at the end of the robotic arm based on the second force information;

[0135] When the net external force exceeds the preset collision trigger threshold, a collision risk is determined to exist;

[0136] In response to the determination of a collision risk, the cooperative control mechanism is activated, sending an obstacle avoidance displacement command to the parallel robot body 6 and a stop preparation signal to the emergency stop module 19.

[0137] At a preset time before the end of the timing window, the second force sensory information is re-acquired and a new net external force is calculated.

[0138] Based on the comparison between the new net external force and the collision trigger threshold, determine whether to send a cancel shutdown preparation signal to the emergency stop module 19.

[0139] In one alternative implementation, the obstacle avoidance cooperative control module 18 is configured as follows:

[0140] Receive the second force information collected by the force acquisition module 5, and calculate the resultant external force of the end effector of the robotic arm based on the second force information;

[0141] When the net external force exceeds the preset collision trigger threshold, a collision risk is determined to exist;

[0142] In response to the determination of a collision risk, a collaborative control mechanism is activated:

[0143] (1) Send a task interruption command to the parallel robot body 6 to pause the currently executing grasping or sorting task, and send an obstacle avoidance displacement command to drive the parallel robot body 6 to perform a reverse obstacle avoidance action.

[0144] (2) Send a shutdown preparation signal to the emergency shutdown module 19 and start the timing window as a safety backup;

[0145] (3) At a preset time before the end of the timing window (e.g., 0.1s before the end of the timing window), send an instruction to the force sensing acquisition module 5 to re-acquire the second force sensing information and calculate the new resultant external force;

[0146] (4) Based on the comparison results between the new net external force and the collision trigger threshold:

[0147] 1) If the new combined external force does not exceed the collision trigger threshold, a stop preparation cancellation signal is sent to reset the timer of the emergency stop module 19 and cancel the stop preparation state, and at the same time, a task recovery command is sent to the parallel robot body 6.

[0148] 2) If the new combined external force still exceeds the collision trigger threshold, maintain the shutdown preparation signal and trigger the emergency shutdown after the timer window expires.

[0149] The control system 12 also includes an emergency stop module 19, which is used to start a timing window according to the stop preparation signal sent by the obstacle avoidance cooperative control module 18. If a stop preparation cancellation signal is received, the timing window is reset and the stop preparation state is cancelled. If the timing window expires and no stop preparation cancellation signal is received, a forced stop is triggered.

[0150] In practice, the abnormal obstacle avoidance-emergency shutdown collaborative method process is as follows:

[0151] 1. Initial state: The robotic arm is in normal material sorting operation. The force sensor monitors the reaction force signal at the end of the robotic arm in real time as a basis for determining whether it has come into contact with an obstacle.

[0152] 2. Abnormal Trigger: Due to the vibration of the sorting table, the material shifted by 6mm, causing the end of the robotic arm to accidentally contact the adjacent crust (obstacle) instead of the target material as expected.

[0153] 3. Signal detection and judgment: The force sensor immediately detects the reaction force generated by this contact, which is 22N, exceeding the preset trigger threshold (20N). The system judges it as a "collision risk that requires emergency handling".

[0154] 4. Dual parallel action start-up:

[0155] Branch 1 (Emergency Stop Preparation): The system sends a "preparation signal" to the emergency stop module 19 and starts a 0.5s timer window. If obstacle avoidance is not completed within 0.5s, the emergency stop module 19 will trigger a stop, forcibly interrupting the robotic arm's movement to avoid damage.

[0156] Branch 2 (Active Obstacle Avoidance): The system synchronously drives the robotic arm to move along the reaction force direction (negative X-axis direction), with a set displacement of 3mm (in actual execution, the displacement can be dynamically set according to the robotic arm speed). The target completes this obstacle avoidance action within 0.3s, physically detaching from contact with the obstacle.

[0157] 5. Obstacle avoidance result judgment: The robotic arm actually completed a 3mm reverse movement within 0.3s, successfully escaping the obstacle. The obstacle avoidance action was completed within the 0.5s timing window of the emergency stop module 19.

[0158] 6. Process recovery: Since obstacle avoidance was successful and the timeout was not exceeded, the emergency stop module 19 was not triggered, the robotic arm was freed from the collision risk, and the normal material sorting process was restored.

[0159] It is understood that the timing triggering relationship between the modules of the control system 12 can be implemented through an event-driven mechanism (such as sending an event to trigger motion control after visual recognition is completed), a data-driven mechanism (such as the collision risk determination unit periodically monitoring the resultant external force), or a hybrid architecture. Those skilled in the art can choose an appropriate implementation method according to the system's real-time requirements and the characteristics of the hardware platform; this embodiment does not limit this approach.

[0160] Example 6

[0161] This embodiment provides a parallel robot sorting device based on vision and force fusion, including the control system 12 of embodiments 1-5, and the device further includes:

[0162] The vision acquisition module 8 is located above the sorting area and is used to acquire image data of the materials to be sorted and send it to the control system 12.

[0163] The force sensing acquisition module 5 is installed on the gripper 9 at the end of the robotic arm of the parallel robot body 6, and is used to collect force sensing information of the interaction between the gripper 9 and the material during the grasping process in real time.

[0164] Parallel robot body 6 is used to transfer materials to the target location to complete sorting.

[0165] Example 7

[0166] like Figure 4 As shown, this embodiment provides a parallel robot sorting control method based on vision and force fusion, applied to the control system 12 described in the above embodiment, including the following steps:

[0167] S101: The visual recognition module 13 responds to the image data of the material to be sorted sent by the visual acquisition module 8 and identifies the category and three-dimensional position information of the material.

[0168] S102: The motion control module 14 responds to the three-dimensional position information sent by the vision recognition module 13, controls the parallel robot body 6 to move to the gripping position, grips the material with an initial gripping force according to the material type, and drives the parallel robot body 6 to transfer the material to the target position.

[0169] S103: The gripping force control module 15 receives the first force information sent by the force sensing acquisition module 5 during the gripping process and extracts the actual gripping force. It performs stability analysis on the actual gripping force and generates a gripping force adjustment command when it determines that there is a gripping abnormality. It dynamically adjusts the gripping force of the gripper 9 so that the actual gripping force is stable within the preset range.

[0170] In one alternative implementation, the method further includes:

[0171] S104, Foreign Object Recognition Module 17, during the stage when the parallel robot body 6 completes the grasping and starts lifting the material, acquires the first force information collected by the force sensing acquisition module 5, extracts the actual load force based on the first force information of the material, and judges whether the current material is a foreign object based on the comparison result of the actual load force and the theoretical load force.

[0172] S105, generate corresponding sorting instructions based on the judgment result: if identified as normal material, the motion control unit continues to drive the parallel robot body 6 to move the material to the target position; if identified as foreign object, control the parallel robot body 6 to perform a safe discarding action.

[0173] like Figure 5As shown, in one optional implementation, the method further includes:

[0174] S106, the obstacle avoidance cooperative control module 18 receives the second force information collected by the force acquisition module 5 during the sorting process, and calculates the resultant external force at the end of the robotic arm based on the second force information; when the resultant external force exceeds the preset collision trigger threshold, it is determined that there is a collision risk.

[0175] S107, in response to the determination of a collision risk, the cooperative control mechanism is activated, sending an obstacle avoidance displacement command to the parallel robot body 6 and a stop preparation signal to the emergency stop module 19;

[0176] S108, at a preset time before the end of the timing window, the second force sensory information is reacquired and the new resultant external force is calculated;

[0177] S109, based on the comparison result of the new resultant external force and the collision trigger threshold, determine whether to send a cancel shutdown preparation signal to the emergency stop module 19.

[0178] Example 8

[0179] This embodiment proposes an electronic device, including: a memory for storing computer programs;

[0180] The processor is used to execute the program stored in the memory to implement the steps of the above embodiment of a parallel robot sorting control method based on vision and force fusion.

[0181] For details on the specific implementation of each step and related explanations, please refer to the aforementioned embodiment of a parallel robot sorting control method based on the fusion of vision and force, which will not be repeated here.

[0182] The memory of the electronic device mentioned in the embodiments of this disclosure may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device.

[0183] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0184] This disclosure also proposes a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the steps of the above-described embodiment of a parallel robot sorting control method based on vision and force fusion. For specific implementation details and explanations of each step, please refer to the foregoing method embodiments; further elaboration is not provided here.

[0185] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0186] It should be understood that the above embodiments are only used to illustrate the technical solutions of this disclosure, and not to limit them; although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A parallel robot sorting control system based on vision and force fusion, wherein the control system (12) is communicatively connected to a vision acquisition module (8), a force acquisition module (5), and the parallel robot body (6), characterized in that, The system includes: The visual recognition module (13) is used to identify the category and three-dimensional position information of the material in response to the image data of the material to be sorted sent by the visual acquisition module (8); The motion control module (14) is used to respond to the three-dimensional position information sent by the vision recognition module (13), control the parallel robot body (6) to move to the gripping position, grip the material with an initial gripping force according to the material type, and drive the parallel robot body (6) to move the material to the target position. The gripping force control module (15) is used to receive the first force information sent by the force acquisition module (5) during the gripping process and extract the actual gripping force, perform stability analysis on the actual gripping force, and generate a gripping force adjustment command when it is determined that there is a gripping abnormality, so as to dynamically adjust the gripping force of the gripper (9) so that the actual gripping force is stable within the preset range. The force sensing acquisition module (5) is set on the gripper at the end of the mechanical arm of the parallel robot body (6) and is used to collect the first force sensing information of the interaction between the gripper and the material during the grasping process in real time. The clamping force control module (15) includes: The clamping force determination unit is configured as follows: During the grasping process, the first force information collected by the force sensing acquisition module (5) is obtained, and the actual gripping force is extracted based on the force components in the X-axis or Y-axis direction. ; Perform static interval verification to determine Is it within the preset material clamping force range? Inside; Perform dynamic trend analysis on multiple actual clamping forces sampled continuously. Perform fluctuation amplitude calculation and trend slope calculation; When static interval verification or dynamic trend analysis triggers abnormal conditions, a clamping force adjustment demand signal is generated. The system also includes: The foreign object identification module (17) is used to acquire the first force information collected by the force acquisition module (5) when the parallel robot body (6) completes the grasping and starts lifting the material, and extracts the actual load force according to the first force information of the material. Based on the comparison result of the actual load force and the theoretical load force, it determines whether the current material is a foreign object; and generates the corresponding sorting instruction according to the judgment result.

2. The parallel robot sorting control system based on vision and force fusion according to claim 1, characterized in that, The clamping force control module (15) further includes: The PID control unit is configured as follows: Receive the clamping force adjustment request signal sent by the clamping force judgment unit; Based on the actual clamping force Fluctuation range The clamping force is iteratively adjusted according to the preset clamping force range using a PID control algorithm, and the adjusted clamping force is calculated in each iteration. The gripping force of the gripper (9) is dynamically adjusted until the actual gripping force F is reached. 实 It remains stable within the preset range.

3. The parallel robot sorting control system based on vision and force fusion according to claim 2, characterized in that, In each iteration cycle, the adjusted clamping force is calculated: ； in: This represents the current actual clamping force. The current fluctuation amplitude is calculated in the fluctuation amplitude judgment. , , These are the proportional coefficient, integral coefficient, and differential coefficient; the adjusted clamping force. Limited to a preset clamping force range Inside.

4. The parallel robot sorting control system based on vision and force fusion according to claim 1, characterized in that, The system also includes: The threshold adaptive adjustment module (16) is used to obtain the state characteristics of the current sorting condition when sorting ends, construct a closed-loop optimization model based on reinforcement learning, and adjust the preset clamping force range. Perform autonomous iterative optimization.

5. The parallel robot sorting control system based on vision and force fusion according to claim 1, characterized in that, The step of extracting the actual load force based on the first force information of the material includes: extracting the force component in the Z-axis direction of the first force information as the actual load force. .

6. The parallel robot sorting control system based on vision and force fusion according to claim 1, characterized in that, The visual recognition module (13) is configured to identify the material category based on the image data of the material to be sorted using a deep learning algorithm.

7. A sorting control method for parallel robots based on vision and force fusion, applied in the sorting control system for parallel robots based on vision and force fusion as described in any one of claims 1-6, characterized in that, Includes the following steps: S101, the visual recognition module (13) responds to the image data of the material to be sorted sent by the visual acquisition module (8) and identifies the category and three-dimensional position information of the material; S102, the motion control module (14) responds to the three-dimensional position information sent by the vision recognition module (13), controls the parallel robot body (6) to move to the gripping position, grips the material with an initial gripping force according to the material type, and drives the parallel robot body (6) to transfer the material to the target position. S103, the gripping force control module (15) receives the first force information sent by the force sensing acquisition module (5) during the gripping process and extracts the actual gripping force. It performs stability analysis on the actual gripping force and generates a gripping force adjustment command when it is determined that there is a gripping abnormality. It dynamically adjusts the gripping force of the gripper (9) so that the actual gripping force is stable within the preset range.

Images