Delta robot sorting method based on visual and force feedback

Abstract

The application discloses a kind of based on vision and force feedback's Delta mechanical arm sorting method, it is related to image processing technical field, including steps: through the embedded vision module of being fixed in the exterior of mechanical arm acquisition target image and extract pixel coordinates, mapping as the plane target coordinate under the base coordinate system of mechanical arm by coordinate transformation;Vertical direction depth coordinate is obtained by ultrasonic ranging module on end effector, forms space target coordinate;Controller is based on the coordinate call kinematics inverse solution algorithm and calculates the expected angle of each active arm, and generates pulse signal and drives stepping motor to make end effector move to target position;Control rudder to close gripper, simultaneously real-time acquisition gripper inner wall pressure diaphragm sensor pressure value, when pressure reaches preset threshold, stop rudder action, complete grabbing.The application realizes the flexible grabbing of automatic vision positioning and force feedback control to target, improves the intelligent degree and grabbing reliability of Delta mechanical arm sorting.

Description

Technical Field

[0001] This invention relates to the field of industrial robot control technology, specifically to a Delta robotic arm sorting method based on vision and force feedback. Background Technology

[0002] Delta parallel robotic arms, due to their compact structure, high speed, and high positioning accuracy, have been widely used in automated production lines in light industries such as food packaging, pharmaceutical sorting, and electronic assembly. Traditional Delta robotic arm sorting systems typically employ industrial cameras in conjunction with industrial control computers for visual guidance. Image processing software identifies the position information of target objects, and the host computer sends the coordinate data to the servo driver to control the robotic arm to perform the grasping and placing operations. While these systems offer stable performance and high accuracy, they are expensive overall and often rely on closed industrial vision libraries and dedicated motion control cards, resulting in complex system integration and high maintenance requirements. In recent years, some studies have attempted to combine embedded vision modules (such as OpenMV and K210) with lightweight robotic arms to reduce system costs and development complexity. However, existing solutions generally suffer from the following shortcomings: vision modules typically use an eye-in-hand mounting method, and the conversion relationship between image coordinates and robotic arm coordinates dynamically changes with the end effector's pose, making calibration cumbersome; the end effector lacks a force feedback mechanism, making it prone to slippage or damage when grasping rigid objects due to excessive gripping force, resulting in poor grasping adaptability; and the control modes are limited, mostly supporting only automatic sorting along preset trajectories, lacking manual intervention methods, making it difficult to cope with sudden anomalies on the production line. Furthermore, existing Delta robotic arm control methods based on stepper motors often employ simple pulse counting control for motion trajectory planning, resulting in poor motion smoothness and a lack of synchronization and coordination mechanisms between the active arms, easily causing end effector jitter. Therefore, how to improve the positioning accuracy, grasping flexibility, and operational agility of Delta robotic arm vision sorting systems while maintaining low cost and ease of development has become a pressing technical problem to be solved in this field. Summary of the Invention

[0003] To improve the positioning accuracy, gripping flexibility, and operational agility of the Delta robotic arm vision sorting system while maintaining low cost and ease of development, this invention proposes a Delta robotic arm sorting method based on vision and force feedback, comprising the following steps: S1: The embedded vision module, which is fixedly installed on the outside of the Delta robotic arm, acquires images of the target to be sorted, identifies and extracts the pixel coordinates of the target in the image pixel coordinate system; S2: Transmit the pixel coordinates to the controller and map the pixel coordinates to the planar target coordinates in the Delta robotic arm base coordinate system through a coordinate transformation algorithm; S3: The distance between the end effector and the target to be sorted is detected by the ultrasonic ranging module installed on the Delta robotic arm end effector, and the vertical target depth coordinates are obtained. These coordinates, together with the planar target coordinates, constitute the spatial target coordinates of the end effector. S4: The controller calls the Delta inverse kinematics algorithm of the robotic arm to calculate the desired angle of rotation required to drive each active arm according to the coordinates of the spatial target. S5: Generate corresponding pulse signals according to the desired angle, and drive the stepper motors connected to the corresponding active arms to rotate, so that the end effector moves to the position indicated by the spatial target coordinates; S6: Controls the servo motor gripper on the end effector to close and collects the gripping pressure value detected by the pressure film sensor installed on the inner wall of the gripper in real time. When the gripping pressure value reaches the preset threshold, the servo motor action is stopped, and the gripping of the target to be sorted is completed.

[0004] This invention utilizes external vision and ultrasonic fusion positioning, inverse kinematics control, and force feedback flexible gripper techniques to achieve autonomous, precise positioning and adaptive grasping of targets by the Delta robotic arm on a low-cost embedded platform.

[0005] Furthermore, in step S1, the embedded vision module is fixedly installed on the outside of the Delta robotic arm body, and the target to be sorted is a single-color object. The center pixel coordinates of the color block are identified and extracted as the pixel coordinates through a color threshold segmentation algorithm.

[0006] Furthermore, in step S2, the coordinate transformation algorithm specifically includes: Establish the affine transformation relationship between the pixel coordinate system and the robot arm base coordinate system. Obtain the transformation matrix through calibration. Multiply the collected pixel coordinates by the transformation matrix to obtain the X and Y plane coordinates in the robot arm base coordinate system.

[0007] Furthermore, in step S4, the Delta robotic arm inverse kinematics algorithm specifically comprises: Establish a static coordinate system with the center of the static platform as the origin, and establish a dynamic coordinate system with the center of the dynamic platform as the origin. Let the distance from the vertex of the static platform to the origin be . The distance from the vertex of the moving platform to the origin is The length of the active arm is From the boom length of Equations are established based on geometric constraints. The angle between the active arm and the stationary platform can be obtained by solving the problem. From the perspective of expectation, where: , , , .

[0008] Furthermore, when invoking the inverse kinematics algorithm, the controller also includes processing the solved desired angle. Amplitude limiting is performed: if the calculated desired angle is less than zero, it is set to zero, and the desired angle is converted into the number of pulse steps of the stepper motor based on the linear relationship between the desired angle and the number of pulses.

[0009] Furthermore, before step S4, there is also a step: the controller performs visual recognition and ultrasonic ranging noise elimination under multi-point sampling and filtering processing on the received spatial target coordinates.

[0010] Furthermore, in step S5, when the stepper motor rotates, a trapezoidal speed curve or a triangular speed curve is used for trajectory planning, and the pulse frequency is adjusted in the acceleration segment, the constant speed segment and the deceleration segment respectively.

[0011] Furthermore, the trajectory planning specifically includes: The movement distance of each active arm from its current position to the target position is calculated. Based on the preset maximum speed and acceleration, a trapezoidal velocity curve is used to generate the position increment within each control cycle, and the pulse output of each active arm is updated synchronously so that each active arm reaches the target position synchronously.

[0012] Furthermore, in step S6, the preset threshold is set by adjusting the potentiometer connected to the pressure film sensor, and the controller uses a comparator circuit to monitor the pressure signal in real time. When the pressure value reaches the preset threshold, the drive signal output of the servo motor is stopped.

[0013] Compared with the prior art, the present invention has at least the following beneficial effects: (1) The present invention proposes a Delta robotic arm sorting method based on vision and force feedback. By fixing the embedded vision module to the outside of the Delta robotic arm body, the spatial target coordinates are obtained by combining the ultrasonic ranging module, and the coordinates are converted into the desired angles of each active arm by using the inverse kinematics algorithm. Then, the end effector is driven by the stepper motor to move to the target position. At the same time, the pressure film sensor is integrated on the inner wall of the gripper to realize the gripping force feedback control, forming a complete vision guidance and flexible gripping linkage mechanism. (2) In the automatic sorting scenario of single color target, the precise mapping from image pixel coordinates to robotic arm spatial coordinates is realized with low-cost embedded hardware, and the force feedback threshold control is used to avoid damage to objects or mechanisms due to excessive clamping force. At the same time, the trajectory planning of trapezoidal or triangular velocity curves helps to improve the stability of robotic arm movement. Overall, the positioning accuracy, automation level and gripping reliability of sorting operations are improved, and the system's dependence on high-performance computing units and expensive servo drives is reduced, making it easy to promote and apply in lightweight, low-cost automated sorting scenarios. Attached Figure Description

[0014] Figure 1 A step diagram of a Delta robotic arm sorting method based on vision and force feedback; Figure 2 This is a schematic diagram of the Delta robotic arm. Figure 3 A simplified mathematical model diagram of the Delta robotic arm; Explanation of reference numerals in the attached diagram: 11-Driving arm, 12-Driven arm. Detailed Implementation

[0015] To address the problems existing in the prior art, this invention proposes a Delta robotic arm sorting method based on vision and force feedback. By integrating a low-cost embedded vision module, an ultrasonic ranging module, and a pressure film sensor, combined with inverse kinematics algorithms and trajectory planning, it simultaneously supports automatic vision-guided sorting and manual remote control operation. This achieves a cost-effective, easy-to-deploy, and safety-protected intelligent sorting solution. Figure 1 As shown, the method mainly includes the following steps: S1: The embedded vision module, which is fixedly installed on the outside of the Delta robotic arm, acquires images of the target to be sorted, identifies and extracts the pixel coordinates of the target in the image pixel coordinate system; S2: Transmit the pixel coordinates to the controller and map the pixel coordinates to the planar target coordinates in the Delta robotic arm base coordinate system through a coordinate transformation algorithm; S3: The distance between the end effector and the target to be sorted is detected by the ultrasonic ranging module installed on the Delta robotic arm end effector, and the vertical target depth coordinates are obtained. These coordinates, together with the planar target coordinates, constitute the spatial target coordinates of the end effector. S4: The controller calls the Delta inverse kinematics algorithm of the robotic arm to calculate the desired angle of rotation required to drive each active arm according to the coordinates of the spatial target. S5: Generate corresponding pulse signals according to the desired angle, and drive the stepper motors connected to the corresponding active arms to rotate, so that the end effector moves to the position indicated by the spatial target coordinates; S6: Controls the servo motor gripper on the end effector to close and collects the gripping pressure value detected by the pressure film sensor installed on the inner wall of the gripper in real time. When the gripping pressure value reaches the preset threshold, the servo motor action is stopped, and the gripping of the target to be sorted is completed.

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the vision sorting method based on a Delta robotic arm provided by this invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that the description herein is for illustrative purposes only and is not intended to limit the scope of protection of this invention.

[0017] The Delta robotic arm sorting method proposed in this invention, based on vision and force feedback, can be applied to a Delta robotic arm sorting system consisting of an STM32 series microcontroller, an OpenMV embedded vision module, an A4988 stepper motor driver, a 42-stepper motor, an end effector (including a servo motor, a pressure film sensor, and an ultrasonic ranging module), and a remote control. The main body of the system is separated from the remote control; the main body is responsible for performing sorting tasks under automatic or manual control, while the remote control uses a LoRa wireless module to operate in manual mode. The implementation process of this vision-based sorting method will be described in detail below using a specific Delta robotic arm hardware platform.

[0018] In this embodiment, as Figure 2 and Figure 3 As shown, the Delta robotic arm employs a mechanical structure with three symmetrically distributed arms connected in parallel, with an angle of 120 degrees between adjacent arms. Each arm includes an active arm 11 and a driven arm 12. The active arm is directly driven by a 42-stepper motor. When the active arm rotates, it drives the driven arm to move via a ball joint or universal joint, thereby controlling the spatial position of the moving platform. The center of the static platform (the plane composed of C1, C2, and C3) is defined as the origin O of the base coordinate system, and the center of the moving platform (the plane composed of A1, A2, and A3) is defined as the origin p of the moving coordinate system. The distance R from the vertices of the static platform (C1, C2, and C3) to the origin O, the distance r from the vertices of the moving platform (A1, A2, and A3) to the origin p, the length L of the active arm (C1B1, C2B2, and C3B3), and the length La of the driven arm (B1A1, B2A2, and B3A3) are all known structural parameters. These parameters are pre-stored in the controller's non-volatile memory for kinematic solution. The controller uses the STM32F103ZET6 chip, which has a 72MHz main frequency, 512KB Flash and rich peripheral interfaces, and can meet the real-time requirements of inverse kinematics solution, trajectory planning and multi-channel pulse output.

[0019] Before the system can perform automatic sorting tasks, initialization is required. After the controller is powered on, it first completes the system clock configuration, GPIO pin function initialization, timer PWM output mode configuration, serial communication parameter settings (baud rate 115200, data bits 8, stop bits 1, no parity), and ADC module initialization. Simultaneously, it performs status checks on all peripheral modules connected to the controller, including communication testing of the OpenMV vision module, trigger response testing of the ultrasonic ranging module, enable and direction control testing of the stepper motor driver, servo motor center angle calibration, initial level reading of the pressure film sensor comparator circuit, and AT command configuration of the LoRa wireless module. After all modules are successfully initialized, the system enters the main loop, waiting for the selection of the operating mode or an external trigger signal.

[0020] In automatic mode, the visual sorting method first acquires images of the target to be sorted using an OpenMV embedded vision module fixedly mounted on the exterior of the Delta robotic arm. In this embodiment, the target to be sorted is a small red ball of a single color. The OpenMV module uses an STM32H750 as the main control chip, and its integrated camera sensor is an OV7725, supporting RGB565 color image output. In the OpenMV IDE development environment, a color threshold range for red is pre-configured, including the upper and lower limits of the L, A, and B components in the LAB color space. During the recognition process, the vision module continuously acquires images and calls the built-in find_blobs function to find all connected regions (color patches) that meet the preset color threshold. To eliminate noise interference, the algorithm only retains the color patch with the largest area as the target. Then, the coordinates of the geometric center of the color patch in the image pixel coordinate system are extracted. The pixel coordinates of the target to be sorted are used. To reduce fluctuations that may occur in a single sampling, the OpenMV module continuously reads the largest color block 10 times. The module calculates the average of these 10 sets of data, and uses the averaged pixel coordinates as the final output value. The module connects to the PA10 (RXD) pin of the STM32 controller via its serial port TXD pin, and sends the pixel coordinate data to the controller according to a custom communication protocol. If no color block matching the threshold is detected in the current frame, the vision module sends a specific error flag data packet to inform the controller that there is currently no target.

[0021] The controller connects to the RXD pin of the OpenMV module via its TXD pin to receive pixel coordinate data packets from the vision module. Upon receiving valid pixel coordinate data, the controller executes a coordinate transformation algorithm to map the pixel coordinates to planar target coordinates in the Delta robotic arm base coordinate system. Specifically, the coordinate transformation algorithm is based on the affine transformation relationship between the pixel coordinate system and the robotic arm base coordinate system. During system deployment, using a calibration board or calibration points with known spatial coordinates, the coordinate values ​​of several calibration points in the pixel coordinate system and their actual coordinate values ​​in the robotic arm base coordinate system are collected. An affine transformation matrix is ​​then fitted using the least squares method. This matrix includes rotation, scaling, and translation parameters. During actual operation, the controller will process the received pixel coordinates... After converting to homogeneous coordinates and multiplying by the transformation matrix, the X-axis and Y-axis plane coordinates in the robot arm's base coordinate system can be obtained. This process eliminates nonlinear errors caused by the installation angle of the vision module, lens distortion, and the relative positional relationship between the camera and the robot arm, resulting in high positioning accuracy in the mapping from image coordinates to physical coordinates.

[0022] While acquiring the coordinates of the planar target, the controller activates the HC-SR04 ultrasonic ranging module installed on the end effector to detect the vertical distance between the end effector and the target to be sorted. The control pin (Trig) and echo pin (Echo) of the ultrasonic ranging module are connected to the controller's general-purpose input / output pins, respectively. The controller outputs a high-level pulse of at least 10 microseconds to the Trig pin, triggering the ultrasonic ranging module to emit 40kHz ultrasonic waves. The ultrasonic ranging module pulls the Echo pin high while emitting the ultrasonic waves and pulls it low upon receiving the echo. The controller uses the timer's input capture function to measure the duration t of the high level on the Echo pin. Based on the speed of sound 340m / s (i.e., 0.0343cm / us), the distance calculation formula is: Distance The unit is cm. The measured distance value is the Z-axis depth coordinate of the end effector in the base coordinate system. The controller combines this Z-axis coordinate with the previously obtained X and Y plane coordinates to form a complete three-dimensional spatial target coordinate (X, Y, Z). It should be noted that in this embodiment, the ultrasonic ranging module is installed below the end effector, and its ranging direction is vertically downward. Therefore, the measured distance directly reflects the vertical distance between the current position of the end effector and the upper surface of the ball to be sorted. When generating the spatial target coordinates, the controller superimposes the current Z-coordinate of the end effector with the measured distance value to accurately calculate the true Z-axis height of the ball to be sorted in the base coordinate system.

[0023] Once the controller obtains the spatial target coordinates, it calls the Delta inverse kinematics algorithm for the robotic arm, which is pre-stored in the program memory, to calculate the desired rotation angles required to drive the three active arms. For example... Figure 3 As shown, the mathematical model of the inverse kinematics algorithm is established based on the aforementioned structural parameters R, r, L, and La. In the static coordinate system O-XYZ, the position vectors of the three static platform vertices Ci (i=1,2,3) are determined by R and the azimuth angle. Decision, among which Let's take 0°, 120°, and 240° respectively. The position vectors of the three moving platform vertices Ai in the moving coordinate system p-xyz are determined by r and the same azimuth angle. The position vector of the origin p in the moving coordinate system in the static coordinate system is the target coordinate (x, y, z). From this, the expressions for the driven arm vectors Ai and Bi can be derived. Based on the geometric constraint |AiBi| = La, meaning the driven arm length remains constant, the following equations are established: , in: , , .

[0024] In the above equation, Let be the angle between the i-th active arm and the stationary platform, which is the desired angle to be solved for. For i=1,2,3, expanding the equation respectively, we can obtain the following about . The quadratic equation in one variable is given. Through trigonometric transformations and algebraic solutions, we can obtain... The closed-form solution. In the actual code implementation, to avoid complex solutions and solutions without physical meaning, the controller selects the following branch during the solution process: , in, These are three intermediate variables obtained after simplifying the kinematic equations, and their specific definitions are as follows: First, the aforementioned kinematic equations Expand and organize into a list about and In the form of... By introducing the trigonometric identity, let the tangent of the half-angle of the i-th active arm be... ,but , Substituting into the equation, it can be transformed into a function about... The quadratic equation of : .in These correspond to the coefficients of the quadratic term, the coefficients of the linear term, and the constant term, respectively. Their expressions are derived from the structural parameters R, r, L, and La, as well as the target coordinates (X, Y, Z) and azimuth angle. Jointly determined. Specifically, for the i-th active arm (i=1,2,3), the following intermediate quantities are defined: , , , in , , The above. These are intermediate parameters that often appear in the inverse kinematics process, and their physical meanings are as follows: It is related to the length of the active arm and the spatial distance. This reflects the projection of the radius difference between the static and dynamic platforms in the horizontal direction. This refers to the target coordinates in the vertical direction. Further, let: , , , Will Substituting into the above formula, the angle between the active arm and the stationary platform can be obtained. (Unit: radians). During the calculation, the square root term... The radicand must be non-negative. If the value is negative due to the coordinates exceeding the workspace, the target point is determined to be unreachable, the controller will refuse to execute the motion command and return an error flag. Calculate... Then, through the arctangent function get Multiply by 2 to get After converting the calculation results from radians to degrees, the controller further... If the calculated amplitude is limited, If the angle is less than 0°, it will be forcibly set to 0°; if it is greater than the maximum allowable angle (the upper limit can be set according to the actual mechanical structure), it will be set to the upper limit value.

[0025] Achieving Expected Angles After calculation, the controller converts the angle into pulse steps based on the microstepping parameters of the stepper motor drive system. In this embodiment, the step angle of the 42-stepper motor is 1.8°, meaning that 200 pulses are needed per revolution when the motor is not microstepped. The A4988 stepper motor driver is configured in 16-microstepping mode, so the total number of pulses per revolution is 200 × 16 = 3200 pulses, and each pulse corresponds to a motor rotation angle of 0.1125°. A 90:20 reduction gear mechanism is also connected between the active arm and the stepper motor, so there is a transmission ratio relationship between the actual rotation angle of the active arm and the rotation angle of the motor. Combining the above parameters, the controller converts the desired angle of each active arm... To convert to the corresponding number of pulse steps, the conversion formula is: number of pulse steps Taking into account the actual range after angle limiting, the number of pulse steps is limited to a safe maximum value to prevent the stepper motor from overshooting the mechanical limit and causing a collision.

[0026] To achieve smooth movement of the end effector, the controller uses a trapezoidal velocity curve for trajectory planning when generating pulse signals. The specific steps are as follows: First, the current positions of the three actuators are read (represented by the cumulative number of pulse steps), and the number of pulse steps required for each actuator to move (the difference between the target position and the current position) is calculated. Then, based on the preset maximum speed (unit: hz / s) and acceleration (unit: hz / s)... 2 The controller calculates the required pulse counts for the acceleration, constant speed, and deceleration phases of each active arm. For scenarios with very short travel distances, the acceleration and deceleration phases may overlap, preventing the maximum speed from being reached. In this case, the algorithm automatically switches to a triangular velocity curve, meaning there are only acceleration and deceleration phases without a constant speed phase. During each timer interrupt (e.g., with a 1ms interrupt period), the controller calculates the required position increment for each active arm within the current control cycle based on the position update formula of the trapezoidal or triangular velocity curve. This increment is then converted into the duty cycle and direction signal of the PWM pulses and output to the STEP and DIR pins of the three A4988 drivers, respectively. To ensure synchronous startup and coordinated movement of the three stepper motors, the controller updates the comparison output registers of the three channels simultaneously at the beginning of each control cycle, aligning the rising edges of the three PWM signals as closely as possible in time. When the cumulative output pulse count of an active arm reaches its target pulse step count, the controller shuts off the PWM output of that channel until all three active arms reach their target positions. This synchronous operation mechanism ensures the relative stability of the moving platform's attitude during spatial movement, avoiding trajectory deviation or mechanism jamming caused by single-axis lag.

[0027] After the end effector moves to the position indicated by the spatial target coordinates, the controller performs a gripping operation. The servo gripper of the end effector is initially in an open state. The controller outputs a pulse width modulation signal to the servo's control signal line, the pulse width of which corresponds to the target rotation angle of the servo. In this embodiment, the control angle range of the servo is 0° to 90°, corresponding to a pulse width range of 500µs to 1500µs. The controller generates a 50Hz reference signal through the PWM output channel of the timer and dynamically adjusts the duty cycle. During the gripper closing process, a pressure diaphragm sensor installed on the inner wall of the gripper detects the contact pressure between the gripper and the ball to be sorted in real time. This pressure diaphragm sensor and an LM393 voltage comparator constitute a simple force detection circuit. The resistance of the diaphragm sensor decreases as the applied pressure increases, causing a change in the voltage division ratio at the comparator input. An adjustable potentiometer is connected to the 1IN- terminal of the comparator to set the reference voltage corresponding to the pressure threshold. When the pressure detected by the diaphragm sensor does not reach the preset threshold, the 1OUT terminal of the comparator outputs a low level; when the pressure reaches or exceeds the threshold, the 1OUT terminal outputs a high level. The controller's general-purpose input / output pins are connected to the comparator output and configured for rising-edge triggered external interrupt mode. During servo closure, the controller continuously reads the comparator output level. Once a high level is detected (i.e., the pressure reaches the threshold), the controller immediately stops outputting PWM signals to the servo, keeping it at the current angle to complete the flexible grasp of the red ball. Furthermore, the controller directly acquires the analog voltage value of the pressure diaphragm sensor's pressure divider point via an ADC module. This value serves as a quantified display of the grasping force and is uploaded to the host computer via a TFT color screen or LoRa wireless module. By adjusting the potentiometer's resistance, users can flexibly set the pressure threshold according to the hardness or weight of the object to be sorted; for example, a lower threshold can be set for grasping more fragile objects, and a higher threshold for grasping heavier objects.

[0028] The Delta-based robotic arm vision sorting method of this invention also supports a manual control mode to enhance the system's application flexibility. The manual mode is implemented through a separate remote control terminal, which is based on an STM32F103C8T6 minimum system board and equipped with a dual-axis joystick module, an OLED display, a buzzer alarm circuit, and a resistor divider circuit for power detection. The remote control terminal communicates with the main unit via a LoRa wireless module. The X-axis and Y-axis potentiometer outputs of the left joystick are connected to the STM32's ADC input channels. The controller determines the joystick's push / pull direction (forward, backward, left, right) by collecting changes in the two ADC values. The Y-axis potentiometer of the right joystick controls the upward and downward movement of the end effector. When the user pushes the left joystick, the remote control controller maps the joystick offset to velocity components in the X and Y axes according to a preset proportional coefficient and sends this information to the main unit controller via the LoRa module. Upon receiving the velocity command, the main unit updates the current target position in real time and calls the inverse kinematics algorithm and trajectory planning function to ensure the robotic arm smoothly follows the joystick movement. When the user releases the left joystick, it automatically returns to center under spring pressure, at which point the speed command returns to zero, and the robotic arm stops moving horizontally. The right joystick uses a non-self-resetting structure; after the user pushes it up or down, it remains at the current angle. The remote control continuously sends Z-axis speed commands based on the deflection angle of the right joystick, and the main unit controls the end effector to rise or fall accordingly. Manual mode also allows the user to directly control the opening and closing of the end effector via independent buttons on the remote control, facilitating gripping and placement operations when automatic recognition fails or manual intervention is required.

[0029] To enhance system safety and intelligence, this embodiment integrates multiple auxiliary detection and protection functions. Limit switches are installed at the extreme movement positions of the three active arms. The normally open contacts of the limit switches are connected to the general-purpose input / output pins of the controller and are configured with internal pull-up resistors. Under normal operation, the limit switches are open, and the controller reads a high level. When any active arm moves to its extreme position and touches a limit switch, the switch closes, and the controller reads a low level. Regardless of whether the system is in automatic or manual mode, the controller periodically scans the level states of the three limit switches. Once any limit switch is detected to be triggered, the controller immediately stops all PWM pulse outputs and pulls the enable pin of the stepper motor driver high (or low, depending on the driver logic), cutting off the power supply to the motor, thereby effectively preventing mechanical damage or motor stall caused by overtravel. Simultaneously, the controller displays the limit alarm information on the display screen and transmits it to the host computer monitoring platform via the LoRa wireless module.

[0030] The system also integrates an SHT40 temperature and humidity sensor, which connects to the controller via an I2C bus. The controller initiates temperature and humidity measurements at regular intervals (e.g., every second), reading raw data from the sensor and converting it to obtain the current ambient temperature in Celsius and relative humidity percentage. This data is displayed in real-time on a local TFT color screen and also encapsulated in a system status data packet, periodically uploaded to a PC-based host computer via a LoRa wireless module. The host computer software uses a general-purpose serial port debugging assistant or a custom graphical interface program to monitor the robotic arm's current 3D coordinates, temperature and humidity data, limit switch status, end effector gripping force feedback value, and system operating mode in real time, facilitating system debugging, fault diagnosis, and operational data recording.

[0031] In summary, the Delta robotic arm sorting method proposed in this invention, based on vision and force feedback, obtains the spatial target coordinates by fixing an embedded vision module to the outside of the Delta robotic arm body, and uses an ultrasonic ranging module to obtain the spatial target coordinates. The coordinates are then converted into the desired angles of each active arm using an inverse kinematics algorithm. Subsequently, a stepper motor drives the end effector to move to the target position. At the same time, a pressure film sensor is integrated into the inner wall of the gripper to realize the gripping force feedback control, forming a complete vision guidance and flexible gripping linkage mechanism.

[0032] In automated sorting scenarios with single-color targets, precise mapping from image pixel coordinates to robotic arm spatial coordinates is achieved with low-cost embedded hardware. Force feedback threshold control prevents damage to objects or mechanisms due to excessive gripping force. Meanwhile, trajectory planning of trapezoidal or triangular velocity curves helps improve the smoothness of robotic arm movement. Overall, this improves the positioning accuracy, automation level, and gripping reliability of sorting operations, reduces the system's dependence on high-performance computing units and expensive servo drives, and facilitates its widespread application in lightweight, low-cost automated sorting scenarios.

[0033] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0034] Furthermore, in this invention, descriptions involving terms such as "first," "second," and "a" are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0035] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0036] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

Claims

1. A Delta robotic arm sorting method based on vision and force feedback, characterized in that, Including the following steps: S1: The embedded vision module, which is fixedly installed on the outside of the Delta robotic arm, acquires images of the target to be sorted, identifies and extracts the pixel coordinates of the target in the image pixel coordinate system; S2: Transmit the pixel coordinates to the controller and map the pixel coordinates to the planar target coordinates in the Delta robotic arm base coordinate system through a coordinate transformation algorithm; S3: The distance between the end effector and the target to be sorted is detected by the ultrasonic ranging module installed on the Delta robotic arm end effector, and the vertical target depth coordinates are obtained. These coordinates, together with the planar target coordinates, constitute the spatial target coordinates of the end effector. S4: The controller calls the Delta inverse kinematics algorithm of the robotic arm to calculate the desired angle of rotation required to drive each active arm according to the coordinates of the spatial target. S5: Generate corresponding pulse signals according to the desired angle, and drive the stepper motors connected to the corresponding active arms to rotate, so that the end effector moves to the position indicated by the spatial target coordinates; S6: Controls the servo motor gripper on the end effector to close and collects the gripping pressure value detected by the pressure film sensor installed on the inner wall of the gripper in real time. When the gripping pressure value reaches the preset threshold, the servo motor action is stopped, and the gripping of the target to be sorted is completed.

2. The Delta robotic arm sorting method based on vision and force feedback as described in claim 1, characterized in that, In step S1, the embedded vision module is fixedly installed on the outside of the Delta robotic arm body. The target to be sorted is a single-color object. The center pixel coordinates of the color block are identified and extracted as the pixel coordinates by a color threshold segmentation algorithm.

3. The Delta robotic arm sorting method based on vision and force feedback as described in claim 1, characterized in that, In step S2, the coordinate transformation algorithm specifically includes: Establish the affine transformation relationship between the pixel coordinate system and the robot arm base coordinate system. Obtain the transformation matrix through calibration. Multiply the collected pixel coordinates by the transformation matrix to obtain the X and Y plane coordinates in the robot arm base coordinate system.

4. The Delta robotic arm sorting method based on vision and force feedback as described in claim 1, characterized in that, In step S4, the Delta robotic arm inverse kinematics algorithm is specifically as follows: Establish a static coordinate system with the center of the static platform as the origin, and establish a dynamic coordinate system with the center of the dynamic platform as the origin. Let the distance from the vertex of the static platform to the origin be . The distance from the vertex of the moving platform to the origin is The length of the active arm is From the boom length of Equations are established based on geometric constraints. The angle between the active arm and the stationary platform can be obtained by solving the problem. From the perspective of expectation, where: , , , .

5. The Delta robotic arm sorting method based on vision and force feedback as described in claim 4, characterized in that, When the controller invokes the inverse kinematics algorithm, it also includes processing the solved desired angle. Amplitude limiting is performed: if the calculated desired angle is less than zero, it is set to zero, and the desired angle is converted into the number of pulse steps of the stepper motor based on the linear relationship between the desired angle and the number of pulses.

6. The Delta robotic arm sorting method based on vision and force feedback as described in claim 1, characterized in that, Before step S4, there is also a step: visual recognition and ultrasonic ranging noise elimination under multi-point sampling and filtering processing of the received spatial target coordinates by the controller.

7. The Delta robotic arm sorting method based on vision and force feedback as described in claim 1, characterized in that, In step S5, when the stepper motor rotates, a trapezoidal speed curve or a triangular speed curve is used for trajectory planning, and the pulse frequency is adjusted in the acceleration segment, the constant speed segment and the deceleration segment respectively.

8. The Delta robotic arm sorting method based on vision and force feedback as described in claim 7, characterized in that, The trajectory planning specifically refers to: The movement distance of each active arm from its current position to the target position is calculated. Based on the preset maximum speed and acceleration, a trapezoidal velocity curve is used to generate the position increment within each control cycle, and the pulse output of each active arm is updated synchronously so that each active arm reaches the target position synchronously.

9. The Delta robotic arm sorting method based on vision and force feedback as described in claim 1, characterized in that, In step S6, the preset threshold is set by adjusting the potentiometer connected to the pressure film sensor, and the controller uses a comparator circuit to monitor the pressure signal in real time. When the pressure value reaches the preset threshold, the drive signal output of the servo motor is stopped.

Images