Consecutive fruit picking control system and method based on Simulink and ROS

Abstract

The invention discloses a consecutive fruit picking control system and method based on Simulink and ROS. The system comprises forward and inverse kinematics models of a mechanical arm based on an MATLAB / Simulink software platform, and forward and inverse solution control system modules of the mechanical arm are established; a network connection and communication mechanism of a Linux operating system and an ROS system is established, data is issued to Simulink according to an ROS message format, and a fruit detection visual system module is constructed; opening and closing of fingers can be controlled according to the state of a picking hand controller, two message formats under an ROS are established, and construction of a picking hand system control module is completed. And according to target positioning and forward and inverse solution models, planning a motion track of the mechanical arm, controlling the tail end of the mechanical arm to operate, and completing the whole process from an approaching point to a target fruit, and picking and collecting the fruit by the mechanical arm. A visual identification system, a picking hand grabbing system and a mechanical arm motion control system are integrated in the whole system, and the problems that subsystems of a traditional picking robot are difficult to integrate, and the development period is long are solved.

Description

Technical Field

[0001] The present invention relates to the field of agricultural robot control, and in particular to a fruit continuous picking control system and method based on Simulink and ROS. Background Art

[0002] With the continuous development of agricultural automation, fruit-picking robots are gradually becoming an important tool to replace manual picking. However, existing fruit-picking robots still have many shortcomings in terms of control accuracy, operating efficiency, and adaptability to complex environments. Especially in orchard environments, faced with the complexity of fruit distribution and changing operating conditions, traditional robot control systems face significant limitations. Currently, many picking robots still use manual programming and debugging, which is time-consuming, labor-intensive, and prone to program errors, making it difficult to achieve multi-arm collaborative operation and precise control. In addition, existing robot control systems often lack effective path planning and obstacle avoidance strategies, resulting in the robot arms being prone to collisions during movement, affecting work quality and picking efficiency. Summary of the Invention

[0003] This paper addresses the challenges of existing fruit-picking robots in complex orchard environments, including insufficient control precision, difficult integration, and low picking efficiency. By developing a fruit-picking control system and method based on Simulink and ROS, this system integrates the modular modeling and simulation advantages of Simulink with the real-time communication and control capabilities of ROS, creating an efficient and intelligent picking control system. Through innovative control strategies and modular design, the robot's adaptability and operating efficiency in complex environments are effectively improved.

[0004] The technical solutions of the present invention are as follows: A fruit continuous picking control system based on Simulink and ROS, including: The robotic arm control module establishes the forward and inverse kinematic models of the robotic arm based on the MATLAB / Simulink software platform. The forward and inverse kinematic models of the control system are used to plan the motion trajectory of the robotic arm. This is used to control the end of the robotic arm to accurately execute the approach, grasping, picking, and collecting operations of the target fruit. The visual recognition module, installed in the Linux operating system, is used to obtain the three-dimensional coordinates and posture information of the fruit in real time through the ROS system, publish it to Simulink according to the ROS message format, and provide the fruit position data required by the robotic arm control module; The picking hand control module controls the opening and closing of the picking hand through the high and low level states of the picking hand controller I / O port. It adjusts the picking hand's operating posture in real time based on the identified fruit position and the robot arm's motion trajectory, so that the fruit can be accurately grasped and safely released. The communication control module is used to establish a network connection and communication mechanism between the Linux operating system and the ROS system, transmit the fruit position data obtained by the visual recognition module to the robotic arm control module, and synchronize the motion instructions of the robotic arm control module to the picking hand control module to enable data interaction and collaborative control between the modules; The integrated control module designs the picking control strategy based on Simulink, integrates the visual recognition module, the robotic arm control module, and the picking hand control module, generates C++ code for the picking control strategy, and directly deploys it to the ROS system to complete the picking work.

[0005] The robotic arm control module: Specifically, it is used to establish the forward and inverse kinematic models of the robotic arm through the DH parameter method, and adopt the principle of minimum state difference for trajectory planning. The principle of minimum state difference compares the difference between the current state of the robotic arm and the target state, and selects the motion path with the least change, so that the end of the robotic arm can accurately locate and grasp the fruit.

[0006] A fruit continuous picking control method based on Simulink and ROS includes the following steps: 1) The robotic arm moves to its initial position: The robotic arm first moves to the set initial position, which is located on the side of the fruit collection box to avoid blocking the camera and improve overall operation efficiency; 2) Set the approach point: When grabbing the target fruit, the robot arm first moves to the set approach point, which is an intermediate position to prevent the robot arm from colliding with other fruits or branches when approaching the target fruit; 3) Path planning and grasping: Calculate the robot arm's motion path using forward kinematics and inverse kinematics models to ensure the end of the robot arm moves smoothly from the approach point to the target fruit location; 4) Execute the grasping action: The picking hand adjusts the grasping posture (vertical or horizontal) at the approach point, closes the fingers and performs a rotating horizontal pulling picking action to ensure the fruit is safely picked; 5) Fruit placement: After picking, the robotic arm moves the fruit to the fruit collection box and places it safely, then returns to its initial position to prepare for the next picking operation. 6) Repeat steps 2 to 5: Continue the picking process until all fruits within the specified range are picked.

[0007] The beneficial technical effects of the present invention are: 1. The combined design of Simulink and ROS simplifies the control system development process and enables high-precision operation of the picking robot in complex orchard environments.

[0008] 2. The modular design facilitates the independent optimization and integration of each function, enhances the scalability and adaptability of the system, and can quickly adapt to the picking needs of different types of orchards.

[0009] 3. Using a customized communication mechanism to achieve real-time data exchange, the efficient collaboration between modules significantly improves the overall performance and operating efficiency of the system, enabling the picking robot to maintain a stable and efficient operating state in a changing environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a structural block diagram of a fruit continuous picking robot motion control system based on Simulink and ROS provided by an embodiment of the present invention.

[0011] Figure 2 It is a structural block diagram of a fruit continuous picking robot motion control system based on Simulink and ROS provided by another embodiment of the present invention.

[0012] Figure 3 The figure is a schematic diagram of the workflow of a fruit continuous picking robot motion control system based on Simulink and ROS provided by one embodiment of the present invention.

[0013] Figure 4 It is a schematic diagram of a forward kinematics Simulink model provided by an embodiment of the present invention.

[0014] Figure 5 It is a schematic diagram of a Simulink model of inverse kinematics based on minimum state difference provided by an embodiment of the present invention.

[0015] Figure 6 It is a schematic diagram of the communication modules of Simulink and ROS of the visual recognition and positioning system provided by one embodiment of the present invention.

[0016] Figure 7 It is a communication module between Simulink and ROS of a robot arm forward and inverse kinematic control system provided by an embodiment of the present invention.

[0017] Figure 8 It is a communication module between Simulink and ROS of the picking hand opening and closing control system provided by an embodiment of the present invention.

[0018] Figure 9 It is a schematic diagram of a fruit continuous picking control strategy model based on Simulink and ROS provided by an embodiment of the present invention.

[0019] Figure 10 This is a schematic diagram of an initial posture control model of a robotic arm provided by an embodiment of the present invention.

[0020] Figure 11 It is a schematic diagram of a picking hand opening control model provided by an embodiment of the present invention.

[0021] Figure 12 It is a schematic diagram of a visual recognition and positioning data acquisition model provided by an embodiment of the present invention.

[0022] Figure 13 It is a schematic diagram of an approach point control model for a robotic arm and a picking hand before grasping a fruit, provided by an embodiment of the present invention.

[0023] Figure 14 It is a schematic diagram of a control model for a robotic arm to move to a position to hold a fruit, provided by an embodiment of the present invention.

[0024] Figure 15 Schematic diagram of a picking hand closing control model provided by one embodiment of the present invention.

[0025] Figure 16 It is a schematic diagram of a twisting-pulling picking control model in which a robotic arm and a picking hand cooperate, provided by an embodiment of the present invention.

[0026] Figure 17 It is a schematic diagram of a control model for the movement of a robotic arm and a picking hand to a fruit collection box provided by an embodiment of the present invention.

[0027] Figure 18 It is a schematic diagram of a picking hand opening control model provided by an embodiment of the present invention.

[0028] Figure 19 It is a schematic diagram of a control model for a picking hand to return to a natural state provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are only some embodiments of the present invention, not all embodiments. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0030] The present invention provides a fruit continuous picking control system based on Simulink and ROS, comprising: The robotic arm control module builds the forward and inverse kinematic models of the robotic arm based on the MATLAB / Simulink software platform. The forward and inverse solutions of the control system module plan the robotic arm's motion trajectory, controlling the end-of-arm to precisely execute operations such as approaching, grasping, picking, and collecting the target fruit. This module leverages Simulink's graphical modeling and simulation capabilities to simplify complex kinematic calculations into drag-and-drop module operations. It also integrates with ROS for real-time data interaction, enabling precise positioning and motion control of the robotic arm in changing environments. Customized control strategies seamlessly integrate with ROS, enabling efficient command response and precise robotic arm operation.

[0031] The visual recognition module, installed in the Linux operating system, acquires the fruit's three-dimensional coordinates and posture information in real time through the ROS system. This information is then published to Simulink using the ROS message format, providing the fruit position data required by the robotic arm control module. This module integrates advanced image processing techniques and machine learning algorithms to accurately identify the position, size, and posture of target fruit in complex backgrounds and under various lighting conditions. Data processing is optimized using parallel computing, significantly improving the speed and accuracy of visual recognition, providing reliable target data for subsequent grasping operations.

[0032] The picking arm control module controls the opening and closing of the picking arm by monitoring the high and low levels of the picking arm's I / O ports. Based on the identified fruit position and the robotic arm's motion trajectory, the picking arm's operating posture is adjusted in real time to ensure precise grasping and safe release of the fruit. This module utilizes flexible grasping and force control strategies based on sensing and feedback from different fruit characteristics to ensure fruit is not damaged during the harvesting process. The picking arm's structural design incorporates bionic principles, adapting to various fruit shapes and improving grasping stability and reliability.

[0033] The communication control module is used to establish a network connection and communication mechanism between the Linux operating system and the ROS system, transmit the fruit position data obtained by the visual recognition module to the robotic arm control module, and synchronize the motion instructions of the robotic arm control module to the picking hand control module, so as to enable data interaction and collaborative control between modules; the Simulink simulation interface provides the function of real-time monitoring and adjustment of the communication process, effectively reducing data transmission delays and improving the overall response speed of the system.

[0034] The integrated control module designs the picking control strategy based on Simulink. It integrates the visual recognition module, the robotic arm control module, and the picking hand control module. The picking control strategy is generated into C++ code and deployed directly into the ROS system to complete the picking process. The communication control module is responsible for real-time communication between Simulink and the ROS system. Using the asynchronous, one-way ROS topic mechanism, modules exchange data using a custom message format. This module utilizes a custom communication protocol and data format to optimize transmission speed and data synchronization, ensuring efficient collaboration among modules in complex operating environments. Simulink's simulation interface provides real-time monitoring and adjustment of the communication process, effectively reducing data transmission latency and improving overall system responsiveness. The integrated control module designs and simulates the picking control strategy in Simulink, encapsulating the recognition, grasping, picking, and placement steps into independent modules. These steps are linked through logical logic to form a closed-loop control system. The integrated control module supports flexible switching between multiple picking strategies, such as planar grasping, rotational grasping, and bimanual coordination, to accommodate different fruit types and conditions. Through simulation testing and optimization adjustments, each control strategy can be verified before deployment to ensure safety and efficiency in actual operation.

[0035] Specifically, the robotic arm control module: Specifically, the system uses the DH parameter method to establish the forward and inverse kinematic models of the robotic arm. It then employs the principle of minimum state difference for trajectory planning. This principle compares the difference between the robotic arm's current state and its target state, selecting the path with the least variance to enable the end-of-arm to accurately locate and grasp the fruit. During path planning, the system dynamically adjusts the trajectory to avoid collisions and interference, taking into account the spatial distribution of the fruit, the robotic arm's kinematic limitations, and the impact of environmental obstacles.

[0036] The robot control module includes dedicated forward kinematics and inverse kinematics models packaged in Simulink: Among them, the forward kinematics model is modularly modeled based on Simulink. The position and posture of the end effector in space are calculated through the angles of each joint of the robotic arm, and it is packaged in Simulink in a graphical and modular way to set the initial posture of the robotic arm, providing an accurate initial state reference for motion path planning; the inverse kinematics model is built and packaged in Simulink. By reversely solving the change values of each joint angle and combining the target position to calculate the motion path of the end of the robotic arm, the simulation and verification functions of Simulink are used to control the precise motion control of the robotic arm from the current posture to the target posture, so as to optimize the path for grasping and collecting fruits.

[0037] The visual recognition module is specifically used to: The 3D spatial coordinates of the fruit are acquired through a depth camera, and the size, shape, and position of the fruit are detected in real time in combination with a fruit recognition algorithm. The detection results are published to Simulink using a custom message format improved based on geometry_msgs / Point, and provided to the robotic arm control module for path planning.

[0038] The communication control module is specifically used to: Adopting the asynchronous one-way ROS Topic communication mechanism, each subsystem publishes and subscribes through a custom message format, realizing data transmission and command synchronization between the visual recognition module, the robotic arm control module, and the picking hand control module; The communication control module uses the interface between Simulink and ROS to exchange data, and monitors and adjusts message transmission through a graphical interface to ensure real-time collaboration and efficient integration of various modules, enhancing the stability and response speed of the system.

[0039] The integrated control module is embedded in Simulink, and the picking control strategy is defined using SwitchCase Block logic. The fruit identification, grasping, picking and placement steps are encapsulated as independent modules, and the steps are connected in series through logical judgment conditions to achieve automated and coherent operations.

[0040] The integrated control module is also used to respond to feedback signals from the visual recognition module, the robotic arm control module, and the picking hand control module, optimize the control process through simulation testing, and generate C++ code to be deployed into the ROS system.

[0041] The harvesting control strategy includes a fully automated process for fruit recognition, robotic arm movement, hand gripping, and fruit placement. Leveraging the integration of Simulink and ROS, the system enables one-click deployment via the ROS Roslaunch tool, supporting automatic parameter adjustment and error detection. Fruit recognition specifically involves setting an approach point before identifying the target fruit, which guides the robotic arm to a safe intermediate position. Robotic arm movement uses an adaptive algorithm to adjust the arm's speed and hand gripping force to achieve non-destructive fruit handling. Simulink and ROS simulation tests are used to optimize parameter configuration for each step.

[0042] The present invention also discloses a fruit continuous picking control method based on Simulink and ROS, which includes the following steps: 1) The robotic arm moves to its initial position: The robotic arm first moves to the set initial position, which is located on the side of the fruit collection box to avoid blocking the camera and improve overall operation efficiency; 2) Set the approach point: When grabbing the target fruit, the robot arm first moves to the set approach point, which is an intermediate position to prevent the robot arm from colliding with other fruits or branches when approaching the target fruit; 3) Path planning and grasping: Calculate the robot arm's motion path using forward kinematics and inverse kinematics models to ensure the end of the robot arm moves smoothly from the approach point to the target fruit location; 4) Execute the grasping action: The picking hand adjusts the grasping posture (vertical or horizontal) at the approach point, closes the fingers and performs a rotating horizontal pulling picking action to ensure the fruit is safely picked; 5) Fruit placement: After picking, the robotic arm moves the fruit to the fruit collection box and places it safely, then returns to its initial position to prepare for the next picking operation. 6) Repeat steps 2 to 5: Continue the picking process until all fruits within the specified range are picked.

[0043] The present invention provides a fruit picking control method and system based on Simulink and ROS. The system is characterized by a unique logic control strategy design, which closely combines the simulation and modular modeling advantages of Simulink with the real-time communication and control functions of ROS. Simulink provides an intuitive graphical modeling environment, allowing control algorithms, path planning and robotic arm motion models to be quickly built and optimized in a modular manner; and through the logic control strategy designed by the present invention, these models and algorithms can be seamlessly integrated into the ROS system, realizing real-time data transmission and precise execution of control commands. This control method not only effectively integrates the dynamic simulation function of Simulink with the distributed computing power of ROS, but also can adjust and optimize the coordination between modules in real time during the picking process, significantly improving the response speed, control accuracy and adaptability to complex environments of the robot system. Through this innovative control strategy, the present invention breaks through the bottleneck of traditional picking robot technology and provides reliable technical support for realizing intelligent and stable automatic fruit picking.

[0044] like Figure 1 、 2 In the figure, an embodiment of the present invention provides a structural block diagram of a fruit continuous picking robot motion control system based on Simulink and ROS, and the fruit continuous picking robot motion control system based on Simulink and ROS includes a hardware structure 100 and a software system 200.

[0045] The hardware structure 100 includes an industrial computer 300, a driver board 400, a sensor 500 and an actuator 600. The driver board 400 includes a power module 410, a driver module 420 and a USB module 430. The software system 200 includes a Simulink simulation platform 210 and a ROS operating platform 220.

[0046] Figure 1 As shown, the industrial computer 300 is used to install the Simulink simulation platform 210 and the ROS operating platform 220, and serves as the core processor of the fruit continuous picking robot motion control system; the industrial computer 300 is connected to the driver board 400 through the USB module 430, and the driver board 400 includes a power module 410 and a driver module 420, wherein the driver module 420 is used to process and execute the control decisions transmitted by the industrial computer 300; the sensor 500 is connected to the industrial computer 300 to transmit the collected three-dimensional information of the fruit to the industrial computer 300; the driver module 420 drives the actuator 600 to operate according to the control instructions of the industrial computer 300; the power module 410 provides power support for other modules on the driver board 400.

[0047] The fruit continuous picking robot motion control system based on Simulink and ROS in the embodiment of the present invention can be divided into two levels: one is the workflow of the software system 200, including the Simulink simulation platform 210 and the ROS operating platform 220; the other is the motion control process of the hardware structure 100 with the industrial computer 300 and the drive module 420 as the core.

[0048] In software system 200, Simulink simulation platform 210 is responsible for the design and simulation of control strategies. It includes a robotic arm control module, a visual recognition module, a picking arm control module, a communication control module, and an integrated control module. ROS operating platform 220 is installed under a Linux operating system (such as Ubuntu) and exchanges data with Simulink simulation platform 210 via ROS communication mechanisms, enabling real-time transmission and execution of control strategies.

[0049] like Figure 2 As shown, actuator 600 includes a robotic arm 700, a picking hand 900, a motor 620, and an air pump 610. Drive module 420 drives motor 620 according to control instructions from industrial computer 300, controlling the movement and grasping of robotic arm 700 and picking hand 900. Air pump 610 provides air pressure support for picking hand 900, enabling precise grasping and release of fruit.

[0050] Further reference Figure 2 The hardware structure 100 also includes a depth camera 500, which obtains the three-dimensional coordinates and posture data of the fruit in real time through the ROS system, and transmits the data to the Simulink simulation platform 210 for path planning and control instruction generation.

[0051] The hardware structure 100 also includes an encoder 630, which is used to provide real-time feedback on the motion state of the motor 620 to ensure precise motion and position control of the robotic arm 700, thereby achieving precise operation during fruit grabbing and placement.

[0052] The hardware structure 100 further includes an input / output interface 450 , which is used to implement data interaction and debugging between modules, thereby ensuring the stability and efficient operation of the entire robot motion control system.

[0053] The hardware structure 100 further includes a power supply module 410 , which provides a stable power supply to all modules to ensure the reliable operation of each component.

[0054] Figure 3 This is a workflow diagram of a fruit continuous picking robot motion control system based on Simulink and ROS provided by an embodiment of the present invention. The method is applied to Figure 1 or Figure 2 The fruit picking robot motion control system based on Simulink and ROS is shown in the figure. This method realizes the automatic picking from "identification - grasping - picking - releasing fruit" through the coordinated work of the vision system, the robotic arm control system and the picking hand control system. Figure 3 The five stages in the code are described in detail: Preparation stage: The system initializes the robot system by starting the Roslaunch node to ensure smooth communication between modules. The industrial computer 300 is connected to the driver board 400 through the USB module 430, and the driver board 400 provides power support for all hardware in the system through the power module 410. The robotic arm 700 and the picking hand 900 act as actuators. Under the control of the driver board, they receive instructions according to the ROS topic message format and move to the preset initial posture to ensure that the robotic arm does not block the camera field of view of the visual system. At this time, the visual system begins to collect image data and identify the position of the target fruit. At the same time, the picking hand module receives instructions and begins to open its fingers to prepare for subsequent grasping. Optionally, in combination with reference Figure 2 The driver board 400 is connected to the industrial computer 300 through a USB interface to ensure that the robotic arm and the picking hand can accurately execute control instructions.

[0055] Target detection stage: The visual system collects environmental images in real time through the depth camera 500, extracts RGB image data, and detects the position of the target fruit through the image processing algorithm. The depth camera 500 is connected to the industrial computer 300 via USB, obtains the three-dimensional coordinate information of the target fruit and transmits it to the robotic arm control system. Through the target detection algorithm, the system detects the target fruit based on the RGB image data, and performs three-dimensional calibration based on the recognition results, converting the coordinate information of the target fruit into a coordinate format that the robotic arm can process, providing accurate position information for subsequent actions. Optionally, in combination with reference Figure 1 , the depth camera 500 is connected to the industrial computer 300 to ensure that the vision system provides real-time and accurate environmental perception data in a complex orchard environment.

[0056] Grasping stage: The robotic arm generates an approach point based on the three-dimensional coordinates of the target fruit, calculates the motion trajectory through the inverse solution model, and executes the motion to the approach point. The industrial computer 300 controls the movement of the robotic arm and the picking hand through the driver board 400 to ensure that the robotic arm avoids collisions with the fruit and surrounding obstacles when approaching the target fruit. The robotic arm adjusts the posture of the end effector at the approach point and ensures that it is aligned with the target fruit, ready to grasp. At this time, the picking hand executes finger closure according to the control instructions to ensure that the target fruit can be accurately grasped. Optionally, in combination with reference Figure 2 ,The optimization algorithm of the robot arm motion path adopts a dynamic ,path adjustment strategy to ensure that the grasping task can be ,performed smoothly even when encountering complex obstacles in the orchard environment.

[0057] Picking stage: When the end of the robotic arm reaches the target fruit, the picking hand performs a horizontal pulling picking mode to ensure that the fruit is accurately grasped. The end joint of the robotic arm performs a rotation action, the x and z coordinates remain unchanged, and the y coordinate is added with a fixed value to ensure that the end of the robotic arm can smoothly complete the picking action. The setting of the rotation angle and translation distance is adjusted according to the position of the target fruit and the position of the fruit collection box to ensure that the picked fruit can be smoothly placed in the fruit collection box. During the whole process, the industrial computer 300 coordinates the actions of the robotic arm and the picking hand through the driver board 400 to ensure the accuracy and stability of the picking process. Optionally, in combination with reference Figure 2 ,The control accuracy of the picking hand can be further optimized by adjusting the ,control module to ensure adaptability to different types of fruits.

[0058] Fruit placement phase: After completing the fruit grabbing, the end of the robotic arm will deliver the fruit to the fruit collection box. During the placement process, the picking hand performs the pressure release action according to the control instructions to ensure that the fruit can be safely released into the fruit collection box and avoid damage to the fruit during the placement process. After the fruit placement operation is completed, the end of the robotic arm returns to the initial position and prepares for the next round of picking operations, ensuring that the system can operate continuously and efficiently. Figure 1The motion path of the robotic arm is adjusted through the optimization algorithm at this stage, which further improves the accuracy and stability of fruit delivery. Especially when the position of the fruit collection box changes, the system can quickly adjust the motion path of the robotic arm.

[0059] Optional, combined with reference Figure 1 、 2 In hardware architecture 100, the system controls the various robot components through the interconnection of an industrial computer 300, a driver board 400, sensors 500, and actuators 600. The industrial computer 300, serving as the system's core control unit, is responsible for exchanging data with each module. The driver board 400 connects to the industrial computer 300 via a USB module 430 and provides power to all hardware in the system via a power module 410, ensuring proper operation of the hardware system. The execution module 440 is responsible for executing the robot's control commands and, through the driver module 420, drives the various joints of the robotic arm 700.

[0060] Optional, combined with reference Figure 2 The Simulink simulation platform 210 and ROS operating platform 220 in software system 200 coordinate the work of the robot arm control, visual recognition, and communication modules, respectively, to ensure the efficient operation of the robotic system. The ROS platform 220 coordinates modules such as Node 1 and Node 2 to perform data transmission and real-time control. The Simulink simulation platform 210 simulates the harvesting task, ensuring that the system has undergone rigorous testing and verification before hardware deployment, maximizing harvesting efficiency and accuracy.

[0061] Combined with reference Figure 4 , which shows a schematic diagram of the robot forward kinematics Simulink model.

[0062] Optional, combined with reference Figure 4 , which shows a schematic diagram of the robot forward kinematics Simulink model. Figure 4 This paper shows how to use Simulink model to perform forward kinematics calculation to determine the position and posture of the end effector of the robot arm. θ ) data, using the forward kinematics algorithm (ForwardSolver_MDH) to calculate and output the end-effector pose (Pos) of the robotic arm. This model outputs position (X, Y, and Z coordinates) and attitude (Roll, Pitch, and Yaw angles), providing precise position data for the end-effector. Using the Simulink platform, each joint angle of the robotic arm affects the position and attitude of the end effector, ensuring precise motion control according to the set targets.

[0063] Optional, combined with reference Figure 4 The forward kinematics model uses Matlab's simulation capabilities to simulate the actual position of the robotic arm at different joint angles, providing data support for subsequent control system optimization. Specifically, theta represents the angle of each joint, while Pos represents the calculated end-point pose, including its coordinates and attitude angle in three-dimensional space. This model can adjust joint angles and motion trajectories in real time, ensuring the accuracy and stability of the robotic arm, especially for effective control when performing tasks in complex environments.

[0064] Combined with reference Figure 5 , which shows a schematic diagram of a Simulink model based on inverse kinematics with minimum state difference.

[0065] Optional, combined with reference Figure 5 , which shows a schematic diagram of a Simulink model based on inverse kinematics with minimum state difference. Figure 5 Demonstrates how to use inverse kinematics to calculate the angles of each joint of the robotic arm ( θ ), thereby achieving precise control of the target position (Pos). The model inputs the position of the robot arm's end effector (such as X, Y, Z coordinates and posture angle) as target data and uses inverse kinematics to calculate the corresponding joint angles to achieve the desired end position.

[0066] The model uses the InverseSolver_MDH module to handle inverse kinematics calculations. AllSolverTheta further optimizes the solution process, ensuring the robot can find the optimal path during motion, minimize joint angle variance, and ensure precise positioning. To further optimize the solution process, the round module is used to round the calculated results to ensure that the joint angle values meet the actual motion constraints of the robot.

[0067] Optional, combined with reference Figure 5 To select the optimal solution, the system uses the "minimum state difference" principle among multiple inverse kinematics solutions. This principle calculates the state difference between each solution (i.e., the difference between the current robot arm state and the state of the upcoming picking action) and selects the solution with the smallest difference. In practice, given a given pose, the inverse kinematics model calculates eight possible solutions. The system then selects the one with the smallest change from the current robot arm state, avoiding excessive joint angle adjustments and unnecessary movement.

[0068] Through this method, the model calculates θ (joint angle) is transmitted to the control system, and the robot arm moves precisely according to this control signal. Figure 2,The output of the inverse kinematics model is controlled through the Simulink simulation platform of the ,software system 200, ensuring that the control instructions can be effectively ,delivered to the hardware through the ROS operating platform, ensuring the ,accuracy and stability of the robotic arm when performing complex tasks.

[0069] Combined with reference Figure 6 , which shows a schematic diagram of the communication modules of Simulink and ROS of the visual recognition and positioning system.

[0070] Optional, combined with reference Figure 6 This module demonstrates how to use ROS and Simulink systems to transmit and process visual data in real time. In this module, the visual system communicates with the Simulink system through the ROS topic / chatter to receive data from the visual sensor (such as the 3D coordinates of the target).

[0071] In ROS, the IsNew signal is used to check whether a new message (Msg) has been received. If so, the data (such as the target's X, Y, Z coordinates, and W value) is transmitted to the Simulink system. This data is written to the Vision Data Store through the message passing mechanism, ensuring that it can be read and used in Simulink in real time for subsequent target positioning and control.

[0072] Optionally, the communication module supports multi-channel data transmission and can handle different visual recognition tasks, such as real-time positioning, obstacle detection, etc., to ensure that the robot can operate accurately in complex environments. Through efficient communication between ROS and Simulink, visual data can be quickly fed back to the control system to achieve real-time dynamic control. Specifically, the vision system will use the geometry_msgs data type in ROS to pass target position information, geometry_msgs / Point is used to describe the target position (x, y, z), geometry_msgs / Quaternion is used to describe the quaternion direction of the target (x, y, z, w), and when necessary, the quaternion is converted into Euler angles (r, p, y) for use in the inverse kinematics model of the robotic arm. Through this mechanism, the output of the vision system can provide the necessary position information (x, y, z) and posture (r, p, y) for the robotic arm. Combined with reference Figure 7 , which shows a schematic diagram of the communication modules of Simulink and ROS for the forward and inverse kinematic control systems of the robotic arm.

[0073] Optional, combined with reference Figure 7This module demonstrates how to transmit data and control commands between ROS and a Simulink system. In this module, ROS publishes a goal message on the topic / rocr6_msgs / Goal, indicating the target position or motion instruction for the robot arm. This message data is passed to the Simulink system via the data bus and further assigned to control parameters for different robot arm joints (A1 to A6).

[0074] In Simulink, data is extracted from the Goal Message and read using the CMD Data StoreRead block. A custom format is then used to convert the data into a form suitable for robotic arm control (such as target joint angles). This data is passed to the control command (Cmd) via the Arm Bus Assignment block and ultimately sent to the robotic arm actuator via the ROS system's publishing mechanism ( / aubo5_msgs / goal).

[0075] Optional, combined with reference Figure 7 The robot arm's joint angle information is communicated using a custom message format. In this system, the ROS standard message library cannot fully meet the complex data transmission requirements, so a custom message format is required. First, create the Goal.msg file to define the robot arm's six joint angles (a1, a2, a3, a4, a5, and a6). These joint angles are in radians (float64 type). Second, define the Feedback.msg file to provide feedback on the status of each robot arm joint. The feedback format includes Joint shoulder_pan_joint, Joint shoulder_lift_joint, Jointtelbow_joint, Joint wrist_1_joint, Joint wrist_2_joint, and Joint wrist_3_joint, also in radians. This is used to monitor and adjust the robot arm's motion status in real time.

[0076] Combined with reference Figure 8 , which shows the communication module diagram of Simulink and ROS of the picking hand opening and closing control system.

[0077] Figure 8This demonstrates how to collaboratively control the opening and closing of a picking hand through Simulink and ROS. The picking hand controls the opening and closing of its fingers by controlling the high and low levels of the controller's I / O ports. When U_DO_00 is 0 and U_DO_01 is 1, the fingers close; when U_DO_00 is 1 and U_DO_01 is 0, the fingers open; when U_DO_00 is 0 and U_DO_01 is 1, the fingers close again; and when U_DO_00 is 1 and U_DO_01 is 1, the fingers release pressure.

[0078] In the Simulink model, this is controlled using the geometry_msgs / Quaternion message format. In this message format, orientation contains four float64 variables: x, y, z, and w. The value of w ranges from 0, 1, and 2, controlling the different states of the picking hand: w = 0 for closed fingers, w = 1 for open fingers, and w = 2 for released fingers.

[0079] This information is transmitted to the robotic arm control system via the ROS publishing mechanism / gripper. A data bus in the Simulink model distributes the state of the picking hand (X, Y, Z coordinates and W angle) to the control module, ensuring that the picking hand at the robotic arm's end effector performs the intended action.

[0080] Combined with reference Figure 9 , which shows a schematic diagram of the fruit continuous picking control strategy model based on Simulink and ROS.

[0081] Optional, combined with reference Figure 9 ,This model modularizes each key step in the picking process by using the SwitchCase Block system, ensuring that the tasks in each stage have clear execution logic. Specifically, Figure 9 The core is to control the order and execution of picking tasks through various "case" blocks.

[0082] Figure 9 The SwitchCase Block structure in Figure 1 triggers different "cases" based on different control conditions. Each "case" block represents a stage of the picking operation. By passing signals, the system is able to complete all operations from step 1 (initialization) to step 9 (fruit placement). These steps are executed in the following order: 1) Initialization (case 0): First, the system enters the initial position to ensure that the robotic arm is in the correct starting position.

[0083] 2) Open fingers (case 1): After the initial pose is completed, the system instructs the picking hand to open its fingers in preparation for grasping.

[0084] 3) Visual data acquisition (case 2): The visual system begins to perform target recognition and obtain the three-dimensional coordinate information of the fruit.

[0085] 4) Approaching the target (case 3): The robotic arm approaches the target fruit based on the feedback from the vision system and prepares to grasp it.

[0086] 5) Grasping the fruit target (case 4): The robotic arm moves to the fruit and grasps the target, ensuring that the fingers are closed.

[0087] 6) Picking mode (case 6): The picking hand performs the horizontal pulling mode to ensure the safe picking of the fruit.

[0088] 7) Move to the fruit collection box (case 7): The picked fruits are transported to the fruit collection box.

[0089] 8) Fruit placement operation (case 8): The robotic arm places the fruit into the fruit collection box and releases its fingers.

[0090] 9) Pressure release operation (case 9): After the fruit is released, the robotic arm performs the pressure release operation to ensure that the fruit falls smoothly into the fruit collection box.

[0091] Optional, combined with reference Figure 3 ,The implementation of this control strategy is completed through the close ,collaboration between ROS and Simulink. Figure 3 The five stages demonstrated (preparation, object detection, grasping, picking, and fruit placement) utilize a message-passing mechanism to coordinate modules, ensuring efficient communication and data synchronization. Each "case" block connects to an external control system via a ROS topic. Every movement and state change of the robotic arm and picking hand is transmitted to the control system in real time, ensuring that the system automatically adjusts to the current state. The steps of each stage are executed sequentially to ensure a precise and consistent picking process, starting from the preparation stage and continuing through object detection, grasping, picking, and fruit placement, ultimately completing the entire operation.

[0092] Optional, combined with reference Figure 9 , which shows a fruit coherent picking control strategy model based on Simulink and ROS. Figure 9 In the system, the picking process is broken down into multiple independent and sequential steps. Each step proceeds to the next only after the previous one is fully executed, ensuring the logic and continuity of the control process.

[0093] Each "case" block represents an independent operation step, covering the entire process from initializing the robot arm's position to placing the fruit. These steps include key links such as target positioning, robot arm movement, and picking hand movements.

[0094] Optional, Figure 9 The SwitchCase Block control strategy design in has the following characteristics: 1) Step modularization: Each step is encapsulated as an independent module, and the modules are linked through data transmission to ensure that the system is executed smoothly according to the predetermined process.

[0095] 2) Real-time monitoring and debugging: By integrating Simulink with ROS, the execution status of each step can be monitored in real time, ensuring accurate execution of operations. If a problem occurs at any stage, the problematic module can be quickly located and corrected.

[0096] 3) Graphical design: The control strategy is designed through a graphical interface, which facilitates development, debugging, and subsequent system expansion and secondary development.

[0097] Combined with reference Figure 10 , which shows a schematic diagram of the initial posture control model of the robotic arm.

[0098] Optional, combined with reference Figure 10 The system first sends the target rotation angles of the six joints to the robot arm through the Goal.msg message format. These angle values are combined with Figure 4 The robot arm adjusts the angle of each joint according to these target angles to achieve the initial posture.

[0099] The symbol 0 in the figure is written to the global variable rad1, representing the starting state of the forward kinematics model. The symbol [A] represents the Goto-From module, which is used to transfer signals between data points within the same subsystem, ensuring accurate signal transmission and effective coordination between modules.

[0100] Optionally, the control model workflow starts in Case [0], and the joint angles of the robot arm are updated according to the target rotation angle. By continuously calculating the angle difference of each joint of the robot arm, when the absolute difference between the angles of all joints and the target angle is less than 0.001 radians, the Case [0] condition is terminated and the system enters Case [1], which starts to execute the next stage of action.

[0101] In Case [1], the system calls the feedback information (Feedback.msg) of the six joints in ROS and performs absolute difference calculations with the target angle in real time. Only when the judgment conditions are met will the next operation be entered. Through this control strategy, the robot arm can accurately adjust the joint angles based on the feedback information to ensure that its posture meets the target requirements.

[0102] Combined with reference Figure 11 , which shows a schematic diagram of the picking hand opening control model.

[0103] Optional, combined with reference Figure 11 The control logic is located in the Case [1] module, which is used to perform the opening operation of the picking hand. The system first encodes the finger opening instruction into a quaternion [0 0 0 2] according to the geometry_msgs / Quaternion message format and writes it into the hand global variable. This variable serves as an intermediate communication medium and can be used by Figure 8 The communication module shown is used to read the data and publish it to the corresponding picking hand control interface through ROS.

[0104] The system then reads the state value in the hand variable and performs a logical judgment on the w element: If the current value meets the set conditions (for example, w ≥ 2), the picking hand opening action is determined to be complete. If w < 3, the system converts the value into a numerical signal and compares it with the threshold to determine whether the opening action meets the judgment criteria for entering the next control state. If the opening state is judged to be valid, the signal is output to the global variable A through the logic judgment module, indicating that the current opening operation is complete and the next control step can be entered.

[0105] Optional, combined with reference Figure 2 The system's hardware structure ensures precise control and stable operation of the picking hand through the coordinated operation of the industrial computer 300, driver board 400, sensor 500, and actuator 600. The industrial computer 300 receives control signals and transmits instructions to the picking hand control module through the driver board 400, ensuring accurate execution of the finger opening movement.

[0106] Combined with reference Figure 12 , which shows a schematic diagram of the visual recognition and positioning data acquisition model.

[0107] Optional, combined with reference Figure 12 , which shows the Simulink implementation structure of the visual recognition and positioning data acquisition model. This model is located in the Case [2] module and is used to receive, judge and filter data from the visual system to control whether to enter the subsequent target detection and pose calculation steps.

[0108] In this module, the system first reads the global variable data written by the vision system through the Vision Data Store Read node. This variable contains the three-dimensional coordinate information output by the vision recognition module. Subsequently, the model performs logical judgments on the coordinate values (such as whether it is less than 1000, whether it is a positive number, etc.) and summarizes the signals that meet the conditions into the selector module. If the judgment logic is valid, the model will write the data value into the new global variable data2 (implemented by the Vision Data2 Store Write module) and generate a control signal to write to variable A at the same time, which will serve as the judgment basis for entering the target approach phase (Case [3]).

[0109] Optional, combined with reference Figure 2 The system ensures the accurate transmission and processing of visual system data through the interconnection and coordination of industrial computer 300, driver board 400, sensor 500 and actuator 600, and further guarantees the stability of target positioning and control. Figure 13 , which shows a schematic diagram of the approach point control model of the robotic arm and picking hand before grasping the fruit.

[0110] Optional, combined with reference Figure 13 ,This model corresponds to the Case [3] module, and is mainly used to ,process the control process of the end of the robot moving from the current ,position to the target approach point.

[0111] In this module, the system first reads the 3D coordinate data of the target fruit using the Vision Data2 Store Read node. The coordinate values are then validated using logical checks. The valid coordinate values are then superimposed with a set of preset offsets (e.g., [1.18, 1.22, 0.3]) to generate the 3D pose information of the approach point. This approach point data is processed by the Reshape module and written to the global variable rad, which serves as input for the subsequent inverse kinematics module.

[0112] The primary purpose of designing the approach point is to reserve a safe distance before executing the grasping action, allowing the robot arm's end effector ample room for posture adjustment as it approaches the target fruit. This design helps reduce the risk of structural interference in complex orchard scenarios, particularly collisions caused by blind spots or obstructions, thereby improving the feasibility of the grasping path and the stability of the system.

[0113] At the same time, the system obtains the current joint angle feedback by subscribing to the ROS topic / aubo5_msgs / feedback and compares the joint-by-joint error with the angle value obtained by inverse solution of the approach point. When the absolute difference of all joints is less than the set threshold (for example, 0.01), it is determined that the robot arm has accurately reached the approach point. The system writes the control variable A, triggering the entry into the Case [4] stage to perform the grasping action.

[0114] Combined with reference Figure 14 , which shows a schematic diagram of the control model for the robotic arm to move to the position of holding the fruit.

[0115] Optional, combined with reference Figure 14 ,This model corresponds to the Case [4] module, which is mainly used to control the end of the ,robotic arm to move further from the approach point to the precise grasping position of ,the target fruit.

[0116] The system uses the Vision Data2 Store Read node to read the 3D coordinate data of the target fruit and corrects the Z-axis coordinates to compensate for the spatial deviation between the end effector and the fruit. The correction value is set by empirical parameters. After the coordinate data passes the validity check, it is combined with the position correction vector to generate the target grasping pose information. This pose information is converted to a new format using the Reshape module and written to the global variable rad, which serves as the input for the inverse kinematics model.

[0117] Pose information is input into Figure 5 In the inverse kinematics model shown, the corresponding 6 joint angles are calculated and output to control the actions of each joint of the robot arm. At the same time, the system obtains the current joint angle status of the robot arm through ROS subscription / aubo5_msgs / feedback topic. The feedback angle is Figure 5 The target angles obtained by solving the inverse kinematics model shown are compared joint by joint, and the system sets a threshold to determine whether the error range meets the control requirements.

[0118] When the absolute difference of all joint angles is less than the set threshold (e.g., 0.01), the system will output a control signal to variable A, which serves as the logical basis for triggering the subsequent Case [5] picking action. This mechanism achieves precise positioning and state closed-loop confirmation before grasping, ensuring the reliability and consistency of the picking action.

[0119] Combined with reference Figure 15 , which shows a schematic diagram of the picking hand closing control model.

[0120] Optional, combined with reference Figure 15 ,This model corresponds to the Case [5] module, which is used to control the picking hand to complete the closing operation after the end of the ,robot arm has reached the fruit grabbing position.

[0121] The control logic of this module is Figure 11 The picking hand opening control model shown in the figure is the same, and also uses the geometry_msgs / Quaternion message format to write the control instructions into the global variable in the form of quaternion and Figure 8 The communication module shown is published to the picking hand 900. The system sets the variable w to the value corresponding to the closed state, and the control signal is transmitted from the industrial computer 300 to the driver module 420 via the USB module 430, thereby controlling the picking hand 900 to perform the finger closing action.

[0122] At the same time, the system subscribes to the ROS topic / aubo5_msgs / feedback to obtain the current position and feedback status of the robotic arm. It then calculates the difference between the Y-axis movement of the picking arm and the set target. When this difference is less than a set threshold (such as 0.01) and the feedback status meets the execution judgment requirements, the system writes a control signal to variable A, which serves as the trigger condition for the subsequent picking mode execution steps.

[0123] Optional, combined with reference Figure 2 The system ensures the precise execution of the picking hand's movements through the collaboration of the industrial computer 300, the driver board 400 and the USB module 430, while maintaining good collaboration with the robotic arm and the visual system. Figure 16 , which shows a schematic diagram of the twisting-pulling picking control model in which the robotic arm and the picking hand cooperate.

[0124] Optional, combined with reference Figure 16 ,This model corresponds to the Case [6] module, which is mainly used to control the end of the robotic arm to execute a specific picking mode after the picking hand has closed and gripped the fruit, thereby detaching and picking the fruit.

[0125] The system reads the end effector pose data (data5) stored in the previous stage from the global variable rad and triggers the current stage's action, triggered by variable A. This action controls the end effector to perform a certain amplitude perturbation in spatial coordinates, forming a combined lateral pulling and rotational motion. Specifically, the model adjusts the end effector pose in real time by setting the translation amount in the X and Y directions, achieving pulling path control for the end effector.

[0126] The robot arm joint control signals are generated by the industrial computer 300 and transmitted to the driver module 420 via the USB module 430. This signals drive the robot arm 700 in the actuator 600 to achieve end-of-line trajectory transformation. The model detects the completion of the disturbance (e.g., whether the translation offset has reached a set value of 0.1) and outputs a determination signal to variable A, which serves as the switching condition for the subsequent release phase.

[0127] Combined with reference Figure 17, which shows a schematic diagram of the control model of the robot arm and picking hand moving to the fruit collection box.

[0128] Optional, combined with reference Figure 17 ,This model corresponds to the Case [7] module, which is mainly used to control the end of the robot arm to move from the current working position to the specified position of the fruit collection box.

[0129] In this model, the system first reads the target pose data stored in the global variable rad (by Figure 5 The target position (collection position) is then written to the global variable collection_position and transmitted to the control system via ROS topic messaging to guide the end-of-arm motion.

[0130] The six joint angles of the robot arm (joint1 to joint6) are calculated separately, and an appropriate feedback control system is used to ensure that the end can accurately move to the designated position of the fruit collection box. Figure 5 The inverse kinematics model shown here comes into play again, calculating the joint angles required for the end-arm to reach the fruit bin based on the target position. During the inverse calculation, the target data (such as X, Y, and Z coordinates and Roll, Pitch, and Yaw angles) is passed as input to the inverse model, which solves for the appropriate joint angles to accurately dock the end-arm with the fruit bin.

[0131] By monitoring the difference between the target position and the actual position in real time, when the error is less than a set threshold (such as 0.1), the robot arm's control signal is written to A, indicating that the target position has been successfully reached. At this point, the control signal ensures that each joint of the robot arm reaches the target position, ensuring that it can successfully complete the docking of the fruit collection box.

[0132] Optional, combined with reference Figure 1 and Figure 2 In hardware architecture 100, the collaborative work of the industrial computer 300, driver board 400, USB module 430, and power module 410 ensures the stability and accuracy of the robotic arm control system. The industrial computer 300, serving as the system's core control unit, communicates with the robotic arm through the driver board 400. The power module 410 provides the necessary power for the entire system, ensuring the proper operation of the hardware. Furthermore, the sensors 500 and actuators 600 work closely with the rest of the system to ensure the robotic arm can accurately perform its tasks.

[0133] Combined with reference Figure 18 , which shows a schematic diagram of the picking hand opening control model.

[0134] Optional, combined with reference Figure 18 , the model is similar to Figure 11 The control model shown is consistent, and both use the geometry_msgs / Quaternion message format to control the opening state of the picking hand. The specific control logic is the same as Figure 11 The description is the same. In the Case[1] module, the picking hand operates according to the set quaternion value. The signal is exchanged with the Simulink system through the ROS topic to control the finger opening state.

[0135] Combined with reference Figure 19 , which shows a schematic diagram of the control model for the picking hand to return to its natural state.

[0136] Optional, combined with reference Figure 19 ,This model is located in the Case [9] module and is used to perform the operation of ,recovering the picking hand to its natural state, marking the completion of a ,picking cycle.

[0137] The system first encodes the instruction of restoring the picking hand to its natural state into a quaternion [0 0 0 3] according to the geometry_msgs / Quaternion message format and writes it into the hand global variable. This variable serves as an intermediate communication medium and can be used by Figure 8 The communication module shown reads the data and publishes it to the picking hand control interface via ROS. At this point, the system reads the state value in the hand variable and performs a logical judgment on the w element: if the w value is greater than or equal to 3, it means that the picking hand has completed the relaxation action.

[0138] Once the relaxation action is completed, the system will update the value of variable A through a logical judgment signal, indicating that the picking cycle has ended, and trigger the re-entry into Case [1] to start a new picking cycle. At this time, the picking hand returns to its initial open state, and the robotic arm is ready to perform the next round of picking tasks.

[0139] A harvesting process control system model was built using the MATLAB / Simulink toolbox. After simulation testing, the control algorithm model automatically generated C++ code for implementation in the project. This process not only shortened the project development cycle and improved development efficiency, but also effectively avoided the repeated debugging required to successfully implement traditional handwritten code. Through the tight integration of Simulink and the ROS system, once the simulation model was verified, the control algorithm could be quickly converted into deployable C++ code for direct application to the robot hardware, further improving system development efficiency and stability.

[0140] It is obvious to those skilled in the art that the above specific examples are only preferred embodiments of the present invention. Therefore, any improvements and changes that may be made by those skilled in the art to certain parts of the present invention still reflect the principles of the present invention and achieve the purpose of the present invention, and all fall within the scope of protection of the present invention.

Claims

1. A fruit continuous picking control system based on Simulink and ROS, characterized in that: include: The robotic arm control module establishes the forward and inverse kinematic models of the robotic arm based on the MATLAB / Simulink software platform. The forward and inverse kinematic models of the control system are used to plan the motion trajectory of the robotic arm. This is used to control the end of the robotic arm to accurately execute the approach, grasping, picking, and collecting operations of the target fruit. The visual recognition module, installed in the Linux operating system, is used to obtain the three-dimensional coordinates and posture information of the fruit in real time through the ROS system, publish it to Simulink according to the ROS message format, and provide the fruit position data required by the robotic arm control module; The picking hand control module controls the opening and closing of the picking hand through the high and low level states of the picking hand controller I / O port. It adjusts the picking hand's operating posture in real time based on the identified fruit position and the robot arm's motion trajectory, so that the fruit can be accurately grasped and safely released. The communication control module is used to establish a network connection and communication mechanism between the Linux operating system and the ROS system, transmit the fruit position data obtained by the visual recognition module to the robotic arm control module, and synchronize the motion instructions of the robotic arm control module to the picking hand control module to enable data interaction and collaborative control between the modules; The integrated control module designs the picking control strategy based on Simulink, integrates the visual recognition module, the robotic arm control module, and the picking hand control module, generates C++ code for the picking control strategy, and directly deploys it to the ROS system to complete the picking work.

2. The fruit continuous picking control system based on Simulink and ROS according to claim 1 is characterized in that: The robotic arm control module: Specifically, it is used to establish the forward and inverse kinematic models of the robotic arm through the DH parameter method, and adopt the principle of minimum state difference for trajectory planning. The principle of minimum state difference compares the difference between the current state of the robotic arm and the target state, and selects the motion path with the least change, so that the end of the robotic arm can accurately locate and grasp the fruit.

3. The fruit continuous picking control system based on Simulink and ROS according to claim 2 is characterized in that: The robot control module includes dedicated forward kinematics and inverse kinematics models packaged in Simulink: The forward kinematics model is modularly modeled based on Simulink. The position and posture of the end effector in space are calculated using the angles of each joint of the robot arm. This model is then graphically and modularly packaged in Simulink to set the initial posture of the robot arm, providing an accurate initial state reference for motion path planning. The inverse kinematics model was built and packaged in Simulink. By reversely solving the change values of each joint angle and combining it with the target position, the motion path of the end of the robotic arm was calculated. Through the simulation and verification functions of Simulink, the precise motion control of the robotic arm from the current posture to the target posture was controlled, thereby optimizing the path for grasping and collecting the fruit.

4. The fruit continuous picking control system based on Simulink and ROS according to claim 1, characterized in that: The visual recognition module is specifically used to: The 3D spatial coordinates of the fruit are acquired through a depth camera, and the size, shape, and position of the fruit are detected in real time in combination with a fruit recognition algorithm. The detection results are published to Simulink using a custom message format improved based on geometry_msgs / Point, and provided to the robotic arm control module for path planning.

5. The fruit continuous picking control system based on Simulink and ROS according to claim 1, characterized in that: The communication control module is specifically used to: Adopting the asynchronous one-way ROS Topic communication mechanism, each subsystem publishes and subscribes to messages in a customized format, thus achieving data transmission and command synchronization between the visual recognition module, the robotic arm control module, and the picking hand control module. The communication control module uses the interface between Simulink and ROS to exchange data, and monitors and adjusts message transmission through a graphical interface to ensure real-time collaboration and efficient integration of various modules, enhancing the stability and response speed of the system.

6. The fruit continuous picking control system based on Simulink and ROS according to claim 1, characterized in that: The integrated control module is embedded in Simulink, and the picking control strategy is defined using SwitchCase Block logic. The fruit identification, grasping, picking and placement steps are encapsulated as independent modules, and the steps are connected in series through logical judgment conditions to achieve automated and coherent operations.

7. The fruit continuous picking control system based on Simulink and ROS according to claim 6, characterized in that: The integrated control module is also used to respond to feedback signals from the visual recognition module, the robotic arm control module, and the picking hand control module, optimize the control process through simulation testing, and generate C++ code to be deployed into the ROS system.

8. The fruit continuous picking control system based on Simulink and ROS according to claim 6, characterized in that: The picking control strategy includes a fully automated control process for fruit recognition, robotic arm movement, picking hand grasping, and fruit placement. This strategy leverages the integration of Simulink and ROS, enabling one-click deployment through the ROS Roslaunch tool, supporting automatic parameter adjustment and error detection.

9. The fruit continuous picking control system based on Simulink and ROS according to claim 6, characterized in that: The fruit recognition method specifically includes: setting an approach point before identifying the target fruit, and the approach point is used to guide the robotic arm to enter a safe intermediate position first; the robotic arm movement is to adjust the movement speed of the robotic arm and the grasping force of the picking hand through an adaptive algorithm to achieve non-destructive processing of the fruit, and use Simulink and ROS simulation tests to optimize the parameter configuration of each link.

10. A fruit continuous picking control method based on Simulink and ROS, characterized in that: The following steps are involved: 1) The robotic arm moves to its initial position: The robotic arm first moves to the set initial position, which is located on the side of the fruit collection box to avoid blocking the camera and improve overall operation efficiency; 2) Set the approach point: When grabbing the target fruit, the robot arm first moves to the set approach point, which is an intermediate position to prevent the robot arm from colliding with other fruits or branches when approaching the target fruit; 3) Path planning and grasping: Calculate the robot arm's motion path using forward kinematics and inverse kinematics models to ensure the end of the robot arm moves smoothly from the approach point to the target fruit location; 4) Execute the grasping action: The picking hand adjusts the grasping posture at the approach point, closes the fingers and performs a rotating horizontal pulling picking action to ensure the fruit is picked safely; 5) Fruit placement: After picking, the robotic arm moves the fruit to the fruit collection box and places it safely, then returns to its initial position to prepare for the next picking operation. 6) Repeat steps 2 to 5: Continue the picking process until all fruits within the specified range are picked.

Images