An intelligent control system and method for a tea bud picking robot

Abstract

The present application discloses an intelligent control system and method for a tea bud picking robot. The scene image information is acquired through a visual analysis module, and the tea bud area to be picked is accurately identified based on a preset neural network model. The tea bud area is divided by a task decomposition module, and each task in a large number of picking tasks is actually assigned to the corresponding robotic arm, and the shortest safe working route of the robotic arm is formulated. The motion control module ensures that the picking range of each robotic arm of the robot can completely cover the area where the tea buds suitable for picking are located, effectively reducing the missed picking rate of tea buds. The present application can coordinate multiple robotic arms of the picking robot to carry out the tea bud picking work, making the mechanized picking have the characteristics of controllable cost, accurate collection, and excellent efficiency. The data calculation and control of the entire system are completely carried out locally, which can ensure the fluency and efficiency of the system, and is conducive to the popularization of the tea picking robot technology.

Description

Technical Field

[0001] This application relates to the technical field of tea-picking robot control, and particularly to an intelligent control system and method for a tea shoot picking robot. Background Art

[0002] The tea industry belongs to a labor-intensive industry. Among them, the picking and pruning of fresh leaves are highly seasonal, with the largest, most concentrated, and highest labor costs. Manual tea picking and pruning account for more than 60% of the tea production cost. As the area of tea gardens continues to increase, the labor gap required for tea production will gradually widen. Relying solely on traditional manual tea picking can no longer meet the market demand. The mechanization of tea picking is the future development trend of tea production. Solving the problem of "difficult tea picking" through mechanization and intelligent picking is of great significance to the development of the tea industry.

[0003] However, different from the picking of other crops, the automatic picking of tea shoots in the field still faces many challenges: the light changes violently in the field tea garden environment, it is difficult to segment images with backgrounds similar to the color of tea shoots, and the occlusion and overlap between leaves result in unsatisfactory recognition effects. In addition, the characteristics of different heights and high growth densities of tea shoots put forward requirements for the harvesting movement efficiency of the robotic arm. At present, most of the end picking actuators of picking robots, especially those for medium-height crops, use a single robotic arm, with slow picking speed and easy damage to tea shoots by the robotic arm, affecting the quality of tea.

[0004] Therefore, there is an urgent need for an intelligent control system and method for a tea shoot picking robot, which can improve the picking efficiency and accuracy of the robot for tea shoots and reduce the cost of tea picking. Summary of the Invention

[0005] This application provides an intelligent control system and method for a tea shoot picking robot to solve the technical problems in the prior art that the picking robot is controlled by a cloud system, overly dependent on the network environment, and the robot only uses a single robotic arm, resulting in slow picking speed and low efficiency.

[0006] This application provides an intelligent control system for a tea shoot picking robot, including a visual analysis module, a task decomposition module, and a motion control module connected in sequence;

[0007] The visual analysis module is used to obtain scene image information, identify the area of tea shoots to be picked in the scene image information based on a preset neural network model, establish a spatial coordinate system, and determine the relative position between the picking robot and the tea shoot area;

[0008] The task decomposition module is used to divide the tea bud area into several picking sub-areas by using a preset division method, determine multiple target picking areas corresponding to each robotic arm according to the spatial positions of the multiple picking sub-areas and the multiple robotic arms of the picking robot, and formulate a planned path based on the relative positions of each robotic arm and the multiple target picking areas.

[0009] The motion control module is used to determine the joint activity data of each robotic arm according to the planned path, and control each robotic arm to pick and suck the tea buds in the target picking area according to the joint activity data along the planned path.

[0010] Furthermore, the visual analysis module includes an image acquisition module, an identification module, and a coordinate establishment module.

[0011] The image acquisition module is used to obtain the scene image information of the current position of the picking robot through a visual sensor.

[0012] The identification module is used to identify the tea bud area to be picked in the scene image information based on the YOLO neural network model.

[0013] The coordinate establishment module is used to establish a spatial coordinate system, determine the spatial positions of the tea bud area and the multiple robotic arms of the picking robot, and establish the positional relationship between the tea bud area and the robotic arms of the picking robot and the picking robot body.

[0014] Furthermore, the task decomposition module includes a partitioning module, an allocation module, and a path planning module.

[0015] The partitioning module is used to divide the tea bud area into several picking sub-areas based on the K-Means algorithm, and determine the area picking task parameters of each picking sub-area.

[0016] The allocation module is used to determine the target picking interval corresponding to each robotic arm based on the auction algorithm according to the area picking task parameters and the characteristic parameters of each robotic arm.

[0017] The path planning module is used to plan the activity path information of each robotic arm according to the spatial position of each robotic arm and the target picking interval by using a preset fusion algorithm to obtain a picking planned path.

[0018] Furthermore, the preset fusion algorithm is constructed based on Q-learning and an optimized genetic algorithm, uses Q-learning to obtain a preliminary picking path, and uses the genetic algorithm to optimize the preliminary picking path to obtain a picking planned path.

[0019] Furthermore, the motion control module includes a model establishment module, a motion analysis module, and an instruction output module.

[0020] A model establishment module for establishing a robotic kinematic model of the picking robot;

[0021] A motion analysis module for performing forward and inverse position solutions on the picking robot according to the planned path and the robotic kinematic model to obtain the joint movement data of each robotic arm of the picking robot;

[0022] An instruction output module for generating control instructions according to the joint movement data of the robotic arm to control the robot to pick and suck the tender tea buds in the target picking area according to the planned path.

[0023] Furthermore, the motion model is a Delta parallel robot kinematic model.

[0024] Furthermore, the motion analysis module is also used to perform singularity analysis and workspace analysis on the parallel robot, and optimize the joint movement data so that the movement range of the robotic arm covers the target picking area.

[0025] Furthermore, the system is built based on the ROS platform.

[0026] On the other hand, the present application also provides an intelligent control method for a tender tea bud picking robot, which is implemented by using any one of the intelligent control systems for tender tea bud picking robots described in the above technical solutions, and includes:

[0027] Obtaining scene image information through a visual analysis module, identifying the area of the tender tea buds to be picked in the scene image information based on a preset neural network model, and establishing a spatial coordinate system to determine the relative position between the picking robot and the tender tea bud area;

[0028] Using a preset partitioning method by a task decomposition module to divide the tender tea bud area into several picking sub-areas, determining multiple target picking areas corresponding to each robotic arm according to the spatial positions of the multiple picking sub-areas and the multiple robotic arms of the picking robot, and formulating a planned path based on the relative positions of each robotic arm and the multiple target picking areas;

[0029] Determining the joint movement data of each robotic arm according to the planned path through a motion control module, and controlling each robotic arm to pick and suck the tender tea buds in the target picking area according to the joint movement data according to the planned path.

[0030] The beneficial effects of the present application are:

[0031] This application provides an intelligent control system for a tea sprout picking robot. The visual analysis module uses common visual sensors to obtain scene image information, and accurately identifies the area of tea sprouts to be picked and the relative position between the picking robot and the tea sprout area based on a preset neural network model. The task decomposition module divides the tea sprout area, actually assigns each task in a large number of picking tasks to the corresponding robotic arm, and formulates the shortest safe working route for the robotic arm. The motion control module ensures that the picking range of each robotic arm of the robot can completely cover the area where the tea sprouts suitable for picking are located, effectively reducing the missed picking rate of tea sprouts and improving the picking efficiency. This application coordinates a variety of relatively independent and complex technologies to jointly serve the same robot body, and operates multiple robotic arms for picking work, making the tea picking robot have the characteristics of controllable cost, accurate collection, and excellent efficiency. The data calculation and control of the entire system are completely carried out locally, which can ensure the fluency and efficiency of the system, and is more conducive to the commercial promotion of the tea picking robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 FIG. is a system architecture diagram of an embodiment of the intelligent control system for a tea sprout picking robot provided by this application;

[0033] Figure 2 FIG. is a structural schematic diagram of the visual analysis module running in an embodiment provided by this application;

[0034] Figure 3 FIG. is a structural schematic diagram of the task decomposition module running in an embodiment provided by this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0036] Before introducing the embodiments of this application, the relevant terms of this application will be explained first.

[0037] YOLO: The full name is You Only Look Once, which is a real-time object detection algorithm based on deep learning. The main idea of YOLO is to use the entire image as the input of the network and directly perform regression on the position and category of the bounding box at the output layer.

[0038] K-Means: The full name is K-Means, which is an unsupervised machine learning algorithm mainly used for data clustering. In K-Means, K represents the number of clusters, and Means represents the average value of the data.

[0039] Auction algorithm: Essentially, it simulates the process of human auction activities. Its algorithm process is like an auction where unallocated bidders simultaneously bid on the target item, thereby increasing the price of the target item. The auction house obtains the bid information of all bidders and sells the target item to the bidder with the highest bid. The final result should ensure the maximization of the interests of all bidders and the auction house as much as possible.

[0040] Robot kinematics: Robot kinematics is a discipline that studies the motion of robots in space, including mathematical models that describe and analyze the motion, speed, acceleration, and position of the robot's end effector (such as a robotic arm or a mobile platform). Kinematics mainly focuses on the geometric characteristics and motion laws of the robot, without involving the mechanics and dynamics of the robot.

[0041] Task partitioning and allocation: Task partitioning and allocation is a method for task allocation in a multi-agent system. In such a system, a task is divided into multiple parts (or "partitions"), and then these parts are assigned to different agents (such as robots) for execution.

[0042] ROS: The full name is Robot Operating System, which is a framework for writing robot software. It is a flexible framework for writing robot software. It is a set of operating system concepts - providing services such as hardware abstraction, low-level device control, implementation of common functions, inter-process message passing, and package management. It also provides a concept of a computational graph, which makes it a distributed computing environment that can run on different machines. The goal of ROS is to make it easy to create complex and powerful robots.

[0043] The inventive concept of this application is introduced below. The disadvantages of the existing automated picking of tea buds are as follows:

[0044] One is the poor visual recognition effect. The recognition of tea buds is greatly affected by light, it is difficult to segment images with backgrounds similar in color to the buds, and the occlusion and overlap between leaves will result in unsatisfactory recognition effects. Compared with traditional machine learning, the current bud and leaf recognition method based on deep learning has good application prospects, but it requires a large number of labeled samples for training, and as the complexity of the network increases, the upgrade of the hardware system is a problem that needs to be solved. In some methods, in order to improve the recognition accuracy of buds, it is necessary to equip multiple different types of sensors. Not only a camera needs to be configured, but even a lidar is required to build a 3D point cloud model, and the use cost is high, so it cannot be widely promoted and applied on a large scale;

[0045] The second is the low picking efficiency of the robotic arm. This is because most of the end picking actuators of existing picking robots use a single robotic arm, and the picking speed is slow. Only a few catties of tea can be picked in a day, far from reaching the efficiency of manual labor. In addition, most of the control systems are control systems based on the cloud computing platform. In this system, only the control components of the sensor and the robot kinematics are deployed on the robot, and the calculations such as recognition and acquisition completely rely on the cloud server connected by wireless network. However, the actual production environment cannot guarantee the stability of the network signal. Placing the key calculation process in the cloud may cause the machine to fall into an infinite loop or stop running, seriously hindering the work efficiency;

[0046] The third is the poor practicability. The existing picking robot control system based on visual recognition technology cannot well combine visual recognition, actual scene calculation, and robotic arm picking implementation control to build a complete system for actual production and cannot be actually put into application;

[0047] The fourth is the poor picking effect. Since the buds of famous and high-quality tea are soft in texture and are distributed randomly, the traditional mechanical picking method has low control accuracy and is likely to damage the famous and high-quality tea by the robotic arm, affecting the quality of the tea.

[0048] To address the above problems, this application improves the visual recognition part, uses a common visual sensor configured in the production operation to obtain image data, accurately identifies the area of the tea buds to be picked in the image data based on a preset neural network model, assigns corresponding picking sub-areas to each end picking actuator of the picking robot with multiple robotic arms, plans the picking paths of each robotic arm, and finally picks the tea buds according to the picking paths, greatly improving the picking efficiency and picking accuracy of the robot.

[0049] Combined with Figure 1 As shown, a specific embodiment of this application discloses an intelligent control system for a tea bud picking robot, including a visual analysis module 101, a task decomposition module 102, and a motion control module 103 that are connected in sequence;

[0050] The visual analysis module 101 is used to obtain scene image information, identify the tea bud regions to be picked in the scene image information based on a preset neural network model, establish a spatial coordinate system, and determine the relative position between the picking robot and the tea bud regions;

[0051] The task decomposition module 102 is used to divide the tea bud regions into several picking sub-regions by using a preset partitioning method, determine multiple target picking regions corresponding to each robotic arm according to the spatial positions of the multiple picking sub-regions and the multiple robotic arms of the picking robot, and formulate a planned path based on the relative positions of each robotic arm and the multiple target picking regions;

[0052] The motion control module 103 is used to determine the joint movement data of each robotic arm according to the planned path, and control each robotic arm to pick and suck the tea buds in the target picking regions according to the joint movement data.

[0053] Compared with the prior art, in this application, the visual analysis module uses a common visual sensor to obtain scene image information, accurately identifies the tea bud regions to be picked and the relative position between the picking robot and the tea bud regions based on a preset neural network model; the task decomposition module divides the tea bud regions, actually assigns each task in a large number of picking tasks to the corresponding robotic arm, and formulates the shortest safe working route for the robotic arm; the motion control module ensures that the picking range of each robotic arm of the robot can completely cover the regions where the tea buds suitable for picking are located, effectively reducing the missed picking rate of the tea buds and improving the picking efficiency. This application coordinates multiple relatively independent and complex technologies to jointly serve the same robot body and operates multiple robotic arms for picking work, making the tea picking robot have the characteristics of controllable cost, accurate collection, and excellent efficiency. The data calculation and control of the entire system are completely carried out locally, which can ensure the smoothness and efficiency of the system and is more conducive to the commercial promotion of the tea picking robot.

[0054] In a preferred embodiment, the system is built based on the ROS platform. The ROS platform is a flexible framework for developing robot software. This software provides a series of tools, libraries, and conventions that can help developers write robot application programs. Through ROS, robot application programs can be quickly built, and the resources and tools of the ROS community can be utilized to accelerate the development process. At the same time, ROS also provides convenience for cross-platform development of robots, enabling robots on different hardware platforms to share and reuse software components.

[0055] It should be noted that the ROS platform is a preferred control platform, and other types of control platforms can also be selected for implementation, such as platforms that provide real-time control and robot application development like Gazebo, V-REP, Webots, etc. In addition, control platforms integrated between different platforms are also applicable.

[0056] Combined with Figure 2 As shown, in a preferred embodiment, the visual analysis module 101 includes an image acquisition module 201, an identification module 202, and a coordinate establishment module 203;

[0057] The image acquisition module 201 is used to obtain the scene image information of the current position of the picking robot through a vision sensor;

[0058] The identification module 202 is used to identify the area of the tea bud to be picked in the scene image information based on the YOLO neural network model;

[0059] The coordinate establishment module 203 is used to establish a spatial coordinate system, determine the spatial positions of the tea bud area and multiple robotic arms of the picking robot, and establish the positional relationship between the tea bud area, the robotic arms of the picking robot, and the picking robot body.

[0060] In a specific embodiment, the image acquisition module is a commonly used ordinary camera, and the area of the tea bud in the image information is identified through a preset visual analysis network model, which can achieve accurate identification of the target to be picked without adding or upgrading hardware, reducing hardware expenditure and maintenance costs.

[0061] In a specific embodiment, using the YOLO neural network model for target detection can achieve real-time detection and rapid positioning of the tea bud area, improving the data processing speed of the system, and thus further enhancing production efficiency. For the recognition model of the tea bud area to be picked, in addition to the recognition network based on the YOLO neural network model, other visual modules with visual recognition capabilities can also be used, such as visual recognition models based on convolutional neural networks, etc.

[0062] In a preferred embodiment, as Figure 3 shown, the task decomposition module 102 includes a partitioning module 301, an allocation module 302, and a path planning module 303;

[0063] The partitioning module 301 is used to divide the tea bud area into several picking sub-areas based on the K-Means algorithm and determine the area picking task parameters for each picking sub-area;

[0064] An allocation module 302, configured to determine a target picking interval corresponding to each robotic arm based on an auction algorithm according to regional picking task parameters and the characteristic parameters of each robotic arm;

[0065] A path planning module 303, configured to plan the picking path of each robotic arm according to the spatial position of each robotic arm and the target picking interval by using a preset fusion algorithm, so as to obtain a picking planned path.

[0066] In a specific embodiment, the regional picking task parameters include the type, difficulty, working hour estimation, etc. of the task, and the characteristic parameters of the robotic arm include the working state, load capacity, spatial position, etc. of the robotic arm.

[0067] The specific method for each robotic arm to participate in the auction algorithm according to the regional picking task parameters is as follows: Each robotic arm expresses its interest in the picking task in the form of a bid according to its own conditions and preferences. Let the bid of the i th robotic arm j be bij . All robotic arms are sorted from high to low according to the bid values, and a bid ranking vector r = r 1, r 2, …, rN is obtained, where ri represents the number of the robotic arm ranked i th. According to the bid ranking, each robotic arm is sequentially asked whether it accepts the task. If a robotic arm does not accept the task, the next robotic arm is asked until a robotic arm accepts the task.

[0068] It should be noted that if multiple robotic arms accept the task simultaneously, the optimal robotic arm is selected according to the matching degree between the robotic arm and the task. The matching degree can be represented by a function f ( j , m ), where j and m represent the numbers of the robotic arm and the item respectively. The matching degree evaluation may involve calculating the distance from the robotic arm to the item, whether the load capacity meets the requirements of the item, etc. Specifically, a weighted average function g ( j , m ) can be defined as follows:

[0069] g ( j , m ) = αf ( j , m ) + (1 - α ) brjj

[0070] Among them, α is the attenuation coefficient, which is used to control the relative weights of the matching degree and the bidding price. Generally, it can be set to 0.5, that is, to balance the contributions of the bidding price and the matching degree; brjj represents the bidding value of the robotic arm j and is weighted according to the bidding ranking.

[0071] According to the weighted average function g ( j , m ), select the robotic arm corresponding to the maximum value to perform the picking task. In the optimal allocation decision, an exponentially weighted moving average can be used to comprehensively consider the bidding price and the matching degree.

[0072] In a preferred embodiment, the preset fusion algorithm is constructed based on Q-learning and an optimized genetic algorithm. The Q-learning is used to obtain a preliminary picking path, and the genetic algorithm is used to optimize the preliminary picking path to obtain a picking planning path.

[0073] Specifically, first use the Q-learning algorithm to train the robotic arm to select the best action in different states to obtain a preliminary active path plan. Then, use the optimized genetic algorithm to optimize and improve these paths to improve the quality and efficiency of the paths. By iteratively performing the alternating operations of Q-learning and the optimized genetic algorithm, gradually optimize the active path of the robotic arm until the optimal picking planning path is obtained. The main steps include:

[0074] Step S11: Define the problem space: Define the problem space of the robotic arm active path planning, including the joint angles of the robotic arm, the position and posture of the end effector, etc.;

[0075] Step S12: Initialize the Q-value table and the population: Initialize an initial Q value for each state-action pair, and at the same time randomly generate an initial population, where each individual represents a possible path planning scheme;

[0076] Step S13: The Q-learning stage, including the following sub-steps:

[0077] Step a1: According to the current state and the Q-value table, select an action to execute. The ε-greedy strategy can be used to randomly select an action with a certain probability to explore new possibilities; in other cases, select the action with the highest Q value;

[0078] Step b1: Execute the action and observe the reward: Execute the selected action and observe the feedback reward given by the environment, that is, calculate the fitness of the current path planning scheme;

[0079] Step c1: Update the Q value: Update the Q value of the previously selected action based on the observed reward. The update formula is as follows:

[0080] Q(s, a) = Q(s, a) + α * (r + γ * maxQ(s', a') - Q(s, a))

[0081] Where Q(s, a) is the Q value of performing action a in state s, r is the obtained reward, s' is the new state after performing the action, a' is the action selected in the new state, α is the learning rate, and γ is the discount factor;

[0082] Step d1: Iterative update, repeatedly execute steps a1 to c1 until convergence or reaching a predetermined number of iterations;

[0083] Step S14: Optimize the genetic algorithm stage, including the following sub-steps:

[0084] Step a2: Selection operation: According to the fitness of each individual, select a part of the better individuals from the current population using the selection operation;

[0085] Step b2: Crossover operation: Through the crossover operation, pair the selected individuals and generate new individuals;

[0086] Step c2: Mutation operation: Perform a mutation operation on the newly generated individuals to introduce a certain degree of randomness to increase the diversity of the population;

[0087] Step d2: Replacement operation: Replace some individuals in the original population with the newly generated individuals;

[0088] Step e2: Iterative update: Repeatedly execute steps a2 to d2 until reaching a predetermined number of iterations;

[0089] Step S15: Combine Q-learning and the optimized genetic algorithm: In each iteration of the genetic algorithm, use the Q-learning algorithm to guide the selection and fitness evaluation of individuals. Specifically, the best fitness value in each state can be stored in the Q-value table of Q-learning and used to calculate the fitness of individuals. After the iterations of Q-learning and the optimized genetic algorithm are completed, select the individual with the highest fitness as the final path planning solution.

[0090] In a preferred embodiment, the motion control module includes a model establishment module, a motion analysis module, and an instruction output module;

[0091] The model establishment module is used to establish the robot kinematic model of the picking robot;

[0092] A motion analysis module, which is used to perform forward and inverse kinematic solutions on the picking robot according to the planned path and the robot kinematic model, and obtain the joint movement data of each robotic arm of the picking robot;

[0093] An instruction output module, which is used to generate control instructions according to the joint movement data of the robotic arm, and control the robot to pick and suck the tea buds in the target picking area according to the planned path.

[0094] In a preferred embodiment, the robot kinematic model is a Delta parallel robot kinematic model. The Delta parallel robot is a special parallel robot system, and this model describes the relationship between the position and posture of the robot end effector and the joint angles of each active arm. In the work of picking tea buds, first define the coordinate system of the Delta robot and establish the relationship between each coordinate system. Through measurement and calibration, the position and posture information of each active arm end effector relative to its corresponding local coordinate system can be obtained. Determine the position and posture of the end effector according to the spatial coordinates of the target picking area, and then solve the joint angles of each active arm.

[0095] In a preferred embodiment, the motion analysis module is further used to perform singularity analysis and workspace analysis on the parallel robot, optimize the joint movement data, and make the movement range of the robotic arm cover the target picking area.

[0096] Specifically, for singularity analysis, the position of the singular point can be determined by calculating the Jacobian matrix and checking whether its determinant is zero. Workspace analysis can calculate the ability of the robotic arm to reach different positions and postures according to conditions such as the rotatable range of the robotic arm joints, the size and posture limitations of the end effector, etc., so as to determine the workspace of the robotic arm. Based on the joint movement model, use an optimization algorithm to find the optimal joint movement data, make the movement range of the robotic arm cover the target picking area, input the optimized joint movement data into the robotic arm, and verify whether its effect meets the requirements. If not, the optimization algorithm can be readjusted or the joint movement model can be modified for iterative optimization to ensure that the movement range of the robotic arm covers the target picking area.

[0097] Furthermore, when a single target picking is completed, it enters the next target picking process in the current area. When all the targets in the area currently photographed by the ceiling camera are picked, use the walking module to move to the next area and repeat the steps of image recognition, area division, and motion control until the picking operation of the tea buds is completed.

[0098] The system of this embodiment uses local visual recognition to establish a coordinate system and transmits the data to the local task allocation path planning module. It allocates the calculated task data to the end effectors of multiple robotic arms to pick tea leaves and drives the traveling system to move after the task is completed. Therefore, the data calculation and control of the entire system are completely carried out locally, which can ensure the fluency and efficiency of the system.

[0099] As a specific embodiment, the robotic arm is modeled through the URDF model (Unified Robot Description Format). URDF is a common robot modeling language. The URDF model defines the structure, linkage relationship, sensors, kinematic information, etc. of the robot and stores them in XML format. The URDF model is commonly used in applications such as robot simulation, path planning, and control design.

[0100] It should be noted that the URDF model is provided by the picking hand (robotic arm) manufacturer or developed independently, which describes the parameters such as the lengths of each link of the robotic arm, relative positions, sizes, etc., and is used for forward and inverse kinematic solutions of the robotic arm. The URDF model is solved through the movelt module of the ROS software to plan the picking trajectory of the robotic arm, inversely solve the joint angles of each motion link, and generate motion control instructions according to the joint angles. The hardware system performs corresponding picking and sucking operations according to the control instructions to complete the picking of tea buds. Through the above method, the robotic arm can be controlled to work with high precision and efficiency, enabling the picking robot to accurately pick tea buds, including picking under the influence of factors such as wind speed, light, and slope.

[0101] The intelligent control system of the tea bud picking robot provided in this embodiment uses the YOLO model to identify and locate tea leaves locally, calculates locally how to allocate the tea leaf collection task to each robotic arm in the optimal way, and then enables each robotic arm to pick and collect tea leaves with a reasonable path and correct posture. It has the advantages of high generalization, strong robustness, and fast response speed. The hardware sensor required for visual recognition is an ordinary camera, with low cost. By organically combining the processes of recognition, analysis, processing, and collection, a full-process production is formed, enabling the system to complete the picking process by one machine in the actual commercial production environment without the need for additional hardware support. This system can control multiple robotic arms for picking, and the picking efficiency in the actual production environment is significantly higher than that of humans, significantly improving the benefits of tea farmers.

[0102] The embodiment of this application also provides an intelligent control method for a tea bud picking robot, which is implemented by using any one of the intelligent control systems of the tea bud picking robot described in the above technical solutions. The method includes:

[0103] Obtain the scene image information through the visual analysis module, identify the tea bud area to be picked in the scene image information based on a preset neural network model, establish a spatial coordinate system, and determine the relative position between the picking robot and the tea bud area;

[0104] Through the task decomposition module, use a preset partitioning method to divide the tea bud area into several picking sub-areas. According to the spatial positions of the multiple picking sub-areas and the multiple robotic arms of the picking robot, determine the multiple target picking areas corresponding to each robotic arm, and formulate a planned path based on the relative positions of each robotic arm and the multiple target picking areas;

[0105] Through the motion control module, determine the joint activity data of each robotic arm according to the planned path, and control each robotic arm to pick and suck the tea buds in the target picking area according to the planned path.

[0106] It should be noted that the various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other.

[0107] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent control system for a tea bud picking robot, characterized in that, It includes a visual analysis module, a task decomposition module, and a motion control module connected in sequence; The visual analysis module is used to obtain scene image information, identify the area of the tea bud to be picked in the scene image information based on a preset neural network model, establish a spatial coordinate system, and determine the relative position between the picking robot and the tea bud area; The task decomposition module is used to divide the tea bud area into several picking sub-areas by using a preset division method, determine multiple target picking areas corresponding to each robotic arm according to the spatial positions of the multiple picking sub-areas and the multiple robotic arms of the picking robot, and formulate a planned path based on the relative positions of each robotic arm and the multiple target picking areas; The task decomposition module includes a zoning module, an allocation module, and a path planning module; The zoning module is used to divide the tea bud area into several picking sub-areas based on the K-Means algorithm and determine the area picking task parameters of each picking sub-area; The allocation module is used to determine the target picking interval corresponding to each robotic arm based on the auction algorithm according to the area picking task parameters and the characteristic parameters of each robotic arm; The specific method is as follows: Each robotic arm expresses its interest in the picking task in the form of a bid according to its own conditions and preferences; let the bid of the i-th robotic arm j be bij; all robotic arms are sorted from high to low according to the bid value to obtain the bid ranking vector r = [r1, r2,..., rN] of the robotic arms, where ri represents the number of the robotic arm ranked i-th. According to the bid ranking, each robotic arm is sequentially asked whether it accepts the task; if a robotic arm does not accept the task, the next robotic arm is asked until a robotic arm accepts the task; When multiple robotic arms accept the task at the same time, the optimal robotic arm is selected according to the matching degree between the robotic arm and the task, and the robotic arm corresponding to the maximum value is selected according to the value of the weighted average function g(j, m) to perform the picking task; among them, the weighted average function g(j, m) is expressed as: g(j, m) = αf(j, m) + (1 - α)brjj Where α is the attenuation coefficient used to control the relative weights of the matching degree and the bid; brjj represents the bid value of robotic arm j, which is weighted according to the bid ranking; The path planning module is used to plan the activity path information of each robotic arm according to the spatial position of each robotic arm and the target picking interval by using a preset fusion algorithm to obtain a picking planned path; the preset fusion algorithm is constructed based on Q-learning and an optimized genetic algorithm, uses Q-learning to obtain a preliminary picking path, and uses the genetic algorithm to optimize the preliminary picking path to obtain a picking planned path; specifically includes: Step S11: Define the problem space: Define the problem space of the robotic arm activity path planning, including the joint angles of the robotic arm, the position and attitude of the end effector; Step S12: Initialize the Q-value table and the population: Initialize an initial Q value for each state-action pair, and at the same time randomly generate an initial population, and each individual represents a possible path planning scheme; Step S13: Alternately operate Q-learning and the optimized genetic algorithm through iteration, including: in each iteration of the genetic algorithm, use the Q-learning algorithm to guide the selection and fitness evaluation of individuals, and optimize the movement path of the robotic arm until the optimal picking planning path is obtained; The motion control module is used to determine the joint motion data of each robotic arm according to the planned path, and control each robotic arm to pick and suck the tea buds in the target picking area according to the joint motion data.

2. The intelligent control system of the tea bud picking robot according to claim 1, characterized in that, The visual analysis module includes an image acquisition module, an identification module, and a coordinate establishment module; The image acquisition module is used to obtain the scene image information of the current position of the picking robot through a visual sensor; The identification module is used to identify the area of the tea buds to be picked in the scene image information based on the YOLO neural network model; The coordinate establishment module is used to establish a spatial coordinate system, determine the spatial positions of the tea bud area and the multiple robotic arms of the picking robot, and establish the positional relationship between the tea bud area, the robotic arms of the picking robot, and the picking robot body.

3. The intelligent control system of the tea bud picking robot according to claim 1, characterized in that, The motion control module includes a model establishment module, a motion analysis module, and an instruction output module; The model establishment module is used to establish the robot kinematics model of the picking robot; The motion analysis module is used to perform forward and inverse position solutions on the picking robot according to the planned path and the robot kinematics model to obtain the joint motion data of each robotic arm of the picking robot; The instruction output module is used to generate control instructions according to the joint motion data of the robotic arm, and control the robot to pick and suck the tea buds in the target picking area according to the planned path.

4. The intelligent control system of the tea bud picking robot according to claim 3, characterized in that, The robot kinematics model is a Delta parallel robot kinematics model.

5. The intelligent control system of the tea bud picking robot according to claim 4, characterized in that, The motion analysis module is also used to perform singularity analysis and workspace analysis on the parallel robot, and optimize the joint motion data to make the movement range of the robotic arm cover the target picking area.

6. The intelligent control system of the tea bud picking robot according to claim 1, characterized in that, The system is built based on the ROS platform.

7. An intelligent control method for a tea bud picking robot, characterized in that, It is implemented by using any one of the intelligent control systems for tea bud picking robots as described in claims 1-6, including: Obtain the scene image information through the visual analysis module, identify the area of the tea buds to be picked in the scene image information based on the preset neural network model, and establish a spatial coordinate system to determine the relative position between the picking robot and the tea bud area; Use the preset division method to divide the tea bud area into several picking sub-areas through the task decomposition module, determine the multiple target picking areas corresponding to each robotic arm according to the multiple picking sub-areas and the spatial positions of the multiple robotic arms of the picking robot, and formulate a planned path based on the relative positions of each robotic arm and the multiple target picking areas; Determine the joint motion data of each robotic arm according to the planned path through the motion control module, and control each robotic arm to pick and suck the tea buds in the target picking area according to the joint motion data.

Images