Autonomous navigation operation method and system for road-rail dual-purpose robot in multi-span greenhouse

Abstract

The invention discloses an autonomous navigation operation method and system for a road-rail dual-purpose robot of a multi-span greenhouse. The method comprises the following steps: acquiring a two-dimensional grating environment map marked with key position labels, planning an overall path, and controlling the robot to operate and implement tasks based on the overall path. The method specifically comprises the steps that the robot autonomously navigates to a track entrance label through a navigation planning module on a road surface, finite-state machine switching is triggered, the navigation planning module is closed, and the robot advances along a track; when a track end label is reached, state switching is triggered, so that the robot retreats along the track; the robot intermittently starts and stops moving on the track according to a preset step pitch, crop images are collected through a second vision unit, and operation is executed; and after the track entrance label is reached, the navigation planning module is started to set the next target position. The system is based on key position label navigation, is high in flexibility, low in failure rate and low in maintenance cost, reduces the operation difficulty through intermittent start and stop, and improves the working efficiency of picking, threshing and the like.

Description

Technical Field

[0001] The present invention relates to the field of agricultural robot navigation technology, and in particular to an autonomous navigation operation method and system for a dual-purpose robot for track and road in a multi-span greenhouse. Background Art

[0002] As an important facility in modern agriculture, multi-span greenhouses are widely used in crop planting and management. Its structure forms an integrated space through multiple greenhouse units connected together. It has excellent insulation, ventilation and environmental control performance, and is suitable for large-scale planting of various economic crops such as fruits, vegetables, and flowers. In multi-span greenhouses, hardened roads and tracks are usually laid to support the operation and operation of agricultural machinery, forming a typical planting operation scene combining tracks and roads. This layout can effectively improve space utilization and operating efficiency, while providing support for the operation of automated equipment such as robots.

[0003] The prior art CN111722631A discloses a method for changing tracks between an autonomous mobile chassis and a multi-span greenhouse chassis, which is mainly guided by navigation lines and turn labels. However, this navigation method has significant limitations. First, navigation lines and turn labels need to be arranged in advance, and a slight change in the environment may cause navigation failure. It also relies on external facilities, has high maintenance costs, and poor flexibility. In addition, the navigation lines impose many restrictions on the robot's path, making it difficult to cope with dynamic task requirements, resulting in low system adaptability, high failure rate, and limited overall efficiency.

[0004] On the other hand, traditional robots need to accurately calculate the operating position, the position of the robot arm and the gripper in operations such as picking fruits and removing leaves. Such calculations usually require high computing resources, which leads to a significant increase in the robot's demand for hardware computing power and a corresponding increase in system costs. At the same time, frequent stops at operating points and insufficient path optimization further limit operating efficiency and fail to meet the needs of modern smart agriculture for efficient and low-cost automation. Summary of the invention

[0005] Purpose of the invention: In order to overcome the deficiencies in the prior art, the present invention provides an autonomous navigation operation method and system for a low-span greenhouse track dual-purpose robot with fewer navigation restrictions, low failure rate, high operating efficiency and low computing power requirements.

[0006] Technical solution: To achieve the above-mentioned purpose, the present invention provides an autonomous navigation operation method of a dual-purpose robot for a multi-span greenhouse, the method comprising:

[0007] Get the environmental map inside the multi-span greenhouse;

[0008] Plan the robot's overall path based on the robot's initial position and task information;

[0009] Control the robot to run and perform tasks based on the planned overall path;

[0010] The environment map includes a two-dimensional grid map with coordinates of key position tags marked; the key position tags include track entrance tags and track end tags;

[0011] The control of the robot to run and perform tasks based on the planned operation path specifically includes:

[0012] Based on the navigation planning module, the robot is autonomously navigated on the road surface to the track entrance label, the finite state machine state transition condition is triggered, the navigation planning module is closed, and the robot moves along the track; when the robot reaches the track end label, the finite state machine state transition condition is triggered, and the robot moves backward along the track;

[0013] When the robot runs on the track, the chassis is controlled to start and stop intermittently on the track according to a preset step distance, and at the same time, the second visual unit collects images of the crops and performs corresponding operations on objects that meet the operating conditions based on the images;

[0014] When the robot reaches the track entry label corresponding to the current track again, the finite state machine state transition condition is triggered, the navigation planning module is opened, and the next target position is set to the coordinates of the track entry label of another track.

[0015] Furthermore, performing corresponding operations on objects meeting the operation conditions based on the image includes:

[0016] Obtaining depth data of the object to be operated and its surrounding facilities and / or branches and vines;

[0017] Based on the depth data, solving the target pose of the gripper;

[0018] Get the coordinate data of multiple stop points closest to the current position of the chassis;

[0019] Based on each coordinate data and the working space of the robot arm, a solution capable of operating the object to be operated is calculated, wherein the solution includes the coordinate data of the stop point and the posture of the robot arm;

[0020] When there are multiple solutions, one of them is selected as the final solution;

[0021] When the number of solutions obtained is 0, a global search for solutions is performed without being limited to the docking points to obtain the intermediate docking points and the corresponding manipulator postures.

[0022] Furthermore, based on the navigation planning module, the robot is autonomously navigated on the road surface to the track entrance tag, triggering the finite state machine state transition condition, closing the navigation planning module, and making the robot move along the track, including:

[0023] Starting a navigation planning module, taking an unpassed track entry tag as the next destination, and planning a path based on the two-dimensional grid map;

[0024] Control the chassis to move along the planned path, obtain the point cloud data collected by the LiDAR, and transmit the obtained LiDAR point cloud information to the AMCL positioning algorithm. Combined with the odometer data fused from the IMU and laser odometer, and the grid map information, estimate the real-time position and posture of the chassis at the current moment, and use the current real-time position and posture of the chassis to correct the driving path of the chassis.

[0025] When the robot's first visual unit captures the track entrance label, the robot's posture is corrected based on the track entrance label, and the ID number of the track entrance label is calculated to trigger the finite state machine state transition condition, close the navigation planning module, and control the robot to move in a straight line along the track.

[0026] Furthermore, each time the first visual unit captures an image of a key position label, it calculates the position and posture of the chassis in the two-dimensional grid map according to the position of the key position label in the image, and feeds it back to AMCL for global repositioning.

[0027] Furthermore, the specific construction method of the environment map includes:

[0028] Using the point cloud data of the laser radar and the IMU data, the LIO-SAM algorithm and the octree map algorithm are used to build a two-dimensional grid map of the greenhouse environment;

[0029] The precise location coordinates corresponding to each of the key location tags entered by the user are received.

[0030] The multi-span greenhouse track dual-purpose robot autonomous navigation operation system includes:

[0031] A robot having a chassis, a robot arm, a gripper, and an autonomous navigation operation method for a multi-span greenhouse track dual-purpose robot; a sensor including a laser radar and an IMU is installed on the chassis; and the robot also includes a first visual unit and a second visual unit;

[0032] The track system comprises a road surface and a plurality of parallel tracks; each track has crop rows on both sides, and key position labels are posted at the entrance, middle and end of each track.

[0033] Beneficial effects: The autonomous navigation operation method and system of the multi-span greenhouse track dual-purpose robot of the present invention have the following beneficial effects:

[0034] (1) On the one hand, the robot autonomously navigates on the road based on a two-dimensional grid map marked with key position labels, and opens and closes the navigation planning module based on a finite state machine. Compared with the navigation method based on navigation lines and turn labels in the prior art, the navigation method in this application has fewer restrictions, a lower failure rate, is more flexible and efficient, can ensure the efficiency of track conversion, and reduce the trouble of subsequent maintenance. On the other hand, the robot starts and stops intermittently on the track according to the preset step distance, which can reduce the difficulty of planning the working position and working posture in operations such as picking and threshing, and improve work efficiency.

[0035] (2) The stop point selection and posture calculation method designed based on the preset step-length intermittent start-stop motion can achieve the optimal selection of stop points and the rapid calculation of the working posture, reduce the difficulty of calculation and selection, reduce the computing power requirements, reduce the number of robot stops, and improve the working efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a schematic diagram of the autonomous navigation system of the track-based dual-purpose robot in a multi-span greenhouse;

[0037] Figure 2 This is the state diagram of the robot traveling on the track;

[0038] Figure 3 It is a flowchart of the autonomous navigation operation method of the dual-purpose robot on tracks in a multi-span greenhouse;

[0039] Figure 4 The present invention is a flowchart of controlling the robot to run and perform tasks based on the planned working path.

[0040] In the figure: 1, robot; 2, road surface; 3, track; 4, crop row; 11, chassis; 12, robot arm; 13, gripper; 1a, chassis base; 1b, drive motor; 1c, ground running wheel; 1d, track running wheel; 5a, track entrance label; 5b, track midpoint label; 5c, track end label. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings.

[0042] The autonomous navigation operation method of the multi-span greenhouse track dual-purpose robot of the present invention is based on Figure 1 The multi-span greenhouse track dual-purpose robot autonomous navigation operation system shown is implemented, specifically, the system includes:

[0043] Robot 1, such as Figure 2 As shown, it has a chassis 11, a robot arm 12, a gripper 13 and implements the following autonomous navigation operation method of the multi-span greenhouse track dual-purpose robot 1; the chassis 11 has a chassis base 1a and walking modules symmetrically installed at the four corners of the chassis base 1a, and the walking module includes a driving motor 1b and coaxially installed ground walking wheels 1c and track walking wheels 1d; in this embodiment, the gripper 13 is a leaf-beating gripper 13, and in other embodiments, it can also be a picking gripper 13; sensors including a laser radar and an IMU are installed on the chassis 11; it also includes a first visual unit and a second visual unit; the first visual unit is an industrial CCD camera, and is used in conjunction with a fill light; the second visual unit is a binocular depth camera.

[0044] The track system includes a road surface 2 and a plurality of parallel tracks 3; each track 3 has crop rows 4 on both sides, and key position labels are posted at the entrance, middle position and tail of each track 3, namely, a track entrance label 5a, a track midpoint label 5b and a track end label 5c.

[0045] like Figure 3 As shown, the autonomous navigation operation method of the multi-span greenhouse dual-purpose robot on track of the present invention includes the following steps S101-S103:

[0046] Step S101, obtaining an environmental map in a multi-span greenhouse;

[0047] Step S102, planning the overall path of the robot based on the initial position and task information of the robot;

[0048] Step S103, controlling the robot to run and perform tasks based on the planned overall path;

[0049] The environment map includes a two-dimensional grid map with coordinates of key position tags marked; the key position tags include track entrance tags and track end tags, and a track midpoint tag is set in the middle of the track, and the track entrance tag, track end tag and track midpoint tag are all AprilTag tags; in step S101, the navigation package is used in the robot operating system ROS to call map_server to load the two-dimensional grid map; in step S102, the overall path planned passes through each track entrance tag;

[0050] like Figure 4 As shown, the above step S103 controls the robot to run and perform tasks based on the planned operation path, specifically including steps S201-S203:

[0051] Step S201: Based on the navigation planning module, the robot is autonomously navigated on the road surface to the track entrance tag, the finite state machine state transition condition is triggered, the navigation planning module is closed, and the robot moves along the track; when the robot reaches the track end tag, the finite state machine state transition condition is triggered, and the robot moves backward along the track;

[0052] Step S202, when the robot is running on the track, the chassis is controlled to start and stop intermittently on the track according to a preset step distance, and at the same time, the second visual unit collects images of the crop, and performs corresponding operations on objects that meet the operating conditions based on the images; in the picking operation, the objects that meet the operating conditions are ripe fruits, and in the leaf threshing operation, the objects that meet the operating conditions are diseased branches and leaves;

[0053] Step S203, when the robot reaches the track entrance tag corresponding to the current track again, the finite state machine state transition condition is triggered, the navigation planning module is opened, and the next target position is set to the coordinates of the track entrance tag of another track.

[0054] In the above autonomous navigation operation method, on the one hand, the robot autonomously navigates on the road based on a two-dimensional grid map marked with key position labels, and opens and closes the navigation planning module based on a finite state machine. Compared with the navigation method based on navigation lines and turn labels in the prior art, the navigation method in this application has fewer restrictions, a lower failure rate, is more flexible and efficient, can ensure the efficiency of track conversion, and reduce the trouble of subsequent maintenance. On the other hand, the robot starts and stops intermittently on the track according to the preset step distance, which can reduce the difficulty of planning the operation position and operation posture in operations such as picking and threshing, and improve the operation efficiency.

[0055] Preferably, the performing of corresponding operations on the objects meeting the operation conditions based on the image in the above step S202 includes the following steps S301-S306:

[0056] Step S301, obtaining depth data of the object to be operated and its surrounding facilities and / or branches and vines;

[0057] Step S302, based on the depth data, calculating the target posture of the gripper; the target posture may be a single posture, or a collection of postures that can realize the operation (that is, it may be a workspace);

[0058] Step S303, obtaining coordinate data of multiple stop points closest to the current position of the chassis; the stop points here are obtained from one end of the guide rail at equal intervals according to the preset step distance, that is, each track has multiple fixed stop points; in this step, the selected stop points include at least one stop point in front of the current moving direction of the chassis and at least one stop point behind the current moving direction; when the chassis itself is at a stop point, the selected stop points also include the stop point where the chassis is currently located;

[0059] Step S304, based on each coordinate data and the working space of the robot arm, a solution capable of operating the object to be operated is calculated, wherein the solution includes the coordinate data of the stop point and the posture of the robot arm;

[0060] Step S305, when there are multiple solutions, select one of them as the final solution;

[0061] In this step, the final solution is selected based on the following principles:

[0062] First, the principle of avoiding retreat, that is, avoiding retreating relative to the current direction of movement, includes the following situations: when the robot moves forward along the track, if the stop point corresponding to the unique solution is on the rear side of the current direction of movement, the corresponding solution will be saved and used as the robot to reach the corresponding stop point for operation when it moves forward to the end of the track and then moves backward; when the robot itself is already moving backward along the track, if the stop point corresponding to the unique solution is on the rear side of the current direction of movement, it will move backward to reach the corresponding stop point and operate on the object to be operated.

[0063] Second, the principle of reducing stops: during actual operation, the second visual unit simultaneously collects depth data of multiple objects to be operated, and calculates the solutions corresponding to each object based on the depth data of each object. According to the principle of minimum stops, all objects are divided into the least number of stops according to the solutions of each object. In this way, operations can be performed on as many objects as possible at the actual stop points.

[0064] When there is still more than one solution that meets the above two principles, the proximity principle is adopted and the nearest stop is selected for operation.

[0065] Step S306, when the number of solutions obtained is 0, a global search for solutions is performed without being limited to the stop points to obtain the intermediate stop points and the corresponding manipulator postures.

[0066] The above-mentioned method of stop point selection and posture solution designed based on intermittent start-stop motion with preset step length can achieve optimal selection of stop points and rapid calculation of working posture based on limited stop points, reduce the difficulty of calculation and selection, reduce computing power requirements, reduce the number of robot stops, and improve working efficiency.

[0067] Preferably, the above step S201, based on the navigation planning module, enables the robot to autonomously navigate on the road surface to the track entrance tag, triggers the finite state machine state transition condition, closes the navigation planning module, and enables the robot to move along the track, including the following steps S401-S403:

[0068] Step S401, start the navigation planning module, take an unpassed track entry tag as the next destination, plan a path based on the two-dimensional grid map, and the planned path includes a global path and a local path;

[0069] In this step, the move_base service in ROS is called to use the A* algorithm and the DWA algorithm to plan the global path and local path of the robot's movement respectively.

[0070] Step S402, control the chassis to move along the planned path, and obtain the point cloud data collected by the laser radar. At the same time, the obtained laser radar point cloud information is transmitted to the AMCL positioning algorithm, and the odometer data fused from the IMU and laser odometer, as well as the grid map information, are combined to estimate the real-time position and posture of the chassis at the current moment, and the driving path of the chassis is corrected using the current real-time position and posture of the chassis;

[0071] Step S403: When the first visual unit of the robot captures the track entrance tag, the robot's posture is corrected based on the track entrance tag, and the ID number of the track entrance tag is solved to trigger the state transition condition of the finite state machine, and the navigation planning module is closed to control the robot to move straight along the track. The track entrance tag is decoded using specific decoding rules based on the AprilTag library.

[0072] Preferably, each time the first visual unit captures an image of a key position tag, the position and posture of the chassis in the two-dimensional grid map is calculated according to the position of the key position tag in the image, and the position and posture are fed back to AMCL for global repositioning. This can significantly improve the positioning accuracy of the chassis at different positions, ensure that the chassis can always adjust its path planning based on the latest accurate position and posture, effectively avoid navigation deviations caused by environmental interference or sensor cumulative errors, and improve the stability and reliability of the system.

[0073] Preferably, the specific method for constructing the environment map includes the following steps S501-S502:

[0074] Step S501, using the point cloud data of the laser radar and the IMU data, using the LIO-SAM algorithm and the octree map algorithm to establish a two-dimensional grid map of the greenhouse environment;

[0075] Step S502: receiving the precise location coordinates corresponding to each of the key location tags entered by the user.

[0076] During implementation, each key position label is manually posted in advance, and the precise position coordinates corresponding to each key position label are obtained by manual precise measurement. In the subsequent navigation operation, the precise position coordinates of the key position label can be corrected when performing global repositioning based on the posture fed back to AMCL to achieve higher accuracy.

[0077] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An autonomous navigation operation method of a dual-purpose robot for a multi-span greenhouse, the method comprising: Get the environmental map inside the multi-span greenhouse; Plan the robot's overall path based on the robot's initial position and task information; Control the robot to run and perform tasks based on the planned overall path; The invention is characterized in that the environment map includes a two-dimensional grid map with coordinates of key position labels marked; the key position labels include track entrance labels and track end labels; The control of the robot to run and perform tasks based on the planned operation path specifically includes: Based on the navigation planning module, the robot is autonomously navigated on the road surface to the track entrance label, the finite state machine state transition condition is triggered, the navigation planning module is closed, and the robot moves along the track; when the robot reaches the track end label, the finite state machine state transition condition is triggered, and the robot moves backward along the track; When the robot runs on the track, the chassis is controlled to start and stop intermittently on the track according to a preset step distance, and at the same time, the second visual unit collects images of the crops and performs corresponding operations on objects that meet the operating conditions based on the images; When the robot reaches the track entry label corresponding to the current track again, the finite state machine state transition condition is triggered, the navigation planning module is opened, and the next target position is set to the coordinates of the track entry label of another track.

2. The autonomous navigation operation method of the dual-purpose robot for multi-span greenhouses according to claim 1 is characterized in that: The performing corresponding operations on the objects meeting the operation conditions based on the image includes: Obtaining depth data of the object to be operated and its surrounding facilities and / or branches and vines; Based on the depth data, solving the target position of the gripper; Get the coordinate data of multiple stop points closest to the current position of the chassis; Based on each coordinate data and the working space of the robot arm, a solution capable of operating the object to be operated is calculated, wherein the solution includes the coordinate data of the stop point and the posture of the robot arm; When there are multiple solutions, one of them is selected as the final solution; When the number of solutions obtained is 0, a global search for solutions is performed without being limited to the docking points to obtain the intermediate docking points and the corresponding manipulator postures.

3. The autonomous navigation operation method of the dual-purpose robot for multi-span greenhouse according to claim 1, characterized in that: Based on the navigation planning module, the robot is autonomously navigated and moved on the road surface to the track entrance tag, triggering the finite state machine state transition condition, closing the navigation planning module, and making the robot move along the track, including: Starting a navigation planning module, taking an unpassed track entry tag as the next destination, and planning a path based on the two-dimensional grid map; Control the chassis to move along the planned path, obtain the point cloud data collected by the LiDAR, and transmit the obtained LiDAR point cloud information to the AMCL positioning algorithm. Combined with the odometer data fused from the IMU and laser odometer, and the grid map information, estimate the real-time position and posture of the chassis at the current moment, and use the current real-time position and posture of the chassis to correct the driving path of the chassis. When the robot's first visual unit captures the track entrance label, the robot's posture is corrected based on the track entrance label, and the ID number of the track entrance label is calculated to trigger the finite state machine state transition condition, close the navigation planning module, and control the robot to move in a straight line along the track.

4. The autonomous navigation operation method of the dual-purpose robot for multi-span greenhouse according to claim 3 is characterized in that: Each time the first visual unit captures an image with a key position label, it calculates the position and posture of the chassis in the two-dimensional grid map according to the position of the key position label in the image, and feeds it back to AMCL for global repositioning.

5. The autonomous navigation operation method of the dual-purpose robot for multi-span greenhouse according to claim 1, characterized in that: The specific construction method of the environment map includes: Using the point cloud data of the laser radar and the IMU data, the LIO-SAM algorithm and the octree map algorithm are used to build a two-dimensional grid map of the greenhouse environment; The precise location coordinates corresponding to each of the key location tags entered by the user are received.

6. The multi-span greenhouse track dual-purpose robot autonomous navigation operation system is characterized by: It includes: A robot having a chassis, a robot arm, a gripper, and an autonomous navigation operation method for a dual-purpose robot for a multi-span greenhouse according to any one of claims 1 to 5; The chassis is equipped with sensors including a laser radar and an IMU; it also includes a first visual unit and a second visual unit; The track system comprises a road surface and a plurality of parallel tracks; each track has crop rows on both sides, and key position labels are posted at the entrance, middle and end of each track.

Images