Autonomous strawberry picking robot and three-dimensional coordinate construction method thereof

Abstract

The invention relates to the technical field of picking equipment, in particular to an autonomous strawberry picking robot and a three-dimensional coordinate construction method thereof. The robot comprises a robot body, the robot body is provided with a binocular stereoscopic vision unit, the binocular stereoscopic vision unit is used for deploying a high-resolution and synchronously-triggered binocular camera, and the baseline distance is optimized according to the target detection range. The method has the multi-shed area autonomous navigation capability, supports omni-directional movement through an omni-directional navigation chassis, namely, four-wheel independent driving and IMU fusion positioning, constructs a multi-shed area topological map in combination with a laser radar and a binocular camera, calculates the optimal shed area crossing path in real time through a dynamic path planning algorithm, can improve the shed area crossing efficiency, and improves the shed area crossing efficiency. Through SLAM mapping and dynamic path planning, the robot can autonomously complete switching of a plurality of shed areas, and the operation range is expanded; and the laser radar and the infrared sensor are used for detecting sudden obstacles in real time, and avoidance is realized in combination with the omnidirectional wheel set.

Description

Technical Field

[0001] The present invention relates to the technical field of picking equipment, and particularly relates to an autonomous strawberry picking robot and a method for constructing its three-dimensional coordinates. Background Art

[0002] Currently, the mechanical structure of strawberry picking robots in the existing technology is as follows: a cross-ridge box-shaped frame is equipped with bilateral six-degree-of-freedom robotic arms, and the end cutting and holding mechanism adopts a spatial cam + laser pair sensor; Vision system: an RGB-D camera (D435i) is combined with an RGB camera, which are respectively used for target positioning and picking point recognition; Navigation system: a four-wheel chassis is combined with magnetic encoding steering, and it relies on vision and distance sensors to move between ridges; Control logic: an embedded AI controller cooperates with the lower computer to achieve multi-threaded control based on the serial port / CAN protocol.

[0003] However, the existing technology has the following objective drawbacks: the camera's judgment of the picking effect is not good, and multi-shed multi-ridge autonomous picking has not been achieved, especially reflected in:

[0004] 1. Navigation limitation: only supports single-shed operation, and manual intervention is required for cross-shed operation, and multi-shed multi-ridge autonomous picking has not been achieved;

[0005] 2. Lack of maturity detection: there is no relevant maturity detection algorithm;

[0006] 3. Lack of dynamic planning: uses a preset path, and the dynamic obstacle avoidance ability is weak;

[0007] 4. Risk of mechanical damage: a two-finger rigid end effector has a high risk of fruit damage. Therefore, we propose an autonomous strawberry picking robot and a method for constructing its three-dimensional coordinates. Summary of the Invention

[0008] The purpose of the present invention is to provide an autonomous strawberry picking robot and a method for constructing its three-dimensional coordinates to solve the problems raised in the above background art.

[0009] To solve the above technical problems, the present invention adopts the following technical solutions:

[0010] An autonomous strawberry picking robot, including a robot body, on which are provided: a binocular stereo vision unit, the binocular stereo vision unit is a binocular camera deployed with high resolution and synchronous triggering, and the baseline distance is optimized according to the target detection range;

[0011] An array of infrared sensor groups, the array of infrared sensor groups covers the near-field area in the moving direction of the robot body through multiple active infrared ranging modules, and supports dynamic obstacle detection;

[0012] An array-type laser radar group, which uses laser radar to implement a SLAM algorithm to create a high-precision map;

[0013] An embedded GPU is provided, wherein the embedded GPU is used for processing multi-source data in real time.

[0014] Preferably, the robot body is also provided with a mobile platform equipped with an encoder and IMU and a weighing and sorting module. The encoder and the mobile platform of the IMU provide odometer information to assist positioning. The weighing and sorting module is a micro weighing sensor that automatically grades and stores.

[0015] Preferably, the binocular camera is horizontally mounted on the top of the robot body, with a baseline distance of 20 cm and a pitch angle covering the main distribution height of the fruit.

[0016] Preferably, the array-type infrared sensor group is deployed in a circular array around the robot body, with an elevation angle of 10°-15° downward, to detect low dynamic obstacles and moving personnel.

[0017] Preferably, an omnidirectional navigation platform is provided on the robot body, and the omnidirectional navigation platform includes a four-drive motor wheel group and a fusion positioning module. The four-drive motor wheel group is independently driven by four wheels to realize movement between ridges and between sheds. The fusion positioning module is a combined positioning of RGBD camera, IMU and laser SLAM.

[0018] Preferably, the robot body adopts a dual-arm collaborative execution system, and the dual-arm collaborative execution system realizes picking through a cross-shaped robotic arm designed with six degrees of freedom.

[0019] A three-dimensional coordinate construction method for an autonomous strawberry picking robot, applicable to an autonomous strawberry picking robot, comprises the following steps:

[0020] S1: Sensor calibration and spatiotemporal synchronization, which includes calibration of the binocular camera and infrared sensor, as well as spatiotemporal synchronization between the various camera sensors.

[0021] S2: Use binocular stereo matching core algorithm to perform stereo calibration, fruit target detection and matching, depth calculation and coordinate mapping;

[0022] S3: Dynamic obstacle detection and multi-source fusion. By processing infrared data, the distance values of each sensor are read in real time. Dynamic targets are detected based on multi-frame data. The rate of change of distance in the same direction between adjacent frames is calculated, and a threshold is set to determine moving objects.

[0023] Preferably, the S3 step further includes a multi-source data fusion strategy step, and the multi-source data fusion strategy step includes:

[0024] Spatial alignment to unify the fruit coordinates of binocular vision and the infrared obstacle coordinates into the global coordinate system;

[0025] Confidence weighting to perform weighted fusion of the depth estimation of binocular vision and infrared ranging.

[0026] Preferably, the multi-source data fusion strategy steps further include dynamic obstacle avoidance decision-making. Using the velocity obstacle method, a collision-free path is generated according to the obstacle motion prediction.

[0027] It can be clearly seen that through the above technical solutions of this application, the technical problems to be solved by this application can surely be solved.

[0028] Meanwhile, through the above technical solutions, the present invention has at least the following beneficial effects:

[0029] The present invention has the ability of autonomous navigation in multiple shed areas. Through the omnidirectional navigation chassis, that is, the four-wheel independent drive and IMU fusion positioning support omnidirectional movement. Combining lidar and binocular cameras to construct a topological map of multiple shed areas, and through the dynamic path planning algorithm to calculate the optimal path across shed areas in real time, it can improve the efficiency across shed areas. Through SLAM mapping and dynamic path planning, the robot can autonomously complete the switching of multiple shed areas, expanding the operation range; and strengthening the obstacle avoidance ability. The lidar and infrared sensors detect sudden obstacles in real time, and combine with the omnidirectional wheel set to achieve avoidance. Description of the Drawings

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0031] Figure 1 It is the operation flowchart of the present invention;

[0032] Figure 2 It is the layout schematic diagram of the binocular camera, lidar and infrared sensor of the present invention;

[0033] Figure 3 It is the schematic diagram of the end collector of the present invention. Detailed Embodiments

[0034] In order to make the purpose, technical solutions and advantages of the present invention clearer, the following will further describe the present invention in detail with reference to the drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, and are not used to limit the present invention.

[0035] Embodiment 1

[0036] Reference Figures 1-3 , an autonomous strawberry-picking robot, including a robot body, on which there are provided: a binocular stereo vision unit, the binocular stereo vision unit is a binocular camera deployed with high resolution and synchronous triggering, and the baseline distance is optimized according to the target detection range;

[0037] An array of infrared sensor groups, the array of infrared sensor groups covers the near-field area in the moving direction of the robot body through multiple groups of active infrared ranging modules, and supports dynamic obstacle detection;

[0038] An array of lidar groups, the array of lidar groups uses lidar to implement the SLAM algorithm to establish a high-precision map;

[0039] An embedded GPU, which is used to process multi-source data in real time.

[0040] The robot body is also provided with a mobile platform equipped with an encoder and an IMU and a weighing and sorting module. The mobile platform with the encoder and IMU provides odometer information for auxiliary positioning, and the weighing and sorting module is a micro weighing sensor for automatic grading and storage.

[0041] The binocular camera is horizontally installed on the top of the robot body, the baseline distance is 20 cm, and the pitch angle covers the main distribution height of the fruits.

[0042] The array of infrared sensor groups is deployed in a circular array around the robot body, and the elevation angle is inclined downward by 10°-15° to detect low dynamic obstacles and moving personnel.

[0043] The robot body is provided with an omnidirectional navigation platform. The omnidirectional navigation platform includes a four-drive motor wheel group and a fusion positioning module. The four-drive motor wheel group is independently driven by four wheels to realize movement between ridges and movement between greenhouses. The fusion positioning module is a combined positioning of an RGBD camera, an IMU and laser SLAM.

[0044] The robot body adopts a two-arm collaborative execution system, and the two-arm collaborative execution system realizes picking through a cross-shaped robotic arm designed with six degrees of freedom.

[0045] Compared with the prior art without dynamic obstacle avoidance logic, the present invention combines an omnidirectional wheel group to achieve avoidance:

[0046] The kinematic model of the omnidirectional wheel group: improving the energy consumption weight factor of the AI algorithm to ensure the path planning efficiency.

[0047] 1. Design of path cost function and energy consumption weight factor

[0048] Path cost function:

[0049] The improved AI algorithm defines the total path cost as a weighted combination of distance cost and energy consumption cost.

[0050] Calculation of energy consumption cost:

[0051] The mobile energy consumption of each node is related to its terrain type. Define the terrain energy consumption coefficient, and give different energy consumption cost weights according to different terrain scenarios.

[0052] 2. Dynamic adjustment logic of energy consumption weight factor

[0053] The weight factor is dynamically adjusted according to the robot state:

[0054] Power sensitivity model:

[0055]

[0056] α is the sensitivity coefficient, β is the baseline weight. The lower the power, the larger the sensitivity model, and the path tends to be energy-saving.

[0057] Task mode intervention:

[0058] Energy-saving mode: Increase ω e , and give priority to low-energy consumption paths.

[0059] Emergency mode: Decrease ω e , and give priority to the shortest path.

[0060] 3. Sliding window method and dynamic map update mechanism

[0061] Definition of sliding window:

[0062] Taking the current position of the robot as the center, define a circular or square area with a radius of r as the local map window.

[0063] Update process:

[0064] 1. Sensor scanning: Real-time obtain the environmental data within the window through LiDAR and camera.

[0065] 2. Change detection: Compare the scanned data with the global map, and mark the newly added obstacles or terrain changes.

[0066] 3. Local map update: Replace the global map within the window with the latest data.

[0067] 4. Path replanning trigger: If the current path passes through the updated area and is blocked, immediately replan the path.

[0068] Dynamic path planning:

[0069] When performing global path planning, give priority to using the latest map data within the window.

[0070] After local update, only recalculate the path within the affected area to reduce the computational overhead.

[0071] 4. Algorithm implementation steps

[0072] 1. Initialization:

[0073] Load the global map and mark the terrain type and obstacles.

[0074] Set according to the initial battery level and task mode.

[0075] 2. Global path planning:

[0076] Use the improved AI algorithm to select nodes.

[0077] For each node expansion, calculate the terrain energy consumption coefficient and the weighted actual cost.

[0078] 3. Movement and real-time update:

[0079] Continuously scan the environment within the sliding window during movement.

[0080] When changes are detected, update the local map and trigger path replanning.

[0081] 4. Weight factor adjustment:

[0082] Periodically check the battery level and update according to the preset strategy.

[0083] If the task mode switches, immediately adjust and replan the path.

[0084] 5. Key advantages

[0085] Energy consumption optimization: Achieve a balance between path length and energy consumption through dynamic weights.

[0086] Real-time performance: The sliding window method reduces the computational load and adapts to dynamic obstacles and terrain changes.

[0087] Robustness: Combine the battery level sensitive model to ensure that the task can still be completed at low battery levels.

[0088] Example 2

[0089] A method for constructing three-dimensional coordinates of an autonomous strawberry picking robot, applicable to an autonomous strawberry picking robot, includes the following steps:

[0090] S1: Sensor calibration and spatio-temporal synchronization, perform calibration of binocular cameras, infrared sensors, and spatio-temporal synchronization between various camera sensors;

[0091] S2: Use the core algorithm of binocular stereo matching for stereo rectification, fruit target detection and matching, and depth calculation and coordinate mapping;

[0092] S3: Dynamic obstacle detection and multi-source fusion. By processing infrared data, the distance values of each sensor are read in real time. Based on multi-frame data, dynamic targets are detected, the distance change rate in the same direction between adjacent frames is calculated, and a threshold is set to determine moving objects.

[0093] The S3 step also includes a multi-source data fusion strategy step, and the multi-source data fusion strategy step includes:

[0094] Spatial alignment, unifying the fruit coordinates of binocular vision and the infrared obstacle coordinates into the global coordinate system;

[0095] Confidence weighting, performing weighted fusion on the depth estimation of binocular vision and infrared ranging.

[0096] The multi-source data fusion strategy step also includes dynamic obstacle avoidance decision-making. Using the velocity obstacle method, a collision-free path is generated according to the prediction of obstacle movement.

[0097] The multi-modal perception fusion uses a clamping mechanical model combined with weighing feedback to avoid over-clamping. The differences from the prior art are:

[0098] I. System hardware architecture design

[0099] 1. Flexible clamping end structure: The contact surface is composed of flexible materials; a distributed pressure sensor array is embedded; a micro weighing sensor is integrated at the end joint of the robotic arm; the electric actuator is equipped with a high-precision displacement encoder.

[0100] 2. Mechanical perception layer:

[0101] Contact pressure perception: The pressure distribution on the contact surface is obtained in real time through flexible piezoresistive sensors;

[0102] Mass perception: Dynamic weighing is realized based on force sensors;

[0103] Deformation monitoring: The elastic body deformation amount is deduced through the coupling relationship between the actuator displacement amount and the contact pressure.

[0104] II. Construction of the clamping mechanical model

[0105] 1. Linear clamping force model, modeling the clamping force as the product of the end effector displacement amount and the clamping force coefficient.

[0106] 2. Dynamic mass correlation model, setting the maximum load-bearing capacity of the clamping force as the form of the product of the safety factor and the current mass plus the base clamping force, and updating the m value in real time through online weighing.

[0107] III. Implementation of the feedback control algorithm

[0108] 1. Double closed-loop control architecture:

[0109] Outer ring (mass-pressure ring): Calculate the target pressure threshold based on the weighing result.

[0110] Inner ring (pressure-displacement ring): Adjust the actuator displacement through PID control.

[0111] 2. Overload protection mechanism:

[0112] Establish a three-level response strategy:

[0113] When the clamping force is greater than 50% of the maximum load capacity, reduce the clamping speed to 50%;

[0114] When the clamping force is greater than 95% of the maximum load capacity, immediately stop the clamping action;

[0115] When the clamping force is greater than 100% of the maximum load capacity, perform a 0.5mm reverse displacement.

[0116] IV. Timing control logic

[0117] 1. Contact detection stage:

[0118] Approach the target at a low speed of 2mm / s;

[0119] Trigger contact confirmation when the pressure of any sensor is greater than 0.1N;

[0120] 2. Pre-clamping stage:

[0121] Linearly increase the clamping force to 30% of the maximum clamping force;

[0122] Synchronously perform mass estimation (3-time moving average filtering);

[0123] 3. Stable clamping stage:

[0124] Adopt variable-gain PID control and suppress pressure fluctuations by applying a 5Hz notch filter.

[0125] 4. Release stage:

[0126] Segmented unloading strategy: Prevent slipping at the end of the last 10% of the stroke with residual pressure.

[0127] Implementation principle of the multi-sensor fusion SLAM mapping and positioning system:

[0128] Lidar (16 lines) + binocular vision + infrared sensor multi-source data fusion, and construct an environmental map with centimeter-level accuracy through the SLAM algorithm based on graph optimization.

[0129] Implementation principle of the omnidirectional navigation chassis and dynamic obstacle avoidance hardware design:

[0130] Four-wheel drive omnidirectional wheel set, supporting lateral translation and in-situ steering, with lidar + infrared sensors for real-time obstacle detection to reduce the response time to sudden obstacles.

[0131] Implementation principle of energy consumption optimization path planning by improving the AI algorithm:

[0132] The path cost function introduces an energy consumption weight factor, and the sliding window method is used for local map update to adapt to dynamic environmental changes.

[0133] The improved AI algorithm defines the total path cost as a weighted combination of distance cost and energy consumption cost.

[0134] Definition of the sliding window of the robot body: Taking the current position of the robot as the center, a circular or square area with a radius of r is defined as the local map window.

[0135] Software update process:

[0136] 1. Sensor scanning: Real-time acquisition of environmental data within the window through LiDAR and cameras.

[0137] 2. Change detection: Compare the scanned data with the global map to mark new obstacles or terrain changes.

[0138] 3. Local map update: Replace the global map within the window with the latest data.

[0139] 4. Path replanning trigger: If the current path passes through the updated area and is blocked, immediately replan the path.

[0140] Dynamic path planning:

[0141] When performing global path planning, preferentially use the latest map data within the window.

[0142] After local update, recalculate the path only within the affected area to reduce the computational overhead.

[0143] Key points of protection: Calculation logic of energy consumption weight factor, dynamic map update mechanism.

[0144] Implementation principle of flexible clamping end and pressure feedback mechanism:

[0145] Closed-loop control of clamping force, integrated with a weighing sensor. Rotary picking mode, using a three-finger flexible robotic claw to rotate and collect fruits to reduce the fruit damage rate.

[0146] Adopt a dual closed-loop pressure feedback control algorithm:

[0147] Outer loop (mass-pressure loop): Calculate the target pressure threshold based on the weighing result;

[0148] Inner loop (pressure-displacement loop): Adjust the actuator displacement through PID.

[0149] Rotary fruit collection design.

[0150] Startup mapping and collection process of the robot body:

[0151] Step 1: Mapping, SLAM initialization → LiDAR scanning → Generation of multi-shed topological map.

[0152] Step 2: Collection, visual target locking → Dynamic path planning → Dual-arm collaborative picking.

[0153] Explanation of importance ranking:

[0154] SLAM mapping and positioning enable cross-shed navigation;

[0155] Omnidirectional chassis and obstacle avoidance hardware;

[0156] Startup and collection process;

[0157] Gripping end design;

[0158] Path planning algorithm.

[0159] Based on the above, it can be seen that:

[0160] The present invention addresses the technical problems: The existing technologies have the following objective drawbacks: The judgment of the picking effect by the camera is not good, and multi-shed and multi-ridge autonomous picking has not been achieved, especially reflected in:

[0161] 1. Navigation limitation: Only single-shed operation is supported, and manual intervention is required for cross-shed operation, and multi-shed and multi-ridge autonomous picking has not been achieved;

[0162] 2. Lack of maturity detection: There is no relevant maturity detection algorithm;

[0163] 3. Lack of dynamic planning: Preset paths are adopted, and the dynamic obstacle avoidance ability is weak;

[0164] 4. Risk of mechanical damage: The two-finger rigid end effector has a high risk of fruit damage; By adopting the technical solutions of the above embodiments and through the above settings, this application will surely solve the above technical problems. At the same time, the following technical effects are achieved:

[0165] The present invention has the ability of multi-shed autonomous navigation. Through the omnidirectional navigation chassis, that is, four-wheel independent drive and IMU fusion positioning support omnidirectional movement, combined with LiDAR and binocular cameras to construct a multi-shed topological map, and through the dynamic path planning algorithm to calculate the optimal cross-shed path in real time, the improvement of cross-shed efficiency can be achieved. Through SLAM mapping and dynamic path planning, the robot can autonomously complete the switching of multiple sheds (manual intervention is required in the existing technology), and the operation range is expanded; as well as the enhancement of the obstacle avoidance ability, LiDAR and infrared sensors detect sudden obstacles in real time, and combined with the omnidirectional wheel set to achieve avoidance.

[0166] In the present invention, unless otherwise clearly specified or defined, terms such as "installed", "connected", "joined", "fixed", etc. shall be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection, an electrical connection, or communicable with each other; it may be directly connected, or indirectly connected through an intermediate medium, and may be the internal connection of two components or the interaction relationship between two components, unless otherwise clearly defined. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0167] Obviously, the embodiments described above are only a part of the embodiments of the present invention, rather than all embodiments. The preferred embodiments of the present invention are given in the drawings, but do not limit the patent scope of the present invention. The present invention can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosed content of the present invention more thorough and comprehensive. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements on some of the technical features. Any equivalent structure directly or indirectly using the content of the specification and drawings of the present invention in other related technical fields shall be within the scope of the patent protection of the present invention by the same token.

Claims

1. An autonomous strawberry-picking robot, characterized in that, It includes a robot body, on which there are provided: a binocular stereo vision unit, which is a binocular camera deployed with high resolution and synchronous triggering, and the baseline distance is optimized according to the target detection range; an array type infrared sensor group, which covers the near-field area in the moving direction of the robot body through multiple active infrared ranging modules and supports dynamic obstacle detection; an array type lidar group, which uses lidar to implement the SLAM algorithm to establish a high-precision map; an embedded GPU, which is used to process multi-source data in real time.

2. The autonomous strawberry picking robot according to claim 1, characterized in that: The robot body is also provided with a mobile platform equipped with an encoder and an IMU and a weighing and sorting module. The mobile platform with the encoder and the IMU provides odometer information for auxiliary positioning, and the weighing and sorting module is a micro weighing sensor for automatic grading and storage.

3. The autonomous strawberry-picking robot according to claim 1, wherein The binocular camera is horizontally installed on the top of the robot body, the baseline distance is 20 cm, and the pitch angle covers the main distribution height of the fruits.

4. The autonomous strawberry-picking robot according to claim 1, wherein, The array type infrared sensor group is deployed in a circular array around the robot body, and the elevation angle is inclined downward by 10°-15° to detect low dynamic obstacles and moving personnel.

5. The autonomous strawberry picking robot according to claim 4, characterized in that: The robot body is provided with an omnidirectional navigation platform, which includes a four-drive motor wheel group and a fusion positioning module. The four-drive motor wheel group is independently driven by four wheels to realize movement between ridges and movement between greenhouses. The fusion positioning module is a combined positioning of an RGBD camera, an IMU, and laser SLAM.

6. The autonomous strawberry-picking robot according to claim 5, wherein, The robot body adopts a two-arm collaborative execution system, and the two-arm collaborative execution system realizes picking through a cross-shaped robotic arm designed with six degrees of freedom.

7. A method for constructing three-dimensional coordinates of an autonomous strawberry-picking robot, applicable to an autonomous strawberry-picking robot described in any one of claims 1-6, characterized in that, It includes the following steps: S1: Sensor calibration and spatio-temporal synchronization, calibrating the binocular camera and the infrared sensor and performing spatio-temporal synchronization between various camera sensors; S2: Adopting the core algorithm of binocular stereo matching for stereo correction, fruit target detection and matching, and depth calculation and coordinate mapping; S3: Dynamic obstacle detection and multi-source fusion, by processing the infrared data, reading the distance values of each sensor in real time, detecting dynamic targets based on multi-frame data, calculating the distance change rate in the same direction between adjacent frames, and setting a threshold to determine moving objects.

8. The method for constructing three-dimensional coordinates of an autonomous strawberry picking robot according to claim 7, wherein: The S3 step also includes a multi-source data fusion strategy step, and the multi-source data fusion strategy step includes: Spatial alignment, unifying the fruit coordinates of binocular vision and the infrared obstacle coordinates to the global coordinate system; Confidence weighting, performing weighted fusion on the depth estimation of binocular vision and infrared ranging.

9. A method for constructing three-dimensional coordinates of an autonomous strawberry-picking robot according to claim 8, characterized in that The multi-source data fusion strategy step also includes dynamic obstacle avoidance decision-making, adopting the velocity obstacle method to generate a collision-free path according to the obstacle motion prediction.

Images