Autonomous greenhouse navigation method and device for seedling feeding and discharging robot and electronic equipment

Abstract

The invention relates to a seedling feeding and discharging robot greenhouse autonomous navigation method and device and electronic equipment, and the method comprises the steps: obtaining multi-source sensor data and semantic information of a breeding workshop, and determining a semantic coarse positioning position according to environment characteristics, so as to form a preliminary semantic positioning frame; based on a ground segmentation and plane fitting algorithm, the relative position relation between the robot and the ground is determined, and the relative position relation comprises the pitch angle, the yaw angle and the offset in the Z-axis direction; deeper feature extraction is carried out on details of the surrounding environment by combining a semantic coarse positioning position and local sensing data collected by a robot in real time; and on the basis of a grid projection method, carrying out accurate matching on local structural features to generate a global pose of the robot relative to the point cloud map, and determining a path plan. Through the multi-source data fusion technology, it is conveniently ensured that the robot can conduct accurate movement, obstacle avoidance and path planning in the complex environment, and collision or misoperation is avoided.

Description

Technical Field

[0001] The present invention relates to the field of robot navigation, and in particular to a method, a device and electronic equipment for autonomous navigation of a seedling loading and unloading robot in a greenhouse. Background Art

[0002] In a greenhouse environment, robots need to accurately locate and navigate in complex spaces. However, greenhouse environments are usually highly dynamic, with complex obstacles and narrow working spaces, and traditional greenhouse seedling robot systems often rely on a single data source, such as GPS or lidar for navigation. However, these technologies may be limited by signal interference in the greenhouse environment (such as weak or failed GPS signals) or complex environmental changes (such as light and temperature and humidity effects). Therefore, they need to be improved. Summary of the invention

[0003] Based on this, it is necessary to provide a greenhouse autonomous navigation method, device and electronic equipment for a seedling loading and unloading robot, in view of the fact that traditional greenhouse seedling robot systems often rely on a single data source, which is easily limited by signal interference or complex environmental changes in the greenhouse environment.

[0004] The present invention provides a method for autonomous navigation in a greenhouse of a seedling loading and unloading robot, the method comprising:

[0005] Acquire multi-source sensor data and semantic information of the breeding workshop, and determine the coarse location of semantics based on environmental characteristics to form a preliminary semantic positioning framework;

[0006] Based on the ground segmentation and plane fitting algorithm, the relative position relationship between the robot and the ground is determined. The relative position relationship includes the pitch angle, yaw angle and offset in the Z-axis direction.

[0007] Combine the semantic coarse positioning position and the local perception data collected by the robot in real time to perform more in-depth feature extraction of the details of the surrounding environment;

[0008] Based on the grid projection method, the local structural features are accurately matched to generate the global pose of the robot relative to the point cloud map and determine the path planning.

[0009] In one embodiment, the acquisition of multi-source sensor data and semantic information of the breeding workshop, and determining the coarse positioning position of the semantics according to environmental characteristics to form a preliminary semantic positioning framework includes:

[0010] Acquire and fuse multi-source sensor data from LiDAR, RGB-D cameras, deep learning-driven vision sensors, and IMUs;

[0011] The convolutional neural network is used to extract features from multi-source sensor data and to build a high-dimensional environmental feature space in combination with the geometric information of the lidar.

[0012] Based on the YOLO target detection algorithm, real-time obstacles are identified and the coarse positioning of semantics is determined.

[0013] In one embodiment, the real-time obstacle identification based on the YOLO target detection algorithm includes:

[0014] Predefine dynamic objects in loading and unloading scenes;

[0015] Input the image frame into the YOLOv8 detector to obtain object category, bounding box and confidence information;

[0016] Based on target detection technology, dynamic points are identified and eliminated;

[0017] Build a semantic map, fuse the object categories identified by YOLOv8 with the map data generated by SLAM, and mark the object categories at specific locations.

[0018] In one embodiment, determining the relative position relationship between the robot and the ground based on the ground segmentation and plane fitting algorithm includes:

[0019] Get the 3D point cloud map generated by the robot chassis laser radar;

[0020] Based on the Patchwork ground segmentation algorithm and the point cloud data collected by the 3D LiDAR sensor, the point cloud is divided into concentric areas of different densities, and plane fitting is performed in each area;

[0021] Estimate the pitch, yaw, and Z-axis offset of the robot relative to the ground.

[0022] In one embodiment, the method of accurately matching local structural features based on a grid projection method to generate a global position of the robot relative to a point cloud map includes:

[0023] Process the point cloud information into an occupancy grid according to a probabilistic voxel transformation and generate a bird's-eye view of the environment by projecting the probabilistic grid;

[0024] Divide the environment into a regular grid, with each cell representing a small area;

[0025] Based on multi-sensor data, the grid areas occupied by obstacles are marked, and the grid areas on the robot path are marked as free areas;

[0026] Perform horizontal plane projection on the point cloud voxel set to generate a 2D grid and calculate the voxel accumulation probability in the vertical direction;

[0027] The generated grid map is combined with real-time perception information and coarse positioning data, and the SuperGlue algorithm based on the attention mechanism is used to select matching point pairs with high similarity.

[0028] According to the posture change, the X-axis and Y-axis offsets and yaw angle of the robot are estimated, and the global posture of the robot relative to the point cloud map is determined.

[0029] In one embodiment, determining the path planning includes:

[0030] According to the kinematic model describing the robot's motion mode and mechanical structure, the robot's motion constraints are determined to limit the robot's optional motion trajectories in path planning;

[0031] Generate a set of potential trajectory candidates based on the robot's kinematic model and motion constraints;

[0032] Evaluate the pros and cons of each candidate trajectory and determine the best trajectory to run in the breeding room.

[0033] In one embodiment, determining the path planning further includes:

[0034] Based on real-time data from multi-source sensors, potential collision risks are identified and response strategies are generated, including speed adjustment, path replanning, and emergency stop when necessary.

[0035] The present invention also provides a seedling loading and unloading robot greenhouse autonomous navigation device, comprising:

[0036] The acquisition module is used to obtain multi-source sensor data and semantic information of the breeding workshop, and determine the coarse positioning position of the semantics according to environmental characteristics to form a preliminary semantic positioning framework;

[0037] A determination module is used to determine the relative position relationship between the robot and the ground based on ground segmentation and plane fitting algorithms. The relative position relationship includes the pitch angle, yaw angle, and offset in the Z-axis direction.

[0038] The extraction module is used to combine the semantic coarse positioning position and the local perception data collected by the robot in real time to perform more in-depth feature extraction of the details of the surrounding environment;

[0039] The matching module is used to accurately match local structural features based on the grid projection method to generate the global pose of the robot relative to the point cloud map and determine the path planning.

[0040] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the greenhouse autonomous navigation method of the seedling loading and unloading robot as described in any one of the above.

[0041] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the greenhouse autonomous navigation method for a seedling loading and unloading robot as described in any one of the above.

[0042] The above-mentioned greenhouse autonomous navigation method, device and electronic equipment of the seedling loading and unloading robot integrates information from different sensors through multi-source data fusion technology, improves the robot's autonomous positioning and navigation capabilities, thereby ensuring that the robot can accurately move, avoid obstacles and plan paths in complex environments to avoid collisions or misoperations. At the same time, by integrating semantic recognition, gravity prior, grid matching, and path planning, while ensuring computing efficiency, the robot's positioning accuracy and stability in complex terrain are further improved, ensuring that the robot can accurately perform loading and unloading operations and can automatically adjust its operating strategies according to environmental changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A schematic diagram of a process flow of a method for autonomous navigation in a greenhouse for a seedling loading and unloading robot in one embodiment;

[0045] Figure 2 A schematic diagram of a flow chart of determining a coarse positioning position of semantics in an embodiment;

[0046] Figure 3 A schematic diagram of a process of identifying real-time obstacles in one embodiment;

[0047] Figure 4 is a schematic diagram of a flow chart of determining the relative position relationship between the robot and the ground in one embodiment;

[0048] Figure 5 It is a schematic diagram of a process of accurately matching local structural features in one embodiment;

[0049] Figure 6 A schematic diagram of a process for determining path planning in an embodiment;

[0050] Figure 7 It is a schematic diagram of the structure of a greenhouse autonomous navigation device of a seedling loading and unloading robot in one embodiment;

[0051] Figure 8 The figure is a diagram showing the internal structure of an electronic device according to an embodiment. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Combine the following Figure 1-Figure 8 The invention describes the autonomous navigation method, device and electronic equipment for a seedling loading and unloading robot in a greenhouse.

[0054] like Figure 1 As shown, in one embodiment, a method for autonomous navigation in a greenhouse of a seedling loading and unloading robot comprises the following steps:

[0055] Step S100, acquiring multi-source sensor data and semantic information of the breeding workshop, and determining the coarse positioning position of the semantics according to environmental characteristics to form a preliminary semantic positioning framework.

[0056] It should be noted that the main body of the seedling loading and unloading robot includes: a drive unit, which is responsible for providing mobility; a control unit, which is responsible for logical decision-making and calculations; a sensor unit, which is used to perceive the external environment; and a communication module, which ensures real-time data exchange with other systems. Among them, multi-sensor fusion perception is the basis of the navigation and positioning system, which can provide the robot with accurate and rich environmental information.

[0057] Through multi-source data fusion, such as combining data from multiple sensors such as image sensors, lidar, IMU (Inertial Measuring Unit), the robot's positioning accuracy and navigation stability in the greenhouse environment are effectively improved, especially in the closed and signal-complex internal environment of the greenhouse, ensuring the reliability of the autonomous navigation system.

[0058] Step S200, based on the ground segmentation and plane fitting algorithm, determine the relative position relationship between the robot and the ground, the relative position relationship includes the pitch angle, the yaw angle and the offset in the Z-axis direction.

[0059] Considering the gravity prior of point cloud data, the ground features are analyzed through ground segmentation and plane fitting algorithms to facilitate accurate estimation of the relative position relationship between the robot and the ground, thereby ensuring the initial accuracy of positioning.

[0060] Step S300 , combining the semantic rough positioning position and the local perception data collected by the robot in real time, performs more in-depth feature extraction on the details of the surrounding environment.

[0061] Further improve the positioning accuracy and reduce errors.

[0062] Step S400, based on the grid projection method, accurately match the local structural features to generate the global posture of the robot relative to the point cloud map and determine the path planning.

[0063] By accurately matching local structural features, it is convenient to improve the accuracy of the coarse pose relative to semantic positioning. Specifically, grid projection is used to measure the offset on the X and Y axes and fine-tune the yaw angle to correct the position deviation of the robot caused by terrain changes or other factors. Ultimately, by accurately matching semantic information and sensor perception data, the robot's high-precision position and posture can be obtained in real time to meet the adaptive navigation needs in the complex environment of the breeding workshop. This method integrates multiple means of semantic recognition, gravity prior and grid matching. While ensuring computational efficiency, it further improves the positioning accuracy and stability of the robot in complex terrain, providing a reliable pose solution basis for loading and unloading tasks in the breeding workshop.

[0064] The greenhouse autonomous navigation method of the seedling loading and unloading robot integrates information from different sensors through multi-source data fusion technology to enhance the robot's autonomous positioning and navigation capabilities, thereby ensuring that the robot can accurately move, avoid obstacles and plan paths in complex environments to avoid collisions or misoperations. At the same time, by integrating semantic recognition, gravity prior, grid matching, and path planning, while ensuring computational efficiency, the robot's positioning accuracy and stability in complex terrain are further improved, ensuring that the robot can accurately perform loading and unloading operations and can automatically adjust its operating strategies according to environmental changes.

[0065] In this embodiment, see Figure 2 , obtain multi-source sensor data and semantic information of the breeding workshop, and determine the coarse positioning position of the semantics according to the environmental characteristics to form a preliminary semantic positioning framework, including:

[0066] Step S110, acquiring and fusing multi-source sensor data of lidar, RGB-D camera, deep learning driven visual sensor and IMU.

[0067] Through multi-source data fusion technology (such as data-level fusion, feature-level fusion or decision-level fusion), the data of multiple sensors such as lidar, RGB-D camera, deep learning-driven visual sensor and IMU are comprehensively utilized to enhance the robot's perception of the environment, ensure that the robot can identify the surrounding plants, obstacles and other important features in real time, provide accurate environmental perception information, and support autonomous decision-making and behavior execution.

[0068] At the same time, through the collaboration of visual recognition technology and robotic arms, the robot can accurately identify, grab and place seedlings. Multi-source data fusion can help the robot perceive the location and status of seedlings in real time, ensuring that the robot can accurately load and unload materials and automatically adjust its operation strategy according to environmental changes.

[0069] Step S120, extracting features from multi-source sensor data through a convolutional neural network, and constructing a high-dimensional environmental feature space in combination with the geometric information of the lidar.

[0070] Significantly improves data accuracy and environmental understanding.

[0071] Step S130, based on the YOLO target detection algorithm, identify real-time obstacles and determine the coarse positioning position of the semantics.

[0072] It is convenient to ensure that robots can operate safely and efficiently in complex agricultural environments.

[0073] In this embodiment, see Figure 3 , based on the YOLO target detection algorithm, identify real-time obstacles, including:

[0074] Step S131, pre-define dynamic objects in the loading and unloading scene.

[0075] Dynamic objects in the loading and unloading scenes include people and plants on the side that appear in the scene.

[0076] Step S132, input the image frame into the YOLOv8 detector to obtain object category, bounding box and confidence information.

[0077] YOLOv8 is the latest version of the YOLO series of real-time object detectors, representing the cutting-edge development of the series in the field of object detection.

[0078] Step S133: identifying and removing dynamic points based on target detection technology.

[0079] By identifying and removing dynamic points, it helps to improve the accuracy of the algorithm, reduce noise, optimize resource utilization, and enhance the stability of the system, reducing its negative impact on SLAM pose estimation and map quality.

[0080] Step S134, construct a semantic map, merge the object categories identified by YOLOv8 with the map data generated by SLAM, and mark the object categories at specific locations.

[0081] For example, identified pedestrians are marked as obstacles, and seedling loading and unloading areas are defined as work areas that need to be approached.

[0082] By representing the relationship between features in the environment as a graph model and performing information dissemination and consistency verification through graph optimization, this method significantly improves the accuracy of map construction in a dynamically changing agricultural environment.

[0083] In response to the needs of autonomous navigation of robots in seedling breeding workshops, a variety of sensors and advanced SLAM algorithms are innovatively integrated, which not only rely on lidar data, but also integrate multi-source data such as cameras and IMUs to complete the construction of high-precision point cloud maps of robots in seedling planting areas, as well as key functions such as stable robot positioning and obstacle recognition. Considering the problems of low computational efficiency, insufficient positioning accuracy and robustness of 3D lidar in the global positioning process, especially the positioning errors that may be caused by viewpoint changes, dynamic object occlusion or changes in weather conditions, the positioning method of semantic maps is introduced, which can effectively alleviate the accuracy impact caused by these factors, thereby improving the accuracy and stability of navigation and meeting the needs of autonomous navigation in the breeding workshop environment.

[0084] In this embodiment, see Figure 4 , based on ground segmentation and plane fitting algorithms, determine the relative position relationship between the robot and the ground, including:

[0085] Step S210, obtaining a 3D point cloud map generated by the robot chassis laser radar.

[0086] It is convenient to use the 3D point cloud map generated by the robot chassis laser radar as a positioning reference in the seedling breeding workshop.

[0087] Step S220 , based on the Patchwork ground segmentation algorithm and the point cloud data collected by the 3D LiDAR sensor, the point cloud is divided into concentric areas of different densities, and a plane fitting is performed in each area.

[0088] Ground probability estimation effectively reduces false alarms and significantly improves the robot's positioning accuracy in complex terrain.

[0089] Step S230, estimating the pitch angle, yaw angle and Z-axis offset of the robot relative to the ground.

[0090] The pitch angle, yaw angle and Z-axis offset are estimated through ground segmentation and plane fitting method to ensure the accuracy of pose estimation.

[0091] In this embodiment, see Figure 5 Based on the grid projection method, the local structural features are accurately matched to generate the global position of the robot relative to the point cloud map, including:

[0092] In step S411 , the point cloud information is processed into an occupancy grid according to a probabilistic voxel conversion, and a bird's-eye view of the environment is generated by projecting the probabilistic grid.

[0093] The bird's-eye view provides an intuitive representation of the environment layout, which helps the robot understand the surrounding environment, perform path planning and obstacle avoidance. The occupancy grid and bird's-eye view can be used to build and update the environment map, providing basic data for automatic navigation and positioning. At the same time, converting point cloud data into occupancy grids and generating a bird's-eye view can reduce the amount of data to a certain extent and reduce storage and transmission costs.

[0094] Step S412, dividing the environment into regular grids, each cell representing a small area.

[0095] Point cloud data usually contains a large number of discrete three-dimensional points that represent the surfaces of objects in the environment. In order to process and analyze this data more efficiently, point clouds can be converted into occupancy grids. An occupancy grid is a data structure that divides a three-dimensional space into multiple regular squares (voxels). Each voxel represents a small area in the space and contains a probability value indicating whether the area is occupied.

[0096] Step S413, based on the multi-sensor data, marking the grid area occupied by the obstacle, and marking the grid area on the robot path as an idle area.

[0097] It is convenient for subsequent path planning and obstacle avoidance detection.

[0098] Step S414 , performing horizontal plane projection on the point cloud voxel set to generate a 2D grid, and calculating the voxel accumulation probability in the vertical direction.

[0099] By performing horizontal plane projection on the point cloud voxel set and calculating the voxel accumulation probability in the vertical direction, it is convenient to reflect the characteristic distribution of the spatial area.

[0100] In step S415, the generated grid map is combined with the real-time perception information and the coarse positioning data, and the SuperGlue algorithm based on the attention mechanism is used to screen the matching point pairs with high similarity.

[0101] SuperGlue is a feature matching algorithm based on graph convolutional neural networks. The algorithm estimates the matching relationship between feature points by solving a differentiable optimal transmission problem, and its cost is predicted by the graph neural network. The core technologies of SuperGlue include the attention graph neural network and the optimal transmission matching module, which can adaptively enhance the global information of feature points and improve the number and accuracy of matches.

[0102] Step S416, estimating the X-axis and Y-axis offsets and yaw angle of the robot according to the posture change, and determining the global posture of the robot relative to the point cloud map.

[0103] The robot's X-axis and Y-axis offsets and yaw angles are estimated based on the posture changes. Combined with the previously estimated pitch angle, yaw angle and Z-axis offset, the robot's global posture relative to the point cloud map is finally obtained. This enables the robot to achieve adaptive navigation in a dynamic and complex breeding workshop environment, providing reliable posture information and path guarantees for autonomous seedling loading and unloading operations.

[0104] Combined with incremental 3D reconstruction technology, a high-resolution three-dimensional environmental model is constructed in real time through a voxel grid generation algorithm, which can effectively reflect crop growth changes and environmental dynamics, providing an important basis for subsequent decision-making.

[0105] In this embodiment, see Figure 6 , determine the path planning, including:

[0106] Step S421, determining the motion constraints of the robot according to the kinematic model describing the motion mode and mechanical structure of the robot, so as to limit the optional motion trajectories of the robot in path planning.

[0107] The path planning problem is modeled as an optimization problem, the goal of which is to find a trajectory suitable for the robot's movement. Trajectory planning combined with the robot's kinematics is to ensure that the robot follows the limitations of its dynamics and mechanical structure during movement to achieve safe and smooth movement.

[0108] Step S422: Generate a set of potential trajectory candidates based on the robot's kinematic model and motion constraints.

[0109] The generated potential trajectory candidate set makes it easier to subsequently select the best path trajectory from the potential trajectory candidate set.

[0110] Step S423, evaluate the pros and cons of each candidate trajectory and determine the best operating trajectory in the breeding workshop.

[0111] Define a cost function to evaluate the pros and cons of each candidate trajectory. The cost function takes into account path length, time, energy consumption, safety, and other factors. The optimization algorithm will try to minimize the cost function to select the best operating trajectory in the breeding workshop and control the robot to move stably in the seedling planting area. The trajectory planning algorithm usually needs to be updated in real time while the robot moves to adapt to changes in the dynamic environment or obstacles. Real-time updates require timely replanning of the path to ensure the safety and efficiency of the robot. The cost function can be expressed as the following weighted sum form:

[0112] C(P)=λ1L(P)+λ2T(P)+λ3E(P)+λ4S(P)+λ5O(P),

[0113] Among them, C(P) is the overall cost function, which represents the evaluation of the candidate trajectory P;

[0114] is the path length cost, which represents the total length of trajectory P;

[0115] T(P)=L(P) / V avg , is the time cost, which represents the time required to complete trajectory P;

[0116] is the energy cost, which represents the energy consumed by the robot to execute trajectory P;

[0117] is the safety cost, which indicates the safety of trajectory P, such as the distance to obstacles, collision risk, etc.;

[0118] is the other factor cost, which represents other factors that may affect trajectory selection, such as the smoothness of the trajectory, the number of turns, etc.;

[0119] λ1, λ2, λ3, λ4, and λ5 are weight coefficients used to adjust the relative importance of each cost item in the total cost.

[0120] Based on the above positioning results, path planning based on grid maps is further carried out to realize the autonomous navigation of the seedling adaptive loading and unloading robot in the breeding workshop environment. The path planning problem is modeled as an optimization problem, aiming to find the optimal trajectory suitable for the robot's movement. In the planning process, the robot's kinematic characteristics are combined to ensure that the trajectory not only meets the constraints of dynamics and mechanical structure, but also achieves safe and smooth movement.

[0121] First, a kinematic model describing the robot's motion mode and mechanical structure is constructed. The constraints of the robot's motion are determined through this model to limit the optional motion trajectories in path planning. On this basis, a potential trajectory candidate set is generated based on the kinematic model and motion constraints, and a cost function is defined for each candidate trajectory. The cost function comprehensively considers path length, time, energy consumption, safety and other key factors as the evaluation criteria of the optimization algorithm. The optimization algorithm will try to minimize the cost function and select the trajectory with the best balance in the planting area, so as to achieve stable movement of the robot in the breeding workshop.

[0122] To adapt to dynamic environments and potential obstacles, the trajectory planning algorithm adopts a real-time update mechanism to quickly adjust the planned path according to environmental changes, ensuring the safety and efficiency of the robot during operation.

[0123] In this embodiment, determining the path planning further includes:

[0124] Step S424, based on the real-time data from the multi-source sensors, identify potential collision risks and generate response strategies, which include speed adjustment, path replanning, and emergency stop when necessary.

[0125] By identifying potential collision risks and generating response strategies, it is convenient to ensure that the seedling adaptive loading and unloading robot avoids collisions and maximizes movement efficiency in the breeding workshop, thereby achieving safe and stable autonomous navigation in the complex breeding workshop environment, ensuring that the robot completes the task efficiently while complying with the collision avoidance strategy.

[0126] The greenhouse autonomous navigation method of the seedling loading and unloading robot integrates multiple sensors and advanced SLAM algorithms to meet the autonomous navigation needs of the seedling adaptive loading and unloading robot in the complex environment of the breeding workshop. By integrating multi-source sensor data such as lidar, camera and IMU, the framework improves the robustness of the navigation system and provides technical support for high-precision point cloud map construction, robot precise positioning and obstacle recognition.

[0127] The robot's front-end odometer integrates multiple methods such as GICP, FLOAM, and FastLIO to obtain real-time motion information and support precise positioning. The GICP algorithm is used to complete point cloud registration, match the newly acquired point cloud data with the local map, and estimate the dynamic position of the robot in the global coordinate system. The back-end optimization is based on the g2o framework, which is used to solve nonlinear optimization problems. It improves the accuracy of the ground by constructing a pose graph, performing loop detection, and keyframe management. Figure 1 In addition, by introducing multiple constraints of the ground, loop and GPS into the system, the cumulative errors caused by narrow environments and light problems are effectively reduced, thereby significantly reducing drift and improving global positioning accuracy.

[0128] In order to overcome the blind spots of lidar in the breeding workshop environment, the framework also combines multiple sensors such as vision, laser, infrared and ultrasonic to enhance the system's comprehensive perception of the surrounding environment, thereby improving the accuracy and robustness of obstacle detection. In addition, multi-sensor fusion provides the robot with an efficient emergency feedback mechanism to ensure timely response and even emergency stop in emergencies to ensure operational safety, and realizes comprehensive detection of obstacles and surrounding targets in the breeding workshop, supports real-time mapping and precise positioning of the area, and further improves the robot's navigation performance in the complex environment of the breeding workshop, providing a solid foundation for autonomous seedling loading and unloading operations.

[0129] The following is a description of the autonomous navigation device for a seedling loading and unloading robot in a greenhouse provided by the present invention. The autonomous navigation device for a seedling loading and unloading robot in a greenhouse described below and the autonomous navigation method for a seedling loading and unloading robot in a greenhouse described above can be referenced to each other.

[0130] like Figure 7 As shown, in one embodiment, a seedling loading and unloading robot greenhouse autonomous navigation device includes an acquisition module 710, a determination module 720, an extraction module 730 and a matching module 740.

[0131] The acquisition module 710 is used to acquire multi-source sensor data and semantic information of the breeding workshop, and determine the coarse positioning position of the semantics according to environmental characteristics to form a preliminary semantic positioning framework.

[0132] The determination module 720 is used to determine the relative position relationship between the robot and the ground based on the ground segmentation and plane fitting algorithm. The relative position relationship includes the pitch angle, the yaw angle and the offset in the Z-axis direction.

[0133] The extraction module 730 is used to combine the semantic coarse positioning position and the local perception data collected by the robot in real time to perform more in-depth feature extraction on the details of the surrounding environment.

[0134] The matching module 740 is used to accurately match local structural features based on the grid projection method to generate the global position of the robot relative to the point cloud map and determine the path planning.

[0135] In this embodiment, the acquisition module 710 is specifically used for:

[0136] Acquire and fuse multi-source sensor data from LiDAR, RGB-D cameras, deep learning-driven vision sensors, and IMUs;

[0137] The convolutional neural network is used to extract features from multi-source sensor data and to build a high-dimensional environmental feature space in combination with the geometric information of the lidar.

[0138] Based on the YOLO target detection algorithm, real-time obstacles are identified and the semantic rough positioning is determined.

[0139] In this embodiment, based on the YOLO target detection algorithm, real-time obstacles are identified, specifically for:

[0140] Predefine dynamic objects in loading and unloading scenes;

[0141] Input the image frame into the YOLOv8 detector to obtain object category, bounding box and confidence information;

[0142] Based on target detection technology, dynamic points are identified and eliminated;

[0143] Build a semantic map, fuse the object categories identified by YOLOv8 with the map data generated by SLAM, and mark the object categories at specific locations.

[0144] In this embodiment, the relative position relationship between the robot and the ground is determined based on the ground segmentation and plane fitting algorithm, which is specifically used for:

[0145] Get the 3D point cloud map generated by the robot chassis laser radar;

[0146] Based on the Patchwork ground segmentation algorithm and the point cloud data collected by the 3D LiDAR sensor, the point cloud is divided into concentric areas of different densities, and plane fitting is performed in each area;

[0147] Estimate the pitch, yaw, and Z-axis offset of the robot relative to the ground.

[0148] In this embodiment, based on the grid projection method, the local structural features are accurately matched to generate the global position of the robot relative to the point cloud map, which is specifically used for:

[0149] Process the point cloud information into an occupancy grid according to a probabilistic voxel transformation and generate a bird's-eye view of the environment by projecting the probabilistic grid;

[0150] Divide the environment into a regular grid, with each cell representing a small area;

[0151] Based on multi-sensor data, the grid areas occupied by obstacles are marked, and the grid areas on the robot path are marked as free areas;

[0152] Perform horizontal plane projection on the point cloud voxel set to generate a 2D grid and calculate the voxel accumulation probability in the vertical direction;

[0153] The generated grid map is combined with real-time perception information and coarse positioning data, and the SuperGlue algorithm based on the attention mechanism is used to select matching point pairs with high similarity.

[0154] According to the posture change, the X-axis and Y-axis offsets and yaw angle of the robot are estimated, and the global posture of the robot relative to the point cloud map is determined.

[0155] In this embodiment, the path planning is determined, specifically for:

[0156] According to the kinematic model describing the robot's motion mode and mechanical structure, the robot's motion constraints are determined to limit the robot's optional motion trajectories in path planning;

[0157] Generate a set of potential trajectory candidates based on the robot's kinematic model and motion constraints;

[0158] Evaluate the pros and cons of each candidate trajectory and determine the best trajectory to run in the breeding room.

[0159] In this embodiment, determining the path planning is also specifically used for:

[0160] Based on real-time data from multi-source sensors, potential collision risks are identified and response strategies are generated, including speed adjustment, path replanning, and emergency stop when necessary.

[0161] The seedling loading and unloading robot greenhouse autonomous navigation device has the following advantages:

[0162] 1. Multi-source data fusion improves navigation accuracy

[0163] Traditional greenhouse seedling robot systems may rely on a single data source, such as simple GPS or lidar for navigation, but these technologies may be limited by signal interference in the greenhouse environment (such as weak or failed GPS signals) or complex environmental changes (such as light and temperature and humidity). The present invention effectively improves the positioning accuracy and navigation stability of the robot in the greenhouse environment through multi-source data fusion (such as combining data from multiple sensors such as image sensors, lidar, inertial measurement units (IMUs)), especially in the closed and signal-complex internal environment of the greenhouse, ensuring the reliability of the autonomous navigation system.

[0164] 2. Strong adaptability and ability to cope with complex environments

[0165] Multi-source data fusion technology can help robots adapt to the dynamically changing greenhouse environment, such as obstacles and environmental changes in the greenhouse. The robot can automatically adjust the path planning by processing multiple sensor data in real time, avoid obstacles, and ensure efficient completion of loading and unloading tasks. Compared with the navigation system of a single sensor, the present invention can effectively overcome the complexity of the greenhouse environment and improve the robot's adaptive ability.

[0166] 3. Intelligent decision-making and path planning optimization

[0167] Based on the fusion of multi-source data, the system can make intelligent decisions and path planning. By integrating multiple data (such as visual data, lidar data, IMU data, etc.), the system can identify and locate crops, equipment, obstacles, etc. in the greenhouse in real time, formulate the optimal navigation path, reduce unnecessary path deviations, ensure that the robot can load and unload materials in a more efficient and precise manner, avoid collisions and redundant paths, and improve overall work efficiency.

[0168] 4. Improve robot autonomy and reduce human intervention

[0169] Traditional greenhouse management robots often require human intervention or rely on external control systems. The navigation system based on multi-source data fusion can achieve a higher degree of autonomy and reduce the need for human intervention. The robot can automatically complete tasks from seedling transportation to loading and unloading without relying on external instructions, which improves work efficiency, reduces labor costs, and improves the automation level of smart agriculture.

[0170] 5. Enhanced robustness and fault tolerance

[0171] The multi-source data fusion system can effectively enhance the robustness of the system by integrating information from different sensors, especially when a single sensor fails or has an error, the data from other sensors can make up for the shortcomings of the failed sensor, thereby improving the fault tolerance of the entire navigation system. In a greenhouse environment, this robustness is particularly important, which can ensure the long-term stable operation of the robot.

[0172] Figure 8 An example of a physical structure diagram of an electronic device is shown. The electronic device may be a smart terminal, and its internal structure diagram may be as follows: Figure 8As shown. The electronic device includes a processor, a memory and a network interface connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for autonomous navigation of a seedling loading and unloading robot in a greenhouse is implemented, and the method includes:

[0173] Acquire multi-source sensor data and semantic information of the breeding workshop, and determine the coarse location of semantics based on environmental characteristics to form a preliminary semantic positioning framework;

[0174] Based on the ground segmentation and plane fitting algorithm, the relative position relationship between the robot and the ground is determined. The relative position relationship includes the pitch angle, yaw angle and offset in the Z-axis direction.

[0175] Combine the semantic coarse positioning position and the local perception data collected by the robot in real time to perform more in-depth feature extraction of the details of the surrounding environment;

[0176] Based on the grid projection method, the local structural features are accurately matched to generate the global pose of the robot relative to the point cloud map and determine the path planning.

[0177] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present invention, and does not constitute a limitation on the electronic device to which the scheme of the present invention is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0178] On the other hand, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, implements a method for autonomous navigation of a seedling loading and unloading robot in a greenhouse, the method comprising:

[0179] Acquire multi-source sensor data and semantic information of the breeding workshop, and determine the coarse location of semantics based on environmental characteristics to form a preliminary semantic positioning framework;

[0180] Based on the ground segmentation and plane fitting algorithm, the relative position relationship between the robot and the ground is determined. The relative position relationship includes the pitch angle, yaw angle and offset in the Z-axis direction.

[0181] Combine the semantic coarse positioning position and the local perception data collected by the robot in real time to perform more in-depth feature extraction of the details of the surrounding environment;

[0182] Based on the grid projection method, the local structural features are accurately matched to generate the global pose of the robot relative to the point cloud map and determine the path planning.

[0183] In another aspect, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, a method for autonomous navigation of a seedling loading and unloading robot in a greenhouse is implemented, the method comprising:

[0184] Acquire multi-source sensor data and semantic information of the breeding workshop, and determine the coarse location of semantics based on environmental characteristics to form a preliminary semantic positioning framework;

[0185] Based on the ground segmentation and plane fitting algorithm, the relative position relationship between the robot and the ground is determined. The relative position relationship includes the pitch angle, yaw angle and offset in the Z-axis direction.

[0186] Combine the semantic coarse positioning position and the local perception data collected by the robot in real time to perform more in-depth feature extraction of the details of the surrounding environment;

[0187] Based on the grid projection method, the local structural features are accurately matched to generate the global pose of the robot relative to the point cloud map and determine the path planning.

[0188] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0189] By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0190] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0191] The above-described embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A method for autonomous navigation of a seedling loading and unloading robot in a greenhouse, characterized in that: The method comprises: Acquire multi-source sensor data and semantic information of the breeding workshop, and determine the coarse location of semantics based on environmental characteristics to form a preliminary semantic positioning framework; Based on the ground segmentation and plane fitting algorithm, the relative position relationship between the robot and the ground is determined. The relative position relationship includes the pitch angle, yaw angle and offset in the Z-axis direction. Combine the semantic coarse positioning position and the local perception data collected by the robot in real time to perform more in-depth feature extraction of the details of the surrounding environment; Based on the grid projection method, the local structural features are accurately matched to generate the global pose of the robot relative to the point cloud map and determine the path planning.

2. The method for autonomous navigation of a seedling loading and unloading robot in a greenhouse according to claim 1, characterized in that: The method of acquiring multi-source sensor data and semantic information of the breeding workshop and determining the coarse positioning position of the semantics according to environmental characteristics to form a preliminary semantic positioning framework includes: Acquire and fuse multi-source sensor data from LiDAR, RGB-D cameras, deep learning-driven vision sensors, and IMUs; The convolutional neural network is used to extract features from multi-source sensor data and to build a high-dimensional environmental feature space in combination with the geometric information of the lidar. Based on the YOLO target detection algorithm, real-time obstacles are identified and the semantic rough positioning is determined.

3. The method for autonomous navigation of a seedling loading and unloading robot in a greenhouse according to claim 2, characterized in that: The YOLO target detection algorithm is used to identify real-time obstacles, including: Predefine dynamic objects in loading and unloading scenes; Input the image frame into the YOLOv8 detector to obtain object category, bounding box and confidence information; Based on target detection technology, dynamic points are identified and eliminated; Build a semantic map, fuse the object categories identified by YOLOv8 with the map data generated by SLAM, and mark the object categories at specific locations.

4. The method for autonomous navigation of a seedling loading and unloading robot in a greenhouse according to claim 3, characterized in that: The method of determining the relative position relationship between the robot and the ground based on the ground segmentation and plane fitting algorithm includes: Get the 3D point cloud map generated by the robot chassis laser radar; Based on the Patchwork ground segmentation algorithm and the point cloud data collected by the 3D LiDAR sensor, the point cloud is divided into concentric areas of different densities, and plane fitting is performed in each area; Estimate the pitch, yaw, and Z-axis offset of the robot relative to the ground.

5. The method for autonomous navigation of a seedling loading and unloading robot in a greenhouse according to claim 4, characterized in that: The grid projection method is based on accurately matching local structural features to generate the global position of the robot relative to the point cloud map, including: Process the point cloud information into an occupancy grid according to a probabilistic voxel transformation and generate a bird's-eye view of the environment by projecting the probabilistic grid; Divide the environment into a regular grid, with each cell representing a small area; Based on multi-sensor data, the grid areas occupied by obstacles are marked, and the grid areas on the robot path are marked as free areas; Perform horizontal plane projection on the point cloud voxel set to generate a 2D grid and calculate the voxel accumulation probability in the vertical direction; The generated grid map is combined with real-time perception information and coarse positioning data, and the SuperGlue algorithm based on the attention mechanism is used to select matching point pairs with high similarity. According to the posture change, the X-axis and Y-axis offsets and yaw angle of the robot are estimated, and the global posture of the robot relative to the point cloud map is determined.

6. The method for autonomous navigation in a greenhouse for a seedling loading and unloading robot according to any one of claims 1 to 5, characterized in that: The determining of the path planning includes: According to the kinematic model describing the robot's motion mode and mechanical structure, the robot's motion constraints are determined to limit the robot's optional motion trajectories in path planning; Generate a set of potential trajectory candidates based on the robot's kinematic model and motion constraints; Evaluate the pros and cons of each candidate trajectory and determine the best trajectory to run in the breeding room.

7. The method for autonomous navigation of a seedling loading and unloading robot in a greenhouse according to claim 6, characterized in that: The determining of the path planning further includes: Based on real-time data from multi-source sensors, potential collision risks are identified and response strategies are generated, including speed adjustment, path replanning, and emergency stop when necessary.

8. A seedling loading and unloading robot greenhouse autonomous navigation device, comprising: The acquisition module is used to obtain multi-source sensor data and semantic information of the breeding workshop, and determine the coarse positioning position of the semantics according to environmental characteristics to form a preliminary semantic positioning framework; A determination module is used to determine the relative position relationship between the robot and the ground based on ground segmentation and plane fitting algorithms. The relative position relationship includes the pitch angle, yaw angle, and offset in the Z-axis direction. The extraction module is used to combine the semantic coarse positioning position and the local perception data collected by the robot in real time to perform more in-depth feature extraction of the details of the surrounding environment; The matching module is used to accurately match local structural features based on the grid projection method to generate the global pose of the robot relative to the point cloud map and determine the path planning.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the greenhouse autonomous navigation method of the seedling loading and unloading robot according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, the steps of the greenhouse autonomous navigation method of the seedling loading and unloading robot described in any one of claims 1 to 7 are implemented.

Images