Multi-sensor fusion greenhouse mobile chassis navigation control system

By fusing pseudo-laser data with lidar data and using Kalman filter and A* algorithm, the inaccurate acquisition problem caused by instability in depth data in the navigation control system is solved, and the accuracy and stability of the navigation system are improved, and the accuracy and stability of the navigation system are enhanced. The robustness and fault tolerance of the system.

CN118938728AInactive Publication Date: 2024-11-12ANHUI SCI & TECH UNIV +1
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410844130.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-27
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing navigation control system leads to inaccurate data acquisition when depth data is unstable, affecting the accuracy and stability of the navigation system.

Method used

The pseudo-laser data is fused with lidar data through the data fusion module, and the timestamp synchronization algorithm and extended Kalman filter are used to establish a state space model, build an environment map and perform path planning, and combine the A* algorithm to generate the optimal path.

Benefits of technology

It improves the accuracy of data acquisition and the stability of the navigation system, enhances the robustness and fault tolerance of the system, and ensures the security and accurate positioning of the mobile platform.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118938728A_ABST
    Figure CN118938728A_ABST
Patent Text Reader

Abstract

The invention relates to the field of data fusion, and discloses a multi-sensor fusion greenhouse mobile chassis navigation control system, which is used for solving the problem of inaccurate data acquisition when depth data of a camera is unstable, and comprises a data fusion module, an environment map construction module, a path planning module and a navigation and obstacle avoidance module, the data fusion module is used for fusing pseudo laser data and laser radar data through data conversion, receiving fused data transmitted by the data fusion module, constructing an environment map according to the fused data, receiving the environment map transmitted by the environment map construction module, performing path planning according to a destination, receiving path planning information transmitted by the path planning module, and performing path planning according to the destination. Navigation and obstacle avoidance are carried out, so that the accuracy of acquired data and the stability of a navigation system are effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of agricultural equipment, and in particular to a multi-sensor fusion greenhouse mobile chassis navigation control system. Background Art

[0002] Room temperature mobile chassis navigation control system refers to a navigation control system applied to robots or other mobile platforms in a room temperature environment. The system integrates multiple sensors and advanced algorithms to ensure that the mobile platform can accurately locate, autonomously navigate and avoid obstacles in indoor or similar environments.

[0003] Multi-sensor fusion refers to the comprehensive processing of data from multiple different types of sensors to obtain more accurate, reliable and comprehensive information. This method utilizes the advantages of each sensor and makes up for the shortcomings of a single sensor, thereby improving the overall performance and robustness of the system.

[0004] The existing technology has the following deficiencies: In the existing navigation control system, data acquisition is performed by collecting surrounding image data through a camera, and feature extraction is performed on the image data. A simultaneous localization and mapping (SLAM) algorithm is used to build an environmental map in real time and locate the mobile chassis. Based on the constructed environmental map and the current position, a path planning algorithm is used to generate the optimal path from the starting point to the target point. However, this data acquisition method will lead to inaccurate data acquisition when the depth data of the camera is unstable.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a greenhouse mobile chassis navigation control system with multi-sensor fusion to solve the problems in the above-mentioned background technology.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A multi-sensor fusion greenhouse mobile chassis navigation control system includes a data fusion module, an environmental map construction module, a path planning module, and a navigation and obstacle avoidance module. The modules are connected through signals. The data processing steps between the modules are as follows:

[0009] A data fusion module, used for fusing pseudo laser data with lidar data through data conversion, and transmitting the fused data to the environment map construction module;

[0010] An environment map construction module is used to receive the fused data transmitted by the data fusion module, construct an environment map according to the fused data, and transmit the environment map to the path planning module;

[0011] The path planning module is used to receive the environment map transmitted by the environment map construction module, plan the path according to the destination, and transmit the path planning information to the navigation and obstacle avoidance module;

[0012] The navigation and obstacle avoidance module is used to receive the path planning information transmitted by the path planning module and perform navigation and obstacle avoidance.

[0013] Preferably, the step of fusing the pseudo laser data with the laser radar data through data conversion is:

[0014] Use an RGB-D camera to obtain a depth image, convert the depth image into pseudo laser data, convert the depth value into a two-dimensional point cloud similar to the lidar through projection, and obtain the two-dimensional point cloud data directly from the lidar;

[0015] Use the timestamp synchronization algorithm to ensure that the pseudo laser data and the lidar data are aligned at the same timestamp, filter the pseudo laser data and the lidar data, remove noise and handle outliers;

[0016] Perform external calibration between sensors for pseudo laser data and lidar data, obtain coordinate transformation matrix, and ensure that pseudo laser data and lidar data are in the same coordinate system;

[0017] The pseudo laser data and lidar data are fused using an extended Kalman filter to build a state space model, and the state estimation is updated using the pseudo laser data and lidar data.

[0018] Preferably, the step of converting the depth image into pseudo laser data is:

[0019] Get depth images from RGB-D cameras;

[0020] Get the camera internal parameters, including focal length f x and f y And the principal point coordinates c x and c y ,These parameters are obtained by camera calibration;

[0021] Each pixel coordinate (u, v) in the depth image and its depth value D(u, v) are converted into a three-dimensional point (X, Y, Z). The calculation formula is: Z = D(u,v);

[0022] Convert the three-dimensional point cloud (X, Y, Z) into two-dimensional pseudo laser data, ignore the Y axis, and only keep the X and Y coordinates to obtain pseudo laser data (X, Y);

[0023] The generated pseudo laser data are filtered to remove points exceeding the maximum distance;

[0024] The point cloud data is divided into voxel sizes and a representative point is retained in each voxel.

[0025] Preferably, the step of performing external calibration between sensors on the pseudo laser data and the laser radar data to obtain a coordinate transformation matrix is:

[0026] A black and white checkerboard calibration plate is used during the calibration process;

[0027] Place the calibration plate and collect pseudo laser data and lidar data at the same time, and record the calibration plate image collected by each sensor and its corresponding timestamp;

[0028] Extracting feature points from the collected calibration plate image, wherein the feature points include corner points and intersection points inside the chessboard;

[0029] Match the feature points of the pseudo laser data and the laser radar data to establish a corresponding relationship between the feature points;

[0030] Using the known size of the calibration plate and the correspondence between the feature points, the camera calibration algorithm is used to calculate the external calibration parameters between the pseudo laser and the laser radar, wherein the parameters include the rotation matrix R and the translation vector T;

[0031] The rotation matrix R and the translation vector T are combined into a coordinate transformation matrix, and the transformation matrix formula is:

[0032] Preferably, the step of using an extended Kalman filter to fuse pseudo laser data and laser radar data to establish a state space model is:

[0033] Initialize the system state vector x and covariance matrix P, set the process noise covariance matrix Q and the measurement noise covariance matrix R;

[0034] Use the system dynamics model to predict the state, calculate the state transfer matrix F and the control input matrix B, and predict the state vector and covariance matrix. The calculation formula is: P - =FPF T +Q;

[0035] Use the measurement model to update the state, calculate the observation matrix H and the measurement residual y, and calculate the Kalman gain, which is calculated by the formula K = P - H T (HP - H T +R) -1, update the state vector and covariance matrix, the calculation formula is P=(I-KH)P - , returns the updated state vector As an estimate of the system state.

[0036] Preferably, the step of performing route planning according to the destination is:

[0037] Receive an environment map, divide the environment into a series of discrete cells, and mark each cell as passable or impassable;

[0038] Determine the locations of the start and end points on the map, and initialize the open list and closed list;

[0039] Use the A* algorithm to find the shortest path from the start point to the end point in the graph.

[0040] Preferably, the steps of using the A* algorithm to find the shortest path from the starting point to the end point in the graph are:

[0041] Step 1: Define the heuristic function h(x) and the cost function g(x), using Manhattan distance, which is calculated as where x goal Represented as the location of the target node, x current Indicates the position of the node currently being processed;

[0042] Step 2: Calculate the total estimated cost of node x, which is calculated as f(x)=g(x)+h(x), add the starting node to the open list, and set g(start)=0 and f(start)=h(start);

[0043] Step 3: Select the node with the smallest f value from the open list as the current node current. If the current node current is the end point, end the search and build the path;

[0044] Step 4: For each neighbor node neighbor of the current node current, calculate the neighbor node g value, the calculation formula is g(neighbor) = g(current) + the cost from the current node to the neighbor node, and calculate the neighbor node f value, the calculation formula is f(neighbor) = g(neighbor) + h(neighbor);

[0045] Step 5: If the neighbor node is in the closed list and the new g value is larger, skip the neighbor node. If the neighbor node is not in the open list, or the new g value is smaller, add the neighbor node to the open list, update the parent node of the neighbor node to the current node, and update the g value and f value of the neighbor node.

[0046] Step 6: Move the current node current from the open list to the closed list and repeat steps 2-5 until the end is found or the open list is empty.

[0047] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0048] The pseudo laser data is fused with the lidar data through data conversion, the fused data transmitted by the data fusion module is received, and the environmental map is constructed based on the fused data. The environmental map transmitted by the environmental map construction module is received, and path planning is performed according to the destination. The path planning information transmitted by the path planning module is received, and navigation and obstacle avoidance are performed, which effectively improves the accuracy of the collected data and the stability of the navigation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0050] Figure 1 It is a schematic diagram of the overall structure of the present invention.

[0051] Figure 2 This is a system structure diagram of the present invention. DETAILED DESCRIPTION

[0052] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present disclosure will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art.

[0053] The present invention provides Figure 1 A multi-sensor fusion greenhouse mobile chassis navigation control system shown includes a data fusion module, an environmental map construction module, a path planning module, and a navigation and obstacle avoidance module. The modules are connected through signals. The data processing steps between the modules are as follows:

[0054] The data fusion module is used to fuse the pseudo laser data with the lidar data through data conversion to improve the accuracy of environmental perception and robot positioning, and transmit the fused data to the environmental map construction module;

[0055] LiDAR and pseudo-laser data provide information from different perspectives and resolutions. Through fusion, more accurate and reliable positioning results can be obtained. LiDAR and pseudo-laser data each have their own advantages. After fusion, they can perceive the environment more comprehensively, including identifying and locating obstacles, detecting environmental changes, etc. A single sensor may be limited by specific conditions (such as LiDAR's performance degrades under strong light). Fusion of multiple sensor data can enhance the robustness and reliability of the system. The fused data can more accurately detect dynamic obstacles, thereby improving the effect of real-time obstacle avoidance and ensuring the safety of the mobile platform. Multi-sensor fusion provides redundancy. When a sensor fails or the data quality degrades, other sensors can still provide useful information and improve the system's fault tolerance.

[0056] In this embodiment, it should be specifically explained that the step of fusing the pseudo laser data with the laser radar data through data conversion is:

[0057] Use an RGB-D camera or a stereo camera to obtain a depth image, and convert the depth image into pseudo laser data. Convert the depth value into a two-dimensional point cloud similar to a laser radar through projection, and obtain two-dimensional point cloud data directly from the laser radar. The RGB-D camera is a camera with depth perception capability, which can simultaneously obtain RGB (red, green, and blue) color images and depth images. Unlike traditional RGB cameras, RGB-D cameras use additional sensors (usually infrared or time-of-flight) to capture the distance information from objects in the scene to the camera, so that each pixel contains not only color information but also distance information. This depth information can provide the three-dimensional spatial structure of objects in the scene, which is of great significance for applications such as robot navigation, virtual reality, augmented reality, and three-dimensional modeling. The depth image is an image that contains not only the color information of each pixel (usually RGB), but also the distance or depth information from each pixel to the camera. The pseudo laser data is a data form similar to laser radar data, but is obtained through an RGB-D camera. Unlike laser radar, pseudo laser data is obtained by converting the depth information in the depth image instead of measuring the laser beam;

[0058] Use a timestamp synchronization algorithm to ensure that the pseudo laser data and the lidar data are aligned at the same timestamp, filter the pseudo laser data and the lidar data, remove noise and outliers, and commonly used filtering methods include mean filtering and Kalman filtering. The timestamp synchronization algorithm is used to ensure that the data generated by different sensors or systems remain consistent in time to facilitate data fusion or other joint processing. The performance of the timestamp synchronization algorithm depends on the accuracy and stability of clock synchronization, so it is necessary to select a clock synchronization protocol or algorithm that suits the system requirements;

[0059] Perform external calibration between sensors for pseudo laser data and lidar data, obtain coordinate transformation matrix, and ensure that pseudo laser data and lidar data are in the same coordinate system;

[0060] The extended Kalman filter is used to fuse pseudo laser data and lidar data, establish a state space model, including state variables such as position and velocity, and update the state estimation using pseudo laser data and lidar data. The extended Kalman filter is a commonly used nonlinear state estimation method used to estimate the state of the system from a series of observation data containing noise. It is an extension of the Kalman filter and is specifically used to process nonlinear system models.

[0061] In this embodiment, it should be specifically explained that the step of converting the depth image into pseudo laser data is:

[0062] Get a depth image from an RGB-D camera or a stereo camera. Assume the depth image is D and has a size of H×W;

[0063] Get the camera internal parameters, including focal length f x and f y And the principal point coordinates c x and c y ,These parameters are usually obtained by camera calibration;

[0064] Each pixel coordinate (u, v) in the depth image and its depth value D(u, v) are converted into a three-dimensional point (X, Y, Z). The calculation formula is: Z = D(u,v);

[0065] Convert the three-dimensional point cloud (X, Y, Z) into two-dimensional pseudo laser data, ignore the Y axis, and only keep the X and Y coordinates to obtain pseudo laser data (X, Y);

[0066] Filter the generated pseudo laser data to remove points that exceed the maximum distance. For example, set a maximum distance threshold and filter out points whose distance exceeds the threshold;

[0067] The point cloud data is segmented according to a certain voxel size, and a representative point (such as the centroid) is retained in each voxel to reduce the number of points.

[0068] In this embodiment, it should be specifically explained that the steps of using the timestamp synchronization algorithm to ensure that the pseudo laser data and the laser radar data are aligned at the same timestamp are:

[0069] Get pseudo laser data and lidar data and record their respective timestamps;

[0070] Find the closest timestamp pair in the two data streams as the initial alignment point;

[0071] Based on the initial alignment point, calculate the timestamp difference between the two data streams;

[0072] Adjust the timestamp of one of the data streams to align it with the timestamp of the other data stream. Assume that the timestamp of the pseudo laser data is t pseudo , the lidar data timestamp is t lidar , the timestamp difference corresponding to the initial alignment point is Δt, and the formula for timestamp synchronization is:

[0073] In this embodiment, it should be specifically explained that the steps of performing external calibration between sensors on the pseudo laser data and the laser radar data and obtaining the coordinate transformation matrix are as follows:

[0074] During the calibration process, a calibration plate of known size is required, usually a black and white checkerboard;

[0075] Place the calibration plate and collect pseudo laser data and lidar data at the same time. Make sure the calibration plate is in the field of view of both sensors, and record the calibration plate image collected by each sensor and its corresponding timestamp;

[0076] Extract feature points from the collected calibration plate image. Commonly used feature points include corner points and intersection points inside the chessboard.

[0077] Match the feature points of the pseudo laser data and the laser radar data to establish a corresponding relationship between the feature points;

[0078] Using the known size of the calibration plate and the correspondence between the feature points, a camera calibration algorithm (such as Zhang's method) is used to calculate the external calibration parameters between the pseudo laser and the lidar, including the rotation matrix R and the translation vector T;

[0079] The rotation matrix R and the translation vector T are combined into a coordinate transformation matrix, which describes the coordinate transformation relationship from the pseudo laser coordinate system to the laser radar coordinate system. The transformation matrix formula is:

[0080] In this embodiment, it should be specifically explained that the steps of using the extended Kalman filter to fuse the pseudo laser data and the laser radar data to establish the state space model are:

[0081] Initialize the system state vector x and covariance matrix P, set the process noise covariance matrix Q and the measurement noise covariance matrix R;

[0082] Use the system dynamics model to predict the state, calculate the state transfer matrix F and the control input matrix B, and predict the state vector and covariance matrix. The calculation formula is: P - =FPF T+Q;

[0083] Use the measurement model to update the state, calculate the observation matrix H and the measurement residual y, and calculate the Kalman gain, which is calculated by the formula K = P - H T (HP - H T +R) -1 , update the state vector and covariance matrix, the calculation formula is P=(I-KH)P - , returns the updated state vector As an estimate of the system state.

[0084] An environment map construction module is used to receive the fused data transmitted by the data fusion module, construct an environment map according to the fused data, and transmit the environment map to the path planning module;

[0085] In this embodiment, it should be specifically explained that the step of constructing the environment map according to the fused data is:

[0086] Initialize a blank map and determine the size and resolution of the map. The map can be a raster map, a point cloud map, or other forms of maps;

[0087] The pseudo laser data and the lidar data are fused into a consistent map using a data fusion algorithm of feature fusion, which is a data fusion method for integrating feature information from different sensors or data sources into a consistent representation. In the process of building an environment map, feature fusion is often used to fuse feature information from multiple sensors into a map to improve the accuracy and richness of the map;

[0088] Continuously update the map based on the robot's movements and real-time changes in sensor data to keep the map synchronized with the environment;

[0089] Optimize the constructed map to improve the accuracy and consistency of the map. The optimization may include closed loop detection and closed loop repair, removal of duplicate information, smoothing of the map, etc. The constructed map is displayed, saved or exported for subsequent navigation, path planning or analysis.

[0090] Pseudo-laser data and LiDAR data each have their own advantages and disadvantages. LiDAR has high accuracy but is limited by field of view and occlusion, while pseudo-laser data has lower accuracy but a wide field of view. By fusing these two types of data, their shortcomings can be compensated and the accuracy and completeness of the map can be improved; fusing pseudo-laser data and LiDAR data can make the map have richer information, including environmental structure and features at different perspectives and resolutions. This additional information can provide a more comprehensive description of the environment, which helps the robot achieve more accurate positioning, navigation, and path planning.

[0091] Since pseudo laser data and lidar data have different characteristics and error sources, fusing them can increase the robustness of the map. The fused map is more adaptable to sensor errors and environmental changes, and can provide a more stable and reliable representation of the environment.

[0092] The path planning module is used to receive the environment map transmitted by the environment map construction module, plan the path according to the destination, and transmit the path planning information to the navigation and obstacle avoidance module;

[0093] In this embodiment, it should be specifically explained that the steps of performing path planning according to the destination are:

[0094] Receive an environment map, which is usually represented in the form of a raster map or a topological map. In a raster map, the environment is divided into a series of discrete cells, each of which is marked as passable or impassable;

[0095] Determine the positions of the start and end points on the map, and initialize the open list and closed list. The open list is used to store the nodes to be evaluated, and the closed list is used to store the nodes that have been evaluated;

[0096] The A* algorithm is used to find the shortest path from the starting point to the end point in the graph. The A* algorithm is a heuristic search algorithm for finding the shortest path from the starting point to the end point in the graph. It combines the advantages of the Dijkstra algorithm and the best-first search, and improves the search efficiency by considering both the actual cost from the starting point to the current node and the estimated cost from the current node to the end point during the search process.

[0097] In this embodiment, it should be specifically explained that the steps of using the A* algorithm to find the shortest path from the starting point to the end point in the graph are:

[0098] Step 1: Define the heuristic function h(x) and the cost function g(x). The heuristic function is expressed as the actual cost from the starting point to the node x, and the heuristic function is expressed as the estimated cost from the node x to the end point. Manhattan distance or Euclidean distance is usually used. The calculation formula is: where x goal Represents the location of the target node or end point, x current Indicates the position of the node currently being processed;

[0099] Step 2: Calculate the total estimated cost of node x, which is calculated as f(x)=g(x)+h(x), add the starting node to the open list, and set g(start)=0 and f(start)=h(start);

[0100] Step 3: Select the node with the smallest f value from the open list as the current node current. If the current node current is the end point, end the search and build the path;

[0101] Step 4: For each neighbor node neighbor of the current node current, calculate the neighbor node g value, the calculation formula is g(neighbor) = g(current) + the cost from the current node to the neighbor node, and calculate the neighbor node f value, the calculation formula is f(neighbor) = g(neighbor) + h(neighbor);

[0102] Step 5: If the neighbor node is in the closed list and the new g value is larger, skip the neighbor node. If the neighbor node is not in the open list, or the new g value is smaller, add the neighbor node to the open list, update the parent node of the neighbor node to the current node, and update the g value and f value of the neighbor node.

[0103] Step 6: Move the current node current from the open list to the closed list and repeat steps 2-5 until the end is found or the open list is empty.

[0104] The navigation and obstacle avoidance module is used to receive the path planning information transmitted by the path planning module and perform navigation and obstacle avoidance;

[0105] In this embodiment, it should be specifically explained that the steps of performing navigation and obstacle avoidance are:

[0106] Represent a path as a series of discrete waypoints, typically with a position (x, y) and a possible direction (θ);

[0107] The extended Kalman filter positioning algorithm is used to estimate the current position and orientation of the robot, and the robot's state is dynamically updated based on sensor data and motion models. The extended Kalman filter (EKF) is an extension of the Kalman filter specifically used for state estimation of nonlinear systems. Unlike the standard Kalman filter, the EKF can handle nonlinear parts in the system model and observation model. It is widely used in robot positioning and navigation to estimate the current position and orientation of the robot;

[0108] Use sensor data to detect obstacles on the path in real time, and use local path planning algorithms (such as DWA (dynamic window method) or VFH (vector field histogram)) to dynamically generate obstacle avoidance paths based on the global path and current obstacle information. The local path planning algorithm is a type of algorithm used by autonomous robots to achieve obstacle avoidance and navigation in dynamic and unknown environments. Unlike the global path planning algorithm, the local path planning algorithm continuously updates the path based on real-time sensor data during the robot's movement to cope with dynamic changes and obstacles in the environment;

[0109] Generate control instructions based on the results of local path planning to control the movement of the robot to follow the path. Common control methods include PID control, pure pursuit algorithm (Pure Pursuit), dynamic window method (DWA), etc., to adjust the speed and direction of the robot to ensure that the robot moves towards the target and avoids obstacles in a safe manner;

[0110] Continuously monitor whether the robot is moving along the planned path, regularly update the path and target points, update obstacle information in real time, and adjust the path based on the latest environmental data;

[0111] Feedback the results of navigation and obstacle avoidance to the control system so that the navigation strategy can be adjusted in real time, recalculating the local path or requesting a global path update if the path is detected to be blocked or a new obstacle is present.

[0112] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A multi-sensor fusion greenhouse mobile chassis navigation control system, including a data fusion module, an environmental map construction module, a path planning module, and a navigation and obstacle avoidance module. The modules are connected by signals, and the data processing steps between the modules are as follows: A data fusion module, used for fusing pseudo laser data with lidar data through data conversion, and transmitting the fused data to the environment map construction module; An environment map construction module is used to receive the fused data transmitted by the data fusion module, construct an environment map according to the fused data, and transmit the environment map to the path planning module; The path planning module is used to receive the environment map transmitted by the environment map construction module, plan the path according to the destination, and transmit the path planning information to the navigation and obstacle avoidance module; The navigation and obstacle avoidance module is used to receive the path planning information transmitted by the path planning module and perform navigation and obstacle avoidance.

2. The multi-sensor fusion greenhouse mobile chassis navigation control system according to claim 1 is characterized by: The steps of fusing pseudo laser data with laser radar data through data conversion are as follows: Use an RGB-D camera to obtain a depth image, convert the depth image into pseudo laser data, convert the depth value into a two-dimensional point cloud similar to the lidar through projection, and obtain the two-dimensional point cloud data directly from the lidar; Use the timestamp synchronization algorithm to ensure that the pseudo laser data and the lidar data are aligned at the same timestamp, filter the pseudo laser data and the lidar data, remove noise and handle outliers; Perform external calibration between sensors for pseudo laser data and lidar data, obtain coordinate transformation matrix, and ensure that pseudo laser data and lidar data are in the same coordinate system; The pseudo laser data and lidar data are fused using an extended Kalman filter to build a state space model, and the state estimation is updated using the pseudo laser data and lidar data.

3. The multi-sensor fusion greenhouse mobile chassis navigation control system according to claim 2 is characterized by: The steps of converting the depth image into pseudo laser data are as follows: Get depth images from RGB-D cameras; Get the camera internal parameters, including focal length f x and f y And the principal point coordinates c x and c y ,These parameters are obtained by camera calibration; Each pixel coordinate (u, v) in the depth image and its depth value D(u, v) are converted into a three-dimensional point (X, Y, Z). The calculation formula is: Z = D(u,v); Convert the three-dimensional point cloud (X, Y, Z) into two-dimensional pseudo laser data, ignore the Y axis, and only keep the X and Y coordinates to obtain pseudo laser data (X, Y); The generated pseudo laser data are filtered to remove points exceeding the maximum distance; The point cloud data is divided into voxel sizes and a representative point is retained in each voxel.

4. The multi-sensor fusion greenhouse mobile chassis navigation control system according to claim 2 is characterized by: The steps of performing external calibration between sensors on pseudo laser data and laser radar data and obtaining a coordinate transformation matrix are as follows: A black and white checkerboard calibration plate is used during the calibration process; Place the calibration plate and collect pseudo laser data and lidar data at the same time, and record the calibration plate image collected by each sensor and its corresponding timestamp; Extracting feature points from the collected calibration plate image, wherein the feature points include corner points and intersection points inside the chessboard; Match the feature points of the pseudo laser data and the laser radar data to establish a corresponding relationship between the feature points; Using the known size of the calibration plate and the correspondence between the feature points, the camera calibration algorithm is used to calculate the external calibration parameters between the pseudo laser and the laser radar, wherein the parameters include the rotation matrix R and the translation vector T; The rotation matrix R and the translation vector T are combined into a coordinate transformation matrix, and the transformation matrix formula is:

5. The multi-sensor fusion greenhouse mobile chassis navigation control system according to claim 2 is characterized by: The steps of using the extended Kalman filter to fuse the pseudo laser data and the laser radar data and establish the state space model are: Initialize the system state vector x and covariance matrix P, set the process noise covariance matrix Q and the measurement noise covariance matrix R; Use the system dynamics model to predict the state, calculate the state transfer matrix F and the control input matrix B, and predict the state vector and covariance matrix. The calculation formula is: P - =FPF T +Q; Use the measurement model to update the state, calculate the observation matrix H and the measurement residual y, and calculate the Kalman gain, which is calculated by the formula K = P - H T (HP - H T +R) -1 , update the state vector and covariance matrix, the calculation formula is P=(I-KH)P - , returns the updated state vector As an estimate of the system state.

6. The multi-sensor fusion greenhouse mobile chassis navigation control system according to claim 1, characterized in that: The steps of performing route planning according to the destination are as follows: Receive an environment map, divide the environment into a series of discrete cells, and mark each cell as passable or impassable; Determine the locations of the start and end points on the map, and initialize the open list and closed list; Use the A* algorithm to find the shortest path from the start point to the end point in the graph.

7. The multi-sensor fusion greenhouse mobile chassis navigation control system according to claim 6, characterized in that: The steps of using the A* algorithm to find the shortest path from the starting point to the end point in the graph are: Step 1: Define the heuristic function h(x) and the cost function g(x), using Manhattan distance, which is calculated as where x goal Represented as the location of the target node, x current Indicates the position of the node currently being processed; Step 2: Calculate the total estimated cost of node x, which is calculated as f(x)=g(x)+h(x), add the starting node to the open list, and set g(start)=0 and f(start)=h(start); Step 3: Select the node with the smallest f value from the open list as the current node current. If the current node current is the end point, end the search and build the path; Step 4: For each neighbor node neighbor of the current node current, calculate the neighbor node g value, the calculation formula is g(neighbor) = g(current) + the cost from the current node to the neighbor node, and calculate the neighbor node f value, the calculation formula is f(neighbor) = g(neighbor) + h(neighbor); Step 5: If the neighbor node is in the closed list and the new g value is larger, skip the neighbor node. If the neighbor node is not in the open list, or the new g value is smaller, add the neighbor node to the open list, update the parent node of the neighbor node to the current node, and update the g value and f value of the neighbor node. Step 6: Move the current node current from the open list to the closed list and repeat steps 2-5 until the end is found or the open list is empty.

Citation Information

Cited By

  • Three-dimensional terrain modeling analysis and adaptive navigation system and method

    CN119594971A

  • Mobile robot positioning method and system based on laser and binocular information and medium

    CN120161476A