Mobile robot for monitoring tomato growth state and fruit maturity

Abstract

The invention discloses a mobile robot for tomato growth state and fruit maturity monitoring, the mobile robot comprises a frame, the top of the frame is provided with a power supply module, a motor driving module, a depth camera module and an airborne computer, and the left side of the bottom of the frame is provided with a steering driving assembly; and two moving driving assemblies are arranged on the right side of the bottom of the vehicle frame. According to the mobile robot for monitoring the tomato growth state and the fruit maturity, moving power is provided for the robot through the two moving driving assemblies, free movement of the robot in the horizontal direction is achieved under the cooperation of the frame and the steering driving assembly, and the depth camera module is arranged in a matched mode; the growth state of the tomato crops and the real-time images of the fruits are effectively and comprehensively obtained, efficient and comprehensive monitoring conditions are provided for monitoring of the growth state of the tomato crops and the maturity of the fruits, the labor cost is reduced, and the advantage of unmanned automatic detection is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural automation, and particularly to a mobile robot for monitoring the growth state and fruit maturity of tomatoes. Background Art

[0002] Traditional manual monitoring: By manually patrolling the fields regularly to record the growth of tomatoes and the degree of fruit maturity. Although this method is intuitive, it is time-consuming and laborious, and is easily affected by human factors, resulting in inaccurate data.

[0003] Fixed camera system based on image processing: Some farms have begun to try using cameras installed in fixed positions to continuously capture images of tomato crops and automatically identify the crop status through image analysis software. However, such systems are limited by the viewing range and cannot cover the entire planting area, and have poor adaptability to dynamic changing environments.

[0004] Aerial monitoring based on drones: Using drones equipped with high-resolution cameras or thermal imaging devices to conduct aerial patrols of tomato planting areas can provide a more comprehensive view. However, the flight time of drones is limited, and the operation is difficult under complex weather conditions; in addition, due to the lack of ground precise positioning ability, the accuracy of the maps generated is often not high.

[0005] Based on the retrieval of the above information, it can be seen that the traditional monitoring and processing methods for tomato crops cannot achieve efficient and accurate monitoring of the growth state and fruit maturity of tomatoes. Therefore, a mobile robot for monitoring the growth state and fruit maturity of tomatoes is specifically proposed, which can move freely in the tomato planting area to achieve timely and accurate grasp of the growth state of tomatoes and accurate monitoring of the fruit maturity. Summary of the Invention

[0006] In view of the deficiencies of the prior art, the present invention provides a mobile robot for monitoring the growth state and fruit maturity of tomatoes, which solves the problem that the traditional monitoring and processing methods cannot achieve efficient and accurate monitoring of the growth state and fruit maturity of tomatoes.

[0007] To achieve the above objectives, the present invention is realized through the following technical solutions: A mobile robot for monitoring the growth state and fruit maturity of tomatoes, including a vehicle frame, on the top of which a power supply module, a motor drive module, a depth camera module and an on-board computer are provided, on the left side of the bottom of the vehicle frame a steering drive assembly is provided, and on the right side of the bottom of the vehicle frame two mobile drive assemblies are provided;

[0008] The depth camera module includes a servo pan-tilt, on the top of which a depth camera is fixedly installed, the servo pan-tilt is fixedly installed on the left side of the top of the vehicle frame, and a lidar is also fixedly installed in the servo pan-tilt.

[0009] The present invention is further configured such that: both of the two mobile driving components include a first mounting frame and a driving motor. A planetary reducer is fixedly mounted on one side of the first mounting frame. The driving motor is fixedly mounted on one side of the planetary reducer, and the output end of the driving motor is fixedly connected to the input end of the planetary reducer. A rear wheel is rotatably mounted on the other side of the first mounting frame, and the output end of the planetary reducer is fixedly connected to one end of the rotating shaft of the rear wheel through a coupling.

[0010] The present invention is further configured such that: the two first mounting frames are respectively fixedly mounted on the front and rear sides of the right side of the bottom of the vehicle frame.

[0011] The present invention is further configured such that: the steering driving component includes a second mounting frame and two T-shaped plates. Positioning rods penetrate through the tops of the two T-shaped plates, and a left front wheel and a right front wheel are respectively rotatably mounted on the opposite sides of the two T-shaped plates. A short connecting column and a long connecting column are respectively fixedly mounted on the right sides of the bottoms of the two T-shaped plates, and a long rod is rotatably mounted on the outer peripheries of the short connecting column and the long connecting column through a ball joint connector.

[0012] The present invention is further configured such that: the two positioning rods are respectively fixedly mounted on the front and rear sides of the left side of the bottom of the vehicle frame, and a buffer spring is sleeved on the outer periphery of the positioning rod. The two ends of the buffer spring are respectively in contact with the opposite sides of the vehicle frame and the T-shaped plate.

[0013] The present invention is further configured such that: a steering motor is fixedly mounted on one side of the second mounting frame. The output end of the steering motor penetrates through the second mounting frame and is fixedly mounted with a rotating plate. One side of the rotating plate is rotatably mounted with a short rod through a ball joint connector, and one end of the short rod is rotatably connected to the outer periphery of the long connecting column through a ball joint connector;

[0014] The second mounting frame is fixedly mounted on the bottom of the vehicle frame.

[0015] The present invention is further configured such that: a cross plate is sleeved and fixedly mounted on the common bottom of the outer peripheries of the two positioning rods, and the cross plate is arranged below the two T-shaped plates.

[0016] The present invention is further configured such that: a cross plate is sleeved and fixedly mounted on the common bottom of the outer peripheries of the two positioning rods, and the cross plate is arranged below the two T-shaped plates.

[0017] The present invention provides a mobile robot for monitoring the growth state and fruit maturity of tomatoes. It has the following beneficial effects:

[0018] The present invention provides motive power for the robot through two mobile drive components. With the cooperation of the vehicle frame and the steering drive component, the robot can move freely in the horizontal direction. Together with the setting of the depth camera module, the growth state of tomato crops and the real-time images of the fruits can be effectively and comprehensively obtained, providing efficient and comprehensive monitoring conditions for the monitoring of the growth state of tomato crops and the maturity of fruits, reducing labor costs, and having the advantage of unmanned automated detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram of the external structure of the present invention;

[0020] Figure 2 is a schematic diagram of the connection structure of the depth camera module and the lidar of the present invention;

[0021] Figure 3 is a schematic diagram of the connection structure of the vehicle frame, the steering drive component and the mobile drive component of the present invention;

[0022] Figure 4 is a schematic diagram of the external structure of the vehicle frame of the present invention.

[0023] In the figure:

[0024] 1. Vehicle frame;

[0025] 2. Power supply module;

[0026] 3. Motor drive module;

[0027] 4. Depth camera module; 401. Servo pan-tilt; 402. Depth camera;

[0028] 5. Onboard computer;

[0029] 6. Steering drive component; 601. Second mounting bracket; 602. T-shaped plate; 603. Positioning rod; 604. Left front wheel; 605. Right front wheel; 606. Short connecting post; 607. Long connecting post; 608. Long rod; 609. Buffer spring; 6010. Steering motor; 6011. Rotating plate; 6012. Short rod; 6013. Horizontal plate;

[0030] 7. Mobile drive component; 701. First mounting bracket; 702. Drive motor; 703. Rear wheel; 704. Planetary reducer;

[0031] 8. Lidar;

[0032] 9. Covering bracket. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention.

[0034] Please refer to Figures 1-4 Figures 1-4 , the embodiments of the present invention provide the following technical solutions: A mobile robot for monitoring the growth state and fruit maturity of tomatoes, including a vehicle frame 1, and a power supply module 2, a motor drive module 3, a depth camera module 4, an on-board computer 5, a steering drive assembly 6, and two mobile drive assemblies 7 provided on the vehicle frame 1. Among them, the vehicle frame 1 is made of 6061 alloy material, with good load-bearing capacity and stability. The power supply module 2 is a 12V lithium battery made of explosion-proof polymer material, used to provide lasting power support. The on-board computer 5 is an X86 architecture on-board computer, equipped with an Ubuntu system, used to run ROS and related software modules.

[0035] As a preferred solution, in order to obtain image data of tomato crops, the depth camera module 4 includes a servo pan-tilt 401. A depth camera 402 is fixedly installed on the top of the servo pan-tilt 401. The depth camera 402 is an Intel D435i depth camera, which has the functions of collecting RGB and depth information, and is used to collect color images and depth information of tomato fruits. The servo pan-tilt 401 is fixedly installed on the left side of the top of the vehicle frame 1, and a lidar 8 is also fixedly installed in the servo pan-tilt 401. Among them, the lidar 8 is used for environmental mapping and positioning.

[0036] As a preferred solution, in order to realize the mobile drive of the robot, the two mobile drive assemblies 7 both include a first mounting bracket 701 and a drive motor 702. The two first mounting brackets 701 are respectively fixedly installed on the front and rear sides of the right side of the bottom of the vehicle frame 1. A planetary reducer 704 is fixedly installed on one side of the first mounting bracket 701. The planetary reducer 704 is a Leisai 57 planetary reducer, used to cooperate with the drive motor 702 to drive the following rear wheel 703 to rotate, ensuring precise control. The drive motor 702 is fixedly installed on one side of the planetary reducer 704, and the output end of the drive motor 702 is fixedly connected to the input end of the planetary reducer 704. A rear wheel 703 is rotatably installed on the other side of the first mounting bracket 701, and the output end of the planetary reducer 704 is fixedly connected to one end of the rotating shaft of the rear wheel 703 through a coupling.

[0037] As a preferred solution, in order to achieve the steering of the robot for convenient movement, the steering drive assembly 6 includes a second mounting bracket 601 and two T-shaped plates 602. The second mounting bracket 601 is fixedly installed at the bottom of the vehicle frame 1. Positioning rods 603 penetrate through the tops of the two T-shaped plates 602. The two positioning rods 603 are respectively fixedly installed on the front and rear sides of the left side of the bottom of the vehicle frame 1. And on the opposite sides of the two T-shaped plates 602, a left front wheel 604 and a right front wheel 605 are respectively rotatably installed. On the right sides of the bottoms of the two T-shaped plates 602, a short connection post 606 and a long connection post 607 are respectively fixedly installed. A long rod 608 is rotatably installed around the outer circumferences of the short connection post 606 and the long connection post 607 through ball head connectors. On one side of the second mounting bracket 601, a steering motor 6010 is fixedly installed. The output end of the steering motor 6010 penetrates through the second mounting bracket 601 and is fixedly installed with a rotating plate 6011. On one side of the rotating plate 6011, a short rod 6012 is rotatably installed through a ball head connector. One end of the short rod 6012 is rotatably connected to the outer circumference of the long connection post 607 through a ball head connector.

[0038] Furthermore, to ensure the stable movement of the robot, a buffer spring 609 is sleeved on the outer circumference of the positioning rod 603. The two ends of the buffer spring 609 are respectively in contact with the opposite sides of the vehicle frame 1 and the T-shaped plate 602. At the bottoms of the outer circumferences of the two positioning rods 603, a cross plate 6013 is sleeved and fixedly installed together, and the cross plate 6013 is arranged below the two T-shaped plates 602.

[0039] The steering method of the robot specifically includes: starting the steering motor 6010, the steering motor 6010 drives the rotating plate 6011 to rotate, the rotating plate 6011 drives the short rod 6012 to pull the long connection post 607 to move, the long connection post 607 drives the connected T-shaped plate 602 to rotate around the positioning rod 603, so that the left front wheel 604 rotates to the left. At the same time, the long connection post 607 pushes the long rod 608 to move the short connection post 606, and the short connection post 606 drives the other T-shaped plate 602 to rotate around the other positioning rod 603, so that the right front wheel 605 rotates to the left. Similarly, when it is necessary to turn to the right, controlling the steering motor 6010 to reverse is sufficient.

[0040] As a preferred solution, to ensure the environmental adaptability of the robot, a covering bracket 9 is fixedly connected to the top of the vehicle frame 1 and around the power supply module 2 and the on-board computer 5.

[0041] As a detailed description, the motor drive module 3 is an STM32F4 series microcontroller, which is used to control the operation of the entire system, including triggering the camera to collect images and data processing. The system is designed with the PyQt framework, which is convenient for user operation and result display. Specifically, it includes:

[0042] A. The Fast-LIO2 algorithm is used for simultaneous localization and mapping. By using the data collected by the lidar, a high-precision map can be quickly generated in a complex environment. Among them, Fast-LIO2 supports incremental updates and dynamic balancing through an incremental k-d tree data structure, thus realizing efficient map maintenance. Fast-LIO2 does not extract features, but directly registers the original point cloud onto the map, improving the accuracy and robustness of the matching. Fast-LIO2 is based on a tightly coupled iterative Kalman filter, and improves the accuracy of localization and mapping through iterative optimization;

[0043] B. Based on the Robot Operating System platform, the move_base package is used for path planning and autonomous navigation to control the motor drive module 3. The move_base package integrates a global path planner and a local path planner to ensure that the mobile robot can move safely and efficiently from the starting point to the target position. The specific steps are as follows:

[0044] Global path planning: The A* algorithm is used for global path planning to generate the optimal path from the starting point to the target point;

[0045] Local path planning: The DWA (Dynamic Window Approach) or other local path planning algorithms are used for local path planning to dynamically adjust the path according to the current environment and avoid obstacles;

[0046] Path execution: The drive motor 702 and the steering motor 6010 are controlled through the motor drive module 3 to make the mobile robot travel along the planned path;

[0047] Feedback and correction: During the driving process, the current position is continuously updated through the lidar 8 and IMU data, and the path is adjusted according to the actual deviation to ensure that the robot can accurately reach the target position;

[0048] C. The two-dimensional image data and depth information of the tomato plants are obtained through the depth camera 402, combined with image processing technology for growth state analysis, and the Mamba-YOLO target detection algorithm is used to accurately identify tomato fruits with different maturities. The methods for identifying the fruit growth state and maturity specifically include:

[0049] Dataset collection: Collect tomato fruit images with different growth states and maturities, and process them as the training dataset;

[0050] Model Training: Using the Mamba-YOLO object detection algorithm, with color image data as the input and the detection results of fruit growth status and maturity as the output. Mamba-YOLO is an improved YOLO network, especially suitable for small object detection, and can accurately identify tomato fruits in different growth states and maturities.

[0051] Mamba-YOLO: Through an improved YOLO network architecture, Mamba-YOLO can effectively detect the position and size of tomato fruits. This algorithm introduces the ODSSBlock module to apply the state space model, enhancing the model's ability to capture global dependencies. At the same time, the LSBlock and RGBlock modules further enhance the model's ability to capture local features and channel expression ability. Mamba-YOLO can not only detect the position of tomato fruits, but also accurately identify tomato fruits in different growth states and maturities based on the color and texture characteristics of the fruits. By training a large amount of labeled data, Mamba-YOLO can achieve high-precision maturity detection.

[0052] System Integration: Integrate the model into the on-board computer 5, input the image into the Mamba-YOLO model through the depth camera 402, and output the model prediction data.

[0053] D. After setting several monitoring points, according to the model prediction data, generate a comprehensive report at each monitoring point, including the growth status of tomato plants and the detection results of fruit maturity. Finally, display the monitoring results through the human-computer interaction interface, and users can intuitively view the growth status of tomato plants and the fruit maturity situation.

[0054] The usage process of the above system after being mounted on the robot hardware proposed in this application specifically includes:

[0055] Startup and Initialization: After turning on the power, the system conducts self-checks to ensure that all components are working properly.

[0056] Mapping and Navigation: Use the Fast-LIO2 algorithm combined with the lidar 8 to collect data for simultaneous localization and mapping to generate an environmental map. Subsequently, perform path planning through the move_base package to enable the mobile robot to autonomously navigate to the specified location.

[0057] Image Acquisition: After reaching the target location, trigger the depth camera 402 and collect the color image data and depth information of tomato plants and fruits.

[0058] Data Processing: Upload the collected data to the on-board computer 5 and analyze it through the growth status monitoring module and the fruit maturity recognition module.

[0059] Result output: The monitoring results are presented through a human-machine interaction interface, including the growth status of tomato plants and the classification results of fruit maturity.

[0060] Data storage: The monitoring data is stored in a local database for subsequent analysis and management.

[0061] Simulation experiment

[0062] In an actual tomato plantation, a complete mobile robot system was built using the above hardware configuration, and sufficient tomato plants and fruit samples were prepared for testing. It was found that through testing multiple monitoring points, the stability and accuracy of the system were verified. The experimental results show that when the system is installed on the robot proposed in the present invention, it can efficiently and accurately complete the monitoring tasks of the growth status of tomato plants and fruit maturity.

[0063] It should be noted that when the mobile robot reaches the designated position, an external trigger signal activates the depth camera 402 to capture the color images and depth information of tomato plants and fruits. The captured images first undergo preprocessing steps such as denoising and cropping to remove background noise and focus on the main body of tomato plants and fruits. For color images, color correction, contrast enhancement, etc. need to be performed for subsequent analysis.

[0064] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A mobile robot for monitoring the growth state of tomatoes and the maturity of fruits, comprising a vehicle frame (1), characterized in that: A power supply module (2), a motor drive module (3), a depth camera module (4), and an on-board computer (5) are provided on the top of the vehicle frame (1). A steering drive assembly (6) is provided on the left side of the bottom of the vehicle frame (1), and two mobile drive assemblies (7) are provided on the right side of the bottom of the vehicle frame (1). The depth camera module (4) includes a servo pan-tilt (401). A depth camera (402) is fixedly installed on the top of the servo pan-tilt (401). The servo pan-tilt (401) is fixedly installed on the left side of the top of the vehicle frame (1), and a lidar (8) is also fixedly installed in the servo pan-tilt (401).

2. The mobile robot for monitoring the growth state and fruit maturity of tomatoes according to claim 1, wherein: Each of the two mobile drive assemblies (7) includes a first mounting bracket (701) and a drive motor (702). A planetary reducer (704) is fixedly installed on one side of the first mounting bracket (701). The drive motor (702) is fixedly installed on one side of the planetary reducer (704), and the output end of the drive motor (702) is fixedly connected to the input end of the planetary reducer (704). A rear wheel (703) is rotatably installed on the other side of the first mounting bracket (701). The output end of the planetary reducer (704) is fixedly connected to one end of the rotating shaft of the rear wheel (703) through a coupling.

3. The mobile robot for monitoring the growth state and fruit maturity of tomatoes according to claim 2, wherein: The two first mounting brackets (701) are respectively fixedly installed on the front and rear sides of the right side of the bottom of the vehicle frame (1).

4. The mobile robot for monitoring the growth state and fruit maturity of tomatoes according to claim 1, characterized in that: The steering drive assembly (6) includes a second mounting bracket (601) and two T-shaped plates (602). Positioning rods (603) penetrate through the tops of the two T-shaped plates (602). A left front wheel (604) and a right front wheel (605) are respectively rotatably installed on the opposite sides of the two T-shaped plates (602). A short connection post (606) and a long connection post (607) are respectively fixedly installed on the right sides of the bottoms of the two T-shaped plates (602). A long rod (608) is rotatably installed around the outer circumferences of the short connection post (606) and the long connection post (607) through a ball joint connector.

5. The mobile robot for monitoring the growth state and fruit maturity of tomatoes according to claim 4, wherein: The two positioning rods (603) are respectively fixedly installed on the front and rear sides of the left side of the bottom of the vehicle frame (1). A buffer spring (609) is sleeved on the outer circumference of the positioning rod (603), and the two ends of the buffer spring (609) are respectively in contact with the opposite sides of the vehicle frame (1) and the T-shaped plate (602).

6. The mobile robot for monitoring the growth state and fruit maturity of tomatoes according to claim 5, characterized in that: A steering motor (6010) is fixedly installed on one side of the second mounting bracket (601). The output end of the steering motor (6010) penetrates through the second mounting bracket (601) and is fixedly installed with a rotating plate (6011). One side of the rotating plate (6011) is rotatably installed with a short rod (6012) through a ball joint connector. One end of the short rod (6012) is rotatably connected to the outer circumference of the long connection post (607) through a ball joint connector. The second mounting bracket (601) is fixedly installed on the bottom of the vehicle frame (1).

7. A mobile robot for monitoring the growth state and fruit maturity of tomatoes according to claim 6, characterized in that: A cross plate (6013) is sleeved and fixedly installed on the common bottom of the outer circumferences of the two positioning rods (603), and the cross plate (6013) is arranged below the two T-shaped plates (602).

8. The mobile robot for monitoring the growth state and fruit maturity of tomatoes according to claim 1, wherein: A covering bracket (9) is fixedly connected to the top of the vehicle frame (1) and located on the outer periphery of the power supply module (2) and the on-board computer (5).

Images