Visual navigation small farmland operation auxiliary robot

Abstract

The invention belongs to the technical field of agricultural robots, and particularly relates to a visual navigation small farmland operation auxiliary robot combining machine vision with image processing. Comprising a mobile platform which is designed to be low and hollow in the middle; independent driving wheels are arranged at four corners, so that the maneuverability is ensured; the control box is suspended behind the platform; the seat is mounted in front of the control box; the control rocker is arranged on the right side of the platform and is responsible for driving control; the mode switching key is located on the left side, and robot control modes are conveniently switched; a basket is assembled at the front part, and a monocular camera is arranged below the basket and is used for capturing road conditions and feeding back the road conditions to a control system; the ultrasonic sensors on the two sides assist in obstacle detection, and driving safety is guaranteed. The multifunctional agricultural machine has multiple functions, can cooperate with farmers to manage low crops, and is matched with the basket design to meet different agricultural auxiliary requirements.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of agricultural robots, and more particularly to a visual navigation small farm operation assistance robot combining machine vision and image processing. Background Art

[0003] When farmers carry out management work on low-growing crops such as peanuts, potatoes, and corn in the seedling stage with a height less than 30 cm, they need to bend down or squat for a long time to operate, which causes a large physical burden on farmers and rapid physical exhaustion, and is not conducive to long-term operation.

[0004] In summary, it is a future trend to use agricultural robots to assist farmers in completing the management work of low-growing crops such as peanuts, potatoes, and corn in the seedling stage, and it is also the only way to promote the mechanization, automation, and intelligentization of low-growing crop management. Summary of the Invention

[0005] The present invention aims to provide a visual navigation small farm operation assistance robot to solve the problems of low work efficiency and inability to work for a long time when farmers carry out management work on low-growing crops, thereby improving the agricultural production efficiency of farmers; in addition, a basket is installed at the front position of the vehicle body, so that the robot has a load-carrying function, ensuring the work efficiency of farmers during the harvest period. To achieve the above object, the technical solutions adopted by the present invention are as follows:

[0006] A visual navigation small farm operation assistance robot, comprising: a vehicle body moving platform, which is the vehicle body part of the robot; a chassis drive system, which is used to provide driving power for the robot; a visual navigation obstacle avoidance system, which is used for path planning, obstacle detection, and obstacle avoidance in the outdoor farm environment; a control mode switching system, which is used for switching the control mode of the robot; a control system, which is used to control the movement and travel of the robot; and a power supply system, which is used to supply power to the electronic components of the whole vehicle.

[0007] In the above solution, the vehicle body moving platform includes a chassis frame, a soft seat, a basket, and a control box. In order to make the robot better adapt to the operation environment of low-growing crop farms and reduce the erosion of the soil during the work process, the present invention has carried out targeted optimization on the design of the vehicle body moving platform: ① Work comfort design: low-profile design and hollowing in the middle. The seat height of the robot moving platform is about 38 cm, which is basically the same as the waist height of the human body in a squatting position, meeting the agricultural labor needs of the robot to assist farmers in low-growing crop fields. The above-mentioned soft seat is installed in the middle position of the vehicle body moving platform, and the hollowing design is carried out at this position to facilitate the management work of the crops directly below when the robot is moving forward; ② Mobility performance design: lightweight and high passability. The mobile platform structure is welded by aluminum alloy, with an empty weight of about 150 kg. It is lightweight and sturdy, exerting little force on the soil. The chassis height is about 37 cm, and the wheelbase between the left and right wheels is 135 cm, meeting the height of low-growing crops and the requirements of domestic crop planting spacing, with good passability. ③ Modular design: The robot as a whole adopts a modular design, which is convenient for later maintenance. The above-mentioned basket is installed in front of the vehicle body mobile platform, enabling the robot to have a stronger load-carrying function and ensuring the operation efficiency of farmers during the harvest period. The above-mentioned control box is suspended behind the vehicle body mobile platform, and the electrical part and control part hardware of the whole vehicle are placed in the control box, making the robot design simple and modular, and at the same time more safe and convenient for maintenance.

[0008] In the above solution, the chassis drive system consists of an Arduino MEGA 2560 development board, a motor driver, and hub motors, whose function is to provide driving power for the robot. Among them, the hub motors are installed on both sides of the above vehicle body mobile platform; the motor drivers are uniformly installed and placed inside the above control box. The motor driver and the hub motor are connected and driven through three-phase lines and Hall lines. The motor driver is connected to the Arduino development board through a TTL to 485 conversion module to complete the chassis drive.

[0009] In the above solution, the visual navigation and obstacle avoidance system mainly consists of a Raspberry Pi 4B, an Arduino MEGA 2560 development board, a monocular camera, and ultrasonic sensors, whose function is to perform path planning, obstacle detection, and obstacle avoidance during the operation of the robot. The implementation method is as follows: The monocular camera is connected to the Raspberry Pi 4B through a USB serial port, and the ultrasonic sensor is connected to the GPIO serial port of the Raspberry Pi 4B through Dupont wires. During the robot's movement, the monocular camera captures road information in real time and transmits it to the Raspberry Pi through the UART communication protocol. Then, using the classic HSV color space image enhancement algorithm in digital image processing technology and combining with the threshold segmentation algorithm, the crop row block in the middle of the image is extracted and the navigation line is fitted, and then combined with the path tracking technology to realize the robot's visual navigation function; the ultrasonic sensor detects the distance between the obstacle information on the front road and the vehicle body in real time, and brakes through the control signal transmitted by the Raspberry Pi to complete the robot's obstacle detection and obstacle avoidance function.

[0010] In the above solution, the control mode switching system mainly consists of a self-locking push-button switch, a rocker module, and a Raspberry Pi. Its function is to enable the robot to switch between two control modes. To improve the working efficiency of the robot, two control modes are set up, namely host computer control and rocker control. The switching between the two control modes is mainly achieved through the self-locking push-button switch. The implementation method is as follows: Connect the self-locking push-button switch to the GPIO serial port of the Raspberry Pi through a Dupont wire. Since the self-locking push-button switch can output digital signals 0 and 1, the user only needs to correspond the digital signals to the control modes. When the digital signal is 0, the control mode is host computer control, and the lower computer Arduino development board drives the robot according to the byte data output by the host computer Raspberry Pi. When the digital signal is 1, the control mode is rocker control, and the lower computer Arduino development board drives the robot according to the analog signal output by the rocker module. The rocker module is connected to the analog signal interface of the above-mentioned lower computer Arduino development board through a Dupont wire so that the lower computer can receive the analog signal.

[0011] In the above solution, the control system consists of a host computer Raspberry Pi 4B and a lower computer Arduino MEGA 2560 development board. Its function is to receive, process, and send data signals. The two are connected by a USB cable and data is transmitted through the UART communication protocol. Among them, the host computer Raspberry Pi 4B is installed inside the control box suspended at the rear of the robot. Its function is to realize control mode switching and host computer control. The components used in the above visual navigation and obstacle avoidance system are connected to the host computer Raspberry Pi 4B. The lower computer Arduino MEGA 2560 development board is also installed inside the control box. Its function is to control the driver and drive the robot chassis. The motor driver of the above chassis drive system and the rocker module of the control mode switching system are connected to the Arduino development board.

[0012] In the above solution, the power supply system consists of a set of ternary lithium batteries and a buck module. It is mainly used for power supply to electronic components such as the Raspberry Pi and the motor driver. The entire power supply system is placed inside the control box for easy later maintenance and repair.

[0013] The implementation method of the above-mentioned vision navigation small-scale farm operation assistance robot for assisting farmers in operating low-growing crops is as follows: When the robot has not entered the farm environment for operation, the rocker control mode is used to improve the robot's traveling efficiency; when the robot enters the farm environment and starts operating, it switches to the host computer control mode to reduce the labor intensity of farmers and improve the farmers' operation efficiency. When the robot is controlled by the host computer, it can collect road crop information through a monocular camera, and combine machine vision technology and path tracking technology to achieve the robot's autonomous navigation and tracking of crop rows. Then, combined with the hollow design in the middle of the vehicle body mobile platform and the seat installed on the vehicle body, it assists farmers to complete cruise management and harvesting operations on low-growing crops in a sitting position, reducing the labor intensity of farmers and improving the operation efficiency.

[0014] The implementation method of the load-carrying function of the above-mentioned vision navigation small-scale farm operation assistance robot is as follows: While the robot is carrying out management work on low-growing crops, the basket can play a certain load-carrying function. For example, when the crops need weeding, the basket can carry some weeding tools; when the crops enter the harvest season, the basket can carry fruits, etc. This function can effectively relieve the physical labor pressure of agricultural practitioners and achieve a substantial increase in labor efficiency.

[0015] The vision navigation small-scale farm operation assistance robot provided by the present invention has a wide range of application scenarios. It can carry out management work on low-growing crops or other seedling-stage crops with a plant height less than 30 cm, and can also provide certain load-carrying convenience during the harvest period. It can play a role in different planting cycles of various crops, significantly improving the utilization rate of the robot and reducing the usage cost of farmers. Compared with many special agricultural machinery that is limited to specific single needs or specific crops, the multi-functional agricultural machinery discussed in the present invention shows a wider applicability and a higher economic return rate. Brief Description of the Drawings

[0016] Figure 1 It is a schematic structural diagram of the vision navigation small-scale farm operation assistance robot of the present invention.

[0017] Figure 2 is Figure 1 front view.

[0018] Figure 3 is Figure 1 front elevation view.

[0019] Figure 4 is Figure 1 rear view.

[0020] Figure 5 is Figure 1 top view.

[0021] Figure 6 It is the three-view drawing of the motor driver.

[0022] Figure 7 It is the system working flow chart.

[0023] In the figure: 1 - seat, 2 - ultrasonic sensor, 3 - monocular camera, 4 - basket device, 5 - in-wheel motor, 6 - motor fixing device, 7 - vehicle body mobile platform, 8 - control box, 9 - seat armrest, 10 - seat fixing device, 11 - middle hollow design, 12 - control box lock, 13 - mode switch, 14 - rocker module, 15 - motor driver, 16 - motor driver power interface, 17 - three-phase wire interface of motor driver, 18 - Hall wire interface of motor driver. Specific implementation manners

[0024] To make the above objects, features and advantages of the present invention more obvious and understandable, the specific implementation manners of the present invention will be described in detail below with reference to the accompanying drawings of the specification.

[0025] In the following description, many specific details are set forth to facilitate a thorough understanding of the present invention. However, the present invention may be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below. Embodiment

[0026] The design intention of the vision navigation small-scale farm operation assistance robot is to serve the fine management of outdoor low-growing crops. In view of the possible damage to the soil structure caused by traditional agricultural machinery during operation, the present invention particularly focuses on the structural optimization and lightweight design of the robot in order to minimize the impact on the soil ecology. After careful adjustment, the external dimensions of the robot are 1400mm * 825mm * 450mm, and the self-weight of the robot is about 150KG. To further improve the adaptability of the robot in complex farm environments, the ground clearance of its chassis is set to 37cm, which is similar to the squatting height of a human body; at the same time, the distance between the left and right wheels of the robot is designed to be 150cm, which conforms to the actual crop planting environment. Farmers' participation in the daily management and protection of low-growing crops with the small-scale farm operation assistance robot will significantly improve work comfort and operation efficiency, while reducing the intensity of manual labor, providing strong technical support for the intelligent transformation of modern agriculture.

[0027] Figure 1It is a structural schematic diagram of a vision-navigation small agricultural operation assistance robot. The small agricultural operation assistance robot includes a vehicle body mobile platform 7. A wheel hub motor 5 of the traveling device is installed below the vehicle body mobile platform 7, and the two are installed and fixed through a motor fixing device 6, which is used to provide driving power for the robot. A control box 8 is suspended behind the vehicle body mobile platform 7, which is used to place the hardware of the electrical part, the power supply system and the control part. A seat 1 is installed in front of the control box. Armrests are installed on both sides of the seat 1. The seat 1 is connected to the vehicle body mobile platform 7 through a seat fixing device 10 to realize the fixed installation of the seat 1. A basket 4 is installed in front of the vehicle body mobile platform 7, so that the robot has a certain function of carrying goods.

[0028] Figure 3 The figure shows the front view of the vision-navigation small agricultural operation assistance robot. The front view of the vision-navigation small agricultural operation assistance robot shows an ultrasonic sensor 2 and a monocular camera 3. As shown in the front view, the monocular camera 3 is installed at the middle position below the basket 4; the ultrasonic sensors 2 are installed on both sides of the monocular camera 3.

[0029] Figure 4 The figure shows the rear view of the vision-navigation small agricultural operation assistance robot. The front view of the vision-navigation small agricultural operation assistance robot shows the control box 8 and the control box lock 12. As shown in the rear view, the control box 8 is installed behind the seat 1; the control box lock 12 is installed at the middle position on the right side of the control box 8 body, which is mainly used for opening and closing the control box.

[0030] Figure 5 The figure shows the top view of the vision-navigation small agricultural operation assistance robot. The top view of the vision-navigation small agricultural operation assistance robot mainly shows the control mode switching button 13 and the rocker module 14. As shown in the top view, the control mode switching button 13 is installed at the aluminum profile position on the left side of the vehicle body mobile platform 7; the rocker module 14 is installed at the aluminum profile position on the right side of the vehicle body mobile platform 7.

[0031] The traveling device is composed of four wheel hub motors 5, which are installed around the vision-navigation small agricultural operation assistance robot to provide strong off-road ability for the robot.

[0032] Figure 6 The figure shows the three views of the motor driver of the vision-navigation small agricultural operation assistance robot. The three views of the motor driver mainly show the positions of the motor driver power supply 16, the three-phase wire interface 17, and the Hall wire interface 18.

[0033] Figure 7The figure shows the workflow of a small agricultural operation assistance robot with visual navigation. After the robot platform is powered on and working, the Raspberry Pi will start for initialization and wait to receive digital signal 0 or 1. When receiving digital signal 0, the robot enters the host computer control mode. The monocular camera and the ultrasonic sensor start to work: the monocular camera begins to collect road condition information, and uploads one frame of image to the Raspberry Pi for image processing every fixed time. After the image processing is completed, a fitted navigation line will be obtained. According to the position relationship between the current pose of the robot and the navigation line, the Raspberry Pi will generate corresponding motor drive instructions and transmit them to the lower computer Arduino development board through the UART communication protocol to complete the drive of the robot platform; at the same time when the camera starts to work, the ultrasonic sensor also starts to work, and detects the information of obstacles in front by continuously transmitting and receiving ultrasonic waves. When there are obstacles that may affect the progress of the robot within 40 cm in front, the Raspberry Pi will issue a braking instruction and the robot enters the braking state, thus ensuring the safety of the driver. When receiving digital signal 1, the robot enters the joystick control mode. At this time, the lower computer Arduino development board will receive the analog signal output from the joystick module to drive the robot.

[0034] The image processing process includes the following steps: First step, perform color space conversion, convert the RGB color space of the original image to the HSV color space; Second step, perform background segmentation work in the HSV color space to segment the crop rows from the ground background; Third step, use morphological processing such as erosion and dilation to denoise the segmented image and enhance the regional connectivity.

[0035] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A visual navigation small farmland operation auxiliary robot, characterized by: The robot has two control modes: host computer control and joystick control. When the robot has not yet entered the farmland environment to operate, the joystick control mode is used to improve the robot's travel efficiency; when the robot enters the farmland environment and starts to operate, it switches to the host computer control mode to free the farmer's hands and improve the farmer's operating efficiency. When the robot is controlled by the host computer, the monocular camera (3) can be used to collect road crop information, and the machine vision technology and path tracking technology can be combined to achieve autonomous navigation of the robot and track crop rows. Combined with the hollow design (11) of the middle part of the vehicle mobile platform (7) and the seat (1) installed on the vehicle body, the robot can assist farmers to complete cruise management and harvesting of low crops in a sitting position, thereby reducing the labor intensity of farmers and improving operating efficiency.

2. The visual navigation small farmland operation auxiliary robot according to claim 1 is characterized in that: include: Control mode switching system, chassis drive system, visual navigation obstacle avoidance system, power supply system and control system.

3. The visual navigation small farmland operation auxiliary robot according to claim 2 is characterized in that: The control mode switching system is composed of a mode switching button (13), a joystick module (14) and a Raspberry Pi (host computer). The mode switching switch (13) is installed on the aluminum profile on the left side of the vehicle body moving platform (7); and the joystick module (14) is installed on the aluminum profile on the right side of the vehicle body moving platform (7). The mode switching switch (13) and the Raspberry Pi are connected via a DuPont line for signal transmission. The switching of the robot control mode described in claim 1 is achieved by relying on this system.

4. The visual navigation small farmland operation auxiliary robot according to claim 2 is characterized in that: The chassis drive system is composed of a wheel hub motor (5), a motor driver (15) and an Arduino development board (lower computer). The wheel hub motor (5) is installed on the left and right sides of the vehicle body moving platform (7) at the front and rear, and is connected to the motor driver (15) through a three-phase line and a Hall line for driving. The motor driver (15) is connected to the Arduino development board through a TTL to 485 conversion module to complete the chassis drive. The motor driver (15), the TTL to 485 module and the Arduino development board are all placed inside the control box (8).

5. The visual navigation small farmland operation auxiliary robot according to claim 2 is characterized in that: The visual navigation obstacle avoidance system is composed of an ultrasonic sensor (2) and a monocular camera (3). The monocular camera (3) is installed below the basket (4) to collect road information; the ultrasonic sensor (2) is installed on both sides of the monocular camera (3) to detect road obstacle information.

6. The visual navigation small farmland operation auxiliary robot according to claim 2 is characterized in that: The power supply system is composed of a group of ternary lithium batteries and a step-down module. After the overall assembly is completed, it is placed in the control box (8) and used to supply power to the electronic components of the entire vehicle.

7. The visual navigation small farmland operation auxiliary robot according to claim 2 is characterized in that: The control system is composed of a Raspberry Pi and an Arduino development board, wherein the Raspberry Pi is respectively connected to the Arduino development board, the ultrasonic sensor (2), the monocular camera (3) and the mode switching switch (13) to complete the movement and motion control of the robot.

8. The visual navigation small farmland operation auxiliary robot according to claim 2 is characterized in that: A control box (8) is suspended at the rear of the vehicle body moving platform (7), and a basket (4) is installed at the front. A seat (1) is installed in front of the control box, seat armrests (9) are installed on both sides of the seat (1), and a seat fixing device (10) is installed below the seat (1), through which the seat (1) is fixed on the vehicle body moving platform (7).

9. The working method of the visual navigation small farmland operation auxiliary robot according to claim 2 is characterized in that The steps include: S1. Start the small farmland operation auxiliary robot, the system enters the initialization state, and starts to send instructions to the control system; S2. The control system receives the command and sends a joystick control command to the control mode switching system, and the robot enters the joystick control mode; S3. Use the joystick to drive the robot into the farmland environment, press the mode switch (13), and switch the control mode to the host computer control; S4. The robot enters the host computer control mode, requests the monocular camera to identify the road crop planting conditions and perform path planning; requests the ultrasonic sensor to identify the road obstacle conditions and perform obstacle avoidance work; S5. The robot starts to operate, and the farmer can manage the crops through the hollow design (11) in the middle of the mobile platform (7).

10. The working method of the visual navigation small farmland operation auxiliary robot according to claim 9 is characterized in that The S4 path planning includes the following steps: S41. The monocular camera identifies the crop row and finds the crop row currently being operated; S42. After finding the crop rows, extract feature points of the relevant crops; S43. After the feature point extraction is completed, the extracted feature points are fitted into a navigation line; S44. After obtaining the navigation line, the robot is driven according to the relative position relationship between the current posture of the robot and the navigation line.

11. The working method of the visual navigation small farmland operation auxiliary robot according to claim 9 is characterized in that The S4 road obstacle recognition and avoidance includes the following steps: S45. The ultrasonic sensor detects obstacle information on the road within 5 meters ahead; S46. After detecting an obstacle, the Raspberry Pi starts calculating the distance between the obstacle and the robot; S47. When the distance is less than the threshold of 40 cm, the robot enters the braking state and combines the monocular camera to plan a new path to achieve the purpose of obstacle avoidance.

Images