Artificial intelligence-based agricultural unmanned ground vehicle system and method thereof

Abstract

The invention relates to the artificial intelligence-based agricultural unmanned ground vehicle system and method thereof for hoeing the field by navigating among crops without the need for GPS and detecting weeds, crop pests and diseases in the field and spraying pesticide.

Description

[0001] ARTIFICIAL INTELLIGENCE-BASED AGRICULTURAL UNMANNED GROUND VEHICLE SYSTEM AND METHOD THEREOF

[0002] Technical Field of the Invention

[0003] The invention relates to the artificial intelligence-based agricultural unmanned ground vehicle system and method thereof for hoeing the field by navigating among crops without the need for GPS and detecting weeds, crop pests and diseases in the field and spraying pesticide.

[0004] State of the Art

[0005] Today, with the development of modern agriculture, the use of technological systems in agricultural areas has become quite widespread. These systems are generally designed to assist farmers in agricultural operations, increase efficiency, and use resources more effectively.

[0006] Unmanned ground vehicles and artificial intelligence technologies used in the agricultural sector have evolved with advances in areas such as agricultural machinery, automated irrigation systems, and analysis of agricultural data.

[0007] Artificial intelligence-based agricultural unmanned ground vehicles are generally designed to perform certain operations automatically in the fields. These may include tasks such as automatic harvesting, fertilization, pesticide spraying, and weed control. These systems offer several advantages to increase the productivity of farm operators and reduce the workforce by using technologies such as sensors, cameras, and artificial intelligence algorithms.

[0008] Artificial intelligence-based agricultural unmanned ground vehicles have significant potential in sustainability, efficiency, and resource use in the agricultural industry. Therefore, the use and development of these technologies is of increasing interest in the agricultural sector. The invention with application number TR2021 / 003997 in the state of the art relates to an artificial intelligence-based autonomous tractor system for autonomous movement of a tractor, comprising a tractor for use in agricultural areas; a steering wheel for steering said tractor in a determined direction; and a drive element configured to move said steering wheel. In this application, the tractor follows a route. However, there is no mention of lateral angle calculation and the ability of the robot to work in fields with offset lateral angles, and the ability of the robot to navigate the entire field by moving between specified coordinates instead of just following a straight line or route

[0009] In the state of the art, systems that perform image processing, pesticide spraying, and hoeing between plants have been encountered. In the state of the art, the lateral angle is calculated according to the start and end coordinates of the crop row. However, in such a system, the use of a GPS is needed for calculating crop rows. In the state of the art, there is no system that detects the crops with the cameras (101) located on the right and left of the robot without using GPS after the robot enters the crop row and calculates the lateral angle according to the lateral tilt of these crops. For this reason, in the detection of crops on the right and left from image data, there is a need for a system that detects weeds, crop pests and diseases in the field, sprays pesticides after detection, and also hoes between crops.

[0010] Summary and Objects of the Invention

[0011] The invention relates to the artificial intelligence-based agricultural unmanned ground vehicle system and method thereof for hoeing the field by navigating among crops without the need for GPS and detecting weeds, crop pests and diseases in the field and spraying pesticide.

[0012] An object of the invention is to enable a robot with known initial coordinates to update its own coordinates based on data received from sensors. Thus, there is no need for an extra GPS system for instantaneous coordinate information.

[0013] Another object of the invention is to calculate the tilt by processing the data from the front camera when the rows between the two plants in the fields proceed in a tilt and to enable the robot to move to the target point with this tilt. Another object of the invention is to enable the detection of objects in the field with a deep learning algorithm in one or more cameras while the robot is moving from one point to another in the field, the calculation of the positions of the detected objects according to the position of the robot, and sorting the objects detected by different cameras in a single row based on these calculations.

[0014] Another object of the invention is to enable pesticide spraying by targeting all objects one by one.

[0015] Description of the Drawings

[0016] Figure 1. The drawing showing the isometric view of the system of the invention.

[0017] Figure 2. The drawing showing the top view of the system of the invention.

[0018] Figure 3. The drawing showing the front view of the system of the invention.

[0019] Figure 4. The image showing the flow diagram of the method of the invention.

[0020] Description of the References in Figures

[0021] 100. Artificial intelligence-based agricultural unmanned ground vehicle system

[0022] 101 . Camera

[0023] 102. Linear motor

[0024] 1000. Entering the coordinates of the start and end of the rows in the field by the user via the interface

[0025] 1010. Receiving the first coordinate entered via the interface as the instantaneous coordinate of the unmanned ground vehicle and calculating the direction of the unmanned ground vehicle using the gyroscope and tilt sensor with the control card 1020. Finding the instantaneous coordinate of the robot by processing the direction of the unmanned ground vehicle and the data from the encoder sensor located on the wheel of the robot together and assigning the number of operations value as zero with the control card

[0026] 1030. Checking whether the number of operations is greater than the number of coordinates with the control card

[0027] 1031. If the number of operations is greater than the number of coordinates, initiating hoeing with the control card 1031.1. Generating a route between the instantaneous coordinate and the initial coordinate and advancing the unmanned ground vehicle to those coordinates with the control card

[0028] 1031.2. Stopping the unmanned ground vehicle when it approaches the initial coordinate with the control card

[0029] 1031.3. Assigning the number of operations determined as zero with the control card and returning to the process step of checking whether the number of operations is greater than the number of coordinates with the control card (1003)

[0030] 1032. If the number of operations is less than the number of coordinates with the control card, determining the initial coordinate, the destination coordinate, and the path between the determined coordinates with the control card

[0031] 1033. Checking whether the number of operations is an even number with the control card

[0032] 1033.1. If the number of operations is an even number, initiating hoeing by starting the linear motor (102) with the control card

[0033] 1033.2. According to the direction determined advancing the unmanned ground vehicle while performing hoeing with the control card

[0034] 1033.3. Increasing the number of operations by one with the control card and returning to the process step of checking whether the number of operations is greater than the number of coordinates with the control card

[0035] 1034. If the number of operations is not an even number, calculating the instantaneous distance between the instantaneous coordinate and the target with the control card

[0036] 1035. Calculating the main distance between the initial coordinate and the target with the control card

[0037] 1036. Finding the error figure by subtracting the instantaneous distance from the calculated main distance with the control card

[0038] 1037. Checking whether the error figure is greater than a determined distance with the control card

[0039] 1038. If the error figure is less than a determined distance, stopping the robot by reducing the robot speed within the control card according to the error figure, increasing the number of control cards by one, and returning to the process step of checking whether the number of operations is greater than the number of coordinates with the control card 1039. If the error figure is greater than a determined distance, receiving the number of objects detected on the right and left camera (101) and the pixel coordinates of the objects on the camera (101) with the control card

[0040] 1040. Setting the object counter value to zero with the control card

[0041] 1041. Checking whether the object counter and the number of detected objects are equal with the control card

[0042] 1042. If the object counter and the number of detected objects are equal, using the tilt data obtained by processing the images received from the front camera (101) with the control card

[0043] 1042.1 . Actuating the linear motor (102) for hoeing with the control board

[0044] 1042.2. Adjusting the unmanned ground vehicle to straighten itself according to the tilt and continue on its route with the control card and returning to the process step of checking whether the error figure is greater than a determined distance with the control card

[0045] 1043. If the object counter and the number of detected objects are not equal, sorting the objects in a single row according to the pixel coordinates on the camera (101 ) with the control card

[0046] 1044. Slowing down and stopping the robot according to the pixel coordinate on the x- axis of the object with the closest sorting result with the control card

[0047] 1045. Directing the middle point on the y-axis of the camera (101) to the y-axis of the object with the servo motor connected to the camera (101 ) detecting the object with the control card

[0048] 1046. With the control card, spraying the liquid in the pesticide chamber for pesticide spraying by actuating the pesticide motor and increasing the object counter value by one and returning to the process step of checking whether the object counter and the number of detected objects are equal with the control card.

[0049] E: Yes

[0050] H: No

[0051] Detailed Description of the Invention

[0052] The invention relates to the artificial intelligence-based agricultural unmanned ground vehicle system (100) for hoeing the field by navigating among crops without the need for GPS and detecting weeds, crop pests and diseases in the field and spraying pesticide. The unmanned ground vehicle system comprises an interface, gyroscope, encoder sensor, tilt sensor, camera (101), hoe, linear motor (102), pesticide chamber, pesticide motor, and control card.

[0053] When the artificial intelligence-based agricultural unmanned ground vehicle system (100) first starts operation, the user must enter the coordinates of the start and end of the rows in the field in order (1 st coordinate 2nd coordinate). For this reason, there is an interface on the unmanned ground vehicle. The interface allows users to enter the start and end coordinates of the row in the field.

[0054] A control card is located on the unmanned ground vehicle. The control card takes the coordinate entered via the interface as the instantaneous coordinate of the robot. Control card, with the data obtained from the gyroscope, tilt sensor, and encoder sensor, detects the position of the vehicle and finds its instantaneous coordinate. The control card measures the total distance traveled by the unmanned ground vehicle according to its speed and the diameter of the wheel. The control card calculates the distance traveled by the unmanned ground vehicle and the distance traveled by the robot in latitude and longitude as a result of multiplying the lateral angle of the unmanned ground vehicle by sine and cosine. Since the Earth is in the shape of an imperfect sphere, the current latitude and longitude coordinates are calculated with the control card by calculating the calculated latitude, longitude distances and initial latitude and longitude coordinates with the Haversine formula, respectively. With this calculation method, the need for a GPS sensor is eliminated.

[0055] The control card detects weeds, plant diseases and pests in the field with deep learning in the images from the right and left cameras (101). The control card applies color filtering, edge detection, and edge density detection operations to images received from the front camera. The control card converts the resulting image into a bird's eye view and creates a line each by averaging the white regions that appear in the right and left sections determined according to the middle point of the image. According to the average of these two lines, it creates a line in the middle section and calculates the lateral angle according to the tilt of that line. A gyroscope is located on the unmanned ground vehicle. The gyroscope provides detection of the direction of the unmanned ground vehicle.

[0056] There is an encoder sensor on the wheel of the unmanned ground vehicle. The encoder sensor detects the speed and direction in which the unmanned ground vehicle is moving. The encoder sensor measures how many turns the wheel makes per second.

[0057] At least one tilt sensor is located on the unmanned ground vehicle. The tilt sensor allows the tilt of the unmanned ground vehicle to be measured.

[0058] Camera (101) placed to the right, left and front of the unmanned ground vehicle is located on the unmanned ground vehicle to take the image of the surrounding objects.

[0059] A linear motor (102) is located on the unmanned ground vehicle. The linear motor (102) allows the hoe system on the unmanned ground vehicle to be moved for activation.

[0060] There is a pesticide chamber on the unmanned land vehicle for spraying pesticides on crops.

[0061] The control card enables the vehicle to be operated so that the unmanned ground vehicle advances to the target coordinate. While the unmanned ground vehicle advances to the target point, the linear motor (102) activates the hoe at the same time. In this way, the unmanned ground vehicle advances to the target point while hoeing. The control card calculates the distance between the instantaneous coordinate of the unmanned ground vehicle and the target coordinate. Then, the control card calculates the distance between the initial coordinate and the target coordinate of the unmanned ground vehicle and finds the margin of error. It checks whether the margin of error is greater than the determined one meter. In the preferred embodiment of the invention, it checks whether it is greater than 0.5 meters. If the margin of error is less than the determined distance, the control card stops the unmanned ground vehicle and reactivates the hoe and performs the same operations for the new route to proceed to the next coordinate. The system sprays pesticide only if the margin of error is greater than the determined distance and the object counter is not equal to the number of objects. When the margin of error is greater than the determined distance, the robot moves in the row between the two plants, and when it is small, the robot performs the maneuvers necessary to enter the other row. The number of objects indicates the number of detected objects, and the object counter indicates the number of pesticide sprayed objects. If the two are not equal to each other, the system detects that the pesticide spraying has not been done and sprays the detected objects. If the margin of error is greater than the determined distance, the control card receives the number of objects detected in the right and left camera (101 ) and the pixel coordinates of the objects on the camera (101 ). If the control card has not detected any object, it calculates the lateral angle with the images from the front camera and allows the unmanned ground vehicle to continue its route by straightening itself according to the tilt. Here, an artificial lane is designed according to the position of the plants on the right and left sides with the image data received from the front camera and lane tracking is performed (for example, if the strip is facing 2 degrees to the left compared to the robot, the robot turns itself 2 degrees to the left), the linear motor (102) located behind the robot helping with hoeing operates to hoe and the robot continues to proceed to the target point.

[0062] If the control card detects objects from the right and left cameras, it sorts the objects in a single row according to the pixel coordinates on the camera (101). According to the pixel coordinate on the x-axis of the object with the closest sorting result, the robot is stopped and the middle point on the y-axis of the camera (101 ) is directed to the y-axis of the object with the servo motor connected to the camera (101 ) detecting the object and the liquid in the pesticide chamber is sprayed by actuating the pesticide motor for pesticide spraying. This process is repeated until the detected objects are finished.

[0063] When all the points on the field to be visited are finished, the unmanned ground vehicle moves from the current coordinate to the initial coordinate while hoeing. When the initial coordinate is reached, the unmanned ground vehicle stops.

[0064] The artificial intelligence-based agricultural unmanned ground vehicle operation method comprises the following process steps;

[0065] Entering the coordinates of the start and end of the rows in the field by the user via the interface (1000), Receiving the first coordinate entered via the interface as the instantaneous coordinate of the unmanned ground vehicle and calculating the direction of the unmanned ground vehicle using the gyroscope and tilt sensor with the control card (1010),

[0066] Finding the instantaneous coordinate of the robot by processing the direction of the unmanned ground vehicle and the data from the encoder sensor located on the wheel of the robot together and assigning the number of operations value as zero with the control card (1020),

[0067] Checking whether the number of operations is greater than the number of coordinates with the control card (1030),

[0068] If the number of operations is greater than the number of coordinates with the control card, initiating hoeing (1031 ),

[0069] Generating a route between the instantaneous coordinate and the initial coordinate with the control card and advancing the unmanned ground vehicle to those coordinates (1031.1 )

[0070] Stopping the unmanned ground vehicle when it approaches the initial coordinate with the control card (1031 .2)

[0071] Assigning the number of operations determined as zero with the control card and returning to the process step of checking whether the number of operations is greater than the number of coordinates with the control card (1003) (1031 .3),

[0072] If the number of operations is less than the number of coordinates with the control card, determining the initial coordinate, the destination coordinate, and the path between the determined coordinates with the control card (1032),

[0073] Checking whether the number of operations is an even number with the control card (1033)

[0074] If the number of operations is an even number, initiating hoeing by starting the linear motor (102) with the control card (1033.1) According to the direction determined advancing the unmanned ground vehicle while performing hoeing with the control card (1033.2),

[0075] Increasing the number of operations by one with the control card and returning to the process step of checking whether the number of operations is greater than the number of coordinates with the control card (1033.3),

[0076] If the number of operations is not an even number, calculating the instantaneous distance between the instantaneous coordinate and the target with the control card (1034),

[0077] Calculating the main distance between the initial coordinate and the target with the control card (1035),

[0078] Finding the error figure by subtracting the instantaneous distance from the calculated main distance with the control card (1036),

[0079] Checking whether the error figure is greater than a determined distance with the control card (1037),

[0080] If the error figure is less than a determined distance, stopping the robot by reducing the robot speed within the control card according to the error figure, increasing the number of control cards by one, and returning to the process step of checking whether the number of operations is greater than the number of coordinates with the control card (1038),

[0081] If the error figure is greater than a determined distance, receiving the number of objects detected on the right and left camera and the pixel coordinates of the objects on the camera (101) with the control card (1039),

[0082] Setting the object counter value to zero with the control card (1040),

[0083] Checking whether the object counter and the number of detected objects are equal with the control card (1041 ), If the object counter and the number of detected objects are equal, using the tilt data obtained by processing the images received from the front camera (101 ) with the control card (1042)

[0084] Actuating the linear motor (102) for hoeing with the control board (1042.1),

[0085] Adjusting the unmanned ground vehicle to straighten itself according to the tilt and continue on its route with the control card and returning to the process step of checking whether the error figure is greater than a determined distance with the control card (1042.2),

[0086] If the object counter and the number of detected objects are not equal, sorting the objects in a single row according to the pixel coordinates on the camera (101) with the control card (1043),

[0087] Slowing down and stopping the robot according to the pixel coordinate on the x-axis of the object with the closest sorting result with the control card (1044)

[0088] Directing the middle point on the y-axis of the camera (101) to the y-axis of the object with the servo motor connected to the camera (101) detecting the object with the control card (1045),

[0089] With the control card, spraying the liquid in the pesticide chamber for pesticide spraying by actuating the pesticide motor and increasing the object counter value by one and returning to the process step of checking whether the object counter and the number of detected objects are equal with the control card (1046).

Claims

CLAIMS1. The artificial intelligence-based agricultural unmanned ground vehicle system (100), characterized by comprising;- at least one interface enabling the user to enter the coordinates of the start and end of the row in the field,- at least one gyroscope detecting the direction of an unmanned ground vehicle,- at least one encoder sensor detecting the speed and direction in which the unmanned ground vehicle is moving,- at least one tilt sensor measuring the tilt of the unmanned ground vehicle,- at least three cameras (101) located in front, right and left of the unmanned ground vehicle for taking images of surrounding objects,- at least one hoe integrated into the unmanned ground vehicle,- at least one linear motor (102) enabling the hoe system on the unmanned ground vehicle to be moved for activation,- at least one pesticide chamber filled with pesticide on the unmanned land vehicle for spraying pesticides on crops,- at least one pesticide motor spraying the liquid in the pesticide chamber for pesticide spraying,- at least one control card in communication with the interface, gyroscope, encoder sensor, tilt sensor, camera (101), linear motor (102), and pesticide motor; detecting the position of the vehicle with the data obtained from the gyroscope, tilt sensor, and encoder sensor by receiving the coordinate entered via the interface as the instantaneous coordinate of the robot; finding the distance traveled by the unmanned ground vehicle; eliminating the need for a GPS sensor by calculating current latitude and longitude coordinates; detecting weeds, plant diseases and pests in the field with deep learning in images from the camera (101); converting the resulting image into a bird's eye view by applying color filter, edge determination, and edge density determination operations to the images received from the camera (101 ) facing the direction of the route taken by the unmanned land vehicle, creating a line each by averaging the white regions that appear in the right and left sections determined according to the middle point of theimage and calculating the lateral angle according to the tilt of that line by creating a line in the middle section according to the average of these two lines; enabling the unmanned ground vehicle to proceed to the target point while hoeing.

2. The artificial intelligence-based agricultural unmanned ground vehicle operation method, characterized by comprising the process steps of;- Entering the coordinates of the start and end of the rows in the field by the user via the interface (1000),- Receiving the first coordinate entered via the interface as the instantaneous coordinate of the unmanned ground vehicle and calculating the direction of the unmanned ground vehicle using the gyroscope and tilt sensor with the control card (1010),- Finding the instantaneous coordinate of the robot by processing the direction of the unmanned ground vehicle and the data from the encoder sensor located on the wheel of the robot together and assigning the number of operations value as zero with the control card (1020),- Checking whether the number of operations is greater than the number of coordinates with the control card (1030),- If the number of operations is greater than the number of coordinates with the control card, initiating hoeing (1031 ),- Generating a route between the instantaneous coordinate and the initial coordinate with the control card and advancing the unmanned ground vehicle to those coordinates (1031 .1 )- Stopping the unmanned ground vehicle when it approaches the initial coordinate with the control card (1031 .2)- Assigning the number of operations determined as zero with the control card and returning to the process step of checking whether the number of operations is greater than the number of coordinates with the control card (1003) (1031 .3),- If the number of operations is less than the number of coordinates with the control card, determining the initial coordinate, the destination coordinate, and the path between the determined coordinates with the control card (1032),- Checking whether the number of operations is an even number with the control card (1033)- If the number of operations is an even number, initiating hoeing by starting the linear motor (102) with the control card (1033.1)- According to the direction determined advancing the unmanned ground vehicle while performing hoeing with the control card (1033.2),- Increasing the number of operations by one with the control card and returning to the process step of checking whether the number of operations is greater than the number of coordinates with the control card (1033.3),- If the number of operations is not an even number, calculating the instantaneous distance between the instantaneous coordinate and the target with the control card (1034),- Calculating the main distance between the initial coordinate and the target with the control card (1035),- Finding the error figure by subtracting the instantaneous distance from the calculated main distance with the control card (1036),- Checking whether the error figure is greater than a determined distance with the control card (1037),- If the error figure is less than a determined distance, stopping the robot by reducing the robot speed within the control card according to the error figure, increasing the number of control cards by one, and returning to the process step of checking whether the number of operations is greater than the number of coordinates with the control card (1038),- If the error figure is greater than a determined distance, receiving the number of objects detected on the right and left camera and the pixel coordinates of the objects on the camera (101) with the control card (1039),- Setting the object counter value to zero with the control card (1040),- Checking whether the object counter and the number of detected objects are equal with the control card (1041),- If the object counter and the number of detected objects are equal, using the tilt data obtained by processing the images received from the front camera (101 ) with the control card (1042)- Actuating the linear motor (102) for hoeing with the control board (1042.1 ),- Adjusting the unmanned ground vehicle to straighten itself according to the tilt and continue on its route with the control card and returning to the process step of checking whether the error figure is greater than a determined distance with the control card (1042.2),- If the object counter and the number of detected objects are not equal, sorting the objects in a single row according to the pixel coordinates on the camera (101 ) with the control card (1043),- Slowing down and stopping the robot according to the pixel coordinate on the x- axis of the object with the closest sorting result with the control card (1044)- Directing the middle point on the y-axis of the camera (101 ) to the y-axis of the object with the servo motor connected to the camera (101) detecting the object with the control card (1045),- With the control card, spraying the liquid in the pesticide chamber for pesticide spraying by actuating the pesticide motor and increasing the object counter value by one and returning to the process step of checking whether the object counter and the number of detected objects are equal with the control card (1046).

3. The artificial intelligence-based agricultural unmanned land vehicle system (100) according to claim 1 , characterized by comprising a control card that finds the distance traveled by the robot in latitude and longitude as a result of multiplying the lateral angle of the unmanned land vehicle by its sine and cosine.

4. The artificial intelligence-based agricultural unmanned land vehicle system (100) according to claim 1 , characterized by comprising a control card that calculates the current latitude and longitude coordinates by calculating the calculated latitude, longitude distances and initial latitude and longitude coordinates with the Haversine formula, respectively.

5. The artificial intelligence-based agricultural unmanned land vehicle system (100) according to claim 1 , characterized by comprising a control card that finds the margin of error by calculating the distance between the instantaneous coordinate of the ground vehicle and the target coordinate and the distance between the initial coordinate and the target coordinate of the unmanned ground vehicle and checks whether it is greater than the determined value.

6. The artificial intelligence-based agricultural unmanned ground vehicle system (100) according to claim 1 or claim 5, characterized by comprising a control card thatstops the unmanned ground vehicle and reactivates the hoe to proceed to the next coordinate, if the margin of error is less than the determined distance.

7. The artificial intelligence-based agricultural unmanned ground vehicle system (100) according to claim 1 or claim 5, characterized by comprising a control card that receives the number of objects detected in the right and left camera (101 ) images and the pixel coordinates of the objects on the camera (101) if the margin of error is greater than the determined distance, and if it has not detected any object, calculates the lateral angle with the images from the front camera (101) and allows the unmanned ground vehicle to continue its route by straightening itself according to the tilt.

8. The artificial intelligence-based agricultural unmanned ground vehicle system (100) according to claim 1 or claim 7, characterized by comprising a control card that sorts the objects in a single row according to the pixel coordinates on the camera (101 ) if it detects objects from the right and left camera (101) images, and according to the pixel coordinate on the x-axis of the object with the closest sorting result, stops the robot and directs the middle point on the y-axis of the camera (101 ) to the y-axis of the object with the servo motor connected to the camera (101 ) detecting the object and sprays the liquid in the pesticide chamber by actuating the pesticide motor for pesticide spraying, and repeats this process until the detected objects are finished.

Images