Agricultural system and method for an agricultural task management

Abstract

A method for an agricultural task management and an agricultural system (100) configured to execute the method including the steps of creating or receiving a main task for an operation in an agricultural field (508), checking the main task whether any information for executing the main task is insufficient, determining the insufficient information, creating a supplementary task for acquiring necessary information, creating an assignment between the supplementary task and an autonomous vehicle (118, 120, 122), receiving the necessary information from the autonomous vehicle (118, 120, 122) and updating the main task based on the necessary information.

Description

AGRICULTURAL SYSTEM AND METHOD FOR AN AGRI CULTURAL TASK MANAGEMENTCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of the filing date of UK Application No. GB2311823.5, filed August 1, 2023, the disclosure of which is hereby incorporated herein in its entirety by this reference.FIELD

[0002] The present disclosure relates generally to an agricultural system and method for an agricultural task management.BACKGROUND

[0003] By usage of an agricultural task management system, agricultural field tasks such as ploughing, seeding, spraying or harvesting can be planned in detail before being executed in an agricultural field. The agricultural task management can be implemented in terms of a farm management information system (FIMS). An operator of the agricultural task management can define offboard (e. g. at home) the upcoming agricultural operations to be done for a specific agricultural field at a specific time by a specific agricultural machine in terms of one or more tasks. Each task comprises information about what is to be done in terms of at least one parameter. For example, a task may define a specific amount (first parameter) of a specific type of material (second parameter) to be applied on a specific type of crop (third parameter) exceeding a specific threshold such as crop height (fourth parameter) at a specific time (fifth parameter), etc. The fully defined tasks can be transferred to an agricultural machine to be executed (semi-)automatically in the agricultural field.BRIEF SUMMARY

[0004] The execution of a field task may fail if a field task could not be sufficiently defined due to an insufficient information. For example, if five parameters such as mentioned above are required for a full definition of a field task, the field task cannot be defined sufficiently if one (or more) of the five parameters is unknown or out of date. Thus, it is an objective to provide an agricultural system and a method for an agricultural task management that enables the definition of a field task in case of insufficient information.

[0005] According to an aspect of the invention, a method for an agricultural task management comprises steps for creating or receiving a main task for an operation in an agricultural field, checking the main task whether any information for executing the main task is insufficient, creating a supplementary task for acquiring necessary information in response to a determined insufficient information, creating an assignment between an autonomous vehicle and the supplementary task for acquiring the necessary information, receiving the necessary information from the autonomous vehicle and updating the main task based on the necessary information.

[0006] The method steps may be executed by a control unit being part of an agricultural system for agricultural task management. The agricultural system may comprise a human machine interface (HMI) for an user interaction. The main task may be any field task comprising at least one agricultural field operation to be performed in an agricultural field such as ploughing, seeding, weeding, fertilizing, spraying, harvesting, baling, swathing, etc. or any field operation to analyze the agricultural field such as determining a soil parameter or a crop parameter (e. g. crop health). The main task may be created by the control unit automatically when executing the method or by an operator operating the HMI of the agricultural system for agricultural task management. The main task may be received by the control unit from an external source communicatively connected with the agricultural system as for example a data base of a server or an agricultural vehicle. I. e., an existing main task may be reused and transferred from the external source to the agricultural system for agricultural task management.

[0007] The main task may be defined by any type of information such as for example the type of field operation(s) to be performed in an agricultural field and one or more parameters required to sufficiently define the main task. For example, a specific amount (first parameter) of a specific type of material (second parameter) to be applied on a specific type of crop (third parameter) exceeding a specific threshold such as crop height (fourth parameter) at a specific time (fifth parameter), etc. may be required for defining sufficiently a main task including a spraying operation. But if at least one parameter is unknown, invalid or indefinite, the main task may lack of a clear definition. Thus, information, hereafter necessary information, may be needed to be acquired in order to provide the at least one parameter for a clear definition of the main task.

[0008] The insufficient information as well as the necessary information may be a field parameter that may comprise a soil parameter and / or a crop parameter. The field parameter may be a parameter to be determined from the air.

[0009] A soil parameter may be for example• level of soil moisture,• temperature of the soil,• content of nutrient in the soil,• content of nitrogen, phosphate or other chemical compounds in the soil, or• level of alkalinity of the soil, etc.

[0010] A crop parameter may be for example• height of crops,• degree of ripeness of crop,• crop health, or• amount of fungal, herbivore or parasite infestation of the crop, etc.

[0011] A parameter to be determined from the air may be for example• humidity,• air temperature,• atmospheric pressure,• an obstacle in the agricultural field, or• a field boundary of the agricultural field, etc.

[0012] The control unit may determine which parameter causes a lack of a clear definition of the main task to determine the insufficient information, for example by means of a consistency check of all information required for the main task. The consistency check may recognize if an information is out of date, out of an expected range or generally not available.

[0013] The control unit may automatically create a supplementary task for gathering the necessary information in response to the determined insufficient information. For example, a main task for a spraying operation in the agricultural field may be insufficiently defined if an appropriate pesticide against a specific infestation of the crop cannot be defined due to unknown information about the infestation. Thus, the type of the infestation of the crop would be the necessary information in case of the insufficient information of an appropriate pesticide to be used for the spraying operation against this infestation. As consequence, a supplementary task may be defined to determine the type of infestation of the crop to determine an appropriate pesticide against this infestation if the type of the pesticide is the insufficient information of a main task for a spraying operation in the agricultural field.

[0014] In general, the supplementary task may comprise a task for determining the necessary information. The necessary information may be a soil parameter or a crop parameter.

[0015] The supplementary task may comprise more details how the soil or crop parameter shall be determined. For example, the infestation of the crop as crop parameter may be determined by taking an image of the invested crop by a camera. The control unit may receive the image and determine the infestation based on an analysis of the image. In dependence of which type of infestation has been determined, an appropriate pesticide may be automatically determined by the control unit. For example, a fungicide may be determined in case of a fungal infestation of the crop.

[0016] The method may comprise a step for determining an autonomous vehicle configured to execute the supplementary task.

[0017] The agricultural system for agricultural task management may comprise at least one autonomous vehicle configured to execute a supplementary task. The autonomous vehicle may be an unmanned agricultural robot or an unmanned aerial vehicle (UAV) such as a drone equipped with a sensor for determining the necessary information. The information of the equipment of the at least one autonomous vehicle may be stored in a database so that the agricultural system may automatically check whether the at least one autonomous vehicle is configured to determine the necessary information. For example, the control unit may check whetherthe autonomous vehicle is equipped with an appropriate sensorto acquire the necessary information. Optionally, the control unit may check whether the autonomousvehicle is available or planned for executing another task. If the at least one autonomous vehicle is configured to determine the necessary information, the autonomous vehicle may be selected for executing the supplementary task. For example, a first autonomous vehicle may be equipped with a camera and a second autonomous vehicle may be equipped with a moisture sensor. Hence, in case of an infestation of the crop being the necessary information, the control unit would determine the first autonomous vehicle due to the camera for taking an image of an infestation of the crop instead of the second autonomous vehicle configured to determine soil moisture. After determining the autonomous vehicle configured to execute the supplementary task, an assignment between this autonomous vehicle and the supplementary task may be created by the control unit.

[0018] The method may comprise a step for determining a target point in the agricultural field for executing the supplementary task.

[0019] The target point may define a local position in the agricultural field where the necessary information can be determined when the supplementary task will be executed. Depending on the insufficient information, a soil parameter, a crop parameter or a field parameter from the air may be required to be determined as necessary information at the target point. If need, more than one target point may be determined to determine the necessary information. For example, the control unit may determine at least one target point where infestation is expected in the agricultural field in case of the infestation of the crop being the necessary information. In case of soil moisture as necessary information for example, the control unit may determine at least a first target point at a position where very high moisture is expected and at least a second point where very low moisture is expected. Based on these points, a distribution of soil moisture within the agricultural field may be estimated by the control unit.

[0020] The method may comprise steps for determining the position of the autonomous vehicle assigned to the supplementary task and determining a travel path from the position of the autonomous vehicle to the target point.

[0021] When the assignment between the autonomous vehicle and the supplementary task has been created by the control unit, the control unit may receive the position of the autonomous vehicle from the autonomous vehicle itself. For example, the autonomous vehicle may comprise a position determination unit such as a global navigation satellite system (GNSS) receiver to determine the position of the autonomous vehicle. Theautonomous vehicle may be communicatively connected with the control unit via a wireless network, e. g. a data cloud, for transferring the position signal of the autonomous vehicle to the control unit. Then, both the position of the autonomous vehicle and the target point are available for the control unit and can be mapped. Based on the mapped position of the autonomous vehicle and the target point, the control unit may search for a route from the mapped position of the autonomous vehicle to the mapped target point to determine the travel path. While searching for the route, the control unit may consider other elements that may be mapped as for example obstacles such as trees or transmission towers, impassible areas such as rivers, traffic, roads, or any other elements that may be considered by a navigation system for routing.

[0022] The method may comprise a step for guiding the autonomous vehicle to the target point.

[0023] When the travel path from the position of the autonomous vehicle to the target point has been determined, the autonomous vehicle may be automatically controlled to move along the travel path to the target point where the supplementary task is to be executed. Based on the position determination unit of the autonomous vehicle, a deviation from the travel path may be recognized. Then, the autonomous vehicle may adjust automatically the driving direction to move back and follow the travel path again. The position determination unit may also signal when the target point has been arrived by the autonomous vehicle.

[0024] The method may comprise a step for initiating the execution of the supplementary task.

[0025] The execution of the supplementary task may be initiated by the autonomous vehicle itself, by the operator operating the computer HMI or automatically by the control unit. The autonomous vehicle may send a signal via the network to the control unit to indicate that it has arrived the target point. In response to the signal received by the control unit, the control unit may send a signal via the network to the autonomous vehicle to start execution of the supplementary task.

[0026] The execution of the supplementary task may be initiated if at least one of the following conditions may be fulfilled: The autonomous vehicle has arrived the target point, the autonomous vehicle has passed a self-test, the weather conditions enable the execution of the supplementary task or a geofence has been activated.

[0027] The conditions may ensure that the supplementary task can be executed successfully or to avoid any interruptions of the supplementary task. The conditions may also ensure a safe execution of the supplementary task without endangering a person. For example, it can be prevented that the supplementary task will be executed at a place other the target point. The autonomous vehicle may perform a self-test to check whether the autonomous vehicle and its sensing unit work properly. The weather forecast may be analyzed to check whether the supplementary task can be executed. For example, a rainy day may disturb the sensing unit of the autonomous vehicle so that the supplementary task may be postponed to a day for which a dry weather is forecasted. Additionally, the supplementary task may be executable after a geofence has been activated. The geofence may restrict the autonomous vehicle to cross a virtual border, e. g. a farmyard boundary, to avoid an unintentional leaving of the agricultural field. Thus, a collision with obstacles outside the field boundary can be prevented.

[0028] The method may comprise a step for executing the supplementary task and for acquiring the necessary information.

[0029] When at least one of the conditions is fulfilled, a vehicle control unit of the autonomous vehicle may execute the supplementary task. Alternatively, the control unit may send instructions via the network to the autonomous vehicle for executing the supplementary task. For example, the supplementary task may comprise a task to take a photo of a crop located at the target point to determine the infestation of the crop as necessary information. Then, the autonomous vehicle is controlled to take a photo of the crop by the sensing unit comprising a camera. Alternatively, the supplementary task may comprise a task to determine the soil moisture at the target point. Then, the autonomous vehicle is controlled to measure the soil moisture by the sensing unit comprising a moisture senor. When the necessary information has been acquired, the control unit may receive the necessary information from the autonomous vehicle via the network.

[0030] The method may comprise a step for updating the main task based on the necessary information.

[0031] The necessary information may be displayed to an operator of the computer HMI to update the main task manually. Alternatively, the control unit may replace the insufficient information by sufficient information based on the necessary information acquired by the autonomous vehicle for updating the main task. For example in case of anappropriate pesticide as insufficient information, the control unit may determine the infestation of the crop based on the received photo and determine an appropriate pesticide based on the determined infestation. Then, the control unit may update the main task to define the usage of the determined pesticide. The necessary information may also be used to optimize a parameter of the main task to enhance the performance of the field operation defined by the main task. Optionally, the main task may comprise a time conflict. So, the necessary information may cause a rescheduling of the main task for executing the main task at another time point for avoiding the time conflict. The necessary information may also replace invalid or missing information of the main task by valid and up-to-date information based on the necessary information. Consequently, the main task may be sufficiently defined by the control unit.

[0032] The method may comprise a step for initiating the execution of the updated main task.

[0033] The operator or the control unit may determine an agricultural vehicle for executing the updated main task. The updated main task may be sent by the control unit via the network to the agricultural vehicle. The operator may drive the agricultural vehicle to the agricultural field for executing the updated main task. The updated main task may be executed manually or automatically.

[0034] The execution of the updated main task may be initiated if at least one of the following conditions may be fulfilled: All necessary information has been acquired, or the agricultural vehicle has arrived the agricultural field.

[0035] The initiation of the execution of the updated main task may be prevented as long as the agricultural vehicle has not arrived the agricultural field or as long as all necessary information has not been acquired. The agricultural vehicle may comprise a position determination unit to detect when the agricultural vehicle has arrived the agricultural field. Then, the initiation may be automatically allowed. Optionally, the execution of the main task may be prevented until all necessary information has been acquired for updating the main task. Additionally, a warning message may be shown to the operator.

[0036] The agricultural system for agricultural task management comprises the agricultural vehicle for operating in the agricultural field, the at least one autonomous vehicle configured to execute a supplementary task, and the control unit configured to carry out at least one of the method steps as described above.

[0037] The agricultural vehicle may by any vehicle or vehicle combination of any type such as a tractor, a harvester, a combine, a sprayer etc. optionally comprising any implement or tool of any type such as a trailer, a baler, a mower, etc. The control unit may be part of a computer system configured for controlling the agricultural task management. Optionally, the agricultural vehicle and / or the at least one autonomous vehicle comprises a control unit for executing one or more method steps of the method described above. For example, a control unit of the autonomous vehicle may execute the supplementary task and the control unit of the agricultural vehicle may execute the updated main task.

[0038] The control unit and the at least one autonomous vehicle may be communicatively connected.

[0039] The agricultural system for an agricultural task management may comprise a network to which the control unit and the at least one autonomous vehicle are connected. Additionally, the control unit of the agricultural vehicle may be connected with the network. Thus, the control unit of the computer system, the control unit of the agricultural vehicle and the vehicle control units of the autonomous vehicles may send signals to the network that may be received from any other control unit connected with the network to communicate with each other. Thus, any information such as the insufficient information and the necessary information can be shared between all connected control units. The network may be designed as a cloud network for exchanging the signals wirelessly. Thus, no manual data transfer is necessary.

[0040] The autonomous vehicle may be an agricultural robot or an unmanned aerial vehicle.

[0041] The agricultural robot may be a wheeled or tracked vehicle to move through the agricultural field. The agricultural robot may provide a determination of necessary information close to the soil or from a soil perspective. Alternatively, the unmanned aerial vehicle may provide a determination of necessary information from the air or from a bird's eye perspective.

[0042] The at least one autonomous vehicle may comprise at least one of the following components: A sensing unit configured to determine a soil parameter, a sensing unit configured to determine a crop parameter, or a sensing unit configured to determine a field parameter from the air.

[0043] Determining a soil parameter may comprisedetermining the soil moisture,• determining the temperature of the soil,• determining the content of nutrient in the soil,• determining the content of nitrogen, phosphate or other chemical compounds in the soil, or• determining the alkalinity of the soil, etc.

[0044] Determining a crop parameter may comprise• determining the height of crops,• determining the degree of ripeness of crop,• determining the crop health, or• determining the amount of fungal, herbivore or parasite infestation of the crop, etc.

[0045] Determining a parameter from the air may comprise• humidity,• air temperature,• atmospheric pressure,• obstacles in the agricultural field, or• field boundary of the agricultural field, etc.

[0046] The determination of a soil parameter may be carried out by the agricultural robot whereas the determination of the parameter from the air may be carried out by the unmanned aerial vehicle. The determination of the crop parameter may be carried out by both the agricultural robot and the unmanned aerial vehicle.

[0047] Within the scope of this application it should be understood that the various aspects, embodiments, examples and alternatives set out herein, and individual features thereof may be taken independently or in any possible and compatible combination. Where features are described with reference to a single aspect or embodiment, it should beunderstood that such features are applicable to all aspects and embodiments unless otherwise stated or where such features are incompatible.BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Several aspects of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0049] FIG. 1 illustrates an agricultural system for an agricultural task management.

[0050] FIG. 2 illustrates a block diagram of an autonomous vehicle.

[0051] FIG. 3 illustrates a simplified view of a control unit.

[0052] FIG. 4 illustrates a schematically an optical sensor.

[0053] FIG. 5 illustrates a farmyard and an agricultural field in combination with the agricultural system for the agricultural task management.

[0054] FIG. 6 illustrates a flow chart of a method for an agricultural task management.DETAILED DESCRIPTION

[0055] FIG. 1 shows an agricultural system 100 for an agricultural task management comprising an agricultural vehicle 112, at least one autonomous vehicle 118, 120, 122, a network 102, and a computer system 104. The network 102 is a wireless network, e. g. implemented as a data cloud, to which the agricultural vehicle 112, the at least one autonomous vehicle 118, 120, 122 and the computer system 104 are connected with for (bidirectionally) exchanging signals to each other. Optionally, one or more entities connected with the network 102 may have a wired connection, e. g. via a data cable or radio. The computer system 104 comprises a computer 106 with an integrated control unit 110 and a computer human machine interface 108, in the following computer HMI 108, for an operator. The computer HMI 108 comprises an output device as a screen for displaying data, e. g. the data received by the network 102, and an input device as a keyboard and a mouse to control the computer 106.

[0056] The agricultural vehicle 112, here exemplarily shown as a tractor, may by any vehicle or vehicle combination of any type such as a harvester, a combine, a sprayer etc. optionally comprising any implement or tool of any type such as a trailer, a baler, a mower, etc. The agricultural vehicle 112 comprises a control unit 116 and a vehicle human machineinterface 114, in the following vehicle HMI 114, for an operator to control the agricultural vehicle 112 and the implement. The vehicle HMI 114 is connected with the control unit 116 and comprises an output device as a display for displaying data, e. g. data received by the control unit 116, and an input device to control a function of the agricultural vehicle 112.

[0057] The at least one autonomous vehicle 118, 120, 122 can be three vehicles as shown in FIG. 1. Each autonomous vehicle may be a vehicle of any type such as an unmanned robot or an unmanned aerial vehicle (UAV), for example in terms of a quadcopter drone. As exemplarily shown in FIG. 1, first and second autonomous vehicles 118 and 120 are unmanned robots and third autonomous vehicle 122 is an UAV. The autonomous vehicles 118, 120 and 122 may receive control commands from the computer system 104 or from the agricultural vehicle 112 via the network 102 for executing a task autonomously without any further human interaction.

[0058] FIG. 2 is a schematical illustration for each of the autonomous vehicles 118, 120 and 122 in terms of a block diagram and shows that each autonomous vehicle 118, 120 and 122 comprises a vehicle control unit 202, a sensing unit 204 and a position determination unit 206.

[0059] The position determination unit 206 provides position and time signals for determining an absolute position of the at least one autonomous vehicle 118, 120, 122 at a specific point of time. The position determination unit 206 may be an inertial measurement unit (IMU) and / or a global navigation satellite system (GNSS) receiver receiving position and time signals from a GNSS such as GPS or Galileo. The IMU may provide additional information about the orientation and movement of the at least one autonomous vehicle 118, 120, 122 for improving the accuracy of the position estimation and the reference points of the GNSS receiver. Based on received position and time signals, the at least one autonomous vehicle 118, 120, 122 can move autonomously from a home point along a travel path to a target point and back.

[0060] The sensing unit 204 of the at least one autonomous vehicle 118, 120, 122 may comprise at least one sensor for gathering any information of an agricultural field, e. g. the agricultural field 508 shown in FIG. 5. The information may be any information or parameter of the agricultural field 508 such as height of crops, degree of ripeness of crop, soil moisture, content of nitrogen, phosphate or other chemical compounds in the soil, alkalinity of the soil, amount of fungal, herbivore or parasite infestation of the crop, etc. The at leastone sensor may be of any type applicable to detect a parameter of the agricultural field 508. For example, the at least one sensor is a LIDAR, an ultrasonic sensor or a camera as exemplarily shown in FIG. 4. In case of two or more sensors, the sensors may be of different type.

[0061] The autonomous vehicles 118, 120 and 122 may be equipped with different sensing units 204. For example, the first autonomous vehicle 118 may be equipped with a sensing unit 204 configured to determine at least one soil parameter such as soil moisture, alkalinity of the soil or the content of nitrogen, phosphate or other chemical compounds in the soil; the second autonomous vehicle 120 may be equipped with a sensing unit 204 configured to determine at least one crop parameter such as height of crops, degree of ripeness of crop or the amount of fungal, herbivore or parasite infestation of the crop; the third autonomous vehicle 122 may be equipped with a sensing unit 204 configured to determine a para meter of or from the air such as an optical sensor (e. g. camera) for capturing images of the agricultural field 508 when flying overthe agricultural field 508. The information of the equipment of each autonomous vehicle 118, 120, 122 is stored in the memory 306 of the control unit 110 by means of a database so that the control unit 110 or any other control unit connected with the network 102 can determine the type of the sensing unit 204 of each autonomous vehicle 118, 120 122.

[0062] The position determination unit 206 and the sensing unit 204 are connected with a vehicle control unit 202 of the at least one autonomous vehicle 118, 120, 122. The vehicle control unit 202 may receive the captured signals of the position determination unit 206 and the sensing unit 204 and process the signals. These signals may be provided by the vehicle control unit 202 to the network 102. Since the other control units 110 and 116 are connected with the network 102, the control units 110 and 116 can receive the signals of the vehicle control unit 202 sent to the network 102. Vice versa, the vehicle control unit 202 of the at least one autonomous vehicle 118, 120, 122 may receive signals from the network 102 that have been provided by the control unit 110 of the computer system 104 or by the control unit 116 of the agricultural vehicle 112. Thus, each control unit communicatively connected to the network 102 can send signals to another control unit communicatively connected with the network 102 and receive signals from any control unit communicatively connected with the network 102.

[0063] FIG. 3 shows a control unit 300 comprising an I / O interface 302, a controller 304 and a memory 306. Control unit 300 represents the control unit 110 of the computer system 104, the control unit 116 of the agricultural vehicle 112 and the vehicle control unit 202 of the at least one autonomous vehicle 118, 120, 122. I. e., the control unit 110, control unit 116 and the vehicle control units 202 comprise each a memory 306, a controller 304 and an I / O interface 302. The control unit 300 may receive and send signals or data via the I / O interface 302. The I / O interface 302 may be a wireless I / O interface or a connector and may be connected with the network 102. So, each of the control units 110, 116 and 202 can be communicatively connected with the network 102 (see FIG. 1) to exchange signals or data between each other. The controller 304 may store the data or signals received by the control unit 300 in the memory 306. The memory 306 may contain additional data or executable computer program products, for example in terms of a computer-implemented method, that may be retrieved, processed or executed by the controller 304. Data or signals resulting from the processing of data or signals or from the execution of a computer program product may be stored to the memory 306 or sent to the I / O interface 302 by the controller 304.

[0064] For example, the vehicle control unit 202 of the at least one autonomous vehicle 118, 120, 122 represented by the control unit 300 may process received signals to control any function of the at least one autonomous vehicle 118, 120, 122. The vehicle control unit 202 may control speed and driving direction of the at least one autonomous vehicle 118, 120, 122 to move the at least one autonomous vehicle 118, 120, 122 through the agricultural field 508 and to perform an operation in the agricultural field 508 such as determining a parameter of the agricultural field 508 at a certain position in the agricultural field 508. In combination with the position determination unit 206, the at least one agricultural vehicle 118, 120, 122 can be guided by the vehicle control unit 202 along a path, e. g. a field path or a travel path. The vehicle control unit 202 detects any deviations between the path and the current position of the at least one autonomous vehicle 118, 120, 122 driving along the path received from the position determination unit 206. In case of a deviation, the vehicle control unit 202 commands a correction action, e. g. an automatic steering action to hold the at least one autonomous vehicle 118, 120, 122 exactly on the path. Such systems are well known as vehicle guidance systems. Analogously to the at least one autonomous vehicle 118, 120, 122, the agricultural vehicle 112 comprises a position determination unit so that the control unit116 represented by control unit 300 can also control speed and driving direction of the agricultural vehicle 112 to guide it along a path.

[0065] FIG. 4 shows an exemplarily embodiment of the sensing unit 204 if the sensing unit 204 is of the type of a 2D-camera, a stereo camera or a time-of-flight (ToF) camera. A ToF camera could provide depth information and improve accuracy of detection and pose estimation. Depending on the type of the sensing unit 204, the sensing unit 204 may capture 2D or 3D images, gray-scale images, color images in any color space as for example in red-green-blue (RGB) color space, or multispectral images.

[0066] The sensing unit 204 may comprise several components such as at least one optical lens 404, an optional filter 406, a detector 408 and a processing circuitry 410. The optical lens 404 may collect and direct light from a field of view 402 of the sensing unit 204 through the filter 406 to the detector 408 and serve to focus and / or magnify images. The at least one optical lens 404 may be of the type of a fisheye lens, a rectilinear lens or any other standard and moderate wide-angle lens. A fish-eye lens may be of the type of a F-theta lens, a F-tan lens, a tailored distortion lens or a fovea lens, for example. A standard lens is typically defined as a lens with a focal length being approximately equal to the diagonal of the detector 408. This results in a field of view 402 that is rather similar to what human eyes see. Moderate wide-angle lenses have shorter focal lengths than standard lenses, typically ranging from 24 mm to 35 mm for full-frame cameras. These lenses offer a wider field of view 402 than standard lenses and can capture more of the scene in the frame. The optional filter 406 passes selected spectral bands such as ultraviolet, infrared or other bands. The detector 408 may be a digital image sensor that converts electromagnetic energy to an electric signal and employs image sensing technology such as charge-coupled device (CCD) technology and / or complementary metal oxide semiconductor (CMOS) technology. The processing circuitry 410 may include a circuitry for amplifying and processing the electric signal generated by the detector 408 to generate image data, which is passed to the one or more computing devices such as the vehicle control unit 202.

[0067] The sensing unit 204 may be moveable so that the pose (i. e. position and / or orientation) of the sensing unit 204 may change. The movement may be determined by a corresponding sensor as for example a position sensor.

[0068] The sensing unit 204 may receive position and time signals from the position determination unit 206 for geo-referencing and time stamping of each captured image. Thedata captured by the sensing unit 204 is logged along with the position and time data gathered by position determination unit 206 allowing an accurate determination of the global position of objects contained in the captured images.

[0069] FIG. 5 shows a farmyard 500 localized by its farmyard boundary 502. On the farmyard 500, there may be several facilities such as a building 504. The computer system 104 is located in the building 504. The agricultural vehicle 112 and the at least one autonomous vehicle, here the first, second and third autonomous vehicle 118, 120 and 122 are parked on the farmyard 500. The third autonomous vehicle 122 has a landing field on the building. Alternatively, the at least one autonomous vehicle 118, 120, 122 may be parked in the agricultural field, e. g. in the headland. The computer system 104 and the vehicles 112, 118, 120 and 122 may be connected to the network 102 as explained above and shown in FIG. 1.

[0070] Next to the farmyard 500, an agricultural field 508 is localized by its field boundary 510. For this agricultural field 508, a headland and several field paths 512 can be defined. The information of the field boundary 510, the headland and the field paths 512 can be stored in the memory 306 of at least one of the control units 110, 116 and / or 202. The field paths 512 can be shared via the network 102 to each vehicle 112, 118, 120 and 122 for guiding each of the vehicles along these paths as described above.

[0071] A road 506, e. g. a public road, extends next to the farmyard 500 and the agricultural field 508. The farmyard 500 comprises an entry connected with the road 506 so that the vehicles 112, 118 and 120 can easily enter the road 506 to drive from the farmyard 500 to the agricultural field 508 and back. The third autonomous vehicle 122 can directly fly from the farmyard 500 to the agricultural field 508. When the third autonomous vehicle 122 has reached the agricultural field 508, the third autonomous vehicle 122 may be guided along one of the field paths 512 as described above whilst the third autonomous vehicle 122 is flying.

[0072] FIG. 6 shows a flow chart of a method for an agricultural task management. The method may be at least partly a computer-implemented method stored as a computer program product in the memory 306 of the control unit 300 representing at least one of the control unit 110 of the computer system 104, the control unit 116 of the agricultural vehicle 112 and the vehicle control unit 202 of the at least one of the autonomous vehicle 118, 120, 122. The control unit 300 is configured to carry out the method. Computer-implemented parts of the method may be executed by the control unit 300. Non-computer-implemented parts may be executed manually or by other components of the agricultural system 100. Themethod is described by way of example of several steps without any restriction in respect of that steps. I. e. the number or the order of steps may be adapted, for example single steps may be excluded and / or added and executed earlier or later than described. The method starts with step S100 and proceeds to step S101.

[0073] At step S101, a main task for an operation in the agricultural field 508 is created. For example, an operator starts a main task management application installed on the memory 306 of the control unit 110 of the computer 106 and defines at least one main task to be performed in the agricultural field 508 by using the computer HMI 108. The main task may define a specific field operation for treating the agricultural field 508 such as ploughing, seeding, weeding, fertilizing, spraying, harvesting, swathing, baling, etc. or any operation to analyze the agricultural field 508 such as determining the soil moisture, determining the temperature of the soil, determining the height of crops, determining the degree of ripeness of crop, determining the content of nitrogen, phosphate or other chemical compounds in the soil, determining the alkalinity of the soil, determining the amount of fungal, herbivore or parasite infestation of the crop, etc. The main task may comprise information which vehicle, vehicle combination or implement is selected to execute the main task, at which time and date the main task is to be executed and where in the agricultural field 508 the main task is to be executed. For example, the main task may comprise information where the selected vehicle needs to enter the agricultural field 508 and how to traverse the agricultural field 508 by defining the order of the field paths 512 along which the vehicle needs to be guided. The main task may comprise additional parameters such as the vehicle speed to traverse the agricultural field 508 when executing the main task, etc. The main task may comprise any condition to be fulfilled to initiate the main task. For example, the main task may define to execute a harvesting operation when the degree of ripeness of crop has reached a predefined level or to execute a spraying operation when the amount of fungal, herbivore or parasite infestation of the crop has exceeded a specific threshold or to execute a fertilizing operation when the content of nitrogen, phosphate or other chemical compounds in the soil has fallen below a predefined threshold. After completion of the creation of the main task, the main task can be stored to the memory 306 of the control unit 110 and shared in the network 102.

[0074] Alternatively, the control unit 110 may receive a main task for the operation in the agricultural field 508 that has been created before. For example, the computer system 104 may show to the operator of the computer system 104 all already existing main taskssaved in the memory 306 of the control unit 110 on the computer HMI 108 to select one main task. When the operator has selected a main task, the control unit 110 receives the selected main task from its memory 306. If the main task is stored in a memory of another control unit communicatively connected with the network 102, e. g. control unit 116 of the agricultural vehicle 112, the control unit 110 may receive the main task from the other control unit via the network 102.

[0075] Then, the method proceeds to step S102 and the control unit 110 checks whether any information for executing the created or received main task is insufficient.

[0076] Information for executing the created or received main task may be insufficient if the information is missing, invalid, outdated or out of an expected range or resolution for example. The insufficient information may also be information that has not been determined before yet or has been lost or deleted. The main task may be inexecutable due to invalid or missing information. In case of insufficient information being out of date or out of an expected range, the main task may be still executable but with reduced performance. The insufficient information may refer to a parameter of the agricultural field 508 such as the soil moisture, the temperature of the soil, the height of crops, the degree of ripeness of crop, the content of nitrogen, phosphate or other chemical compounds in the soil, the alkalinity of the soil, the amount of fungal, herbivore or parasite infestation of the crop, etc.

[0077] If all information is present and sufficient for executing the main task, the method steps to step S115 for initiating the execution of the main task analogously as described for the execution of the updated main task. If any insufficient information has been recognized by the control unit 110, the method proceeds to step S103. At step 5103, the control unit 110 determines which information is insufficient. For example, the main task comprises a condition fora spraying operation to spray a pesticide when the amount of fungal, herbivore or parasite infestation of the crop has exceeded a specific threshold. But the information of the current amount of fungal, herbivore or parasite infestation of the crop has not been determined before and is therefore missing. Thus, the control unit 110 cannot determine whether the condition is fulfilled (or not) since a comparison with the threshold is not possible due to the insufficient information of the current amount of fungal, herbivore or parasite infestation of the crop. According to another example, the soil moisture may be the insufficient information in case of a main task defining a maximum vehicle speed in dependence of the soil moisture. So, the control unit 110 will recognize that the maximumvehicle speed for the selected vehicle to perform the main task cannot be determined due to the insufficient information of the soil moisture. According to another example, the main task may define a plough operation in the agricultural field 508 wherein settings of the plough shall be automatically adjusted by the control unit 110 according to the level of soil moisture. But the control unit 110 will detect if settings could not be adjusted due an insufficient information of the soil moisture. According to another example, the main task may define a spraying operation depending on the type of the infestation of the crop. But the control unit 110 will detect if the appropriate pesticide can not be defined for the main task when the information about the infestation of the crop is insufficient. I. e., the control unit 110 needs the necessary information about the infestation of the crop to determine whether an herbicide, a fungicide or an insecticide shall be used as pesticide and to determine the amount of pesticide to be applied on the crop. Hence, the control unit 110 can identify the insufficient information based on an incomplete or incorrect definition of the main task or may identify the insufficient information by means of a plausibility check or consistency check.

[0078] The method proceeds to step S104 and the control unit 110 creates a supplementary task for acquiring necessary information in response to the determined insufficient information. Depending on which information is insufficient a different supplementary task is created by the control unit 110. For example, if the insufficient information is an outdated field parameter, the control unit defines a supplementary task to acquire an up-to-date parameter as necessary information. If the insufficient information is out of an expected range or resolution, the control unit defines a supplementary task to acquire a refined parameter of the agricultural field 508 as necessary information. In case of an insufficient information in respect of the agricultural field 508, the control unit 110 defines a supplementary task to analyze the agricultural field 508 by means of sensing an appropriate parameter of the agricultural field 508 as necessary information. The field parameter may be a soil parameter such as soil moisture, alkalinity of the soil or the content of nitrogen, phosphate or other chemical compounds in the soil or a crop parameter such as height of crops, degree of ripeness of crop or the amount of fungal, herbivore or parasite infestation of the crop for example. The field parameter can also be a field parameter to be captured from the air. The control unit 110 may define a supplementary task to sense a parameter of the soil, to sense a parameter of the crop or to capture a parameter of the agricultural field 508 from the air. For example, the control unit 110 may define a supplementary task to sense thesoil moisture of the agricultural field 508 as a soil parameter if the soil moisture is the insufficient information. Alternatively, the control unit 110 may define a supplementary task to sense the amount of herbivore infestation of the crop as crop parameter if the amount of herbivore infestation of the crop is the insufficient information. Or, the control unit 110 may define a supplementary task to capture an image of the agricultural field 508 from the air to determine a specific parameter of the agricultural field 508 such as the number of crop rows, the distance between the crop rows, etc.

[0079] Then, the method proceeds to step S105 and the control unit 110 determines an autonomous vehicle 118, 120, 122 configured to execute the supplementary task. Depending on the supplementary task, only a specific vehicle out of the three autonomous vehicles 118, 120 and 122 may be applicable to execute the supplementary task. If the necessary information is a specific parameter of the agricultural field 508, the applicable autonomous vehicle needs to be equipped with the corresponding sensing unit 204 for determining the parameter of the agricultural field 508. The control unit 110 checks which of the three autonomous vehicles 118, 120, 122 is equipped with the appropriate sensor sensing unit 204 by accessing the database stored in the memory 306 of the control unit 110 that contains information about the sensing units of all autonomous vehicles 118, 120 and 122. Hence, the control unit 110 determines the first autonomous vehicle 118 if the supplementary task requires to determine a parameter of the soil or the second autonomous vehicle 120 if the supplementary task requires to determine a parameter of the crop or the third autonomous vehicle 122 if the supplementary task requires to determine a parameter of the agricultural field 508 from the air.

[0080] Then, the method proceeds to step S106 and the control unit 110 creates an assignment between an autonomous vehicle 118, 120, 122 to the supplementary task for acquiring the necessary information. The supplementary task is assigned to that autonomous vehicle 118, 120 122 which is equipped with the appropriate sensing unit 204 for executing the supplementary task. Hence, the control unit 110 assigns the supplementary task to the first autonomous vehicle 118 if the supplementary task requires to determine a parameter of the soil or to the second autonomous vehicle 120 if the supplementary task requires to determine a parameter of the crop or to the third autonomous vehicle 122 if the supplementary task requires to determine a parameter of the agricultural field 508 from the air. For reasons of clarity, it is exemplarily assumed for the following description that thecontrol unit 110 creates an assignment between the supplementary task and the first autonomous vehicle 118.

[0081] The method proceeds to step S107 and the control unit 110 determines at least one target point 518 in the agricultural field 508 for executing the supplementary task (see FIG. 5). For example, the target point 518 may define a specific point in the agricultural field 508 at which a soil parameter, a crop parameter or a field parameter from the air shall be determined. For reasons of clarity, it is exemplarily assumed for the following description that the soil moisture of the agricultural field 508 is the insufficient information. I. e., the soil moisture needs to be determined at target point 518. Step S107 may be repeated by the control unit 110 to determine additional target points for gathering additional information of the agricultural field 508 at different positions. The additional information may be the same parameter to be determined at target point 518 or a different parameter of the agricultural field 508. The target points may be stored in the memory 306.

[0082] The method proceeds to step S108 and the control unit 110 determines the position of the autonomous vehicle being configured to execute the supplementary task and to which the supplementary task has been assigned. The control unit 110 may send a request to the network 102 to receive the position of the pertaining autonomous vehicle. Since the assignment has been created between the supplementary task and the first autonomous vehicle 118 as mentioned above, the vehicle control unit 202 of the first autonomous vehicle 118 responds to the request by sending its positional information provided by its position determination unit 206 to the control unit 110 via the network 102. As can be seen in FIG. 5 for example, the first autonomous vehicle 118 is positioned at home point 516. The control unit 110 receives the positional information comprising the home point 516 and stores this information in the memory 306.

[0083] The method proceeds to step S109 and the control unit 110 determines a travel path from the position of the determined autonomous vehicle to the target point. The travel path defines a route along which the autonomous vehicle can be automatically guided to move from its home point to the target point (and optionally back). As can be seen in FIG. 5, the control unit 110 determines a travel path 514 from the home point 516 of the first autonomous vehicle 118 to the target point 518 in the agricultural field 508 determined before. The control unit 110 may consider map information stored in the memory 306 for determining the travel path 514. The map information may contain roads, impassable areassuch as a river, obstacles such as buildings, trees or rocks, topographic parameters of an area, etc. The control unit 110 may optionally consider a road 506 to reduce or avoid unnecessary offroad driving of the first autonomous vehicle 118. The travel path 514 leads from the road 506 into the agricultural field 508. Since the target point 518 is located on a field path 512, the travel path 514 leads also into the corresponding field path. The determined travel path 514 can be sent to the network 102.

[0084] Then, the method proceeds to step S110 for guiding the autonomous vehicle to the target point. For example, the vehicle control unit 202 of the first autonomous vehicle 118 receives the travel path 514 from the network 102 and navigates the first autonomous vehicle 118 along the travel path 514 from the home point 516 to the target point 518. Based on the positional information provided from the vehicle control unit 202, the vehicle control unit 202 can determine any positional deviation of the first autonomous vehicle 118 from the travel path 514 and initiate a corrective action such as steering the first autonomous vehicle 118 to follow the travel path 514 again. Alternatively, the control unit 110 may undertake the navigation functionality of the vehicle control unit 202 to guide the first autonomous vehicle 118 along the travel path 514. For example, the control unit 110 receives the positional information provided by the position determination unit 206 of the first autonomous vehicle 118 via the network 102. Based on the positional information, the control unit 110 determines any positional deviation of the first autonomous vehicle 118 from the travel path 514, determines a corrective action and sends the corrective action via the network 102 to the vehicle control unit 202 of the first autonomous vehicle 118. Then, the vehicle control unit 202 carries out the corrective action received from the control unit 110.

[0085] The method proceeds to step Sill and for initiating the execution of the supplementary task. The execution may be initiated by the control unit 110 sending a corresponding signal to the vehicle control unit 202 of the first autonomous vehicle 118 via the network 102. Optionally, the supplementary task may have been transferred from the control unit 110 to the vehicle control unit 202 of the first autonomous vehicle 118 via the network 102 so that the vehicle control unit 202 is configured to initiate the supplementary task. The execution of the supplementary task may be initiated if at least one condition is fulfilled. The condition may be that the first autonomous vehicle 118 has arrived the target point 518, the autonomous vehicle 118 has passed a self-test, e. g. to check the functionality of the sensing unit, the weather conditions (sunshine, rain, temperature, etc.) enable theexecution of the supplementary task or that a geofence for the first autonomous vehicle 118 has been activated, for example.

[0086] Then, the method proceeds to step S112 for executing the supplementary task and acquiring the necessary information. The supplementary task can be executed by the control unit 110 sending corresponding signals to the vehicle control unit 202 of the first autonomous vehicle 118 via the network 102. Alternatively, the supplementary task may be executed by the vehicle control unit 202 of the first autonomous vehicle 118 if the supplementary task has been transferred to it as mentioned above. For example, the supplementary task may require to determine the soil moisture of the agricultural field 508 at the target point 518. Thus, the control unit 110 or the vehicle control unit 202 controls the sensing unit 204 of the first autonomous vehicle 118 accordingly for acquiring the necessary information of the soil moisture when the first autonomous vehicle 118 has arrived at the target point 518. For example, the sensing unit 204 can be controlled to move a soil moisture sensor in the soil at the target point 518 and to measure a value representing the soil moisture. When the measurement has been finished, the necessary information of the soil moisture acquired by the sensing unit 204 may be stored in the memory 306 of the vehicle control unit 202.

[0087] The method proceeds to step S113 and the control unit 110 receives the necessary information from the first autonomous vehicle 118. For example, the control unit 110 sends a request to the vehicle control unit 202 of the first autonomous vehicle 118 via the network 102 to transfer the necessary information via the network 102 to the control unit 110. The vehicle control unit 202 of the first autonomous vehicle 118 responds to the request and sends the necessary information of the soil moisture to the control unit 110.

[0088] The method proceeds to step S114 and the control unit 110 updates the main task based on the necessary information. Alternatively, the control unit 110 may show an indication at the computer HMI that the necessary information has been received to inform the operator that he can update the main task manually. The necessary information, here the soil moisture for example, will be integrated in the main task to complete the definition of the main task. As a result, the updated main task includes all information so that the main task can be executed. Hence, the main task may be optimized by up-to-date parameters, rescheduled or both.

[0089] The method proceeds to step S115 for initiating the execution of the updated main task. The updated main task may require to perform a field operation with the agricultural vehicle 112 in the agricultural field 508, e. g. to plough the agricultural field 508 while moving along the field paths 512. The execution of the main task may be initiated if at least one condition is fulfilled. The condition may be that the agricultural vehicle 112 has arrived the agricultural field 508 or that all necessary information has been acquired, for example. Based on the necessary information, settings of the agricultural vehicle 112 or the implement, e. g. the plough, may be adjusted. The main task is sent to the network 102 by the control unit 110. The control unit 116 of the agricultural vehicle 112 receives the main task from the network 102 and may control the agricultural vehicle 112 or the implement for the adjustment of the settings. When the operator has moved the agricultural vehicle 112 to the agricultural field 508, the vehicle control unit 202 starts and executes the plough operation with the adjusted settings in dependence of the necessary information of the soil moisture. Alternatively, the agricultural vehicle 112 may drive autonomously to the agricultural field 508 similar to the first autonomous vehicle 118. The position determination unit of the agricultural vehicle 112 determines when the agricultural vehicle 112 has reached the agricultural field 508 and sends a corresponding signal to the control unit 116 or to the network 102. In reaction of the signal, the control unit 110 or the control unit 116 may start and execute the plough operation with the adjusted settings in dependence of the necessary information of the soil moisture.

[0090] The method is described by way of example of the first autonomous vehicle to which the supplementary task has been assigned to and of the soil moisture as insufficient information. Analogously, any other insufficient information mentioned in the description could be determined as insufficient information by the control unit 110. Analogously to the first autonomous vehicle 118, the second or the third autonomous vehicle 120, 122 could be determined as appropriate autonomous vehicle for executing a supplementary task to gather another necessary information for which the second or third autonomous vehicle would be configured for.

[0091] Finally, the method proceeds to step 5116 and ends. The method may be restarted with step S100.

[0092] All references cited herein are incorporated herein in their entireties. If there is a conflict between definitions herein and in an incorporated reference, the definition herein shall control.

Claims

CLAIMSWhat is claimed is:

1. A method for an agricultural task management, comprising:Creating or receiving a main task for an operation in an agricultural field (508);Checking the main task whether any information for executing the main task is insufficient;Creating a supplementary task for acquiring necessary information in response to a determined insufficient information;Creating an assignment between the supplementary task and an autonomous vehicle (118, 120, 122) for executing the supplementary task;Receiving the necessary information from the autonomous vehicle (118, 120, 122); and Updating the main task based on the necessary information.

2. The method of claim 1, wherein the insufficient information is a field parameter comprising a soil parameter or a crop parameter.

3. The method of claim 1 or 2, wherein the supplementary task comprises determining a soil parameter or determining a crop parameter.

4. The method of any one of the preceding claims, comprising the step:Determining an autonomous vehicle (118, 120, 122) configured to execute the supplementary task.

5. The method of claim 4, comprising the step:Determining a target point (518) in the agricultural field (508) for executing the supplementary task.

6. The method of claim 5, comprising the steps:Determining the position of the autonomous vehicle (118, 120, 122) assigned to the supplementary task; and determining a travel path (514) from the position of the autonomous vehicle (118, 120, 122) to the target point (518).

7. The method of claim 5 or 6, comprising the steps:Guiding the autonomous vehicle (118, 120, 122) to the target point (518).

8. The method of any one of the preceding claims, comprising the step: Initiating the execution of the supplementary task.

9. The method of claim 8, wherein the execution of the supplementary task is initiated if at least one of the following conditions is fulfilled:The autonomous vehicle (118, 120, 122) has arrived the target point (518); the autonomous vehicle (118, 120, 122) has passed a self-test; the weather conditions enable the execution of the supplementary task; a geofence has been activated.

10. The method of any one of the preceding claims, comprising the step:Executing the supplementary task; and acquiring the necessary information.

11. The method of any one of the preceding claims, comprising the step: Initiating the execution of the updated main task.

12. The method of claim 11, wherein the execution of the main task is initiated if at least one of the following conditions is fulfilled:All necessary information has been acquired.

13. An agricultural system (100) for an agricultural task management, comprising:An agricultural vehicle (112) for operating in the agricultural field (508); at least one autonomous vehicle (118, 120, 122) configured to execute a supplementary task; and a control unit (110) configured to carry out a method of any one of claims 1 to 12.

14. The agricultural system (100) of claim 13, wherein the control unit (110) and the at least one autonomous vehicle (118, 120, 122) are communicatively connected.

15. The agricultural system (100) of claim 13 or 14, whereinthe autonomous vehicle (118, 120, 122) is an agricultural robot or an unmanned aerial vehicle.

16. The agricultural system (100) of any one of claims 13 to 15, wherein the at least one autonomous vehicle (118, 120, 122) comprises at least one of the following components: a sensing unit (204) configured to determine a soil parameter; a sensing unit (204) configured to determine a crop parameter; a sensing unit (202) configured to determine a field parameter from the air.

Images