Agricultural harvesting machine monitoring

Abstract

Systems and methods for monitoring the operation of an agricultural harvesting machine, comprising utilising an event based sensor mounted or otherwise associated with the agricultural harvesting machine to determine crop material parameter(s), here a measure of grain present in a material flow from an outlet of the machine. Operation of one or more operable components of the agricultural harvesting machine may be controlled in dependence on the determined crop material parameter(s), e.g. to improve grain yield.

Description

TITLEAGRICULTURAL HARVESTING MACHINE MONITORINGCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] Not applicable.FIELD

[0002] Embodiments of the present disclosure relate generally to systems and methods for monitoring operation of an agricultural harvesting machine, such as a combine harvester.BACKGROUND

[0003] Agricultural harvesters such as combine harvesters, or "combines", work to cut crop material from a field before separating the grain from the material other than grain (MOG) (referred to interchangeably as "residue") on board. Generally, the grain is transferred to a grain bin of the combine (where it may be temporarily stored) and the MOG is deposited back onto the field. A second operation may be performed to gather the deposited MOG, or the MOG may be used as a fertilizer for the soil in the field.

[0004] Inevitably, some grain material may not be separated completely from the MOG and as such may be contained within the material which is spread onto the field. An operator is therefore always looking to monitor the levels of grain material in the expelled material and optimize operational conditions of the combine, e.g. the cleaning and processing settings to increase the grain yield and reduce losses. Typically, an operator may physically get out of the operator cab and inspect the floor behind the combine to assess a measure of grain loss. However, being able to effectively monitor, in real-time, a measure of grain present within the material deposited onto the field would beneficially assist the operator in optimizing the efficiency of the combine operation and reduce downtime of the machine for manual inspection.

[0005] Grain material successfully separated from the MOG and stored in the grain bin, as discussed above, may subsequently be unloaded from the harvester into an adjacent collectionvehicle - e.g. a trailer towed by a tractor moving alongside the combine. It is believed a measure of grain quantity or quality, and / or a measure of the efficiency of this unloading process may be advantageously obtained by monitoring these unloading or spreading processes.

[0006] However, to date, no complete solution has been provided to any of the above problems.

[0007] It is therefore an aim of an embodiment or embodiments of the system to provide systems and methods to monitor properties of the crop material as it is unloaded or expelled from the combine, e.g. straw deposited by a spreader tool of a harvesting machine, or grain material unloaded by an unloading auger.BRIEF SUMMARY

[0008] In an aspect of the invention there is provided a system for monitoring the operation of an agricultural harvesting machine, the system comprising: an event based sensor; and one or more controllers, configured to: receive sensor data from the event based sensor indicative of a measure of material present within a material flow from an outlet of the agricultural harvesting machine; analyse the sensor data to: identify one or more crop material components within the material flow; and determine a crop material parameter for the identified crop material component(s), the crop material parameter quantifying a measure of grain within the material within the material flow; and output one or more control signals for controlling operation of one or more operable components of or otherwise associated with the agricultural harvesting machine in dependence on the determined crop material parameter.

[0009] Advantageously, the present disclosure utilises an event based sensor for monitoring operation of the harvesting machine, and in particular identifying individual crop material components, specifically grain in the material leaving the outlet of the machine. The functionality of an event based sensor is particularly suited to such a use case as the data obtained therefrom can be used to accurately identify individual crop components as they move through a sensing region of the event based sensor. Parameters of the crop material components can then be determined, once identified in the sensor data, and used to infer a measure of, forexample, grain present within the material flow and hence the efficiency of the operation of operable components of the harvesting machine.

[0010] When used herein and throughout the term "event based sensor" is intended to include a sensor having multiple individual sensing elements or pixels which are individually triggerable in dependence on one or more variations in the signal output associated with those elements. The one or more variations result from a change in the light incident on said sensing elements, resulting in a change in amplitude in the output signal of those sensing elements. By setting relevant thresholds for said variations, "events" can be triggered in dependence on a given change in incident light on said sensing elements. Such a change may indicate movement in the observed environment, for example. Such sensor types may also be referred to as a neuromorphic camera, a silicon retina or dynamic vision sensor.

[0011] The outlet of the harvesting machine may include an outlet of a crop transfer device such as an unloading spout (e.g. on a forage harvester) or auger (e.g. on a combine harvester) for transferring material from the harvester, e.g. to a collection vehicle or grain pile adjacent or following the harvester, in use. The outlet of the harvesting machine may include a spreader tool for distributing material, e.g. material other than grain ("MOG") onto a field behind the harvesting machine, in use, as will be appreciated. When used herein and throughout the term "spreader tool" is intended to cover multiple different spreader tool types, including rotor based spreaders for distributing residue material over a desired distance, but also tools (e.g. tailboards) for distributing straw or other crop material in a swath orthe like behind the machine.

[0012] The one or more controllers may be configured to apply a clustering process to the sensor data. The clustering process may be applied to the sensor data to identify and associate triggers at multiple sensing elements of the event based sensor. The one or more controllers may be configured to identify one or more material components in dependence on the associated sensing element triggers. This may, for example, include associating sensing element triggers which occur substantially simultaneously, and / or for sensing elements which trigger which are proximal to one another. For instance, the clustering process may comprise a timewise and / or location based clustering process to identify and associate the sensing elements triggered by a common material component.

[0013] The one or more controllers may be configured to determine a measure of a length, size or shape associated with an identified material component. This may, for instance, comprise a count of the number of adjacent and associated sensing elements for a given material component. This may comprise a calculated distance, within the sensor data, between two or more associated sensing elements. The one or more controllers may be configured to compare the determined length or size with a threshold length or size. The threshold length or size may comprise a predetermined threshold length or size, which may be programmable, for example, to distinguish between different material component types in the monitored material flow. The one or more controllers may be configured to categorize the identified material component as a crop material component in dependence on the comparison of the determined length or size measure with the threshold length or size. In embodiments, the one or more controllers may be configured to categorize the identified material component as a grain material component in dependence on the determined length or size measure being less than a predetermined expected or maximum length or size. Advantageously, MOG which may typically be longer or larger - e.g. straw pieces - may be discounted in this manner.

[0014] The crop material parameter quantifies a measure of grain within the material within the material flow. This may include a count of individual grain components within the sensor data. The one or more controllers may be configured to determine a flow rate of grain components in the scene, e.g. a count of individual identified grain components per unit time. The one or more controllers may be configured to extrapolate a count of identified grain components in the sensor data, which may provide a count of material visible to the sensor -e.g. on the outer surface the material flow, to a crop material parameter which quantifies a volumetric measure of the grain within the material flow. This may include a volumetric flow rate, for example. The one or more controllers may utilise a known relationship between the visible surface area of the material flow and the volume of the material flow, e.g. by knowing the outlet size and / or expected flow characteristics through said outlet to extrapolate the calculation from the received sensor data.

[0015] The crop material parameter may comprise a measure of a speed associated with one or more of the identified grain components. The one or more controllers may beconfigured to compute an average speed of identified grain components within the sensor data. The speed may provide information relating to an unload rate of the grain material, for example, where the event based sensor is mounted at or proximal to the unloading auger of the machine.

[0016] The crop material parameter may comprise a measure of a velocity or an acceleration or deceleration associated with the one or more identified components. A characteristic acceleration may be indicative of a crop material type or unload rate, for example.

[0017] The one or more controllers may be configured to compare the determined crop material parameter with an expected crop material parameter. The expected crop material parameter may be dependent on one or more controllable operating conditions of a crop processing component of the agricultural harvesting machine. For instance, an expected speed of the crop material components may be dependent on an operating speed of crop processing apparatus of the harvesting machine. An expected speed of the crop material components may be dependent on an operating speed of the unloading auger of the machine. The one or more controllers may be configured to generate and output the control signal for controlling the operable component(s) in dependence on the comparison.

[0018] The one or more controllers may be configured to control application of a filter to the output signals from individual sensing elements of the event based sensor, e.g. to remove triggers thereof due to background motion. For example, this may be due to the relative movement of the ground causing changes in the light incident on the sensing elements as the machine performs a task, or due to structural vibration and / or component motion during said tasks. Advantageously, applying one or more filters to the analogue output signal from the sensing elements may control the number of individual triggers to be processed by the event based sensor and / or one or more processing units associated therewith.

[0019] Additionally or alternatively, the one or more controllers may be configured to apply a filter to the sensor data. Where the processing capabilities of the controllers is high enough it may be possible to receive all triggers and apply said filter(s) digitally to remove or discard unwanted triggers.

[0020] The filter may comprise a threshold for a signal parameter associated with the sensing elements, which may include amplitude or rate of change of amplitude of the outputsignal. The threshold may be definable and / or be dependent on the agricultural task (to be) performed by the agricultural machine.

[0021] The filter may be dependent on a ground speed of the agricultural harvesting machine and / or an operational speed of one or more components of or associated with the machine. The filter may be dependent on the crop material type. For instance, grain material may move at a different speed to MOG in the sensor data, and as such a speed based filter may be applied to discount MOG or other material from the sensor data.

[0022] The filter may comprise a mask applied to a sensing region of the sensor, e.g. to exclude regions of the sensing region which do not correspond to the outlet and / or a material flow associated therewith.

[0023] The one or more operable components of or otherwise associated with the agricultural harvesting machine may include a user interface. The one or more controllers may be configured to control output of an indicator indicative of the determined crop material parameter(s) via the user interface. For example, the one or more controllers may be configured to control generation of a graphical representation of a grain loss measure or yield as determined from the material flow from the outlet in the manner discussed herein.

[0024] The user interface may comprise a display screen, which may be provided as part of a user terminal of the machine, e.g. in an operator cab of the machine. Additionally or alternatively, the user interface may be provided by a portable user device, e.g. one carried by a user / operator during operation of the machine. The portable user device may comprise a smartphone, tablet computer, laptop or the like.

[0025] The one or more controllers may be configured to control a forward speed of the agricultural harvesting machine in dependence on the determined crop material parameter(s). Advantageously, the forward speed of the machine may be utilised to control the feed speed of material into / through the agricultural harvesting machine, in turn controlling a load on the crop processing components thereof. Advantageously, where a high or relatively high level of grain is present within the material expelled from the spreader tool of the machine, as determined through analysis of the sensor data from the event based sensor, the one or morecontrollers may control a reduction in a forward speed of the machine to reduce the load on the processing components of the machine in an attempt to reduce the grain loss.

[0026] In embodiments, the component(s) of the agricultural harvesting machine may comprise a header for the machine. The one or more controllers may be configured for controlling an operational speed of elements of the header, a lift mechanism for the header for controlling an operational position or height of the header, and / or an operational state of the header in dependence on the crop material parameter(s). Again, such control actions may be taken to control the load put on the processing components of the machine to control grain loss measured at a spreader tool of the machine.

[0027] In embodiments the component(s) of the agricultural harvesting machine may comprise a crop processing apparatus of the machine. The one or more controllers may be configured to adjust an operational speed of components of the crop processing apparatus to control the speed at which crop material is processed and moved through the agricultural harvesting machine in dependence on the crop material parameter(s). Here, suitable adjustments may be made to the processing components themselves to increase the grain separation efficiency, for example, where a high or relatively high grain loss is observed at the spreader tool.

[0028] The event based sensor may be mounted or be mountable on the machine. For example, the sensor may be mounted or be mountable on the rear of the machine, at or proximal to an outlet of the spreader tool of the machine. Alternatively, the event based sensor may be mounted or be mountable at or proximal to an unloading auger of the machine, for example, having a field of view which encompasses material expelled, in use, from an outlet of the unloading auger. The event based sensor may be mounted or be mountable at or proximal to one or more internal components of the machine, e.g. a conveyor, elevator or the like, and configured for obtaining data indicative of an internal operation of the machine.

[0029] The one or more controllers may collectively comprise an input (e.g. an electronic input) for receiving one or more input signals. The one or more input signals may comprise sensor data from the event based sensor. The one or more controllers may collectively comprise one or more processors (e.g. electronic processors) operable to execute computerreadable instructions for controlling operation of the control system, for example, identify the crop material component(s) and / or determine the crop material parameter(s). The one or more processors may be operable to generate one or more control signals for controlling operation of the one or more operable components. The one or more controllers may collectively comprise an output (e.g. an electronic output) for outputting the one or more control signals.

[0030] A further aspect provides an agricultural machine comprising the system as described hereinabove. The agricultural machine may comprise a harvesting machine, such as a combine harvester.

[0031] A further aspect provides a method of monitoring the operation of an agricultural harvesting machine, comprising: utilising an event based sensor to obtain sensor data from indicative of a measure of material present within a material flow from an outlet of the agricultural harvesting machine; analysing the sensor data to: identify one or more crop material components within the material flow; determining a crop material parameter for the identified crop material component(s), the crop material parameter quantifying a measure of grain within the material within the material flow; and controlling operation of one or more operable components of or otherwise associated with the agricultural harvesting machine in dependence on the determined crop material parameter.

[0032] The method may comprise performing any one or more of the functionalities of the one or more controllers of the system described hereinabove.

[0033] A further aspect of the invention provides computer software comprising computer readable instructions which, when executed by one or more electronic processors, causes performance of a method in accordance with any aspect described herein.

[0034] A yet further aspect of the invention provides a computer readable medium having the computer software of the preceding aspect of the invention stored thereon.

[0035] Within the scope of this application it should be understood that the various aspects, embodiments, examples, and alternatives set out herein, and individual features thereof may be taken independently or in any possible and compatible combination. Where features are described with reference to a single aspect or embodiment, it should be understood that suchfeatures are applicable to all aspects and embodiments unless otherwise stated or where such features are incompatible.BRIEF DESCRIPTION OF THE DRAWINGS

[0036] One or more embodiments of the invention / disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0037] FIG. 1 is a schematic side cross-sectional view of an agricultural harvester embodying aspects of the present disclosure;

[0038] FIG. 2 is a schematic view of an embodiment of a system of the present disclosure;

[0039] FIG. 3 illustrates sensor data obtained by a sensing arrangement forming part of aspects of the present disclosure;

[0040] FIG. 4 is an image of residue material obtained utilising a camera illustrating the operational use of embodiments discussed herein; and

[0041] FIG. 5 is a schematic illustration of an alternative embodiment of the present disclosure.DETAILED DESCRIPTION

[0042] The present disclosure relates to systems and methods for monitoring the operation of an agricultural harvesting machine, here a harvester 10. An event based sensor 30 is utilised to obtained sensor data which is indicative of a measure of material present within a material flow from an outlet of the harvester 10, in the illustrated embodiment in FIG. 1 a spreader tool 22 for spreading residue material, e.g. MOG, onto the field behind the harvester, in use. Controller(s) 102 is configured to analyse the sensor data to identify one or more crop material components, specifically grain components in the material flow, and determine a crop material parameter for the identified crop material component(s) which is indicative of a measure of the amount of grain present within the material flow. A measure of grain present in the material flow from the spreader tool 22 may advantageously provide an indication of thelevel of grain loss associated with the operation. Operation of one or more operable components of or otherwise associated with the harvester 10 (e.g. a user interface 32 or components of the spreader tool 22, or processing systems 16 of the harvester, for instance) can be controlled based on determined parameters, in the manner discussed herein.Harvester

[0043] With reference to FIG. 1, an agricultural machine in the form of a harvester 10 is shown which embodies aspects of the present disclosure.

[0044] The harvester 10 is coupled to a header 12 which is operable, in use, to cut and gather a strip of crop material as the harvester 10 is driven across a field / area to be harvested during a harvesting operation. A conveyor section 14 conveys the cut crop material from the header 12 into a crop processing apparatus 16 operable to separate grain and non-grain (i.e. material other than grain (MOG) or residue material (used interchangeably herein)) as will be appreciated. It is noted here that apparatus for separating grain and non-grain material are well- known in the art and the present invention is not limited in this sense. The skilled person will appreciate that numerous different configurations for the crop processing apparatus may be used as appropriate. Clean grain separated from the cut crop material is collected in a grain bin 18, which may be periodically emptied, e.g. into a collection vehicle, storage container, etc. utilising unloading auger 20. The remaining non-grain material (MOG) / residue material is separately moved to a spreader tool 22 which is operable in use to eject the non-grain material or MOG from the rear of the harvester 10 and onto the ground. In Figure 1, this is represented by arrow 24 which illustrates the MOG being ejected rearwards from the harvester 10. The harvester 10 also includes a chopper tool 25 positioned between the crop processing apparatus 16 and the spreader tool 22 and operable, in use, to cut the residue material before it is spread by the spreader tool 22.

[0045] The harvester 10 also typically includes, amongst other features, an operator cab 26, wheels 28, engine (not shown) and a user interface in the form of a display terminal 32 provided within the operator cab 26.Event Based Sensor

[0046] As will be discussed in detail herein, the harvester 10 additionally includes a sensor in the form of an event based sensor 30. The sensor 30 is mounted to a rear of the harvester 10 in the illustrated embodiment and has a field of view which encompasses an operating region of the spreader tool 22, specifically here the region of the working environment directly behind the harvester 10 into which material is spread by the spreader tool 22. Event based sensor 30 is used, by a control system 101 of the harvester 10, to identify material components in a material flow associated with the spreader tool 22 in the manner discussed herein.

[0047] In a variant, the sensor may be positioned with a field of view F which encompasses grain being unloaded via unloading auger 20 into a collection vehicle 50 moving alongside the harvester 10, in use - see FIG. 5. In this variant, the event based sensor 30' is use, by a control system 101 of the harvester 10 to identify grain components in the material flow G associated with the unloading auger 20 and determine, for example, a measure of the unload rate of material from the auger 20 to determine whether the unloading is occurring at an expected rate for given operational settings of the harvester, for example.

[0048] Event based sensor 30 has multiple individual sensing elements (pixels) which are individually triggerable in dependence on one or more variations in the signal output associated with those elements. The one or more variations result from a change in the light incident on said sensing elements, resulting in a change in amplitude in the output signal of those sensing elements. By setting relevant thresholds for said variations, "events" can be triggered in dependence on a given change in incident light on said sensing elements. The variations can be used as an indication of movement in the observed environment, for example, by monitoring successive triggers of adjacent sensing elements, and inferring movement in the observed scene based thereon.

[0049] Operation of the event based sensor 30 may be controlled based on the given scenario. For instance a filter may be applied to the sensing elements to control at what magnitude of change / variation said sensing elements are triggered. This may be utilised to remove background events from the data. For instance, in the presently illustrated embodiment using the sensor 30 on the harvester 10 during a harvesting operation, motion of the ground withrespect to the harvester 10 may otherwise trigger the sensing elements. Accordingly, a filter can be applied based on the ground speed of the harvester 10 to reduce or eliminate false triggers thereof. In alternative arrangements, the operational speed of the components of the harvester 10, e.g. a rotational speed of elements of the spreader tool 22, may be used to determine the filter to be applied to the sensor data to remove cyclical or repetitive motion from the scene - e.g. in order to identify anomalies or changes in that motion indicative of an operational issue - or to hone in on expected speeds of material specifically grain components in the material flow, e.g. by excluding objects in the sensor data moving more slowly or faster than an expected grain speed.System

[0050] FIG. 2 illustrates system 101 further. As shown, the system incorporates a control system 100 here having a single controller 102. The controller 102 includes an electronic processor 104, an electronic input 106 and electronic outputs 108, 110, 114. The processor 104 is operable to access a memory 112 of the controller 102 and execute instructions stored therein to perform the steps and functionality of the present invention, for example to output control signals 109, 111, 115 via the outputs 108, 110, 114 for controlling operation of operable components of the machine, including processing components of the harvester 10, an output for updating data stored on a remote server 34, e.g. a mapped representation of the crop parameter(s), or for controlling the display terminal 32, for example to provide an graphical indication to an operator of the harvester 10 illustrative of the determined crop parameter(s).

[0051] The processor 104 is operable to receive sensor data via input 106 which, in the illustrated embodiment, takes the form of input signals 105 received from the event based sensor 30. As described in detail herein, the event based sensor 30 has a sensing region rearward of the harvester 10, with the sensor data received from the sensor 30 being indicative of a measure of crop material present within the sensing region. Using this information, the processor 104 is operable to identify one or more crop material components within the material flow from the spreader tool 22. This may include application of a filter to the sensor data, and / or performance of a clustering process to extract data relating to individual crop material pieces from background data. As discussed herein, a filter may be applied to the sensing elements to control at whatmagnitude of change / variation said sensing elements are triggered. This may be utilised to remove background events from the data. For instance, in the presently illustrated embodiment using the sensor 30 on the harvester 10 during a harvesting operation, motion of the ground with respect to the harvester 10 may otherwise trigger the sensing elements. Accordingly, a filter can be applied based on the ground speed of the harvester 10 to reduce or eliminate false triggers thereof. A clustering process can be applied to the sensor data to identify sensing element triggers which relate to common crop material components. This clustering may be a timewise and / or location based clustering. For instance, data may be clustered timewise to identify (and preferably track movement of) individual material components (e.g. pieces of straw or other crop material) in the sensor data. Identified material components having a length which exceeds a given or predetermined threshold may be discounted. Such sensor data may correspond to pieces of straw or other MOG, and thereby discounting or filtering such data may leave only grain components in the sensor data. Clustering may also be applied in dependence on a determined flow velocity of material (e.g. as determined through application of an optical flow measurement of the sensor data) which may be used to further filter out non-grain material components in the scene data. For instance, grain may be expected to move at a different speed in the sensor data compared with MOG.

[0052] The processor 104 is configured to determine a crop material parameter in the manner described herein. Specifically, the processor 104 is analyse the sensor data received from the event based sensor 30, processed (e.g. filtered and / or clustered as discussed above) to determine a crop material parameter which is indicative of a measure of grain material present within the material flow. As discussed herein, this can simply include a measure or count of individual grain components within the sensor data. In the illustrated embodiment, the count of grain material components associated with the spreader tool 22 advantageously provides a measure of grain loss, that being grain being spread onto the field rather than separated and collected in the grain bin of the harvester 10. Advantageously, the sensor data can therefore be used to determine a quality parameter indicative of the operational effectiveness or efficiency of crop processing components of the harvester 10. For instance, a high level of grain present withinthe material flow from the spreader tool 22 may indicate an inefficient processing of crop material by the harvester 10 and a high grain loss and lower yield.

[0053] The processor 104 may be configured to extrapolate the measured grain count to determine or estimate a volumetric crop material parameter. As will be appreciated, material visible to the sensor - e.g. on the outer surface the material flow - may not be a complete representation of the grain material contained within the material flow. Accordingly, the processor 104 can utilise a known relationship between the visible surface area of the material flow and the volume of the material flow, e.g. by knowing the outlet size of the spreader tool 22 and / or expected flow characteristics through said outlet to extrapolate the calculation from the received sensor data.

[0054] Output 108 is operably coupled to a control unit associated with the crop processing apparatus 16 of the harvester 10. The control system 101 is operable to control operation of the crop processing apparatus in dependence on the determined crop material parameter(s), determined in the manner discussed herein. For instance, where a high or relatively high level of grain is present within the material flow from the spreader tool 22, the control system may control, through generation and output of control signals 109, the crop processing apparatus 16, for example, to adjust an operational speed, setup, clearance or the like to adjust a level or processing applied to the collected crop material. This may include controlling a fan speed, sieve oscillating speed, spacing or the like, as will be appreciated in an attempt to lower the level of grain present in the analysed material flow.

[0055] Output 110 is operably coupled to the display terminal 32 of the harvester 10. Here, the control system 101 is operable to control operation of the display terminal 32, e.g. through output of control signals 111 in order to display operational data to an operator of the harvester 10 relating to the operation of the control system 101. Specifically, the control system 101 may be operable to control the display terminal 32 to display to the operator a graphical representation of the crop material parameter(s) as determined by processor 104, sensor data from the event based sensor 30, image data obtained from a camera on the harvester 10, or other useful information. The graphical representation can include an indication of a grain loss measure.

[0056] In some variants, the display terminal 32 may also be operable to receive a user input from the operator, and in such instances the output 110 may act as an input for receiving that user input at the processor 104. The user input may relate to a requested or target parameter, for example a threshold grain loss at which automated adjustments may be made to the crop processing apparatus 16.

[0057] Output 114 is operably coupled to a database hosted on remote or cloud server 34. Here, the output 114 forms a communication module for communicating data offboard from the harvester 10. Alternatively, output 114 may utilise a communication module of the harvester 10 and may be operably connected to server 34 indirectly thereby. The control system 101 may utilise server 34 to store data relating to the determined crop material parameter(s). This may provide an accessible mapped representation of the environment illustrating any problem areas, for example, where a high level of grain loss was experienced or measured. This may be cross- referenced with operational settings for the harvester 10 at those locations in order to better inform later harvesting operations in the same regions. In embodiments the control system 101 may be operably connected to a positioning module of the harvester 10, and the location of the harvester 10 within the environment may be linked to determined crop material parameters to generate a mapped environment containing data indicative of, e.g. a representation of grain loss across the environment.

[0058] In further examples, the control system 101 may be operable to control further operational components of the harvester 10. This may include controlling a forward speed of the harvester 10, for example, or operational settings for a header operably coupled to the harvester 10.Sensor Data

[0059] To be FIG. 3 illustrates a representation of sensor data obtained during operation of the harvester 10. Here, "triggered" sensing elements of the event based sensor 30 are shown indicative of movement in the region of the environment corresponding to the relevant pixel in the displayed sensor data. FIG. 4 illustrates a similar scene utilising a camera based system with residue material shown being spread behind the rear of a harvester 10. Thisis inherently noisy and near impossible to identify individual crop components from the scene data alone.

[0060] As shown, through suitable processing of the sensor data via filters and the like, only those triggers in response to movement of crop material within the observed region is present in the generated representation. This removes background noise / triggers from the data enabling further features to be extracted from the data, including the crop material parameter(s) discussed herein.

[0061] In FIG. 3, a group of crop material components C is highlighted. In the sensor data, this is represented as a plurality of individual or closely clustered triggered sensing elements. Through suitable processing, e.g. in the manner discussed herein, individual grain components can be identified in the scene, or determined from e.g. straw components in the sensor data showing as substantially straight and relatively longer groups or clusters of triggered sensing elements. From this, a crop material parameter in the form of a count of grain material components can be computed. This snapshot can be extrapolated to a volumetric measure of the amount of grain in the material flow by assuming a roughly uniform distribution of grain through the material flow, as discussed above. In this manner, the sensor data provided by the event based sensor 30 can be used to determine a relative measure of an amount of grain present, or a grain loss associated with the harvester 10.

[0062] In a further variant, an optical flow analysis may be performed on the sensor data, from which it may be possible to extract flow parameters for any given identified crop material component, including a speed and / or direction of the material component. The flow parameters may provide information relating to the flow rate of a crop material component. This may be particularly beneficial in the embodiment shown in FIG. 5, where the event based sensor 30' is used to monitor material flow from the unloading auger 20 as discussed above. The optical flow analysis may provide information relating to the flow rate of material from the auger 20, and hence an unload rate of material from the combine 10. This may be compared with an expected unload rate, e.g. as set by the operator, or as an input into a planning tool which, for example, might utilise information such as a "time-to empty" the grain tank 18 of the harvester10 or "time to fill" the grain cart 50 into which the material is being unloaded for operational planning purposes.General

[0063] Any process descriptions or blocks in flow diagrams should be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of the embodiments in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present disclosure.

[0064] It will be appreciated that embodiments of the present invention can be realized in the form of hardware, software or a combination of hardware and software. Any such software may be stored in the form of volatile or non-volatile storage such as, for example, a storage device like a ROM, whether erasable or rewritable or not, or in the form of memory such as, for example, RAM, memory chips, device, or integrated circuits or on an optically or magnetically readable medium such as, for example, a CD, DVD, magnetic disk or magnetic tape. It will be appreciated that the storage devices and storage media are embodiments of machine-readable storage that are suitable for storing a program or programs that, when executed, implement embodiments of the present invention. Accordingly, embodiments provide a program comprising code for implementing a system or method as set out herein and a machine readable storage storing such a program. Still further, embodiments of the present invention may be conveyed electronically via any medium such as a communication signal carried over a wired or wireless connection and embodiments suitably encompass the same.

[0065] All references cited herein are incorporated herein in their entireties. If there is a conflict between definitions herein and in an incorporated reference, the definition herein shall control.

Claims

CLAIMSWhat is claimed is:

1. A system for a system for monitoring the operation of an agricultural harvesting machine, the system comprising: an event based sensor; and one or more controllers, configured to: receive sensor data from the event based sensor indicative of a measure of material present within a material flow from an outlet of the agricultural harvesting machine; analyse the sensor data to: identify one or more crop material components within the material flow; and determine a crop material parameter for the identified crop material component(s), the crop material parameter quantifying a measure of grain within the material within the material flow; and output one or more control signals for controlling operation of one or more operable components of or otherwise associated with the agricultural harvesting machine in dependence on the determined crop material parameter.

2. A system of claim 1, wherein the outlet of the harvesting machine includes an outlet of a crop transfer device for transferring material from the harvesting machine; or includes a spreader tool for distributing material onto a field behind the harvesting machine, in use.

3. A system of claim 1 or claim 2, wherein the one or more controllers are configured to determine a measure of a length or size associated with an identified material component.

4. A system of claim 3, wherein the one or more controllers are configured to: compare the determined length or size with a threshold length or size; and categorize the identified material component as a grain material component in dependence on the determined length or size measure being less than a predetermined expected or maximum length or size.

5. A system of any preceding claim, wherein the crop material parameter quantifies a measure of grain within the material within the material flow as a count of individual grain components within the sensor data.

6. A system of claim 5, wherein the one or more controllers are configured to determine a flow rate of grain components in the scene.

7. A system of claim 5 or claim 6, wherein the one or more controllers are configured to extrapolate a count of identified grain components in the sensor data to a crop material parameter which quantifies a volumetric measure of the grain within the material flow.

8. A system of any preceding claim, wherein the crop material parameter comprises a measure of a speed associated with one or more of the identified grain components.

9. A system of claim 8, wherein the one or more controllers are configured to compute an average speed of identified grain components within the sensor data.

10. A system of claim 9, wherein the one or more controllers are configured to compute an unload rate of the grain material in dependence on the computed average speed.

11. A system of any preceding claim, wherein the one or more controllers are configured to compare the determined crop material parameter with an expected crop material parameter.

12. A system of claim 11, wherein the expected speed of the crop material components is dependent on an operating speed of the unloading auger of the machine; and wherein the one or more controllers are configured to generate and output the control signal for controlling the operable component(s) in dependence on the comparison.

13. A system of any preceding claim, wherein the one or more controllers are configured to control application of a filter to the output signals from individual sensing elements of the event based sensor; and wherein the filter comprises a threshold for a signal parameter associated with the sensing elements which is definable and / or be dependent on the agricultural task (to be) performed by the agricultural machine.

14. A system of claim 13, wherein the filter is dependent on a ground speed of the agricultural harvesting machine and / or an operational speed of one or more components of or associated with the machine.

15. A system of claim 13 or 14, wherein the filter is dependent on the crop material type.

16. A system of any preceding claim, wherein the one or more operable components of or otherwise associated with the agricultural harvesting machine include a user interface; and the one or more controllers are configured to control output of anindicator indicative of the determined crop material parameter(s) via the user interface.

17. A system of any preceding claim, wherein the one or more controllers are configured to control a forward speed of the agricultural harvesting machine in dependence on the determined crop material parameter(s).

18. A system of any preceding claim, wherein the component(s) of the agricultural harvesting machine comprise a header for the machine; and the one or more controllers are configured for controlling an operational speed of elements of the header, a lift mechanism for the header for controlling an operational position or height of the header, and / or an operational state of the header in dependence on the crop material parameter(s).

19. A system of any preceding claim, wherein the component(s) of the agricultural harvesting machine comprise a crop processing apparatus of the machine; and the one or more controllers are configured to adjust an operational speed of components of the crop processing apparatus to control the speed at which crop material is processed and moved through the agricultural harvesting machine in dependence on the crop material parameter(s).

20. A system of any preceding claim, wherein the event based sensor is mounted or is mountable on the rear of the machine, at or proximal to an outlet of the spreader tool of the machine; or wherein the event based sensor is mounted or is mountable at or proximal to an unloading auger of the machine.

21. An agricultural machine comprising the system of any preceding claim.

22. A method of monitoring the operation of an agricultural harvesting machine, comprising:utilising an event based sensor to obtain sensor data from indicative of a measure of material present within a material flow from an outlet of the agricultural harvesting machine; analysing the sensor data to: identify one or more crop material components within the material flow; and determine a crop material parameter for the identified crop material component(s), the crop material parameter quantifying a measure of grain within the material within the material flow; and controlling operation of one or more operable components of or otherwise associated with the agricultural harvesting machine in dependence on the determined crop material parameter.

Images