Agricultural harvesting machine monitoring

Abstract

Systems and methods for monitoring the operation of an agricultural harvesting machine, which includes utilising an event based sensor mounted or otherwise coupled to the agricultural machine to obtain a measure of material present within a material flow from a spreader tool of the machine. Crop material components within the material flow are identified and used to calculate a swath quality parameter for a resultant swath. Operation of one or more operable components of or otherwise associated with the agricultural harvesting machine can then be controlled based on the swath quality parameter.

Description

AGRICULTURAL HARVESTING MACHINE MONITORINGCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] Not applicable.FIELD

[0002] Embodiments of the present disclosure relate generally to systems and methods for monitoring operation of an agricultural harvesting machine, such as a combine harvester.BACKGROUND

[0003] Agricultural harvesters such as combine harvesters, or "combines", work to cut crop material from a field before separating the grain from the material other than grain (MOG) (referred to interchangeably as "residue") on board. Generally, the grain is transferred to a grain bin of the combine (where it may be temporarily stored) and the MOG is deposited back onto the field. A second operation may be performed to gather the deposited MOG, or the MOG may be used as a fertilizer for the soil in the field.

[0004] Where the MOG includes straw specifically, e.g. when harvesting grain (wheat, barley, etc.), the straw is usually discharged onto the field at the rear end of the combine, either in the form of a swath to be later taken up by a baler for further processing, or in chopped form over the width of the cutting path. The straw can be damaged during the threshing and separating process, and the extent of this damage largely depends on operational conditions of the combine, in addition to the properties of the crop itself. It is therefore important to consider the operational conditions of the combine when harvesting and its effect on the MOG, and specifically the straw quality.

[0005] It is therefore an aim of an embodiment or embodiments of the system to provide systems and methods to monitor properties of the crop material, e.g. straw, deposited by a spreader tool of a harvesting machine.BRIEF SUMMARY

[0006] In an aspect of the invention there is provided a system for monitoring the operation of an agricultural harvesting machine, the system comprising: an event based sensor; and one or more controllers, configured to receive sensor data from the event based sensor indicative of a measure of material present within a material flow from a spreader tool of the agricultural harvesting machine; analyse the sensor data to: identify one or more crop material components within the material flow; determine a crop material parameter for the identified crop material component(s); and calculate a swath quality parameter in dependence on the crop material parameters; and output one or more control signals for controlling operation of one or more operable components of or otherwise associated with the agricultural harvesting machine in dependence on the determined swath quality parameter.

[0007] Advantageously, the present disclosure utilises an event based sensor for monitoring operation of the harvesting machine, and in particular identifying individual crop material pieces in the material deposited by the machine. The functionality of an event based sensor is particularly suited to such a use case as the data obtained therefrom can be used to accurately identify individual crop components as they move through a sensing region of the event based sensor. Parameters of the crop material components can then be determined, once identified in the sensor data, and used to infer a measure of, for example, a quality of the crop material components, the trajectory of those components and hence the makeup of the resultant swath, which may include quality parameters including a profile, uniformity, crop material quality, etc.

[0008] When used herein and throughout the term "event based sensor" is intended to include a sensor having multiple individual sensing elements or pixels which are individually triggerable in dependence on one or more variations in the signal output associated with those elements. The one or more variations result from a change in the light incident on said sensing elements, resulting in a change in amplitude in the output signal of those sensing elements. By setting relevant thresholds for said variations, "events" can be triggered in dependence on a given change in incident light on said sensing elements. Such a change may indicate movementin the observed environment, for example. Such sensor types may also be referred to as a neuromorphic camera, a silicon retina or dynamic vision sensor.

[0009] When used herein and throughout the term "spreader tool" is intended to cover multiple different spreader tool types, including rotor based spreaders for distributing residue material over a desired distance, but also tools (e.g. tailboards) for distributing straw or other crop material in a swath or the like behind the machine. This can also include the outlet of a swather or windrower which deposits cut crop material in a swath or windrow.

[0010] The one or more controllers may be configured to apply a clustering process to the sensor data. The clustering process may be applied to the sensor data to identify and associate triggers at multiple sensing elements of the event based sensor. The one or more controllers may be configured to identify one or more material components in dependence on the associated sensing element triggers. This may, for example, include associating sensing element triggers which occur substantially simultaneously, and / or for sensing elements which trigger which are proximal to one another. For instance, the clustering process may comprise a timewise and / or location based clustering process to identify and associate the sensing elements triggered by a common material component.

[0011] The one or more controllers may be configured to determine a measure of a length associated with an identified material component. This may, for instance, comprise a count of the number of adjacent and associated sensing elements for a given material component. This may comprise a calculated distance, within the sensor data, between two or more associated sensing elements. The one or more controllers may be configured to compare the determined length with a threshold length. The threshold length may comprise a predetermined threshold length, which may be programmable, for example, to distinguish between different material component types in the monitored material flow. The one or more controllers may be configured to categorize the identified material component as a crop material component in dependence on the comparison of the determined length measure with the threshold length. In embodiments, the one or more controllers may be configured to categorize the identified material component as a straw material component in dependence on the determined length measure exceeding a predetermined straw threshold length.

[0012] The crop material parameter may comprise a measure of the length of the identified crop material component(s). The length of the identified crop material component(s) may advantageously provide an indication of an operational condition or function of, e.g. a chopper or cutting tool, of the harvesting machine. The one or more controllers are configured to utilise a spatial relationship between the event based sensor and a spreading area of the spreader tool to obtain a measure of length for the crop material component which comprises an absolute measure of its length. In other embodiments, the system comprises a distance sensor and the one or more controllers may be configured to utilise distance data from the distance sensor and a length determined from the sensor data from the event based sensor to determine an absolute measure of the length of the crop material component. The distance sensor may comprise a transceiver type sensor, e.g. a LIDAR sensor, or a stereo based vision system configured for providing distance data for objects within the sensing region of the event based sensor.

[0013] The crop material parameter may comprise a measure of a relative length associated with the identified crop material component(s). The relative length may comprise a length measure comparable with an expected or average length measure determined across multiple identified crop material components. In this way, a uniformity of the crop material components can be quantified, again being indicative of an operating condition or function of the crop processing components of the agricultural harvesting machine.

[0014] The crop material parameter may comprise a measure of a shape or structure of the identified crop material component(s). The measure of a shape or structure of the component(s) may include quantifying a straightness of the identified crop material component(s). This can include a count of a number of bends or folds in the identified crop material component(s).

[0015] The crop material parameter may comprise a measure of a speed associated with the identified crop material component(s).

[0016] The crop material parameter(s) may comprise a measure of a trajectory of the respective crop material component(s). The trajectory measure may be indicative or be utilised to compute a location within the working environment where the crop material component(s)come to rest - a "resting location" - e.g. its position within (or outside) of a formed swath or windrow. In this way, the crop material parameter(s) can relate to a swath quality parameter relating to the formation of the swath.

[0017] The one or more controllers may be configured to perform an optical flow measurement on the received sensor data. The optical flow measurement may provide a crop material parameter in the form of a flow parameter comprising a measure of a direction, speed or velocity distribution associated with material movement within the sensing region of the event based sensor. The optical flow measurement may provide the measure of the trajectory of the crop material component. The one or more controllers may be configured to compute the resting location of the crop material component(s) in dependence on the optical flow measurement, and optionally utilise the resting location in the determination of the swath quality parameter.

[0018] The one or more controllers may be configured to compile the sensor data from the event based sensor in an event histogram. The histogram may correspond to a measure of a number of activations of individual sensing elements. The histogram may be generated over a time period during operation of the machine. The duration of the time period may be user definable or may be pre-programmed. The one or more controllers may be configured to generate a representation of the event histogram, e.g. for output by an operably coupled user interface in the manner discussed herein. Advantageously, an indication of an overall shape and / or size of a resultant swath profile may be inferred from the event histogram.

[0019] The swath quality parameter may comprise a measure of a width of the formed swath, e.g. as determined from crop material parameters including a trajectory and / or resting location of individual crop material components. The one or more controllers may be configured to compare the measure of the width of the resultant swath with one or more control parameters, such as the width of the harvesting machine or header coupled thereto, for example, or the intake width of a further machine, e.g. a baling machine for later collecting (and baling) the crop material in the swath. The one or more controllers may be configured to control operation of the spreader tool of the agricultural harvesting machine in dependence on this comparison - e.g. to adjust operation thereof to extend or reduce the width of the observed distribution in dependence on the control parameter(s) as required.

[0020] The swath quality parameter can comprise a measure of the uniformity of the swath across its width. This may comprise a comparison of the intensity of the histogram / heatmap across swath width indicative of how evenly the crop material components are spread or distributed by the spreader tool. This may comprise a measure of a swath profile or contour.

[0021] The swath quality parameter can comprise information indicative of a crop component quality. This may include, for example, a measure of a length, shape or structure of the crop components forming the swath. The one or more controllers may be configured to quantify the crop component quality in dependence on the determined crop material parameter(s), and associate this with a specific swath location, for example, providing an indication of the swath quality at that location.

[0022] The one or more controllers may be configured to control application of a filter to the output signals from individual sensing elements of the event based sensor, e.g. to remove triggers thereof due to background motion. For example, this may be due to the relative movement of the ground causing changes in the light incident on the sensing elements as the machine performs a task, or due to structural vibration and / or component motion during said tasks. Advantageously, applying one or more filters to the analogue output signal from the sensing elements may control the number of individual triggers to be processed by the event based sensor and / or one or more processing units associated therewith.

[0023] Additionally or alternatively, the one or more controllers may be configured to apply a filter to the sensor data. Where the processing capabilities of the controllers is high enough it may be possible to receive all triggers and apply said filter(s) digitally to remove or discard unwanted triggers.

[0024] The filter may comprise a threshold for a signal parameter associated with the sensing elements, which may include amplitude or rate of change of amplitude of the output signal. The threshold may be definable and / or be dependent on the agricultural task (to be) performed by the agricultural machine.

[0025] The filter may be dependent on a ground speed of the agricultural harvesting machine and / or an operational speed of one or more components of or associated with the machine.

[0026] The filter may comprise a mask applied to a sensing region of the sensor, e.g. to exclude regions of the sensing region which do not correspond to the spreader tool and / or a material flow associated therewith.

[0027] The one or more operable components of or otherwise associated with the agricultural harvesting machine may include a user interface. The one or more controllers may be configured to control output of an indicator indicative of the determined swath quality parameter via the user interface.

[0028] The user interface may comprise a display screen, which may be provided as part of a user terminal of the machine, e.g. in an operator cab of the machine. Additionally or alternatively, the user interface may be provided by a portable user device, e.g. one carried by a user / operator during operation of the machine. The portable user device may comprise a smartphone, tablet computer, laptop or the like. The one or more controllers may be configured to control generation of a graphical representation of the determined swath quality parameter. The graphical representation may comprise data from one or more further sensors, e.g. a camera system. The graphical representation may include a representation of the swath quality parameter. This may include an overlay of a graphic or other generated representation indicative of the swath quality parameter. This may be provided, for example, as an overlay over on a representation of the working environment of the machine or preferably the operating region of the spreader tool.

[0029] The one or more components of or otherwise associated with the agricultural harvesting machine may include a database, e.g. a remote server or cloud based server, for storing information indicative of the determined crop material parameter(s) and / or swath quality parameter(s). The one or more controllers may be communicable with the database, e.g. utilising a communication module of the agricultural harvesting machine, or a dedicated communication module forming part of the system.

[0030] The one or more controllers may be configured to map the determined swath quality parameter. The system may comprise, or the controller(s) be communicable with, a positioning system for providing location data indicative of the location of the agricultural harvesting machine within a working environment. The one or more controllers may be configured to utilise the location data to associate the determined swath quality parameter(s) with a location within the working environment, thereby generating a mapped representation of the swath quality within the environment. Such information may advantageously be used in subsequent harvesting operations, e.g. a baling operation. For instance, a bale collection operation may be planned in accordance with the mapped representation, to collect the same or similar quality crop material across a bale, e.g. by generating a collection plan which routes the baler to areas within the environment of consistent swath quality.

[0031] The one or more controllers may be configured to control a forward speed of the agricultural harvesting machine in dependence on the determined swath quality parameter. Advantageously, the forward speed of the machine may be utilised to control the feed speed of material into / through the agricultural harvesting machine, in turn controlling a load on the crop processing components thereof.

[0032] In embodiments, the component(s) of or otherwise associated with the agricultural harvesting machine may comprise a header for the machine. The one or more controllers may be configured for controlling an operational speed of elements of the header, a lift mechanism for the header for controlling an operational position or height of the header, and / or an operational state of the header in dependence on the swath quality parameter.

[0033] In embodiments the component(s) of the agricultural harvesting machine may comprise a crop processing apparatus of the machine. The one or more controllers may be configured to adjust an operational speed of components of the crop processing apparatus to control the speed at which crop material is processed and moved through the agricultural harvesting machine in dependence on the swath quality parameter.

[0034] The crop processing apparatus may include a chopper tool. The one or more controllers may be configured to control operation of the chopper tool in dependence on the swath quality parameter, e.g. to adjust a cutting length of the crop material passing therethrough.

[0035] The component(s) of the agricultural harvesting machine may comprise a straw guide apparatus, including one or more guide plates, for controlling a distribution of crop material components from the spreader tool. The one or more controllers may be configured to control operation of the straw guide apparatus in dependence on the swath quality parameter, e.g. to adjust a width of distribution of crop material. This may include adjusting an angle of the one or more guide plates, for example, to change an angle of deflection of crop material as it is distributed by the spreader tool.

[0036] The event based sensor may be mounted or be mountable on the machine. For example, the sensor may be mounted or be mountable on the rear of the machine, or on an unloading auger of the machine, for example.

[0037] The one or more controllers may collectively comprise an input (e.g. an electronic input) for receiving one or more input signals. The one or more input signals may comprise sensor data from the event based sensor. The one or more controllers may collectively comprise one or more processors (e.g. electronic processors) operable to execute computer readable instructions for controlling operational of the control system, for example, identify the crop material component(s) and / or determine the crop material parameter(s). The one or more processors may be operable to generate one or more control signals for controlling operation of the one or more operable components. The one or more controllers may collectively comprise an output (e.g. an electronic output) for outputting the one or more control signals.

[0038] A further aspect provides an agricultural machine comprising the system as described hereinabove. The agricultural machine may comprise a harvesting machine, such as a combine harvester.

[0039] A further aspect provides a method of monitoring the operation of an agricultural harvesting machine, comprising: receiving sensor data from an event based sensor mounted or otherwise coupled to the agricultural machine, the sensor data being indicative of a measure of material present within a material flow from a spreader tool of the agricultural harvesting machine; analysing the sensor data to: identify one or more crop material components within the material flow; determine a crop material parameter for the identified crop material component(s); and calculate a swath quality parameter in dependence on the crop materialparameters; and controlling operation of one or more operable components of or otherwise associated with the agricultural harvesting machine in dependence on the determined swath quality parameter.

[0040] The method may comprise performing any one or more of the functionalities of the one or more controllers of the system described hereinabove.

[0041] A further aspect of the invention provides computer software comprising computer readable instructions which, when executed by one or more electronic processors, causes performance of a method in accordance with any aspect described herein.

[0042] A yet further aspect of the invention provides a computer readable medium having the computer software of the preceding aspect of the invention stored thereon.

[0043] Within the scope of this application it should be understood that the various aspects, embodiments, examples and alternatives set out herein, and individual features thereof may be taken independently or in any possible and compatible combination. Where features are described with reference to a single aspect or embodiment, it should be understood that such features are applicable to all aspects and embodiments unless otherwise stated or where such features are incompatible.BRIEF DESCRIPTION OF THE DRAWINGS

[0044] One or more embodiments of the invention / disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0045] FIG. 1 is a schematic side cross-sectional view of an agricultural harvester embodying aspects of the present disclosure;

[0046] FIG. 2 is a schematic view of an embodiment of a system of the present disclosure;

[0047] FIG. 3 illustrates sensor data obtained by a sensing arrangement forming part of aspects of the present disclosure;

[0048] FIG. 4 is an image of residue material obtained utilising a camera illustrating the operational use of embodiments discussed herein; and

[0049] FIG. 5 is a graphical representation of mapped data generated utilising embodiments discussed herein.DETAILED DESCRIPTION

[0050] The present disclosure relates to systems and methods for monitoring the operation of an agricultural harvesting machine, here a harvester 10. An event based sensor 30 is utilised to obtained sensor data which is indicative of a measure of material present within a material flow from a spreader tool 22 of the harvester 10. Controller(s) 102 is configured to analyse the sensor data to identify one or more crop material components, e.g. straw, within the material flow, and determine a crop material parameter for the identified crop material component(s). As discussed herein, this may include a parameter indicative of material quality, from which an operational efficiency or effectiveness of crop processing apparatus of the harvester 10 may be inferred. The crop material parameter is utilised to calculate a swath quality parameter which, as discussed hereinbelow, can be used to generate a mapped environment of swath quality across a working environment, e.g. for use in subsequent operational planning, or for real time adjustment of the operation of the harvester 10, e.g. where an increase in swath quality may be obtainable. Operation of one or more operable components of or otherwise associated with the harvester 10 (e.g. a user interface 32 or components of the spreader tool 22, for instance) can be controlled based on determined parameters, in the manner discussed herein.Harvester

[0051] With reference to FIG. 1, an agricultural machine in the form of a harvester 10 is shown which embodies aspects of the present disclosure.

[0052] The harvester 10 is coupled to a header 12 which is operable, in use, to cut and gather a strip of crop material as the harvester 10 is driven across a field / area to be harvested during a harvesting operation. A conveyor section 14 conveys the cut crop material from the header 12 into a crop processing apparatus 16 operable to separate grain and non-grain (i.e. material other than grain (MOG) or residue material (used interchangeably herein)) as will be appreciated. It is noted here that apparatus for separating grain and non-grain material are well- known in the art and the present invention is not limited in this sense. The skilled person willappreciate that numerous different configurations for the crop processing apparatus may be used as appropriate. Clean grain separated from the cut crop material is collected in a grain bin 18, which may be periodically emptied, e.g. into a collection vehicle, storage container, etc. utilising unloading auger 20. The remaining non-grain material (MOG), here specifically straw material is separately moved to a spreader tool 22 which is operable in use to distribute the crop material from the rear of the harvester 10 and onto the ground. In Figure 1, this is represented by arrow 24 which illustrates the crop material being distributed rearwards from the harvester 10. Here, the spreader tool 22 is provided in the form of a tailboard 22 for the harvester 10 having guide plates in the form of straw guides for controlling the distribution of the crop material from the tailboard 22. As discussed herein, the straw guides are controllable for controlling said distribution, as required. The harvester 10 also includes a chopper tool 25 positioned between the crop processing apparatus 16 and the spreader tool 22 and operable, in use, to cut the crop material before it is distributed by the spreader tool 22.

[0053] The harvester 10 also typically includes, amongst other features, an operator cab 26, wheels 28, engine (not shown) and a user interface in the form of a display terminal 32 provided within the operator cab 26.Event Based Sensor

[0054] As will be discussed in detail herein, the harvester 10 additionally includes a sensor in the form of an event based sensor 30. The sensor 30 is mounted to a rear of the harvester 10 in the illustrated embodiment, and has a field of view which encompasses an operating region of the spreader tool 22, specifically here the region of the working environment directly behind the harvester 10 into which material is distributed by the spreader tool 22. Event based sensor 30 is used, by a control system 101 of the harvester 10, to identify material components in a material flow associated with the spreader tool 22 in the manner discussed herein.

[0055] Event based sensor 30 has multiple individual sensing elements (pixels) which are individually triggerable in dependence on one or more variations in the signal output associated with those elements. The one or more variations result from a change in the light incident on said sensing elements, resulting in a change in amplitude in the output signal of thosesensing elements. By setting relevant thresholds for said variations, "events" can be triggered in dependence on a given change in incident light on said sensing elements. The variations can be used as an indication of movement in the observed environment, for example, by monitoring successive triggers of adjacent sensing elements, and inferring movement in the observed scene based thereon. In further arrangements the overall sensor output for a given time period may be used - e.g. forming of a histogram or heat map of triggers - to obtain a measure of an overview of a given process - e.g. shape / size of flowpath or material distribution, etc.

[0056] Operation of the event based sensor 30 may be controlled based on the given scenario. For instance a filter may be applied to the sensing elements to control at what magnitude of change / variation said sensing elements are triggered. This may be utilised to remove background events from the data. For instance, in the presently illustrated embodiment using the sensor 30 on the harvester 10 during a harvesting operation, motion of the ground with respect to the harvester 10 may otherwise trigger the sensing elements. Accordingly, a filter can be applied based on the ground speed of the harvester 10 to reduce or eliminate false triggers thereof. In alternative arrangements, the operational speed of the components of the harvester 10, e.g. an operational speed of elements of the spreader tool 22 or chopper tool 25, may be used to determine the filter to be applied to the sensor data to remove cyclical or repetitive motion from the scene - e.g. in order to identify anomalies or changes in that motion indicative of an operational issue.

[0057] In the present case, a clustering process may be applied to the sensor data. This clustering may be a timewise and / or location based clustering to identify triggers at multiple sensing elements which correlate to the same physical observation. For instance, data may be clustered timewise to identify (and optionally track movement of) individual material components (e.g. pieces of straw or other crop material) in the sensor data. In further examples, clustering may be applied in dependence on a determined flow velocity of material (e.g. as determined through application of an optical flow measurement of the sensor data) which may be used to indicate regions within the field of view of the sensor where material may be travelling at different speeds to other regions or compared with an expected flow parameter. Regionswithin the sensing area where no triggers are seen (in areas where they would be expected) may be indicative of a plugging of material or other fault preventing material flow in that region.System

[0058] FIG. 2 illustrates system 101 further. As shown, the system incorporates a control system 100 here having a single controller 102. The controller 102 includes an electronic processor 104, an electronic input 106 and electronic outputs 108, 110, 114. The processor 104 is operable to access a memory 112 of the controller 102 and execute instructions stored therein to perform the steps and functionality of the present invention, for example to output control signals 109, 111, 115 via the outputs 108, 110, 114 for controlling operation of operable components of the machine, including the spreader tool 22, for updating data stored on a remote server 34, e.g. a mapped representation of the crop and / or swath quality parameter(s), or for controlling the display terminal 32, for example to provide an graphical indication to an operator of the harvester 10 illustrative of the determined parameter(s).

[0059] The processor 104 is operable to receive sensor data via input 106 which, in the illustrated embodiment, takes the form of input signals 105 received from the event based sensor 30. As described in detail herein, the event based sensor 30 has a sensing region rearward of the harvester 10, with the sensor data received from the sensor 30 being indicative of a measure of crop material present within the sensing region. Using this information, the processor 104 is operable to identify one or more crop material components within the material flow from the spreader tool 22. This may include application of a filter to the sensor data, and / or performance of a clustering process to extract data relating to individual crop material pieces from background data. As discussed herein, a filter may be applied to the sensing elements to control at what magnitude of change / variation said sensing elements are triggered. This may be utilised to remove background events from the data. For instance, in the presently illustrated embodiment using the sensor 30 on the harvester 10 during a harvesting operation, motion of the ground with respect to the harvester 10 may otherwise trigger the sensing elements. Accordingly, a filter can be applied based on the ground speed of the harvester 10 to reduce or eliminate false triggers thereof. In alternative arrangements, the operational speed of the components of the harvester 10, e.g. an operational speed of elements of the spreader tool 22 or chopper tool 25, may beused to determine the filter to be applied to the sensor data set a trigger threshold which is related to an expected flow velocity for the crop material components.

[0060] A clustering process can be applied to the sensor data to identify sensing element triggers which relate to common crop material components. This clustering may be a timewise and / or location based clustering. For instance, data may be clustered timewise to identify (and preferably track movement of) individual material components (e.g. pieces of straw or other crop material) in the sensor data. Clustering may be applied in dependence on a determined flow velocity of material (e.g. as determined through application of an optical flow measurement of the sensor data) which may be used to indicate regions within the field of view of the sensor indicative of crop material being spread by the harvester 10.

[0061] For each of the identified crop material components, the processor 104 determines a crop material parameter in the manner described herein. Specifically, the processor 104 is analyse the sensor data received from the event based sensor 30 to determine crop material parameters which may include a measure of the length for one or multiple crop material components, e.g. looking for a uniformity of length of crop material components. The crop material parameters may include a measure of a shape and / or structure of the crop material components, optionally including a measure of the "straightness" of the crop material component, and / or a speed or trajectory of the crop material components as determined through an optical flow analysis of the sensor data for identified crop material components.

[0062] Advantageously, the sensor data can be used to determine a quality parameter for the crop material indicative of the operational effectiveness or efficiency of crop processing components of the harvester 10. For instance, a substantially consistent length of cut of the crop material components may be indicative of an acceptable operation of a chopper tool 25 of the harvester 10, that being providing chopped crop material with a consistent length for, e.g. a later baling process. Where the length of the identified crop material varies significantly, this could be indicative of a fault with the chopper tool, for example. A shape or structure of the crop material component may indicate a level of processing of the crop material by the harvester 10. A material piece having multiple bends, kinks etc. may indicate an over working of the crop material component by the machine which could be detrimental depending on what that crop material isto be used for. The speed, flow characteristics, etc. advantageously provide information relating to the location in the working environment where the material is spread. This in turn can be used to generate a measure of parameters relating to, e.g. a formed swath. In the present case, where the crop material is discharged by the spreader tool 22 to form a swath, the crop material parameter is utilised to calculate a swath quality parameter which can be used, for example, to map the quality of the swath across an environment which may feed into subsequent operational planning.

[0063] Output 108 is operably coupled to the spreader tool, here the tailboard 22, or more specifically a local control unit for the tailboard 22. The control system 101 is operable to control operation of the tailboard 22, specifically operational components thereof in the form of first and second straw guides 23a, 23b in dependence on the determined swath quality parameter, determined in the manner discussed herein. For instance, where the swath quality parameter relates to a measure of a width or profile of the swath formed by crop material distributed by the tailboard 22, the control system 101 may control, through generation and output of control signals 109, the tailboard 22 and specifically the straw guides 23a, 23b to, for example, adjust a distribution in first and second directions corresponding to the first and second straw guides 23a, 23b. This may advantageously be used to increase or decrease a width of the formed swath, for example.

[0064] Output 110 is operably coupled to the display terminal 32 of the harvester 10. Here, the control system 101 is operable to control operation of the display terminal 32, e.g. through output of control signals 111 in order to display operational data to an operator of the harvester 10 relating to the operation of the control system 101. Specifically, the control system 101 may be operable to control the display terminal 32 to display to the operator a graphical representation of the swath quality parameter as calculated by processor 104, sensor data from the event based sensor 30, image data obtained from a camera on the harvester 10, or other useful information. In some variants, the display terminal 32 may also be operable to receive a user input from the operator, and in such instances the output 110 may act as an input for receiving that user input at the processor 104. The user input may relate to a requested or targetparameter, for example a desired swath width, height, profile etc., made by the operator of the harvester 10.

[0065] Output 114 is operably coupled to a database hosted on remote or cloud server 34. Here, the output 114 forms a communication module for communicating data offboard from the harvester 10. Alternatively, output 114 may utilise a communication module of the harvester 10 and may be operably connected to server 34 indirectly thereby. The control system 101 may utilise server 34 to store data relating to the calculated swath quality parameter, e.g. for later retrieval for operational planning of subsequent operations, e.g. a baling operation. In embodiments the control system 101 may be operably connected to a positioning module of the harvester 10, and the location of the harvester 10 within the environment may be linked to calculated swath quality parameters to generate a mapped environment containing data indicative of, e.g. a crop material quality across the environment.

[0066] A representation of an example mapped environment is provided in FIG. 5. FIG. 5 shows a representation of a working environment, of field, F, having a plurality of swaths / windrows formed by the harvester 10. Information relating to the calculated swath quality parameter for the swaths at different locations within the environment F is stored at server 34 in such a mapped representation of the environment. Specifically here, the swath quality parameter calculated across the field is categorized into first, second and third swath quality parameters 200a, 200b, 200c. This could, for instance, be categorized in dependence on a quantification of the uniformity (e.g. of length) of the crop material components forming the given swath. This could be a quantification of a crop material quality, e.g. based on a determination of a crop material parameter relating to a level of processing applied to the crop material by the harvester 10 during the harvesting operation - e.g. looking for excessive bends or crimping of the material pieces.

[0067] The graphical representation shown in FIG. 5 may be presented directly to an operator, e.g. a farm manager looking to plan subsequent operations in the field. The information may be utilised to determine a baling strategy, e.g. to plan baling such that bales of substantially uniform quality can be formed. The swath quality map may be used to associate or determine a bale quality parameter based on the section of swath corresponding to a particular bale. Thisinformation may be linked with the relevant bale utilising known technologies, including the use of identification tags or the like, for example.

[0068] In further examples, the control system 101 may be operable to control further operational components of the harvester, e.g. to actively change a crop material quality parameter in real time. For instance, the control system 101 may be operably coupled to crop processing apparatus 16 of the harvester 10, for adjusting a degree of processing applied to the crop material as it passes through the harvester 10. This may be performed, in dependence on crop material parameters relating to the shape or structure of the identified crop material components.Sensor Data

[0069] To be FIG. 3 illustrates a representation of sensor data obtained during operation of the harvester 10. Here, "triggered" sensing elements of the event based sensor 30 are shown indicative of movement in the region of the environment corresponding to the relevant pixel in the displayed sensor data. FIG. 4 illustrates a similar scene utilising a camera based system with residue material shown being spread behind the rear of a harvester 10. This is inherently noisy and near impossible to identify individual crop components from the image data alone.

[0070] As shown, through suitable processing of the sensor data via filters and the like, only those triggers in response to movement of crop material within the observed region is present in the generated representation. This removes background noise / triggers from the data enabling further features to be extracted from the data, including the crop material parameter(s) discussed herein.

[0071] In FIG. 3, a crop material component, C is highlighted. In the sensor data, this is represented as a substantially straight line of adjacent triggered sensing elements. From this, a crop material parameter in the form of a measure of the length of the identified material component can be computed. This can be a count of the number of adjacent sensing elements triggered determined to correspond to the crop material component. In an extension of this, a further sensing capability may be introduced with suitable data fusion thereof to obtain a measure of the distance of the crop material component from the event based sensor 30, thisdistance being used to compute an absolute length of the crop material component. The additional sensing capability may include a transceiver based system, such as LIDAR, for instance. As discussed, a relative length measure may be calculated, relating to the length of the identified crop material components across the sensor data, which may provide information relating to the uniformity of cut of the material by the chopper tool 25.

[0072] In a further variant, a crop material parameter in the form of a measure of the shape or structure of the crop material component may be computed. This may, for example, quantify a number of bends or kinks in a given material component. As discussed herein, this may be indicative of a level of crop processing provided by the harvester 10, and suitable control actions may be taken dependent thereon.

[0073] In a further variant, an optical flow analysis may be performed on the sensor data, from which it may be possible to extract flow parameters for any given identified crop material component, including a speed, direction or even mass of the material component (based on speed and length of the relevant component. The flow parameters may provide information relating to the trajectory of a crop material component, including thereby information relating to a resting location for any given material component in the working environment. This may be particularly beneficial to determine whether material discharged by the spreader tool 22 is forming a distribution, e.g. full spread pattern or swath operation, with desired characteristics, e.g. width and / or profile. Information on the resting location for given material components may also be linked to the mapped dataset stored at server 34. In the illustrated examples provided here, the trajectory of identified crop material components is used to determine a working width of the swath being formed by the harvester 10. As discussed above, this can then subsequently be used to notify the operator through user interface 32 of the observed width, any adjustments to be made etc. or indeed automate control of the tailboard 22, and straw guides 23a, 23b thereof to adjust the swath width based on the observations.General

[0074] Any process descriptions or blocks in flow diagrams should be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process, and alternateimplementations are included within the scope of the embodiments in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present disclosure.

[0075] It will be appreciated that embodiments of the present invention can be realized in the form of hardware, software or a combination of hardware and software. Any such software may be stored in the form of volatile or non-volatile storage such as, for example, a storage device like a ROM, whether erasable or rewritable or not, or in the form of memory such as, for example, RAM, memory chips, device or integrated circuits or on an optically or magnetically readable medium such as, for example, a CD, DVD, magnetic disk or magnetic tape. It will be appreciated that the storage devices and storage media are embodiments of machine-readable storage that are suitable for storing a program or programs that, when executed, implement embodiments of the present invention. Accordingly, embodiments provide a program comprising code for implementing a system or method as set out herein and a machine readable storage storing such a program. Still further, embodiments of the present invention may be conveyed electronically via any medium such as a communication signal carried over a wired or wireless connection and embodiments suitably encompass the same.

[0076] All references cited herein are incorporated herein in their entireties. If there is a conflict between definitions herein and in an incorporated reference, the definition herein shall control.

Claims

CLAIMSWhat is claimed is:

1. A system for monitoring the operation of an agricultural harvesting machine, the system comprising: an event based sensor; and one or more controllers, configured to: receive sensor data from the event based sensor indicative of a measure of material present within a material flow from a spreader tool of the agricultural harvesting machine; analyse the sensor data to: identify one or more crop material components within the material flow; determine a crop material parameter for the identified crop material component(s); and calculate a swath quality parameter in dependence on the crop material parameters; and output one or more control signals for controlling operation of one or more operable components of or otherwise associated with the agricultural harvesting machine in dependence on the determined swath quality parameter.

2. A system as claimed in claim 1, wherein the one or more controllers may be configured to:apply a clustering process to the sensor data to identify and associate triggers at multiple sensing elements of the event based sensor; and identify one or more material components in dependence on the associated sensing element triggers.

3. A system as claimed in claim 1 or claim 2, wherein the crop material parameter comprises one or more of: a measure of the length of the identified crop material component(s); a measure of a shape or structure of the identified crop material component(s); and / or a measure of a speed associated with the identified crop material component(s).

4. A system as claimed in any preceding claim, wherein the crop material parameter(s) comprises a measure of a trajectory of the respective crop material component(s); and wherein the trajectory measure is utilised by the one or more controllers to compute a resting location within the working environment where the crop material component(s) come to rest.

5. A system as claimed in claim 4, wherein the one or more controllers are configured to: perform an optical flow measurement on the received sensor data, providing a crop material parameter in the form of a flow parameter comprising a measure of a direction, speed or velocity distribution associated with material movement within the sensing region of the event based sensor; andcompute the resting location of the crop material component(s) in dependence on the optical flow measurement.

6. A system as claimed in claim 5, wherein the swath quality parameter comprises a measures of a width of the formed swath.

7. A system as claimed in claim 6, wherein the one or more controllers are configured to: compare the measure of the width of the resultant swath with one or more control parameters; and control operation of the spreader tool of the agricultural harvesting machine in dependence on this comparison.

8. A system of any preceding claim, wherein the swath quality parameter comprises a measure of a swath profile or contour.

9. A system of any preceding claim, wherein the swath quality parameter comprises information indicative of a crop component quality; including one or more of a measure of a length, shape or structure of the crop components forming the swath.

10. A system of claim 9, wherein the one or more controllers are configured to quantify the crop component quality in dependence on the determined crop material parameter(s), and associate this with a specific swath location providing a swath quality parameter which comprises an indication of the swath quality at that location.

11. A system of any preceding claim, wherein the one or more operable components of or otherwise associated with the agricultural harvesting machine includes a user interface; and wherein the one or more controllers are configured to controloutput of an indicator indicative of the determined swath quality parameter via the user interface.

12. A system of any preceding claim, wherein the one or more components of or otherwise associated with the agricultural harvesting machine includes a database for storing information indicative of the determined crop material parameter(s) and / or swath quality parameter(s); and wherein the one or more controllers are be communicable with the database for storing information thereon.

13. A system of claim 12, wherein the system comprises or the controller(s) are communicable with a positioning system for providing location data indicative of the location of the agricultural harvesting machine within a working environment; and wherein the one or more controllers are configured to utilise the location data to associate the determined swath quality parameter(s) with a location within the working environment, thereby generating a mapped representation of the swath quality within the environment.

14. A system of any preceding claim, wherein the one or more controllers are configured to control a forward speed of the agricultural harvesting machine in dependence on the determined swath quality parameter.

15. A system of any preceding claim, wherein the component(s) of or otherwise associated with the agricultural harvesting machine comprises a header for the machine; and wherein the one or more controllers are configured for controlling an operational speed of elements of the header, a lift mechanism for the header for controlling an operational position or height of the header, and / or an operational state of the header in dependence on the swath quality parameter.

16. A system of any preceding claim, wherein the component(s) of or otherwise associated with the agricultural harvesting machine comprise a crop processing apparatus of the machine; and wherein the one or more controllers are configured to adjust an operational speed of components of the crop processing apparatusto control the speed at which crop material is processed and moved through the agricultural harvesting machine in dependence on the swath quality parameter.

17. A system of claim 16, wherein the crop processing apparatus includes a chopper tool; and wherein the one or more controllers are configured to control operation of the chopper tool in dependence on the swath quality parameter.

18. A system of any preceding claim, wherein the component(s) of the agricultural harvesting machine comprises a straw guide apparatus, including one or more guide plates, for controlling a distribution of crop material components from the spreader tool; and wherein the one or more controllers are configured to control operation of the straw guide apparatus in dependence on the swath quality parameter.

19. An agricultural machine comprising the system of any preceding claim.

20. A method of monitoring the operation of an agricultural harvesting machine, comprising: receiving sensor data from an event based sensor mounted or otherwise coupled to the agricultural machine, the sensor data being indicative of a measure of material present within a material flow from a spreader tool of the agricultural harvesting machine; analysing the sensor data to: identify one or more crop material components within the material flow; determine a crop material parameter for the identified crop material component(s); andcalculate a swath quality parameter in dependence on the crop material parameters; and controlling operation of one or more operable components of or otherwise associated with the agricultural harvesting machine in dependence on the determined swath quality parameter.

Images