Pest detection system and method

Abstract

A pest detection system and method, the method comprising: training a machine learning model to determine one or more pest parameters; generating image data using the camera module; analyzing the image data with a processing unit using a machine learning model to determine one or more pest parameters; and storing the one or more pest parameters on a computer readable medium.

Description

TECHNICAL FIELD

[0001] The present disclosure is for a system and method for detecting pests, in particular agricultural pests (e.g. wireworms). BACKGROUND

[0002] Pests are known to damage property, in particular agricultural crops. It is necessary to detect the presence of pests in order to take appropriate action.

[0003] Pest detection can be labour intensive, for example requiring a human to physically inspect an area to determine the presence or absence of pests. Further still, some pests, for example subterranean pests (e.g. pests that live in the soil) are not easily detected.

[0004] Wireworms are the larvae of click beetles (e.g. Agriotes lineatus, Agriotes obscurus and Agriotes sputator) and are a species of pest. Wireworms are known to damage the roots and other subterranean parts of many agricultural crops, including potatoes. As wireworms live in the soil, they are difficult to detect, meaning that agricultural crops can be inadvertently planted in infected areas, resulting in significant damage to the crops.

[0005] There is a need for an improved system and method for detecting pests. SUMMARY

[0006] A pest detection system is disclosed, comprising: a camera module configured to generate image data; and a processing unit configured to receive the image data, to determine one or more pest parameters associated with the image data using a machine learning model; and to record the one or more pest parameters to a computer readable medium.

[0007] The pest detection system can further comprise a soil agitator, and the camera module can be configured to capture images of soil agitated by the soil agitator.

[0008] The one or more pest parameters can comprise at least one of a presence or absence of a pest, a quantity of a pest, or a type of a pest.

[0009] The machine learning model can be trained using training data comprising images of surrogates for pests.

[0010] The pest can be a wireworm species, and the surrogate can be a mealworm species.

[0011] The machine learning model can be trained using training data comprising at least one image formed of an array of pixels, the image can be divided into tiles, each tile comprising a subset of the pixels in the image, and the machine learning model can be trained using the tiles.

[0012] The processing unit can be configured to determine the pest parameter by: dividing the image captured by the camera module into overlapping tiles; assigning a confidence value to each pixel in each tile such that each pixel is assigned a plurality of confidence values; and averaging the confidence values for each pixel.

[0013] The one or more pest parameters can comprise a number of pests, and the processing unit can be configured to determine the number of pests by: flattening the confidence values for each pixel that falls within a threshold, determining coordinates of local confidence value peaks in the array of pixels, and counting a number of local peaks.

[0014] A pest detection method is also disclosed, comprising: training a machine learning model to determine one or more pest parameters; generating image data using a camera module; analyzing the image data with the processing unit using the machine learning model to determine the one or more pest parameters; and storing the one or more pest parameters onto a computer readable medium.

[0015] The method can further comprise: agitating the soil, and capturing an image of the agitated soil with the camera module.

[0016] The one or more pest parameters can comprise at least one of a presence or absence of a pest, a number of pests, or a type of pest.

[0017] The machine learning model can be trained using training data comprising an image of a surrogate for a pest.

[0018] The pest can be a species of mealworm, and the surrogate can be a species of yellow mealworm.

[0019] The machine learning model can be trained using training data comprising at least one image formed of an array of pixels, the image can be divided into tiles, each tile comprising a subset of the pixels in the image, and the machine learning model can be trained using the tiles.

[0020] The processing unit can be configured to determine the pest parameter by: dividing the image captured by the camera module into overlapping tiles; assigning a confidence value to each pixel in each tile such that each pixel is assigned a plurality of confidence values; and averaging the confidence values for each pixel.

[0021] The one or more pest parameters can comprise a number of pests, and the processing unit can be configured to determine the number of pests by: flattening the confidence values of each pixel falling within a threshold, determining coordinates of local confidence value peaks in the array of pixels, and counting the number of local peaks. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order that the disclosure can be more readily understood, preferred embodiments thereof will be described below, by way of example only, with reference to the accompanying drawings, in which: Figure 1 A schematic diagram illustration of a pest detection system is shown; Figure 2 A schematic diagram illustration of a pest around a crop is shown; Figure 3 A schematic diagram illustration of a vehicle is shown; Figure 4 A schematic diagram illustration of a vehicle is shown; Figure 5 A schematic diagram illustration of a pest detection system mounted to a vehicle is shown; Figure 6 A pest detection system mounted to a vehicle is shown; Figure 7a and Figure 7b An image of a pest is shown; Figure 7c An image of a non-pest is shown; Figure 8 A schematic diagram illustration of a camera module is shown; Figure 9 A schematic diagram illustration of a camera module is shown; Figure 10 A schematic diagram illustration of a processing unit is shown; Figure 11 A schematic diagram illustration of an output unit is shown; Figure 12 A schematic diagram illustration of a storage subsystem is shown; Figure 13 A schematic diagram illustration of a communication subsystem is shown; Figure 14 A schematic diagram illustration of a processing unit is shown; Figure 15 A schematic diagram illustration of a power subsystem is shown; Figure 16 and Figure 17 A housing and a housing mounting bracket are shown; Figure 18 A schematic diagram illustration of a remote management system is shown; Figure 19 andFigure 20 A training image split into tiles is shown; Figure 21a and Figure 21b An RGB image and an HED colorized image of a pest are shown; Figure 22 Images of a pest in various color spaces are shown; Figure 23 A pest detected by a pest detection system is shown; Figure 24 A heat map of a pest is shown; Figure 25 A histogram of detected pests at different confidences is shown; Figure 26 A tractor is shown in a schematic illustration; Figure 27 A tractor is shown in a schematic illustration; and Figure 28 A vehicle is shown. DETAILED DESCRIPTION

[0023] The disclosed technology includes a pest detection system 1 and method. The pest detection system 1 and method can be configured to determine one or more pest parameters (e.g., a presence or absence of a pest 6 (see, e.g., FIG. 1), a type of pest 6, a number of pests 6, and / or a location of a pest 6). Figure 2

[0024] The pest 6 or each pest 6 detected by the system 1 can be an organism that causes damage to the crop 2. The crop 2 can include an underground harvestable item 22, in particular an underground fruit or vegetable, which can thus be a root vegetable. At least a portion of the harvestable item 22 can be located underground. The underground harvestable item 22 can be entirely located underground (e.g., potatoes), or can include a portion that is underground and a portion that is above ground (e.g., onions). The harvestable item 22 can also be referred to as an agricultural product. Examples of such a crop 2 include potatoes, sugar beets, carrots, turnips, taros, cassavas, yams, ginger, onions, and the like. The crop 2 can be, in particular, a potato crop. Thus, the pest 6 can be an organism that damages a potato crop.

[0025] ​The pest 6 or each pest 6 can be an animal. The pest 6 or each pest 6 can be a subterranean pest. The pest 6 or each pest 6 can be a pest that lives in soil. The pest 6 or each pest 6 can typically be found underground. Thus, the pest 6 or each pest 6 can cause damage to an underground portion of the crop 2 (e.g., the harvestable items 22 underground) or generally roots (even if harvestable portions of the crop 2 are above ground, e.g., apples, grapes, lychees, etc.). The pest 6 or each pest 6 can live in the ground or soil at one stage of its life cycle, but can live primarily above ground at another stage of its life cycle.

[0026] The pest 6 or each pest 6 can include or be a larva. Thus, the pest 6 or each pest 6 can cause damage to an underground portion of the crop 2 when in a juvenile form. The pest 6 or each pest 6 can include or be a wireworm, also known as an elaterid larva. The pest 6 or each pest 6 can particularly include a member of the elateridae family (e.g., striped wireworm, dark wireworm, and / or banded wireworm).

[0027] The pest 6 or each pest 6 can include a disease associated with the crop 2 (e.g., potato blight in the case of potatoes), and the system 1 can thus be configured to detect symptoms of the disease, e.g., a color or shape of a portion of the crop 2 (e.g., a leaf and / or a root and / or a fruit). The pest 6 or each pest 6 can include a fungal pest 6, e.g., a mold, also known as mildew.

[0028] The system 1 can be configured to detect multiple pests 6 that can be from different families. For example, the system 1 can be configured to detect one or more larval pests, and one or more non-larval pests. Additionally or alternatively, the system 1 can be configured to detect multiple different larval pests and / or non-larval pests. The system 1 can be configured to detect at least one subterranean pest 6, and can also be configured to detect at least one above-ground pest 6. The system 1 can be configured to detect at least one animal pest 6, and can also be configured to detect at least one non-animal pest 6 (e.g., a fungal pest).

[0029] The system 1 can be configured to detect a plurality of pests 6 and to identify the type of the detected pests 6. For example, in the case where the system 1 is configured to detect a plurality of pests 6 that are larvae (e.g., wireworms and sciarid fly larvae), the system 1 can be configured to identify which type of pest 6 has been detected (or, in some cases, it can be both or all of the pests 6). In versions of the system 1 that are configured to detect only one type of pest 6, the system 1 can still identify the type of the detected pest 6 (or this can be inherent when a pest 6 is detected).

[0030] The system 1 can include one or more of: an imaging subsystem 11, a storage subsystem 12, a communication subsystem 13, a location subsystem 14, a power subsystem 15, and a processing subsystem 16.

[0031] The location subsystem 14 can be configured to determine a current location of the pest detection system 1 (or a portion thereof). For example, the location can be a longitude and latitude. In some embodiments, the location can be a location relative to a fixed geographic location.

[0032] The storage subsystem 12 can be configured to store one or more pest parameters and / or a current location (as determined by the location subsystem 14, if provided). The storage subsystem 12 can be further configured to store other information, as will be apparent from this description.

[0033] The communication subsystem 13 can be configured to transmit one or more pest parameters and / or a current location (as determined by the location subsystem 14, if provided) to a remote management system 3. Collectively, the pest detection system 1 and the remote management system 3 (if provided) can be referred to as a pest detection and management system 4.

[0034] The power subsystem 15 can be configured to provide power to one or more (or all) components of the pest detection system 1.

[0035] The pest detection system 1 can be configured to be carried, in whole or in part, by the vehicle 100, and thus can be configured to be installed, in whole or in part, to the vehicle 100. The pest detection system 1 can be configured such that it is installable to a variety of different vehicles 100. Such vehicles 100 can include tractors, harvesters, all-terrain vehicles, passenger cars, trucks, and the like. The system 1 can be installed to an implement (e.g., a plow (or a plough), a cultivator, a planter, and the like) that is towed by the vehicle 100.

[0036] The system 1 can include a tillage machine 50, which can be configured to till a ground medium (e.g., soil) - see, for example,Figure 5 The tiller 50 can have an active position in which the tiller 50 is configured to till the ground medium and a passive position (e.g. a raised position) in which the tiller 50 is configured not to till the ground medium. This can facilitate transportation of the tiller 50. Thus, the tiller 50 can comprise one or more tillage members 51 configured to at least partially extend into the ground medium (e.g. soil) and, in use, to agitate the ground medium.

[0037] Examples of such tillers 50 include ploughs, harrows, cultivators, rotary tillers, stone cleaners, and ploughshares of harvesters (e.g. root vegetable harvesters such as potato harvesters, carrot harvesters, etc.). One or more of the subsystems 11-16, or at least a portion thereof, can be mounted to the tiller 50. For example, at least a portion of the imaging subsystem 11 (e.g. the camera module 111) can be mounted to the tiller 50.

[0038] The or each tillage member 51 can comprise (or can be) a tool configured to till the ground medium (e.g. soil). Thus, the tillage member 51 can comprise a disc, a tine, a tine, or a ploughshare for tilling soil. The tiller 50 can be towed by the vehicle 100 and / or can be mounted to the vehicle 100. In some versions, the tiller 50 can comprise one or more ground engaging wheels 501.

[0039] The tiller 50 can also be referred to as a soil agitator 50, as the purpose of the tiller 50 (or the soil agitator 50) is to agitate the ground medium (typically soil). This agitation is typically mechanical agitation, which is caused by movement of the implement (tillage member 51) through the ground medium or soil. Thus, the action of the tiller 50 is to invert at least a portion of the soil (which can thereby expose one or more pests 6).

[0040] The vehicle 100 can comprise a plurality of ground engaging wheels 101 and / or tracks configured to support the body 102 of the vehicle 100 over the ground. The vehicle 100 can comprise an engine 103 configured to drive rotation of one or more of the ground engaging wheels 101 and / or tracks, and / or configured to drive operation of one or more other components of the vehicle 100 (this can also be the case for a towed vehicle 100).

[0041] In some embodiments, the vehicle 100 is a towed vehicle 100 (e.g. in Figure 3 from a towing vehicle (e.g. from a tractor 300). In some embodiments, the towing vehicle (e.g. the tractor 300) comprises an engine 301 and an electrical system 302 (see, for example,Figure 27 ), which can be mechanically and electrically coupled to the vehicle 100, respectively.

[0042] The vehicle 100 can include a cab 104, which can be part of the body 102, and from which an operator can control operation of the vehicle 100, including, for example, steering and control of operation of the engine 103. Reference to the cab 104 of the vehicle 100 can be reference to a cab or control panel of a towed vehicle 100, or to the cab 104 of a self-propelled vehicle 100, and is interpreted in embodiments including such vehicles to encompass the cab 303 of a towing vehicle (e.g., tractor 300).

[0043] The vehicle 100 can include an electrical system 105 (see, e.g., Figure 26 ) configured to provide electrical power to one or more parts of the vehicle 100 (e.g., to drive operation of one or more electric motors). The electrical system 105 can include one or more batteries for storing electrical power and / or an alternator or other electrical generator coupled to the engine 103 to generate electrical power. In embodiments including a towed vehicle (e.g., tractor 300), the electrical system 105 of the vehicle 100 can be electrically coupled to the electrical system 302 of the tractor 300 (or other vehicle) to allow the vehicle 100 to be at least partially powered by the tractor 300 (or other vehicle).

[0044] The vehicle 100 can be configured for harvesting one or more harvestable items 22. As described, the one or more harvestable items 22 can include a tuber vegetable (e.g., potato or carrot) that is buried in the ground.

[0045] Accordingly, the vehicle 100 can include a share 106 configured to lift the one or more harvestable items 22 from the ground (i.e., from the soil) and transport the one or more items 22 (which are now one or more harvested items 21) toward a container 107 (see, e.g., Figure 4 ). The share 106 is an example of a tillage member (or soil agitator) 51.

[0046] The vehicle 100 can include a conveyer 108 to transport the one or more harvested items 21 from the share 106 toward the container 107.

[0047] The conveyer 108 can be in the form of a spaced series of slats or strips oriented perpendicular to the direction of travel of the conveyer 108, such that soil and other debris can pass between the slats or strips. In some versions, the conveyer 108 can include a belt, which can thus be a transport belt.

[0048] The vehicle 100 can comprise a picking table 109 (see, for example Figure 28 ), which can be part of the conveyor 108 (i.e. can be a generally flat section of the conveyor 108 that is accessible by one or more pickers). The or each harvested item 21 can be transported by the vehicle across the picking table 109, and one or more pickers can manually remove stones and other large debris. The picking table 109 is generally upstream of the container 107 with respect to movement of the one or more harvested items 21 by the vehicle 100 (the coulter 106 is downstream of the picking table 109 and the conveyor 108). The picking table 109 can comprise a canopy.

[0049] From the picking table 109 and / or the conveyor 108, the one or more harvested items 21 can be transported to the container 107. For example, this transportation can comprise use of one or more further conveyors 110.

[0050] In some embodiments, the container 107 is carried by a second vehicle (not shown) that drives alongside the vehicle 100. The second vehicle can be a self-propelled vehicle, such as a tractor that tows a trailer, with the container 107 supported on the tractor. Thus, the second vehicle can comprise a tractor that is generally the same as or similar to the tractor 300.

[0051] In some embodiments, the container 107 is carried by the vehicle 100.

[0052] As will be appreciated, the form of the vehicle 100 can vary, but can comprise a section through which the one or more harvested items 21 pass (or otherwise travel) relative to that section - for example, in their movement driven by the conveyor 108, the picking table 109, or the one or more further conveyors 110.

[0053] Thus, the vehicle 100 can be a harvester, and the pest detection system 1 can be used to detect pests 6 while harvesting a crop.

[0054] In some versions, the vehicle 100 can be used to cultivate the ground medium before a crop is planted, and the pest detection system 1 can thus be used to detect pests before a crop is planted. A user can use information provided by the pest detection system 1 to take action before a crop is planted, for example to apply a chemical treatment (e.g. a pesticide, fungicide, and / or herbicide), or to select a location to plant a crop based on one or more pest parameters determined by the pest detection system 1.

[0055] The pest detection system 1 may be carried by or entirely within the vehicle 100. In some versions, a portion of the pest detection system 1 may be carried by the vehicle 100, and a portion of the pest detection system 1 may be located elsewhere (e.g., away from the vehicle 100).

[0056] Imaging subsystem 11 may include camera module 111. Camera module 111 may be configured to acquire one or more images of the ground medium. Camera module 111 may be specifically configured to acquire one or more images of the ground medium after it has been tilled by tiller 50. Thus, camera module 111 may be configured to acquire one or more images of the stirred soil. Camera module 111 may additionally or alternatively be configured to acquire one or more images of the harvested crop 21. Thus, system 1 may be configured to detect pests 6 in the ground medium and / or on the harvested crop 21. In some versions, camera module 111 may be configured to acquire images of crop 2, which may include the aboveground parts of crop 2 (e.g., one or more leaves of crop 2).

[0057] In some versions, camera module 111 can be configured to acquire one or more images of the ground surface formed by the ground medium. As described above, tiller 50 can be configured to agitate the ground medium, and therefore camera module 111 can be configured to acquire one or more images of the agitated ground medium. Thus, camera module 111 can be configured to acquire one or more images of the agitated ground. Therefore, camera module 111 can be mounted to vehicle 100, which may include mounting camera module 111 to tiller 50 such that the operating components (e.g., lenses) of camera module 111 face downwards or toward the ground.

[0058] Therefore, the camera module 111 can acquire images of the tilled ground medium after the tiller 50 has finished tilling, such as... Figure 5 As illustrated in the diagram. These images can be obtained by analyzing the ground (such as...) Figure 5 It can be acquired by imaging (as shown), or by imaging a portion of the ground medium carried by vehicle 100 (e.g., soil transported on transport aircraft 108).

[0059] The system 1 can be configured such that the images of the tilled ground medium are acquired by the camera module 111 within a threshold period of time of the ground medium being tilled by the tiller 50 (in other words, within a threshold period of time of the soil being agitated by the soil agitator 50). This can be a period of time in which the pests 6 that are dug up by the tiller 50 are visible. It will be appreciated that the pests 6 that are dug up by the tiller 50 will typically burrow into the earth (soil) soon after being dug up, meaning that they can only be seen by the camera module 111 for a limited period of time.

[0060] The threshold period of time can be up to about 5 minutes, about 4 minutes, about 3 minutes, about 2 minutes, about 1 minute, about 50 seconds, about 40 seconds, about 30 seconds, about 25 seconds, about 20 seconds, about 15 seconds, about 10 seconds, about 5 seconds, or about 1 second. The system 1 can be configured such that the images of the tilled ground medium are acquired when the ground medium is being tilled (in other words, when the soil is being agitated). Thus, the system 1 can be configured such that at least a portion of the one or more tilling members 51 is located within the field of view of the camera module 111.

[0061] In some versions, the imaging subsystem 11 can comprise a plurality of camera modules 111. In such cases, a first camera module 111 can be configured to acquire images of the tilled ground medium. In versions in which the vehicle 100 is a harvester, a second camera module 111 can be configured to acquire images of the harvested items 21. In some versions, a first camera module 111 can be configured to acquire images of a first portion of the tilled ground medium, and a second camera module can be configured to acquire images of a second portion of the tilled ground medium (and the second portion can be different to the first portion). Likewise, for systems 1 having more than two camera modules 111, each camera module 111 can be configured to acquire one or more images of a portion of the tilled ground medium, and each portion can be different.

[0062] The at least one camera module 111 can be a visible light camera module. The at least one camera module 111 can be an infrared camera module. The at least one camera module 111 can be a multispectral camera module. The at least one camera module 111 can be a hyperspectral camera module.

[0063] One example of a multispectral camera module is the CMS-S camera by Silios Technologies. Examples of hyperspectral cameras include the Model 4250 VNIR camera by HinaLea and the Specim IQ camera. One example of a suitable visible light camera module is the Alvium 1800 u158-c camera.

[0064] In versions that include a hyperspectral camera, the hyperspectral camera can be configured to acquire a hyperspectral image in a spectral band (i.e. a range of wavelengths) in which the reflectance of the pest 6 peaks or exceeds a predetermined threshold, referred to as the optimal spectral band. Thus, the pest detection method can include determining a hyperspectral property of the pest 6 or each pest 6. The hyperspectral property of the pest 6 or each pest 6 can include the spectral band (wavelength or range of wavelengths) in which the reflectance of the pest 6 exceeds a predetermined threshold or peaks. This can correspond to the spectral band in which the pest 6 stands out or contrasts with the background (see, for example Figure 7a ).

[0065] The hyperspectral image data generated by the hyperspectral camera can also include image data corresponding to wavelengths outside of the spectral band corresponding to the pest 6. The hyperspectral camera can be configured to send the hyperspectral image data to the processing unit 114, and the processing unit can be configured to filter the hyperspectral image data such that only the hyperspectral image data corresponding to the spectral band associated with the pest 6 is used to determine the one or more pest parameters. In other versions, the processing unit 114 can be configured to use the hyperspectral image data corresponding to spectral bands falling outside of the optimal spectral band for the pest 6 to determine the pest parameters in addition to the optimal spectral band. Thus, there can be no filtering of the hyperspectral image data in such versions.

[0066] In the case of wireworms, the spectral band of 920 nm to 950 nm is particularly reflective (see, for example Figure 7a and Figure 7b where the pest 6 is a wireworm). The hyperspectral images acquired in this spectral band show low reflectance for non-pest 61 (e.g. earthworms) (see Figure 7c ). Thus, the hyperspectral camera can be configured to acquire hyperspectral image data in the 920 nm to 950 nm spectral band, and the processing unit 114 can be configured to use the hyperspectral image data corresponding to the 920 nm to 950 nm spectral band to determine the one or more pest parameters.

[0067] In cases where the system 1 is configured to detect a plurality of different pests 6, each type of pest 6 can have a respective optimal spectral band, and the hyperspectral camera can be configured to acquire a hyperspectral image that includes each optimal spectral band. Likewise, the processing unit 114 can be configured to use the hyperspectral image data corresponding to the optimal spectral band for each pest 6 to determine the pest parameters, which can include filtering the hyperspectral image data such that only the image data corresponding to the optimal spectral band for the pest 6 or each optimal spectral band for the pest 6 is used to determine the one or more pest parameters.

[0068] In versions that use a multispectral camera, the multispectral camera can be configured in a manner similar to a hyperspectral camera, and the processing unit 114 can be configured to process multispectral image data in a manner similar to processing hyperspectral image data (e.g., using multispectral image data corresponding to a spectral band of 920 nm to 950 nm to determine one or more pest parameters).

[0069] The imaging subsystem 11 can include a camera module mount 11m. In versions having multiple camera modules 111, the imaging subsystem 11 can include a corresponding plurality of camera module mounts 11m.

[0070] The camera module mount 11m can be configured to secure the camera module 111 to the vehicle 100, which can include securing the camera module 111 to the tiller 50. In versions where the vehicle 100 is a harvester, the camera module mount 11m can be configured to secure the camera module 111 relative to a portion of the harvester through which (or otherwise relative to which) one or more harvested items 21 travel. In some embodiments, the camera module mount 11m can be configured to secure the camera module 111 relative to the conveyor 108, the pick table 109, or one or more additional conveyors 110.

[0071] The camera module mount 11m can be configured to secure the camera module 111 such that a portion of the crop 2, a portion of the ground medium, a portion of the tilled ground medium, and / or a harvested item 21 passes through the field of view of the camera module 111.

[0072] The camera module mount 11m can be configured to secure the camera module 111 such that the ground medium, the tilled ground medium, and / or one or more harvested items 21 pass beneath the camera module 111, and the camera module mount 11m can therefore secure the camera module 111 above the ground, the conveyor 108, the pick table 109, or one or more additional conveyors 110. Thus, in these embodiments, one or more harvested items 21 transported by the conveyor 108, the pick table 109, or one or more additional conveyors 110 can be resting on the conveyor 108, the pick table 109, or one or more additional conveyors 110 as they are transported.

[0073] The camera module mount 11m can include a shroud 112 (see Figure 6The shroud 112 is configured to reduce the amount of ambient light in which operational parts of the camera module 111 (e.g. operational components of the camera module 111 including the lens) are exposed. Thus, the shroud 112 can reduce glare (and / or infrared interference, see below) on the operational components of the camera module 111.

[0074] The camera module mount 11m can comprise an upright portion 11u. The upright portion 11u can be configured to be fixed to a part of the vehicle 100 and / or a part of the cultivator 50. The upright portion 11u can comprise a mounting bracket for mounting to the vehicle 100 and / or the cultivator 50.

[0075] The camera module mount 11m can comprise a crossbeam portion 11b. The crossbeam portion 11b can be mounted to the upright portion 11u. The crossbeam portion 11b can be detachable from the upright portion 11u. The crossbeam portion 11b can extend generally horizontally in use. The position of the crossbeam portion 11b relative to the upright portion 11u can be adjustable, such that the height of the camera module 111 is adjustable. For example, the crossbeam portion 11b can slide relative to the upright portion 11u. Thus, the distance between the camera module 111 and the surface below the camera module 111 can be adjustable. The crossbeam portion 11b can be configured to attach to the shroud 112 and / or the camera module 111, and thus can comprise a mounting bracket configured to attach to the shroud 112 and / or the camera module 111.

[0076] The camera module 111 can be mounted about 20-200 cm above the surface to be imaged (e.g. the ground medium or the conveyer carrying the harvested items 21), optionally about 20 to 150 cm above the surface, optionally about 30 to 120 cm above the surface, optionally about 40 to 110 cm above the surface, optionally about 50 to 100 cm above the surface, optionally about 50 to 90 cm above the surface, optionally about 60 to 80 cm above the surface. Such ranges can strike a good balance between maximizing the field of view and ensuring that the harmful organisms 6 are still detectable in the captured images.

[0077] Other mounting systems are possible, and can be customized to match the configuration of the vehicle 100 and / or the cultivator 50.

[0078] The camera module 111 can comprise a stereo camera 111a (see e.g. Figure 8 ), configured to capture images of its field of view. The stereo camera 111a can be configured to capture pairs of images substantially simultaneously, to provide parallax, to provide depth information.

[0079] In some embodiments (see, for example Figure 9 ), the camera module 111 can comprise a camera 111c configured to capture visible light images, and can comprise an infrared emitter 111d and an infrared receiver 111e. The infrared emitter 111d can comprise an infrared laser that emits a light output that is directed through one or more optical elements (e.g. a diffraction grating) to spread the emitted light over a relatively wide area (e.g. the field of view of the camera module 111) - e.g. in the form of a speckle pattern. The infrared receiver 111e can be configured to capture images of the field of view of the camera module 111 in the infrared spectrum. By comparing the emitted infrared light pattern with the received infrared light pattern captured by the infrared receiver 111e, depth information can be determined by analysing the distortion of the infrared light (i.e. the distorted pattern) - e.g. by generating a parallax map. For example, the Kinect （RTM） camera by Microsoft Corporation employs this technique.

[0080] The imaging subsystem 11 (e.g. the camera module 111) can comprise an illumination device 111b configured to illuminate the field of view (and can illuminate the interior of the shroud 112). The illumination device 111b can be configured to provide visible light illumination. The illumination device 111b can additionally or alternatively be configured to provide infrared illumination, which can comprise illumination in the optimal spectral band for the pest 6. Where the system 1 is configured to detect a plurality of different pests 6, the illumination device 111b can provide illumination in the optimal spectral band for each pest 6. Thus, the illumination device 111b can provide illumination across a wide spectrum, which can comprise visible illumination and / or infrared illumination. A plurality of illumination devices 111b can provide illumination of different wavelengths (e.g. one can provide visible light illumination, and one can provide infrared illumination).

[0081] The camera module 111 and / or the illumination device 111b (if provided separately) can be provided with a protective casing configured to shield the camera module 111 from one or more of a shielding fluid (e.g. water), dust, and vibration. The protective casing can have an internal lining to support the camera module 111 and / or the illumination device 111b. The protective casing can have at least a portion that is transparent so as to allow the camera module 111 to capture images therethrough.

[0082] The illumination device 111b can enable the pest detection system 1 to be used in low ambient light conditions, and can provide a consistent illumination level for the camera module 111. Thus, this can facilitate more consistent images from the camera module 111 (e.g. in terms of the exposure of the images).

[0083] The imaging subsystem 11 can comprise a processing unit 114 (see, for exampleFigure 10 The processing unit 114 can be configured to receive signals representing one or more images captured by the camera module 111 (the camera module 111 is configured to output such signals to the processing unit 114). Therefore, the processing unit 114 and the camera module 111 are communicatively connected. This communication connection can be via a wireless or wired communication system.

[0084] Processing unit 114 includes processor 114a, which is configured to execute one or more instructions that may be stored as a computer program on the memory of processing unit 114b. The memory 114b may include volatile memory and / or non-volatile memory. Processing unit 114 may be, for example, a computer system, and may be a laptop computer, desktop computer, tablet computer, mobile (cellular) phone, minicomputer, or the like.

[0085] In some embodiments, one or more instructions are stored on a non-transitory computer-readable medium.

[0086] The processing unit 114 can be configured to store images or portions of images captured by the camera module 111 and sent to the processing unit 114 as image data in the memory 114b. When executed, the one or more instructions can perform one or more methods on the image as described herein.

[0087] In some embodiments, the pest detection system 1 may include an output unit 115 (see, for example...) Figure 11 Output unit 115 can be configured to provide output to a user or operator indicating aspects of the operation of pest detection system 1. Thus, output unit 115 can be communicatively coupled to processing unit 114 and / or camera module 111—each of which can be configured to control output unit 115 or a portion thereof to transport information about aspects of its operation. Output unit 115 may include a visual output element 115a, such as light (which may be a light-emitting diode) or a display screen. Output unit 115 may include an audio output element 115b, which may include a sound generator (e.g., a buzzer, a bell, or a speaker). Output unit 115 can provide one or more of the following indications: pest detection system 1 is powered; pest detection system 1 is operating; camera module 111 is capturing images; there is an error in the operation of pest detection system 1; data is being transmitted from communication subsystem 13 or storage subsystem 12, or the like.

[0088] As described herein, in some embodiments, the imaging subsystem 11 may be communicatively coupled to the storage subsystem 12 (see, for example...). Figure 12). The processing unit 114 can be communicatively coupled to the storage subsystem 12 (e.g., the coupling can be via a wired data bus).

[0089] The storage subsystem 12 can be configured to receive data from the imaging subsystem 11 (e.g., from the processing unit 114) and store the data onto a storage device 121 of the storage subsystem 12. The storage device 121 can be a computer-readable medium. In some embodiments, the storage device 121 can be removable from one or more other parts of the pest detection system 1. Thus, for example, the storage device 121 can comprise a flash drive (also known as a pen drive or memory stick). The flash drive can comprise non-volatile memory for storing data and a communication element to enable the flash drive to communicate with a computing device (e.g., the processing unit 114). The storage subsystem 12 can comprise a communication port 122 (e.g., a universal serial port socket / plug) configured to selectively mate with a corresponding communication power source (e.g., plug / socket) for the storage device 121 to enable transfer of data from the processing unit 114 (or other parts of the imaging subsystem 11) to the storage device 121.

[0090] The storage subsystem 12 can be configured to receive data from the storage device 121 and communicate the data to the imaging subsystem 11 (e.g., to the processing unit 114 and / or the camera module 111). The data can comprise, for example, software or firmware updates, license keys, and the like.

[0091] The communication subsystem 13 (see, e.g., Figure 13 ) can be communicatively coupled to the imaging subsystem 11. The processing unit 114 can be communicatively coupled to the communication subsystem 13 (e.g., via a wired data bus).

[0092] Thus, the communication subsystem 13 can be configured to receive data from the imaging subsystem 11 (e.g., from the processing unit 114) and communicate (i.e., transfer) the data (or a portion thereof) to the remote management system 3. The communicative coupling between the communication subsystem 13 and the remote management system 3 can be over a wired and / or wireless communication network. The network can comprise portions that use a number of different protocols and communication mechanisms. For example, the communication subsystem 13 can be configured to communicate with the remote management system 3 using one or more of: a cellular network, a wide area network (e.g., the Internet), and a local area network (e.g., using an Ethernet （RTM） or WiFi （RTM） ) network. Thus, in some embodiments, the communication subsystem 13 comprises a network I / O module 131, which can be in the form of a circuit configured to manage and enable the communicative coupling with the remote management system 3.

[0093] The communication subsystem 13 can include a cache memory 132 configured to provide temporary local storage for data prior to transmission to the remote management system 3.

[0094] Accordingly, the communication subsystem 13 can be configured to store data in the cache memory 132 when a communication link with the remote management system 3 is unavailable. For example, if a wireless link is present in a communication network between the communication subsystem 13 and the remote management system 3, the wireless link can not always be available - for example, the vehicle 100 can move in and out of range of the wireless link. Accordingly, the cache memory 132 can provide temporary storage for data, which is then transmitted to the remote management system 3 when the link is available.

[0095] The communication subsystem 13 can be configured to receive data from the remote management system 3 and communicate the data to the imaging subsystem 11 (e.g., to the processing unit 114 and / or the camera module 111). The data can include, for example, software or firmware updates, license keys, and the like.

[0096] The location subsystem 14 (see, e.g., Figure 14 ) can be configured to determine its geographic location, and thus the geographic location of at least a portion of the pest detection system 1 (e.g., the camera module 111). The location subsystem 14 can be configured to determine the location of the vehicle 100. The location subsystem 14 can include, for example, a satellite-based location system module 141 (e.g., a global positioning system receiver, or a receiver for one or more of GLONASS, Galileo, Beidou, IRNSS (NAVIC), or QZSS). In some embodiments, the location subsystem 14 can be configured to receive signals transmitted from one or more beacons, and can use the received signals (e.g., through triangulation) to determine its location relative to the beacons. Accordingly, the location subsystem 14 can be configured to determine and output location data representative of the geographic location of the location subsystem 14 (and thus of the pest detection system 1, or at least its camera module 111).

[0097] The location subsystem 14 can be communicatively coupled to the imaging subsystem 11 (e.g., to the processing unit 114), and can be configured to send data including the geographic location determined by the location subsystem 14 to the imaging subsystem 11 (e.g., to the processing unit 114). In some embodiments, the location subsystem 14 is also configured to communicate current time information to the imaging subsystem 11 (e.g., to the processing unit 114).

[0098] The power subsystem 15 (see, e.g., Figure 15The power subsystem 15 can be configured to provide power to one or more other components of the pest detection system 1. For example, the power subsystem 15 can be configured to provide power to: the imaging subsystem 11 (e.g., to one or both of the camera module 111 and the processing unit 114), and / or the storage subsystem 12, and / or the communication subsystem 13, and / or the positioning subsystem 14.

[0099] The power subsystem 15 can include a connector 151 configured to be coupled to the electrical system 105 of the vehicle 100. Thus, the power subsystem 15 can provide power to the pest detection system 1 from the vehicle 100.

[0100] The power subsystem 15 can include one or more batteries 152 configured to provide power to the pest detection system 1.

[0101] In some embodiments, the one or more batteries 152 are used to provide an uninterrupted power supply, such that variations in power from the electrical system 105 of the vehicle 100 can be compensated for by using power from the one or more batteries 152. In some embodiments, the one or more batteries 152 are used in combination with the electrical system 105 of the vehicle 152 to provide power to the pest detection system 1, the pest detection system 1 having a higher voltage than the voltage provided by the electrical system 105.

[0102] In some embodiments, the pest detection system 1 is electrically isolated from the electrical system 105 of the vehicle 100, and thus the power subsystem 15 provides all of the power required to operate the pest detection system 1 without using power from the electrical system 105 of the vehicle 100 (e.g., this can be achieved using one or more batteries 152).

[0103] In some embodiments, the power subsystem 15 is configured to determine when the vehicle 100 is operating - e.g., when the engine 103 is running - and can initiate the pest detection system 1 based on the determined operation of the vehicle 100. When the vehicle 100 stops operating, the power subsystem 15 can then be further configured to detect this change and trigger a shutdown (i.e., a power down) of the pest detection system 1. This determination can be based on the connection of the power subsystem 15 to the electrical system 105 and / or can be based on the connection of the power subsystem 15 to a control bus of the vehicle 100.

[0104] In some embodiments, the pest detection system 1 or portions thereof are disposed within a housing 16 (see, e.g., Fig. 1). Figure 16 and Figure 17). The housing 16 can be configured to inhibit or substantially prevent ingress of fluids (e.g., water) and / or dust, and / or can protect against vibration. In some embodiments, the housing 16 can house the processing unit 114. In some embodiments, the housing 16 can house one or more of at least a portion of the storage subsystem 12, at least a portion of the communication subsystem 13, at least a portion of the location subsystem 14, and at least a portion of the power subsystem 15.

[0105] The housing 16 can carry one or both of the communication port 12 and the connector 151 in an outer wall thereof. In some embodiments, the housing 16 defines at least one port through which a cable of the camera module 111 and / or the location subsystem 14 can pass - in such embodiments, these parts or portions thereof of the pest detection system 1 are located outside of the housing 16. As will be appreciated, the satellite-based location system module 141 can need to be located outside of the housing 16 in order to be able to receive signals for its operation. In some embodiments, the satellite-based location system module 141 or an antenna thereof can be located, for example, in or adjacent to the camera module 111. In other embodiments, the satellite-based location system module 141 can be located elsewhere on the vehicle 100 and / or the implement 50.

[0106] In some embodiments, the housing 16 can carry one or more elements of the output unit 115 - e.g., a visual output element 115a and / or an audio output element 115b - in an outer wall thereof.

[0107] The housing 16 can include a housing mounting bracket 161 for securing the housing 16 to a portion of the vehicle 100 and / or the implement 50. Thus, in some embodiments, the housing mounting bracket 161 can include a lip 161a (e.g., in the form of a right-angled cross-beam) configured to engage a generally horizontal cross-beam or other member of the vehicle 100 and / or the implement 50. The lip 161a can be positioned toward an upper portion of the housing 16. The housing mounting bracket 161 can include a hook-like member 161b configured to engage a generally vertical cross-beam or other member of the vehicle 100 and / or the implement 50. The hook-like member 161b can be movable between an extended position in which the generally vertical cross-beam can be moved into a channel defined by the hook-like member 161b and a retracted position in which the generally vertical cross-beam is captured within the channel. Movement of the hook-like member 161b can be achieved through the use of a threaded member 161c that engages a threaded portion of the hook-like member 161b (see, e.g., FIG. 1 1). Figure 17 .

[0108] The remote management system 3 (see, e.g., FIG. 1) can be configured to receive data from the pest detection system 1 and / or the vehicle 100 and / or the implement 50. The remote management system 3 can be configured to store the received data and / or to transmit the received data to a remote location (e.g., a remote server or other computing device). Figure 18The remote management system 3 can be communicatively coupled to the pest detection system 1 via a communication subsystem 13, and the coupling can be wired or wireless. In some embodiments, the remote management system 3 is located remotely from the pest detection system 1 and the vehicle 100. In some embodiments, the remote management system 3 is located remotely from the pest detection system 1 but on or within the vehicle 100 (or tractor 300).

[0109] The remote management system 3 can be communicatively coupled to the pest detection system 1 via a communication subsystem 13, and the coupling can be wired or wireless. In some embodiments, the remote management system 3 is located remotely from the pest detection system 1 and the vehicle 100. In some embodiments, the remote management system 3 is located remotely from the pest detection system 1 but on or within the vehicle 100 (or tractor 300).

[0110] The remote management system 3 can be configured to send software, software updates, firmware, firmware updates, license keys, licensing information, and the like to the pest detection system 1. For example, the pest detection system 1 can be configured to install or update software or firmware based on the received data.

[0111] The remote management system 3 can be a laptop, desktop computer, tablet, mobile (cellular) phone, or the like. In some embodiments, the remote management system 3 comprises a server, and can comprise multiple servers and other computing devices.

[0112] As mentioned above, in versions where the tiller 50 is part of a harvester, the pest detection system 1 can be fixed to the harvester at one location such that the one or more harvested items 21 pass through the field of view of the imaging subsystem 11 (e.g., of the camera module 111). Such fixation can be achieved through the use of a mounting bracket 113.

[0113] In some embodiments, the pest detection system 1 is positioned such that the distance from the lowest portion of the shroud 112 to the surface over which the one or more harvested items 21 are or will be passing is between 10 cm and 30 cm.

[0114] During operation of the harvester, the harvester will typically move within the field 200 along a row of harvestable items 22. In the case where the harvestable items 22 are root vegetables, the harvestable items 22 are typically buried. The share 106 (i.e., the tiller 50) of the harvester 100 lifts the harvestable items 22 (which result in the harvested items 21) from the ground into the harvester 100. In doing so, the share 106 tills or agitates the soil.

[0115] Thus, one or more harvested items 22 will typically pass through the harvester and through the field of view of the imaging subsystem 11 on the conveyor, where they can be carrying one or more pests 6, or in the soil being transported with the harvested items 22. Such pests 6 can thus be detected by the pest detection system 1 during harvesting.

[0116] Similarly, in versions where the cultivator 50 is not part of a harvester, the cultivator 50 will typically be moved within the field 200 (e.g. when it is being towed by the vehicle 100) to cultivate the soil of the field 200, and can move along rows of crops (e.g. in the case of an inter-row cultivator).

[0117] The method of operation of the pest detection system 1 is controlled by instructions executed by the processing unit 114 - also as described herein - and in this regard any reference to method steps should be interpreted as encompassing instructions which when executed cause those method steps to occur.

[0118] The pest detection method implemented by the pest detection system 1 can comprise at least a training phase and an operational phase.

[0119] The training phase can comprise training a machine learning model to detect one or more pests 6. The machine learning model can be a neural network, optionally a convolutional neural network (e.g. VGG16). The machine learning model can run on the processing unit 114. Thus, the machine learning model can be stored in the memory 114b and can run on the processor 114a. Other examples of suitable machine learning models include AlexNet, DenseNet and ResNet models.

[0120] The machine learning model can be trained using a training dataset, which can be stored in the storage subsystem 12 and / or the memory 114b. The training dataset can comprise images of one or more pests 6 to be detected by the pest detection system 1. The training dataset can comprise a mix of images where no pests 6 are present, images where a single pest 6 is present and / or images where multiple pests 6 (which can be of the same type or different types) are present.

[0121] The training images can simulate environmental conditions expected in use (e.g. including soil and / or plants).

[0122] The training dataset may include images of substitutes for pest 6 that will be detected by pest detection system 1. Images of the substitutes can be used to train a machine learning model to detect pest 6. Therefore, the substitutes may be visually similar to or indistinguishable from pest 6. For example, the substitute may be an artificial replica of pest 6 (e.g., formed of plastic material), or it may be a visually similar or indistinguishable species. This may be advantageous when pest 6 is not readily available. For example, in the case of pest 6 being a wireworm, the substitute may be a mealworm. More specifically, the wireworm pest 6 species to be detected may be striped click beetle, dark click beetle, and / or brown click beetle, and the substitute mealworm species may be mealworm.

[0123] The training dataset may include images acquired by camera module 111. Therefore, the training phase may include acquiring images using camera module 111. Camera module 111 may be configured to acquire images of the ground medium for training a machine learning model. The training phase may include seeding pest 6 in the training environment and acquiring images of pest 6 using camera module 111. The training phase may also include seeding a substitute for pest 6 in the training environment and acquiring images of the substitute using camera module 111. Therefore, during the training phase, imaging subsystem 11 may be mounted on vehicle 100 and / or tiller 50 as described herein.

[0124] The training images can be modified before being used to train the machine learning model. Therefore, the processing unit 114 can be configured to modify or transform the training images before they are used to train the machine learning model. The modified images can be stored in the training dataset.

[0125] Training images can be modified by splitting each image into multiple subframes (also called tiles) (see example). Figure 19 and Figure 20 An unmodified image frame consists of an array of pixels with a width and a height. An image can be modified by splitting the frame into a set of subframes or tiles. Each subframe or tile can include a subset of the pixels that make up the unmodified image frame. For example, in... Figure 19 and Figure 20 In the example shown, the image frame has been split into four equal subframes or tiles, one of which is in Figure 20 As shown in the figure, and its corresponding to Figure 19 The area marked "c" in the middle. Overlaid on... Figure 19The grid on the image of Figure 1 indicates how the image can be split into four sub-frames or tiles by halving the width and height of the frame. The image can be split into sub-frames or tiles that have equal pixel area, but in some versions can also be split into unequal tiles.

[0126] The training image can be split into at least 2 tiles, at least 4 tiles, at least 10 tiles, at least 16 tiles, at least 20 tiles, at least 50 tiles, at least 64 tiles, at least 100 tiles, at least 200 tiles, at least 256 tiles, at least 400 tiles, at least 512 tiles, at least 600 tiles, at least 700 tiles, at least 800 tiles, at least 896 tiles, at least 900 tiles, or at least 1,000 tiles. The training image can be split into tiles by dividing the width of the image by a first factor and dividing the height of the frame by a second factor. The first factor and the second factor can be equal, for example, the first factor and the second factor can be 2, 4, 8, 16, or 32. The first factor and the second factor can be unequal, for example, the first factor can be 32 and the second factor can be 28. If the number of tiles selected, or the width and height of the array, and the factors by which they are to be divided do not allow each tile to have an equal number of pixels, then some of the tiles can include a different number of pixels.

[0127] It has been found that generating tiles from unmodified image frames is particularly effective for training a machine learning model, in which the width of the frame is divided by 32 and the height of the frame is divided by 28 to generate 896 tiles. Machine learning models that are able to reliably detect the harmful organism 6 can also be implemented using fewer tiles, for example by dividing the width and height by 16 to generate 256 tiles.

[0128] The precision and recall rates for the selected neural network and tiling method are shown in Table 1 below, in which the system 1 is trained to detect the harmful organism 6 is the golden needle.

[0129]

[0130] Splitting the training images into sub-frames or tiles increases the amount of space occupied by the pest 6 in the tile. The space occupied by the pest 6 can be measured, for example, by comparing the number of pest pixels in a frame or sub-frame to the total number of pixels in the frame or to the number of non-pest pixels in the frame. For a complete, unmodified image frame containing one instance of the pest 6, the pixels forming the image can be dominated by non-pest pixels. For a sub-frame or tile, the pest 6 pixels will occupy a larger portion of the total pixels present (for a sub-frame containing the pest 6). In this example, the remaining sub-frames or tiles generated from the original image will not contain any pest 6. It has been found that splitting the training images into tiles in this way can improve precision and recall compared to a training dataset using unmodified training images.

[0131] The training images included in the training dataset (whether unmodified or split into tiles) can include ground truth data. Thus, the machine learning model can learn to detect the pest 6 based on the ground truth data provided in the training dataset. The training dataset can be split into classes based on the ground truth data associated with the images included in the training dataset. Each of the training images can be associated with a ground truth number of pest 6, and the training images can be classified based on the number of pest 6 visible in each image (which can be a tile as described). The number of pest 6 in a given image can be determined as the number of center points of ground truth bounding box labels within that image.

[0132] In some versions, the training images can be manually annotated by labeling the pest 6 visible in the images with bounding boxes. The use of bounding boxes enables the object detection model to be trained using the training dataset, and enables the pest 6 to be located within a frame (e.g., when the frame is blocked or a sliding window analysis is performed), as described in more detail below.

[0133] Thus, a first class of training images can be control images in which no pest 6 is present. A second class of training images can contain one and only one pest 6 in each image. The classes can continue with increasing numbers of pests shown in each image (e.g., 2, 3, 4, etc.). The training dataset can contain about an equal number of control images and pest-containing images (e.g., the number of control images can be within about 10% of the number of pest-containing images). Thus, this can provide a balanced training dataset that is not dominated by any particular class of images.

[0134] The training dataset can include one or more synthetic images. A synthetic image is an image that has been modified to depict an entity that is not present in the original image. A synthetic image can be created by inserting a depiction of a pest 6 into an image. For example, a depiction of a pest 6 can be inserted into an image that does not contain any pests 6 to create a synthetic image that includes at least one pest. Multiple pests 6 can also be inserted in such a synthetic image. In this way, training data can be generated using training images that include any number of pests 6 as needed to train the machine learning model. Thus, the use of synthetic images can alleviate the need to obtain a large number of real images of pests 6 for training the machine learning model. However, in some versions, the training dataset does not include any synthetic images (but can still include subframes or tiles).

[0135] The training images can use the RGB color space. Additionally or alternatively, the training images can use the BGR, Haematoxylin-Eosin-DAB (HED), HSV, and / or CIElab color spaces. Thus, the training dataset can include a set of training images that can include one or more subsets of RGB images, BGR images, HED images, HSV images, or CIElab images.

[0136] The training images can include HED images in which the intensity of the H channel is in the range of 70-90%, optionally 75-85%, optionally 78-82%, and optionally about 80%. The intensity of the D channel can be greater than 75%, greater than 85%, greater than 90%, greater than 95%, about 95 to 99%, or about 99%. In particular, the intensity of the H channel can be about 80% and the intensity of the D channel can be about 99%.

[0137] Figure 21a An RGB image of a pest 6 that is a wireworm is shown against a soil background. Figure 21b The same image is shown with HED conversion, where the H channel intensity is 80% and the D channel intensity is 99%. Under these conditions, the wireworm 6 forms a strong contrast with the soil background (in Figure 22 In the middle, the soil is typically yellow and the wireworm 6 is typically blue). Similarly, Figure 22 a shows an RGB image of a wireworm pest 6, and Figure 22 b shows the corresponding image with HED staining.

[0138] The training images can include one or more channels of a color space, and can not use all channels associated with the color space. For example, in the case of the HSV color space, the images can use the full HSV color space (see Figure 22c) H channel only (see FIG. 6A) Figure 22 d) S channel only (see FIG. 6B) Figure 22 e) V channel only (see FIG. 6C) Figure 22 f) H and S channels only (see FIG. 6D) Figure 22 g) H and V channels only (see FIG. 6E) Figure 22 h) or S and V channels only (see FIG. 6F) Figure 22 i). Similarly, for the CIElab color space, the image can use the full CIElab color space (see FIG. 7A) Figure 22 j) I channel only (see FIG. 7B) Figure 22 k) a channel only (see FIG. 7C) Figure 22 l) b channel only (see FIG. 7D) Figure 22 m) I and a channels only (see FIG. 7E) Figure 22 n) I and b channels only (see FIG. 7F) Figure 22 o) or a and b channels only (see FIG. 7G) Figure 6 p).

[0139] It has been found that the RGB, CIElab-l, CIElab-lb, HED, and HSV-V color spaces produce particularly successful results in terms of precision and recall of the respective trained machine learning models.

[0140] The machine learning model can be trained and evaluated using k-fold cross-validation (e.g., 5-fold cross-validation). The k-fold cross-validation can be used to validate the machine learning model.

[0141] In an example, the machine learning model is trained using a VGG16 architecture for binary classification and trained for 15 epochs with a batch size of 25 using a BCEWithLogits loss function and an SGD (stochastic gradient descent) optimizer (learning rate of 0.001 and momentum of 0.9).

[0142] The training phase can include tilling the ground medium and capturing images of the tilled ground medium using the camera module 111. The tiller 50 can be used to till the ground medium. The pests 6 and / or surrogates for the pests 6 can be seeded in the ground medium such that tilling the ground medium causes the pests 6 and / or surrogates for the pests 6 to be exposed to the camera module 111.

[0143] Figure 23An example of a tillage machine 50, in this case a cultivator with tillage members 51, is shown, being towed by a vehicle 100, i.e. a tractor. The imaging subsystem 11 is mounted to the tillage machine 50 via a camera module mount 11m. The camera module 111 is configured to acquire images of the ground medium as the tillage machine 50 is tilled. Thus, as the vehicle 100 moves around the training environment, in this case a field 200, a training dataset in view can be acquired by capturing images with the camera module 111, the training dataset comprising images of the ground with and without the presence of the pests 6 or a proxy for the pests 6. The pests 6 or proxy for the pests 6 can be distributed in the path of the vehicle 100 (and thus in the path of the camera module 111) to ensure that the training dataset includes images of the pests 6 or proxy for the pests 6.

[0144] The machine learning model can be trained to determine one or more pest parameters and to classify the image based on the determined pest parameters. The one or more pest parameters can be determined by the machine learning model in the operating phase of the pest detection method. The one or more pest parameters can be used to classify the image.

[0145] The pest parameter can be the presence or absence of the pests 6. Thus, the machine learning model can be trained to determine whether any pests 6 are present in a given image and, optionally, to classify the image accordingly. This can be a binary classification; for example, if no pests 6 are detected, the image can be assigned to a first class and if at least one pest 6 is detected, the image can be assigned to a second class. In such a version, the machine learning model can not necessarily attempt to count the number of pests 6 detected. Thus, in such a version, a value of 0 can indicate that no pests 6 are detected and a value of 1 can indicate that at least one pest 6 is detected.

[0146] The pest parameter can be the number of pests 6 detected. Thus, the machine learning model can be trained to count the number of pests 6 present in a given image. The image can then be classified into different classes depending on the number of pests 6 detected, for example “none” where no pests 6 are detected; “low” where up to ten pests 6 are detected, and “high” where more than ten pests 6 are detected.

[0147] The pest parameter can be a type of pest 6 detected. This is particularly relevant where the pest detection system 1 is configured to detect multiple pest types. Thus, the machine learning model can be trained to determine the pest type of each pest detected. The image can then be classified into different categories according to the type of pest 6 detected, for example “none”, “pest A”, “pest B”, and “both”, where the system 1 is configured to detect two pests (pest A and pest B) (e.g. an example of a golden needle and a fungus gnat larva).

[0148] It has been found that simply binary classifying the image into two classes (presence of a pest and absence of a pest) can give particularly accurate results.

[0149] As described below, a combination of binary image classification and image patching can be used to estimate the number of pests 6 present.

[0150] In an operational phase, the camera module 111 can generate image data (corresponding to an image captured by the camera module 111) and transmit the image data to the processing unit 114. The processing unit 114 can use the machine learning model to determine one or more pest parameters associated with the image data, and can optionally classify the image data based on the determined pest parameters.

[0151] The camera module 111 can be configured to capture images periodically (i.e. at predetermined time intervals). The camera module 111 can be configured to capture an image approximately every 0.05-1 second (optionally 0.1-0.8 seconds, optionally 0.1-0.5 seconds, optionally 0.2-0.4 seconds, optionally 0.3 seconds).

[0152] The image data generated by the camera module 111 can be stored in association with a corresponding location determined by the location subsystem 14. Thus, each image captured by the camera module 111 can be stored in association with a corresponding location. For example, the image and location data can be stored in the storage subsystem 12 or the memory 114b. For example, each image can be associated with GPS coordinates.

[0153] The system 1 (optionally the imaging subsystem 11) can comprise a light meter which can be configured to determine a light level associated with an image captured by the camera module 111. For example, the light meter can output a light level in lux. For example, the light meter can output the light level to the processing unit 114. Thus, each image captured by the camera module 111 can be stored in association with a corresponding light level, and optionally can additionally be stored with a corresponding location.

[0154] In versions that include a harvester, the camera module 111 can be instructed to capture one or more images of one or more harvested items 21 as they pass through the field of view of the imaging subsystem 11. In this regard, the camera module 111 can be instructed by the processing unit 114.

[0155] The tiller 50 can be configured to till as described herein, and the camera module 111 can be instructed to capture one or more images of the tilled ground medium. In this regard, the camera module 111 can be instructed by the processing unit 114. In versions that do not include a tiller 50, the camera module 111 can be instructed to capture one or more images of: the ground medium, or the crop, or other locations where pests 6 can be detected. The imaging subsystem 11 can be moved by the vehicle 100 across a target area (e.g., a field) to acquire images of the target area such that pests 6 present in the target area can be detected by the pest detection system 1.

[0156] The one or more images captured by the camera module 111 can be communicated (i.e., transmitted) to the processing unit 114, which is configured to process the one or more images represented by the image data through execution of respective instructions. This processing can include determining one or more pest parameters using a machine learning model.

[0157] In some versions, the pest detection method can use multiple machine learning models to determine one or more pest parameters. For example, a first machine learning model can use a first model architecture (e.g., VGG16), and a second model can use a second model architecture (e.g., ResNet101). Alternatively, both models can have the same architecture (e.g., VGG16), but they can be trained using different training data sets. For example, the first model can be trained using RGB training images, and the second model can be trained using CIElab-l training images. In such versions, the models can each use a respective image format to determine one or more pest parameters in an operational phase (e.g., the RGB model will use RGB images, and the CIElab-l model will use CIElab-l images).

[0158] The machine learning model or each machine learning model can output a confidence value associated with the pest parameter or each pest parameter. For example, where the machine learning model is configured to determine the presence or absence of the pest 6, the machine learning model can output a floating point number between 0 and 1. A number closer to 1 can indicate a higher confidence that the associated image shows the pest 6 (and vice versa). For binary image classification, the image can be classified based on the confidence value determined by the machine learning model. The image can be classified based on a threshold, where images associated with a confidence value above the threshold are placed into a first category (e.g., containing the pest) and images associated with a confidence value below the threshold are placed into a second category (e.g., not containing the pest). The threshold can be predetermined or can be dynamically changed. For example, the threshold can be 0.5. The image and the confidence value associated therewith can be stored together so that the image can be classified later.

[0159] Similarly, where the machine learning model is configured to determine the number of pests 6 present in a given image, the machine learning model can output a confidence value associated with the number of pests. Where the machine learning model is configured to determine the type of pest 6 in the image, the confidence value can be associated with the type of pest predicted by the machine learning model.

[0160] The machine learning model can use a sliding window approach to analyze the image data (e.g., to determine one or more pest parameters). In this approach, the image frame to be analyzed can be divided into tiles or sub-frames. The size of each tile can be the same as the size of the tiles provided in the training data set. The tiles can overlap, such that a given pixel can belong to more than one tile. The degree of overlap between tiles can be about 70-90%, about 75-95%, or about 80%. Additionally, zero padding borders can be applied around the image frame to be analyzed, which can be proportional to the degree of overlap between tiles.

[0161] The machine learning model can analyze each tile of the image and can assign a confidence value to each pixel in the tile being analyzed. For a binary classification (e.g., into the classes of “pest present” or “pest not present”), the confidence value output can be the same for each pixel in a given tile. For example, a value of 0.7 can be assigned to each pixel in a tile. Using overlapping tiles, each pixel will be assigned multiple confidence values as the sliding window is passed over the image. For example, approximately 25 confidence values can be assigned to each pixel in an image frame. The confidence values for a given pixel can be averaged (e.g., mean averaged), providing a single confidence value for each pixel in the image frame. In versions where multiple machine learning models are used to analyze the image, the outputs from the models can also be averaged, providing a single confidence value for a given pixel.

[0162] Accordingly, the machine learning model can output a confidence value for each pixel in a given image frame. A threshold can be applied to the confidence values, and all values within the threshold can be flattened (e.g., all values below the threshold can be flattened to 0, or equivalently, all values above the threshold can be flattened to 1; in one example, the threshold can be 0.5). The coordinates of the local peaks in the array of pixels can then be determined, for example, using the Scikit-Image library. Accordingly, the local peaks can represent a prediction of the presence of a pest 6 at that location within the frame. By counting the number of local peaks present in a given frame, an estimate of the number of pests present in the frame can be made.

[0163] As noted above, the same image can be analyzed by multiple machine learning models, and the outputs from these models can be averaged to improve the accuracy of the output.

[0164] Figure 25 One example of the output from the described sliding window analysis is shown in FIG. 13, where the red boxes indicate true bounding box annotations, the white rings indicate the capture area around the bounding box to count a prediction as true during model evaluation, the red dots indicate prediction peaks at 0.5 < confidence < 0.6, the purple dots indicate prediction peaks at 0.6 < confidence < 0.7, the dark blue dots indicate prediction peaks at 0.7 < confidence < 0.8, the cyan dots indicate prediction peaks at 0.8 < confidence < 0.9, and the green dots indicate prediction peaks at 0.9 < confidence < 1.

[0165] One or more pest parameters can be output and stored in the memory 114b, and / or can be output by the processing unit 114 to the output unit 115 (for display to an operator), the storage subsystem 12 (for storage on the storage device 121), and / or the communication subsystem 13 (for transmission to the remote management system 3).

[0166] In some versions, the image data can be sent to the remote management system 3 for processing, and the remote management system 3 can use a machine learning model to determine one or more pest parameters. Thus, the processing unit used to run the machine learning model can be remote from the camera module 111, for example.

[0167] In some embodiments, the processing unit 114 and / or the remote management system 3 can be configured to generate one or more tabular representations, or graphical representations, or cartographic representations of the one or more pest parameters.

[0168] For example, the processing unit 114 and / or the remote management system 3 can be configured to generate a table identifying the number of pests 6 detected in a predetermined geographical area (e.g. a field). The area can be set by a user.

[0169] The processing unit 114 and / or the remote management system 3 can be configured to generate a chart, for example a histogram (see for example Figure 24 ), showing the number of pest detections made at different levels of confidence.

[0170] The processing unit 114 and / or the remote management system 3 can be configured to generate a map based on the locations at which the respective images were taken, showing the number of pests 6 detected (which can include showing the presence or absence of pests 6, i.e. a number of zero or non-zero). The map can be in the form of a “heat map”, in which the number of pests 6 at different geographical locations is represented by different colours on the map. ​ One example of such a map is shown in Figure 6, in which green squares indicate that no pests 6 (in this case, wireworms) are present in that area, yellow squares indicate that one to three pests 6 are present, and red squares indicate that four or more pests 6 are present.

[0171] The date and / or time can be associated with the image data and / or the generated representations. In versions that include a harvester or tiller 50, the date and / or time can indicate the date and / or time at which the soil was harvested or tilled, and can be stored in the same location as the associated data or representations, in general.

[0172] In some embodiments, the conditions at one or more locations can be assessed based on the outputted pest parameters and actions taken accordingly. For example, crops can be planted in areas where no pests 6 are detected. Crops can not be planted in areas where pests 6 are detected. Pesticides (e.g., insecticides, herbicides, or fungicides) can be applied to areas where pests 6 are detected, and the application can be automatic. For example, a sprinkler can be caused to output at a location where pests are detected, the sprinkler can have an associated location subsystem, and the sprinkler can be configured to spray pesticides at the location where pests are detected.

[0173] One or more of these resulting actions can be implemented in an automated manner - e.g., by instructing devices by the remote management system 3 to perform the resulting actions.

[0174] As used in this specification and claims, the terms "comprises," "comprising," "includes," and "including" and variations thereof mean that the specified features, steps or components are included. These terms are not to be interpreted in an exclusionary sense.

[0175] The application can also broadly include parts, elements, steps, examples and / or features individually or collectively referred to as a portion, element, step, example and / or feature. Any of the portions, elements, steps, examples and / or features can be utilized in any other embodiment whether or not such embodiment is specifically described in the specification. In particular, one or more features of any of the embodiments described herein can be combined with one or more features of any of the other embodiments described herein.

[0176] Any feature disclosed in any one or more of the documents cited herein can be sought to be protected.

[0177] Although certain example embodiments of the application have been described herein, the scope of the appended claims is not intended to be limited to the specific embodiments described herein. The claims should be interpreted to cover all equivalents falling within the literal scope of the claims.

Claims

1. A pest detection system comprising: a camera module configured to generate image data; and a processing unit configured to receive the image data, determine one or more pest parameters associated with the image data using a machine learning model, and record the one or more pest parameters on a computer readable medium.

2. The pest detection system of claim 1, further comprising a soil agitator, wherein the camera module is configured to capture images of soil agitated by the soil agitator.

3. The pest detection system of any one of the preceding claims, wherein the one or more pest parameters comprise at least one of a presence or absence of a pest, a quantity of a pest, or a type of a pest.

4. The pest detection system of any one of the preceding claims, wherein the machine learning model is trained using training data comprising images of surrogates for pests.

5. The pest detection system of claim 4, wherein the pests are a species of wireworm and the surrogates are a species of yellow mealworm.

6. The pest detection system of any one of the preceding claims, wherein the machine learning model is trained using training data comprising at least one image formed from an array of pixels, wherein the image is divided into tiles, each tile comprising a subset of the pixels in the image, and the tiles are used to train the machine learning model.

7. The pest detection system according to any one of the preceding claims, wherein the processing unit is configured to determine the pest parameter by dividing an image captured by the camera module into overlapping tiles; a confidence value is assigned to each pixel in each tile, such that each pixel is assigned a plurality of confidence values; and the confidence values for each pixel are averaged.

8. The pest detection system of claim 7, wherein the one or more pest parameters comprise a quantity of pests, and the processing unit is configured to determine the quantity of pests by: flattening the confidence values for each pixel falling within a threshold value, determining coordinates of local confidence value peaks in the array of pixels, and counting a number of local peaks.

9. A pest detection method comprising: training a machine learning model to determine one or more pest parameters; generating image data using a camera module; analyzing the image data using the machine learning model with a processing unit to determine the one or more pest parameters; and storing the one or more pest parameters onto a computer readable medium.

10. The pest detection method according to claim 9, further comprising: agitating soil, and capturing images of the agitated soil with the camera module.

11. The pest detection method of claim 9 or 10, wherein the one or more pest parameters comprise at least one of a presence or absence of a pest, a quantity of a pest, or a type of a pest.

12. The pest detection method of any one of claims 9 to 11, wherein the machine learning model is trained using training data comprising images of surrogates for pests.

13. The pest detection method of claim 12, wherein the pest is a species of mealworm and the surrogate is a species of yellow mealworm.

14. The pest detection method of any one of claims 9 to 13, wherein the machine learning model is trained using training data comprising at least one image formed from an array of pixels, wherein the image is divided into tiles, each tile comprising a subset of the pixels in the image, and the machine learning model is trained using the tiles.

15. The pest detection method of any one of claims 9 to 14, wherein the processing unit is configured to determine the pest parameter by dividing an image captured by the camera module into overlapping tiles, assigning a confidence value to each pixel in each tile such that each pixel is assigned a plurality of confidence values, and averaging the confidence values for each pixel.

16. The pest detection method of claim 15, wherein the one or more pest parameters comprise a number of pests, and the processing unit is configured to determine the number of pests by flattening the confidence values for each pixel falling within a threshold value, determining coordinates of local confidence value peaks in the array of pixels, and counting the number of local peaks.

Images