Calibration methods for agricultural vision systems

Abstract

A method of calibrating a camera in a camera module for use in a vision system of a moveable agricultural equipment is described. The method comprises determining a first or second calibration matrix, determining a next calibration matrix which is another one of the first or second calibration matrices or a third calibration matrix and then using the two determined calibration matrices to calculate the remaining one of the set of the first, second and third calibration matrices. The first calibration matrix describes a transform from a coordinate system of the camera to a coordinate system of the camera module, the second calibration matrix describes a transform from the coordinate system of the camera module to the coordinate system of the moveable agricultural equipment and the third calibration matrix describes a transform from the coordinate system of the camera to the coordinate system of the moveable agricultural equipment.

Description

204007.00019 / ECORO-11CALIBRATION METHODS FOR AGRICULTURAL VISION SYSTEMSTechnical Field

[0001] The present invention relates to calibration of vision systems used in agriculture for the treatment of the ground or objects on the ground such as plants.Background

[0002] Vision systems are widely used as main sensing modality in a broad range of agricultural equipment performing automatized operations in agricultural environments like open fields, greenhouses, vineyards or orchards. Such vision systems are installed on the agricultural machines and allow to capture 2D or 3D images of the scene with its objects to be processed. Depending on the kind of sensor (2D camera with various spectral bands such as RGB, monochrome, IR, multispectral or a mix of them, or 3D cameras using stereovision, time of flight, structured light, laser triangulation or LIDAR), each image may contain information of colour and distance. These images are then processed by an image analysis system to extract the parameters of the objects and their position in space, whether in a 2D or a 3D representation. The parameters (shape, colour, etc.) of the objects may, for example, be used to perform object recognition, and the position of the objects in 2D or 3D representation may be used to orientate a subsequent processing tool in the right position to perform an operation on the object. The kind of operations performed by agricultural machinery on the various objects of the scene can be extremely broad. Examples include: the targeted and selective application of agrochemicals by means of a spot spray system; the application of a radiative beam on a plant or an organism to destroy it (e.g. a laser beam for weeding or killing bugs); a mechanical action to remove part or the totality of the object (e.g. removing a small weed, or cutting a leaf or a branch for pruning); the collection of part or the totality of the objects (e.g. harvesting fruits, or collecting stones on the ground); and the targeted and accurate insertion on the ground of seeds or other agricultural input.

[0003] One common usage of vision systems in agriculture is the application of agrochemicals. Agrochemicals, whether to promote growth (e.g. fertilisers), inhibit growth (e.g. herbicides) or prevent diseases or plagues (fungicides, insecticides, etc.), are typically applied to plants in liquid form using spraying. The agrochemicals are sprayed through nozzles which may be mounted on a spray bar. The spray bar may be mounted on a vehicle (e.g. a tractor or robot) or mounted on a device that is towed by a vehicle. Spot spraying applies droplets of liquids on specific and predetermined locations through the use of valves (e.g. electromechanically controlled valves) which can switch the flow of the agrochemical on and off rapidly. For spot spraying to be effective, the spot sprays must be accurately positioned relative to the plants.2111826-5 1204007.00019 / ECORO-11

[0004] More generally, all previously cited examples of agricultural operations require a vision system able to provide the position of the objects in the 2D or 3D coordinate system of the machine with precision. This precision is needed to operate the above-mentioned processing tools with precision. In order to provide this provision, the vision systems need to be calibrated and may need to be recalibrated at least periodically.

[0005] The embodiments described below are not limited to implementations which solve any or all of the disadvantages of known agricultural equipment that performs automatized operations such as spot spraying systems.Summary

[0006] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0007] A method of calibrating a camera in a camera module for use in a vision system of a moveable agricultural equipment is described. The method comprises determining a first or second calibration matrix, determining a next calibration matrix which is another one of the first or second calibration matrices or a third calibration matrix and then using the two determined calibration matrices to calculate the remaining one of the set of the first, second and third calibration matrices. The first calibration matrix describes a transform from a coordinate system of the camera to a coordinate system of the camera module, the second calibration matrix describes a transform from the coordinate system of the camera module to the coordinate system of the moveable agricultural equipment and the third calibration matrix describes a transform from the coordinate system of the camera to the coordinate system of the moveable agricultural equipment.

[0008] A first aspect provides a method of calibrating a camera fixed in a camera module for use in a vision system of a moveable agricultural equipment for performing actions on objects on a cultivated environment, said agricultural equipment comprising the vision system, a body on which are fixed the camera module and an actuating system, wherein the vision system is arranged to acquire images of the cultivated environment ahead of the actuating system using the camera module, the method comprising: determining a first or second calibration matrix, wherein the first calibration matrix describes a transform from a coordinate system of the camera to a coordinate system of the camera module and the second calibration matrix describes a transform from the coordinate system of the camera module to the coordinate system of the moveable agricultural equipment; determining a next calibration matrix, wherein the next calibration matrix is another one of the first or second calibration matrices or a third 2111826-5 2204007.00019 / ECORO-11calibration matrix, wherein the third calibration matrix describes a transform from the coordinate system of the camera to the coordinate system of the moveable agricultural equipment; using the determined two calibration matrices to calculate a remaining one of a set of the first, second and third calibration matrices; assigning the first calibration matrix to the camera module; and assigning the second calibration matrix to the body of the moveable agricultural equipment.

[0009] A second aspect provides a method comprising: reading a first calibration matrix assigned to a camera module fitted in a movable agricultural equipment, wherein the first calibration matrix describes a transform from a coordinate system of the camera in the camera module to a coordinate system of the camera module and the movable agricultural equipment is configured to perform actions on objects on a cultivated environment; reading a second calibration matrix assigned to a body of the movable agricultural equipment, wherein the second calibration matrix describes a transform from the coordinate system of the camera module to the coordinate system of the movable agricultural equipment; calculating a third calibration matrix by combining the first and second calibration matrices, wherein the third calibration matrix describes a transform from the coordinate system of the camera to the coordinate system of the movable agricultural equipment; applying the third calibration matrix to transform coordinate data for objects on the cultivated environment; and using the transformed coordinate data to perform the actions on the objects on the cultivated environment using the movable agricultural equipment.

[0010] A third aspect provides a computing device comprising: a processor; and a memory arranged to store computer executable instructions that, when executed by the processor, cause the computing device to implement any of the methods described herein.

[0011] The methods described herein may be performed by software in machine readable form which may be stored on a tangible storage medium e.g. in the form of a computer program comprising computer program code means adapted to perform all the steps of any of the methods described herein when the program is run on a computer and where the computer program may be embodied on a computer readable medium. Examples of tangible (or non-transitory) storage media include disks, thumb drives, memory cards etc. and do not include propagated signals. The software can be suitable for execution on a parallel processor or a serial processor such that the method steps may be carried out in any suitable order, or simultaneously.

[0012] This acknowledges that firmware and software can be valuable, separately tradable commodities. It is intended to encompass software, which runs on or controls “dumb” or standard hardware, to carry out the desired functions. It is also intended to encompass2111826-5 3204007.00019 / ECORO-11software which “describes” or defines the configuration of hardware, such as HDL (hardware description language) software, as is used for designing silicon chips, or for configuring universal programmable chips, to carry out desired functions.

[0013] The preferred features may be combined as appropriate, as would be apparent to a skilled person, and may be combined with any of the aspects of the invention.Brief Description of the Drawings

[0014] Embodiments of the invention will be described, byway of example, with reference to the following drawings, in which:

[0015] Figure 1 A is a schematic diagram of an example of a vision system used in an agricultural equipment performing an action in an agricultural context;

[0016] Figure 1B is a schematic diagram of an example of a spot spraying system.

[0017] Figure 2A is a schematic diagram of a first example of a spot spraying system carried behind a tractor;

[0018] Figure 2B is a schematic diagram of a second example of a spot spraying system carried on a vehicle with its own wheels, and with a different view compared to the first example spot spraying system shown in Figure 2A;

[0019] Figure 2C is a schematic diagram showing a part of Figure 2A or Figure 2B in more detail;

[0020] Figure 3 is a graphical representation of the calibration matrices used in the improved calibration methods described herein;

[0021] Figure 4 is a flow diagram of a first example of the improved calibration method as described herein;

[0022] Figure 5 is a flow diagram showing a first example use of the calibration data generated using the method of Figure 4;

[0023] Figure 6 shows a first example arrangement for determining the value of the calibration matrix T1 fora particular camera module;

[0024] Figure 7 shows a first example arrangement for determining the value of the calibration matrix T2 for a particular body of a spot spraying system;2111826-5 4204007.00019 / ECORO-11

[0025] Figures 8 and 9 are schematic diagrams showing the use of a calibration jig on a spraying equipment carried by a tractor to determine the value of calibration matrix T2 orT3; and

[0026] Figure 10 illustrates various components of an example spot spray control system in the form of a computing-based device.

[0027] Common reference numerals are used throughout the figures to indicate similar features.Detailed Description

[0028] Embodiments of the present invention are described below by way of example only. These examples represent the best ways of putting the invention into practice that are currently known to the Applicant although they are not the only ways in which this could be achieved. The description sets forth the functions of the example and the sequence of steps for constructing and operating the example. However, the same or equivalent functions and sequences may be accomplished by different examples.

[0029] As described above, agricultural equipment performing automatized operations on objects in agricultural environments like open fields, greenhouses, vineyards or orchards require a vision system able to provide the positions of the objects in the 2D or 3D coordinate system of the machine with precision. This precision is needed to operate the processing tools with precision and various different examples of processing tools have been described above. The positions of the objects are defined using a coordinate system, generally cartesian (x, y, z coordinates for the three orthogonal dimensions of space), the origin and axis directions being generally defined on a reference, fixed point on the machine, for instance the machine frame. This coordinate reference system is generally called the machine coordinate system. In this system are generally referenced the position of the devices performing the agricultural operation, such as a mechanical tool, a spot spraying system, a robotic arm or a laser with orientable beam. Regarding the vision system, it has its own coordinate system, defined by the coordinate system of the sensor used, for instance a (x, y) coordinate of a pixel in the image of a camera, or a (x, y, z) coordinate of a pixel in the image of a 3D camera or a LIDAR. This coordinate system is generally called the camera coordinate system, and its origin and axis orientation differ completely from the origin and axis orientation of the machine coordinate system. When a sensor, for instance a camera, is installed on the agricultural machine, a basic function that the system shall provide is to transform with precision the position of a point in the camera coordinate system to the position of the same point in the machine coordinate system, on which the position of the device performing the agricultural operation (which may be referred to as an actuating device 2111826-5 5204007.00019 / ECORO-11or element) is referenced. This coordinate transform operation can be done using a large variety of mathematical operations. However, such a transform is a fundamental element of any agricultural system equipped with a vision system and a device performing an agricultural operation on an object. It provides an accurate known relationship between the positions of objects to be processed in the real world and features of the images captured by the vision system and therefore enables the accurate transfer of the detected position of an object seen by the vision system to the device performing the operation.

[0030] The coordinate transform cannot generally be determined from the images alone because the actuating elements are generally outside the field of view of the vision system so that the vision system can scan the area of a field, or other cultivated surface, ahead of the actuating elements. Errors in the coordinate transform result in a mismatch between the desired and real location of the action performed, and this can reduce the efficiency of the operation (e.g. only part of the object is processed) or even have adverse effects (e.g. a crop plant is destroyed in place of a weed). For instance, on a spot spraying equipment, mismatch in the placement of the nozzles will reduce efficiency (e.g. where the actual spray area is larger than the target spray area) and effectiveness (e.g. by not spraying parts of the target spray area). Given that the smallest plants that require spraying may be only a few millimetres wide (when viewed from above), the correct placement of the nozzle, or any actuating element, is required.

[0031] There are several ways to determine the coordinate transform operation from one sensor coordinate system, for instance a camera, to the machine coordinate system. One way is to fix the camera at a known position (x, y, z) on the machine coordinate system, and with a known axis orientation (three angles, one for each axis), these six position parameters being defined by the mechanical fixture of the camera on the machine. In this case, the coordinate transform is known from the mechanical design. Another way is through a process called extrinsic camera calibration. In this process are placed in the field view of the camera several points whose 3D positions are known in the machine coordinate system. For this a calibration tool is generally used, consisting in its simple form of a plate of the size of the field of view of the camera, the plate being placed in a precise and reproducible known position in the machine coordinate system. The plate contains many visual calibration marks easy to detect and position in the camera coordinate system. Then, a calibration software routine is run, consisting of calculating the mathematical function needed to convert the position of these calibration marks from the image coordinate system to the machine coordinate system. This mathematical function is the coordinate transfer function.

[0032] This extrinsic camera calibration process is called extrinsic because it does not cover the coordinate transforms needed inside the camera. Indeed, cameras have an intrinsic 2111826-5 6204007.00019 / ECORO-11coordinate transform which results from the fact that the optical system is not perfect and as a result the real position of an object on an object plane does not correspond exactly with the projected position of this object on the image (or sensor) plane. Optical systems introduce various errors such as geometry distortion, coma, astigmatism, chromatic aberrations, spherical aberrations, etc. which can be corrected if the corresponding coordinate transform of these errors are known.

[0033] Extrinsic calibration is however a delicate operation requiring specific calibration tools such as briefly described above. These tools must be placed in an accurate and reproducible position in the machine coordinate system to provide an accurate camera calibration. This is not an issue in a well-controlled environment such as a machine factory, where the initial extrinsic camera calibration is done, but can be a challenge for the machines in field operation requiring a new camera extrinsic calibration, where the calibration tools are not available and the precision of their placement cannot be guaranteed. Indeed, in some cases it is necessary to perform a new extrinsic camera calibration, for instance if the mechanical support of the camera has been geometrically deformed following a shock.

[0034] Another difficulty with external camera calibration occurs when a camera needs to be replaced. The problem lies in the fact that the mechanical reference of the image sensor itself may differ from the mechanical reference of the fixture points of the camera enclosure. There may be slight differences in position and orientation caused by component fabrication tolerances between two cameras. In case of replacement of a camera, or a vision system enclosure containing cameras, these differences may require a new extrinsic camera calibration, an operation which is not easy to do in field.

[0035] The present invention aim is to provide new extrinsic calibration methods for vision systems that facilitate or even avoid the extrinsic calibration in specific situations. As the present invention provides methods for extrinsic calibration, for the rest of this description, the term “calibration”, when used alone, will refer to the extrinsic calibration. Vision systems requiring calibration such as described in this invention can be found in machines for soil preparation, for sowing plants, for fertilizer application, for crop protection (i.e applying agrochemicals), for weeding (mechanical, thermal, chemical, radiative or any other means), for harvesting, etc.

[0036] Described herein are improved methods of calibrating the vision system of an apparatus for processing an action on objects in the worked field, generally plants, such as shown in Figure 1 A. The apparatus may, for example, be a spot spraying system, such as the system shown in Figure 1B. In other examples, the apparatus may use other methods for the treatment of plants, whether wanted (i.e. crops) or unwanted (i.e. weeds), e.g. laser-based2111826-5 7204007.00019 / ECORO-11systems which use directed laser beams to eradicate weeds, or mechanical tools to eradicate weeds. In yet other examples, the apparatus may collect objects in the field, for instance parts or the totality of plants for harvesting, or stones in the ground to remove them. The calibration methods described herein are two-stage calibration methods and are particularly suited to calibrate the vision system after fabrication, i.e. once it has left the manufacturing facility and arrived at the location where it is to be used (e.g. at a farm). The calibration methods described herein also provide an improved method of calibrating the vision system of an automated agricultural processing system (such as a spot spraying system) after repairs or upgrades, without requiring the system to be returned to the manufacturing facility or other controlled environment. The methods described herein may be used to calibrate a single camera or more than one camera.

[0037] Whilst most of the examples described below relate to a spot spraying system, the same principles are also applicable to other types of plant-treatment, soil-treatment or other object-treatment systems.

[0038] Figure 1 A shows a schematic diagram of an agricultural equipment 100 configured to perform an action on objects previously detected by a vision system. The agricultural equipment 100 comprises a control system 102, an actuating system 108 and a vision system 106. The actuating system 108 may, for example, be a plant treatment system, such as a spot spraying system or laser weeding system, or object collection system. It will be appreciated that the agricultural equipment 100 may comprise other elements not shown in Figure 1 , such as one or more sensors (e.g. an accelerometer, a GPS receiver, a sensor, etc.).

[0039] The vision system 106 is configured to scan a portion of a field ahead of the actuating system 108 passing over the area and identify target objects. Depending upon the type of agricultural action, the target object may be a desired plant (i.e. a crop) or an undesired plant (i.e. a weed), or a portion of a crop plant to be processed (e.g. a fruit to be harvested), or another kind of organism to be processed (e.g. a bug to be removed), or any other object to be processed (e.g. a stone that is to be collected). The vision system 106 comprises a camera module 162 and an image analysis system 164. The camera module 162 comprises one or more cameras and optionally one or more other sensors, a light source and control hardware for the camera and lighting. In various examples, the camera module 162 may comprise one or several imaging device, which may utilize any suitable technology for 2D or 3D plant or object detection, including but not limited to RGB cameras, multispectral cameras, hyperspectral cameras, infrared cameras, thermal imaging devices, time-of-flight cameras, stereo cameras, or any other imaging technology capable of capturing visual or spatial data relevant to identifying plants or objects found in agricultural environments. Dependent upon the width of the field of view of the camera module 162 (where the width is defined as being 2111826-5 8204007.00019 / ECORO-11perpendicular to the direction of motion of the agricultural equipment), the vision system 106 may comprise more than one camera module 162 so that the combined field of view of the camera modules extends at least the full width of the actuating system 108. The image analysis system 164 is arranged to process the data captured by the camera module(s) 162 and output data defining a target object to be processed (e.g. to perform image analysis and plant / object detection). The output data defining a target object comprises position data for the target object (e.g. the 3D coordinates of the edges of target object) and may also include other information, such as the vertical distance of the target object from the actuating system in case of a spraying equipment, and the target object type.

[0040] The control system 102 is configured to generate control signals for the actuating system 108 based on input received from the vision system 106. Whilst the vision system 106 and control system 102 are shown as separate elements in Figure 1A, it will be appreciated that they may share common components (e.g. processing capabilities) or they may be combined. The actuating system 108 is configured to perform an agricultural action on the target objects and various examples of agricultural actions are described above. The actuating system 108 may comprise a plurality of actuating devices which operate in a coordinated manner (under the control of the control system 102) to perform the agricultural action on the target actions.

[0041] Figure 1B is a variation of the agricultural equipment 100 shown in Figure 1A and described above in which the actuating system 108 is a spray assembly 104 and hence the equipment is a spot spraying system. The spray assembly 104 comprises a plurality of nozzles 110 mounted on a spray bar 112. The nozzles 110 are mounted at a regular spacing, s, along the spray bar 112. Each nozzle 110 has an associated electromechanical valve 114 that is positioned between the nozzle 110 and the spray bar 112. These electromechanical valves 114 can be switched on and off rapidly and precisely to control the flow of fluid from each nozzle 110 and generate each spot spray. A spot spray refers to the fluid output from a nozzle 110 in a single valve opening (i.e. between the valve opening and subsequently closing again). The time that the valve 114 is open and generating a single spot spray may be referred to as a spot duration or opening duration, t, and may be in the region of 3-20 ms. The spray bar 112 is in fluid communication with a tank and pressure system 116 via an optional inlet electromechanical valve 118. The tank and pressure system 116 comprises a tank for holding the agrochemical that is to be sprayed and a pressure system that generates and controls the pressure, p, at which the agrochemical is supplied from the tank to the spray bar 112 and ultimately to the nozzles 110. The volume of fluid output in a single spot spray is a function of the opening duration, the nozzle design (e.g. nozzle diameter) and the pressure and the area that the single spot spray covers (and hence the dose per unit area) is also dependent upon the distance between the nozzle and the surface (e.g. the ground). The inlet 2111826-5 9204007.00019 / ECORO-11valve 118 enables the tank and pressure system 116 to be isolated from the spray bar 112, e.g. for maintenance purposes.

[0042] It will be appreciated that the spray assembly 104 may differ from that shown in Figure 1B. For example, the valves 114 may be integrated into the spray bar 112, there may be more than one spray bar 112 and / or the inlet valve 118 may be omitted.

[0043] The spot spray control system 122 generates control signals for the valves 114 associated with the nozzles 110 in the spray assembly 104 based on input received from the vision system 106. Whilst the vision system 106 and spot spray control system 122 are shown as separate elements in Figure 1 , it will be appreciated that they may share common components (e.g. processing capabilities) or they may be combined.

[0044] Figure 2A shows a schematic diagram of a first example of agricultural equipment 200, which is a spot spraying system. The agricultural equipment 200, like that shown in Figure 1B, comprises a spot spray control system, a spray assembly and a vision system, although only some elements of these are visible in Figure 2A. In particular, Figure 2A shows the nozzles 110 and spray bar 112 as well as a camera 202 that is part of the camera module 162 in the vision system 106. The tank and pressure system, the spot spray control system and other parts of the vision system may be located within the body 204 of the agricultural equipment 200. The body 204 of the agricultural equipment may also be referred to as the frame (or chassis) of the agricultural equipment, or the frame may be a part of the body 204 (e.g. the body 204 may comprise the frame and an outer covering). In this example, the agricultural equipment 200 is towed behind a vehicle 206 (e.g. a tractor) and the direction of travel of the vehicle, when moving forwards, is marked by an arrow 208. As shown in Figure 2A, the vision system scans a portion of a field 210 ahead of (i.e. before) the spray bar 112 passing over the area, i.e. the area being sprayed 212 is behind the area being scanned 210. The area being scanned 210 is the area within the field of view of the camera 202, but the nozzles 110 are outside the field of view of the camera 202. As the distance between the scanned portion of the field 210 and the area being sprayed 212 is fixed, the time delay between the scanning of the field and the spray bar passing over the area can be calculated if the forward speed of the vehicle 206 is known. This is taken into consideration when generating control signals for the mechanical valves 114 in order to spatially synchronize the spot sprays and the target objects.

[0045] Figure 2B is a schematic diagram of a second example of agricultural equipment 200 and using a different view than the first example shown in Figure 2A. The view in Figure 2B is from the rear of the agricultural equipment 200 such that the spray bar 112 and nozzles 110 are visible. Also shown in Figure 2B is the camera 202 which is mounted at pre-defined2111826-5 10204007.00019 / ECORO-11camera fixation points 220 within the camera module 162. The camera module 162 is itself mounted at pre-defined camera module fixation points 222 on the body 204 of the agricultural equipment 200 (e.g. on the frame of the agricultural equipment 200). The viewing cone 224 of the camera 202 is also shown in both Figure 2A and Figure 2B.

[0046] Figure 2C is a schematic diagram showing a part of Figures 2A and 2B in more detail. In the perspective view of Figure 2C, the undulating nature of the ground 229, along with the varying distances of the target objects 230 from the spray bar 112 can be seen. The spray bar 112 is positioned substantially parallel to the ground (which may be a field or other cultivated surface). As shown in Figure 2C, some target objects may have one portion 232 that is closer to the spray bar 112 than another portion 234 of the same target object. Figure 2C also shows some spot sprays 236 being applied to target objects 230.

[0047] In order that the actions performed by the actuating system 108 (or actuating devices within the actuating system 108), for instance spot sprays that are produced by a spot spraying system, are accurately positioned relative to the target objects identified by the vision system 106, the camera 202 of the vision system 106 must be calibrated, as explained earlier. The goal of the calibration process is to define the direct correspondence between a 3D point of the real world expressed in the coordinate system of the agricultural equipment and the corresponding 3D point in an image captured by the camera 202 in a vision system 106 expressed in the coordinate system of the camera. If the camera 202 is able to provide the 3D information of every point of the scene under the form of a depth map, then the 3D position of the objects in the camera coordinate system are known. If the camera is a 2D sensor and there is no other sensor providing the depth information, then an assumption of the unknown distance of each point to the camera must be made, for example by assuming that all the points are located on a plane whose distance to the camera is estimated, for instance the ground of a field. The 3D point in the real world may be defined relative to the body 204 of the agricultural equipment 200 and hence to the position of the actuating system 108 (e.g. to the nozzles 110). The camera calibration is divided into intrinsic and extrinsic parameters, where the intrinsic parameters relate to the internal characteristics of the camera that affect how it captures images, and models all errors related to the optical system projecting the points of the scene into a sensor pixel. This includes geometrical distortions, lenses aberrations, etc. In contrast the extrinsic calibration parameters of a camera relate to the external characteristics that define the camera's position and orientation in the 3D world such as the rotation and translation vectors that describe the camera's pose relative to a known reference frame. The extrinsic calibration procedure hence determines a coordinate transform that transforms coordinates from the coordinate system of the camera R1 , 226 to the coordinate system of the agricultural equipment R3, 227. The coordinate transform may be written as a calibration matrix T3.2111826-5 11204007.00019 / ECORO-11

[0048] Typically, calibration of the vision system 106, and in particular determination of the coordinate transform T3, requires determination of the mathematical relation between a point in the camera coordinate system R1 , 226, and the same point in the agricultural equipment coordinate system R3, 227. Such points are not chosen randomly, but must be easily identified by the camera, and their position must be accurately controlled in the agricultural equipment coordinate system. They are usually placed on a large and heavy calibration mat, which may be several metres square and made of rigid material that ensures flatness of the mat, which is slid under the agricultural equipment 200 (e.g. between the wheels on the ground 229) and into the field of view of the camera 202. Its position in the agricultural equipment coordinate system must finally be accurately controlled, because every optical marker position in the 3D coordinates of the agricultural equipment must be known with the highest accuracy. Whilst this can be performed well in a controlled manufacturing environment, performing it at the location where the agricultural equipment 200 is used (e.g. on a farm) is difficult. The conditions are not well controlled (e.g. the temperature may vary wildly dependent upon the time of year and the ground 229 may not be flat) and the calibration mat is cumbersome. This can lead to inaccuracies in the calibration process. Even if an agricultural equipment 200 is calibrated before it leaves the manufacturing facility, calibration ‘in situ’ cannot be totally avoided because it may be required if the equipment or part of the equipment was subject to a geometrical deformation after a shock, or if a faulty camera or camera module has to be replaced. Difficulties in performing the calibration can lead to increased equipment downtime (e.g. whilst waiting for receipt of a calibration mat following maintenance), or loss in the precision of the actuating operation.

[0049] The improved methods of calibration of an agricultural equipment described herein can be described with reference to Figure 3. Instead of determining calibration matrix T3 using a calibration mat, the calibration process is broken into two separate calibration steps, and hence coordinate transforms, expressed by calibration matrices T1 and T2, where T3 is calculated by combining T1 and T2, e.g. T3=T1+T2. In this context, the + symbol describes a combination operation, and not necessarily an addition in the mathematical definition. The combination operation may be of additive or multiplicative nature. The first of these two new calibration matrices T1 , describes the transform from the coordinate system of the camera R1 , 226 to the coordinate system of the camera module R2, 228. It is worth to mention that, in general, T1 includes only the extrinsic calibration. Nevertheless, it is also possible that T1 combines both intrinsic and extrinsic calibration parameters, although they are usually separately determined and applied. The second of these two new calibration matrices T2, describes the transform from the coordinate system of the camera module R2, 228 to the coordinate system of the agricultural equipment R3, 227 (which may also be referred to as the vehicle coordinate system).2111826-5 12204007.00019 / ECORO-11

[0050] The separation between a camera coordinate system and a camera module coordinate system is at the heart of the improved methods of calibration described herein. Indeed, in practice a camera (or in its most simple form, a camera sensor) must be in all cases be placed mechanically inside a camera module, which can be a simple camera enclosure, ora more complex camera housing with water ingress protections, optical glasses, illumination devices, image processing, etc. This mechanical placement suffers from the same imprecisions that the mechanical placement of the camera module on the frame of the agricultural equipment. However, the mechanical stability of these two placements is different. For the case of the camera, its placement inside the camera module will not change with time, because it is protected from external shocks. However, if the camera module is changed, the camera calibration must be done again because of potential differences in mechanical placement of the camera from one camera module to another. On the contrary, for the case of the placement of the camera module on the agricultural equipment frame, the mechanical design of both the camera modules and the agricultural equipment frame can be done with precisely machined reference points (for instance mechanical pins) that guarantee that even when being replaced, camera modules will be mounted in an accurate and reproducible 3D position on the machine, as long as the machine frame and camera module fixtures are not subject to deformations after a shock.

[0051] Figure 4 shows a flow diagram of the improved calibration method as described herein. The method comprises determining a first calibration matrix from measurements of a sub-set of the agricultural equipment (block 402) where the determined calibration matrix is either T1 orT2. Various methods of determining T1 and T2 are described in more detail below. The method further comprises determining a second calibration matrix from measurements (block 404) where the second calibration matrix is one of T1 , T2 and T3 which has not already been determined. The method then uses the two calibration matrices that have been determined from measurements (one in block 402 and a second, different, one in block 404) to compute the third one of T1 , T2 and T3 (block 406). Having obtained all three calibration matrices, T1, T2 and T3 (in blocks 402-406), calibration matrix T1 is assigned to the camera module 162 in the agricultural equipment used in the calibration method (block 408) and calibration matrix T2 is assigned to the body 204 of the agricultural equipment used in the calibration method (block 410). The calibration matrices that are assigned in the calibration method shown in Figure 4 are then used when generating the coordinate data for target objects at the time of action, as can be described with reference to Figure 5. The action of assigning the calibration matrices (in blocks 408 and 410) comprises storing the calibration matrices in a manner that links the calibration matrix to the particular entity to which it is assigned (i.e. the camera module or body). This may comprise storing the matrix in memory within the particular entity or storing the matrix in memory along with an identifier for the particular entity, as described below.2111826-5 13204007.00019 / ECORO-11

[0052] Figure 5 is a flow diagram showing use of the calibration data generated using the method of Figure 4. When generating the coordinate data for target objects at the time of action using the agricultural equipment (e.g. at the time of spraying for a spot spraying system), the value of T1 that is assigned to the particular camera module 163 that is within the vision system 106 of the agricultural equipment is read (block 502). This calibration matrix, T1 , may be stored in the vision system 106 (e.g. in memory within the vision system 106), within the control system 102 or in a remote data store (e.g. a cloud based data store) indexed by a unique identifier for the particular camera module 163. Similarly, the value of T2 that is assigned to the particular body 204 of the agricultural equipment is read (block 504). As the body 204 is an integral part of the agricultural equipment and cannot be swapped out easily, the calibration matrix T2 which is assigned to the body 204 may be associated with an identifier for the body or with an identifier for the agricultural equipment itself. This calibration matrix, T2, may be stored in the control system 102 or in a remote data store (e.g. a cloud based data store) indexed by a unique identifier for the particular agricultural equipment body 204. Having read the appropriate values of T1 and T2 for the combination of elements within the agricultural equipment, the value of T3 is calculated by combining T1 and T2 (block 506) and is then used to transform coordinate data for target objects which is generated by the vision system 106. The matrices may be combined (in block 506) using summing or multiplying dependent upon the nature of the values. These steps of the method of Figure 5 may be performed by the vision system 106 or by the control system 102.

[0053] Having generated the transformed coordinate data (in block 508), the transformed coordinate data for the target objects may be used to generate control signals for the actuating device to perform action on the target objects (block 510) and then the generated control signals may be used by the actuating device to perform an action on the target objects (block 512). The generation of the control signals (which may also be referred to as ‘actuator commands’) may be performed by the control system 102 and the action (in block 512) may be performed by the agricultural equipment as a whole. For instance, in the case of spot spraying system, the generation of the pattern of spot sprays (in block 510) may be performed by the spot spray control system 122 and the spraying (in block 512) may be performed by the spot spraying system as a whole.

[0054] By splitting out the calibration method as shown in Figure 4 and assigning calibration matrices to individual parts of the agricultural equipment, if a camera module in agricultural equipment is replaced, e.g. because it has failed or for other reasons, then it is not necessary to perform recalibration of the entire agricultural equipment, but instead the value of T1 for the replacement camera module need only be read (in block 502) instead of the value of T1 for the replaced camera module. As described above, as the camera module 162 is mounted within the body 204 of the spot spray system at pre-defined fixing positions 222 (e.g. pre- 2111826-5 14204007.00019 / ECORO-11defined fixing positions on the frame of the spot spray system), replacing the camera module 162 does not affect the value of T2 that is used.

[0055] Figure 6 shows a first example arrangement for determining the value of the calibration matrix T1 for a particular camera module 162 (e.g. in block 402 or 404 of Figure 4). This method is performed without the entire agricultural equipment (e.g. there is no body 204) and is performed using reference objects that can be accurately mapped in the coordinate system of the camera R1, 226 by acquiring images of these objects and determining their position in the image. The calibration matrix T1 can be generated using known coordinates of the objects in the camera module coordinate system R2, 228. In the example shown in Figure 6, the camera module 162 is mounted on a calibration bench 600 using camera module fixation points 222 that correspond to those used when mounting the camera module within the agricultural equipment. In Figure 6 the reference objects are a camera calibration mat 604 which is in a known position relative to the calibration bench 600 (i.e. in a known position in the coordinate reference system of the calibration bench, e.g. in a fixed position in the calibration bench coordinate system 602 resting upon a base plate 606 of the calibration bench 600). In some examples the camera calibration mat 604 may be fixed to, or an integral part of, the calibration bench 600. This has the effect that the coordinate system of the camera module 228 and the coordinate system of the calibration bench 602 may be the same. The calibration mat 604 provides a known arrangement of calibration markers and in other examples, the known arrangement of calibration markers may be provided by something other than a calibration mat.

[0056] By determining calibration matrix T1 using a calibration bench as shown in Figure 6, it reduces errors in the measurements taken (e.g. because the environment is controlled and repeatable, compared to making in-vehicle measurements) and this improves the accuracy of the calibration matrix.

[0057] While the calibration bench 600 is typically used to determine the extrinsic calibration matrix T1 of the camera module, it also allows to perform an initial or repeated intrinsic calibration of the camera 202, placed in its camera module 162, if needed, for example in the case the camera’s optics has been manipulated or changed, or to account for distortions potentially coming from the window of the camera module 162. A dedicated arrangement of calibration targets, fixed to the calibration bench 600, can be used for this purpose.

[0058] Figure 7 shows a first example arrangement for determining the value of the calibration matrix T2 for a particular body 204 of an agricultural equipment (e.g. in block 402 or 404 of Figure 4). In the example shown in Figure 7, the agricultural equipment is a spot spraying system, however, in other examples it may be a different type of agricultural2111826-5 15204007.00019 / ECORO-11equipment. This method is performed using a calibration module 702 that is fitted in place of the camera module 162 using the same module fixing positions 222. As the calibration module 702 is fitted in the same place as the camera module 162 it has the same coordinate system R2, 228, as the camera module. The calibration module 702 may use optical or mechanical methods to measure the relative position of the calibration module 702 to objects that have fixed known position in the vehicle coordinate system R3, 227. These objects may be parts of the actuating system (e.g. the spray bar 112) or a calibration jig that is attached to the actuating system (e.g. to the spray bar 112) or to the body 204 of the agricultural equipment (e.g. to the frame of the agricultural equipment).

[0059] Where the calibration module 702 uses optical methods, the calibration module 702 may comprise one or more lasers that can be used to measure distances and angles between the calibration module 702 and parts of the actuating system (e.g. the spray bar 112 or individual nozzles 110). In example, the parts of the actuating system (e.g. the spray bar 112 or nozzles 110) may have integrated photodetectors which detect incident light emitted by a laser within the calibration module 702. Given the known position of the photodetector and the laser within the calibration module and the angle of the emitted laser beam when it is incident on the photodetector, the calibration matrix T2 can be determined (e.g. using the detected geometry). Instead of integrating photodetectors into parts of the actuating system (e.g. the spray bar or nozzles), a calibration jig, comprising the photodetectors may be fixed to the actuating system (e.g. to the spray bar or nozzles) to enable calibration in this way. In other examples, the calibration module 702 may comprise both lasers and photodetectors and the actuating system (e.g. the spray bar or nozzles), or a calibration jig fitted to the actuating system (e.g. the spray bar or nozzles), may comprise reflective surfaces. Given the known position of the photodetector and the laser within the calibration module, the position of the reflective surfaces and the angle of the emitted laser beam when it is reflected back onto the photodetector, the calibration matrix T2 can be determined (e.g. using geometry). In another example, the calibration module 702 may comprise a wide-angle camera so that the actuating devices within the actuating system (e.g. the spray bar 112 or nozzles 110) are within the field of view of the camera in the calibration module 702. In a further example, the calibration module 702 may use mechanical methods such as high precision rules and / or retractable cable gauges (also referred to as linear cable encoders) to accurately measure the relative positions and geometry between the calibration module and the actuating system (e.g. the spray bar 112 or individual nozzles 110).

[0060] Where the second calibration matrix that is determined in the method of Figure 4 (in block 404) is T3, this may be determined using a calibration mat placed under the agricultural equipment, as described above. Alternatively, a calibration jig 802 may be used instead of the calibration mat, as can be described with reference to Figures 8 and 9. In Figure 8, the 2111826-5 16204007.00019 / ECORO-11calibration jig 802 is shown in its retracted position. The calibration jig 802 is a mechanical structure which is firmly mounted in a fixed position to the body 204 of the agricultural equipment, which in the example shown is a spot spraying system. In the example shown in Figure 8, the calibration jig 802 is mounted to the actuating system (to the spray bar 112); however, in other examples it may be fixed in other positions to the body 204 where there is a fixed, known relationship between the position of the calibration jig 802 and the position of the actuating system or a part thereof (e.g. to the spray bar 112). When the calibration jig 802 is retracted, the field of view of the camera 202 is not obscured and the agricultural equipment can be used to perform its action on the target objects (e.g. a spot spraying system can be used for spraying plants), as shown in Figure 8. Figure 9, in contrast shows the calibration jig 802 in its extended form such that it extends into the field of view of the camera 202 in the camera module 162. Calibration objects 902 may be permanently attached to the surface of the calibration jig that is visible to the camera 202 or alternatively these calibration objects 902 may be attached to this surface in pre-defined positions when the calibration jig 802 is extended to perform calibration.

[0061] The determination of calibration matrix T3 using the calibration jig 802 and calibration objects 902 is performed in a similar manner to using the calibration mat (as described above) but without requiring the insertion of a cumbersome calibration mat underneath the agricultural equipment. Consequently, use of the calibration jig 802 is more suited for ‘in situ’ calibration (i.e. calibration at the location where the agricultural equipment is used, rather than in the manufacturing environment) than the calibration mat. As the position of the calibration objects 902 is known in the vehicle coordinate system, R3227, these positions can be mapped to the positions of the objects in the images captured by the camera 202, thereby determining calibration matrix T3. Calibration matrix T2 can then be calculated I computed with the knowledge of T1 and T3 using the appropriate matrix combination operation, where T1 is the calibration matrix assigned to the camera module 162.

[0062] The calibration jig 802 shown in Figures 8 and 9 may in addition, or instead, be used to determine calibration matrix T2 in a variation of the arrangement shown in Figure 7 and described above. In such a variation, the camera module 162 may use optical methods to measure distances and angles between the camera module 162 and the calibration objects 902 on the calibration jig.

[0063] In the examples described above there is a single position at which a camera module 162 can be mounted using fixation points 222 within the body 204 of the agricultural equipment. In some examples of agricultural equipment, there may be more than one set of fixation points 222, which enables a camera module 162 to be mounted in a plurality of different positions and / or for multiple camera modules to be mounted at the same time. In 2111826-5 17204007.00019 / ECORO-11such examples, there will be a plurality of T2 calibration matrices, one corresponding to each of the possible camera module positions. As a result, the method of Figure 5 is modified such that the T2 calibration matrix that is read (in block 504) is the calibration matrix that correspond to the position in which the camera module is mounted.

[0064] Where there are multiple camera modules 162 mounted at different positions, a value of T3 will be separately calculated for each camera module 162, e.g. using T1 for the camera module and T2 for the fixing position of the module and then the corresponding value of T3 will be applied to coordinate data generated from images from each of the camera modules. For example, if there are 6 camera modules, there will be one T1 calibration matrix for each one, e.g. denoted T1 (1), T1(2),...T1(6). There will also be 6 fixing positions, with one T2 calibration matrix for each one, e.g. denoted T2(1), T2(2),...T2(6). Consequently, there will be 6 calculated T3 calibration matrices, e.g. denoted T3(1), T3(2),...T3(6), where T3(1) is calculated by combining T1 (1 ) and T2(1), T3(2) is calculated by combining T1(2) and T2(2), etc. Calibration matrix T3(1) is then used to transform coordinate data generated by the camera module 1 , calibration matrix T3(2) is then used to transform coordinate data generated by the camera module 2, etc.

[0065] Although the methods of calibration are described above in relation to determination of the extrinsic calibration parameters for the agricultural equipment, as described above, where a calibration bench is used to determine calibration matrix T1 , e.g. as shown in Figure 6, this calibration bench may be used to determine both calibration matrix T1 and the intrinsic calibration parameters of the camera 202.

[0066] Whilst in the description above, the use of a calibration jig 802 to determine calibration matrix T3 is described in the context of the improved calibration method shown in Figure 4, it will be appreciated that the calibration jig 802 may be used instead of the calibration mat to perform calibration independently of the methods of Figures 4 and 5.

[0067] Although the methods have been described above mostly with reference to a spot spraying apparatus, the methods and apparatus (e.g. calibration jigs and modules, etc.) are also applicable to other types of plant-treatment or object-treatment systems used for automated agricultural operations.

[0068] Figure 10 illustrates various components of an example computing-based device 1000 that may be configured to implement the method of Figure 4 or Figure 5. Where the computing-based device 1000 implements the method of Figure 5, the computing-based device 1000 may be part of the vision system 106 or correspond to the control system 102, 122 shown in Figures 1 A and 1 B.2111826-5 18204007.00019 / ECORO-11

[0069] Computing-based device 1000 comprises one or more processors 1002 which may be microprocessors, controllers or any other suitable type of processors for processing computer executable instructions to control the operation of the device in order to perform the methods described herein (e.g. as shown in Figures 4 and 5). The one or more processors can be programmable (e.g., a central processing unit (CPU) or a microcontroller), a field programmable gate array (FPGA), DSP, ASICs, PLC and / or one or more ARM processors, etc. In some examples, for example where a system on a chip architecture is used, the processors 1002 may include one or more fixed function blocks (also referred to as accelerators) which implement a part of the method of controlling a spot spray system in hardware (rather than software or firmware). Platform software comprising an operating system 1004 or any other suitable platform software may be provided at the computing-based device to enable application software 1006, such as software that implements the methods described herein, to be executed on the device (e.g. calibration software that performs one or more steps of the method of Figure 4 and / or coordinate transformation software to perform the method of Figure 5).

[0070] The computer executable instructions may be provided using any computer-readable media that is accessible by computing-based device 1000. Computer-readable media may include, for example, computer storage media such as memory 1008 and communications media. Computer storage media, such as memory 1008, includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transport mechanism. As defined herein, computer storage media does not include communication media. Although the computer storage media (memory 1008) is shown within the computing-based device 1000 it will be appreciated that the storage may be distributed or located remotely and accessed via a network or other communication link (e.g. using communication interface 1010). The memory 1008 may be arranged to store data used by the methods described herein.

[0071] The communication interface 1010 may be arranged to receive data used in the methods described herein such as remotely stored calibration matrices (in blocks 502 and 504 of the method of Figure 5). The communication interface 1010 may also be arranged out2111826-5 19204007.00019 / ECORO-11output the transformed coordinate data for target objects (as generated in block 508 of Figure 5).

[0072] The computing-based device 1000 may also comprise an input / output interface 1012 arranged to output display information to a display device 1014 which may be separate from or integral to the computing-based device 1000. In addition, or instead, the display information may be output via the communication interface 1010 to a remote display device 1014. The display information may provide a graphical user interface. The input / output interface 1012 may also be arranged to receive and process input from one or more devices, such as a user input device 1016 (e.g. data identifying which of a plurality of mounting positions a camera module is mounted on and / or identifiers for the camera module that is mounted). This user input may be used to adjust parameters of the method or provide inputs. In an embodiment the display device 1014 may also act as the user input device 1016 if it is a touch sensitive display device.

[0073] The term 'computer' is used herein to refer to any device with processing capability such that it can execute instructions. Those skilled in the art will realize that such processing capabilities are incorporated into many different devices and therefore the term 'computer' includes PCs, servers, mobile telephones, personal digital assistants and many other devices.

[0074] Those skilled in the art will realize that storage devices utilized to store program instructions can be distributed across a network. For example, a remote computer may store an example of the process described as software. A local or terminal computer may access the remote computer and download a part or all of the software to run the program.Alternatively, the local computer may download pieces of the software as needed, or execute some software instructions at the local terminal and some at the remote computer (or computer network). Those skilled in the art will also realize that by utilizing conventional techniques known to those skilled in the art that all, or a portion of the software instructions may be carried out by a dedicated circuit, such as a DSP, programmable logic array, or the like.

[0075] Any range or device value given herein may be extended or altered without losing the effect sought, as will be apparent to the skilled person.

[0076] It will be understood that the benefits and advantages described above may relate to one embodiment or may relate to several embodiments. The embodiments are not limited to those that solve any or all of the stated problems orthose that have any or all of the stated benefits and advantages.2111826-5 20204007.00019 / ECORO-11

[0077] Any reference to 'an' item refers to one or more of those items. The term 'comprising' is used herein to mean including the method blocks or elements identified, but that such blocks or elements do not comprise an exclusive list and a method or apparatus may contain additional blocks or elements.

[0078] The steps of the methods described herein may be carried out in any suitable order, or simultaneously where appropriate. Additionally, individual blocks may be deleted from any of the methods without departing from the spirit and scope of the subject matter described herein. Aspects of any of the examples described above may be combined with aspects of any of the other examples described to form further examples without losing the effect sought.

[0079] It will be understood that the above description of a preferred embodiment is given by way of example only and that various modifications may be made by those skilled in the art. Although various embodiments have been described above with a certain degree of particularity, or with reference to one or more individual embodiments, those skilled in the art could make numerous alterations to the disclosed embodiments without departing from the spirit or scope of this invention.2111826-5 21

Claims

204007.00019 / ECORO-11Claims1. A method of calibrating a camera (202) fixed in a camera module (162) for use in a vision system (106) of a moveable agricultural equipment (100, 200) for performing actions on objects on a cultivated environment, said agricultural equipment comprising the vision system, a body (204) on which are fixed the camera module and an actuating system (108), wherein the vision system is arranged to acquire images of the cultivated environment ahead of the actuating system using the camera module, the method comprising:determining a first or second calibration matrix (402), wherein the first calibration matrix (T1) describes a transform from a coordinate system of the camera (226) to a coordinate system of the camera module (228) and the second calibration matrix (T2) describes a transform from the coordinate system of the camera module (228) to the coordinate system of the moveable agricultural equipment (227);determining a next calibration matrix (404), wherein the next calibration matrix is another one of the first or second calibration matrices or a third calibration matrix, wherein the third calibration matrix (T3) describes a transform from the coordinate system of the camera (226) to the coordinate system of the moveable agricultural equipment (227);using the determined two calibration matrices to calculate a remaining one of a set of the first, second and third calibration matrices (406);assigning the first calibration matrix (T1) to the camera module (408); and assigning the second calibration matrix (T2) to the body (204) of the moveable agricultural equipment (410).

2. The method according to claim 1 , wherein determining the first calibration matrix comprises:mounting the camera module (162) on a calibration bench (600) in a known position in the coordinate reference system of the calibration bench (602); anddetermining the first calibration matrix from measurements of positions of reference objects at known positions relative to the calibration bench.

3. The method according to claim 2, wherein the reference objects comprise a known arrangement of calibration markers in a known position relative to the calibration bench.

4. The method according to claim 2, wherein the reference objects comprise a camera calibration mat (604) in a known position relative to the calibration bench.2111826-5 22204007.00019 / ECORO-115. The method according to any of the preceding claims, wherein determining the second calibration matrix comprises:mounting a calibration module (702) to the body (204) of the moveable agricultural equipment in place of the camera module; anddetermining the second calibration matrix from measurements of positions of objects having a fixed known position in the vehicle coordinate system.

6. The method according to claim 5, wherein the objects having a fixed known position in the vehicle coordinate system are a part of the actuating system (108) or a calibration jig that is attached to the actuating system.

7. The method according to claim 5, wherein the objects having a fixed known position in the vehicle coordinate system are a calibration jig (802), retractably mounted to the actuating system (108) and wherein the calibration jig is extended to perform the measurements and retracted when performing the actions on objects on the cultivated environment.

8. The method according to any of the preceding claims, wherein determining the third calibration matrix comprises:determining the third calibration matrix from measurements of positions of objects having a fixed known position in the vehicle coordinate system.

9. The method according to claim 7 or 8, wherein the calibration jig is extended into a field of view of the camera to perform the measurements.

10. The method according to any of claims 7-9, wherein the calibration jig (802) comprises a plurality of calibration objects (902) on a surface of the calibration jig that is visible to the camera (202).

11. The method according to any of claims 2-10, wherein the measurements are performed optically.

12. The method according to any of claims 2-8, wherein the measurements are performed mechanically.

13. The method according to any of claims 1-12, wherein the movable agricultural equipment is a movable spray assembly (100, 200) and wherein the actuating system is a spray assembly (104) comprising a spray bar (112) and an array of nozzles (110), each nozzle configured to produce a jet of agrochemical directed towards the cultivated surface.2111826-5 23204007.00019 / ECORO-1114. The method according to any of claims 1-12, wherein the movable plant treatment system is a movable laser weeding assembly and wherein the actuating system is an assembly of one or more laser units and each laser unit comprises a laser source and a means of laser beam guidance configured to emit a laser beam directed towards the cultivated surface.

15. A method comprising:reading a first calibration matrix (T1) assigned to a camera module (162) fitted in a movable agricultural equipment (502), wherein the first calibration matrix (T1) describes a transform from a coordinate system of the camera (226) in the camera module to a coordinate system of the camera module (228) and the movable agricultural equipment is configured to perform actions on objects on a cultivated environment;reading a second calibration matrix (T2) assigned to a body (204) of the movable agricultural equipment (504), wherein the second calibration matrix (T2) describes a transform from the coordinate system of the camera module (228) to the coordinate system of the movable agricultural equipment (227);calculating a third calibration matrix (T3) by combining the first and second calibration matrices (506), wherein the third calibration matrix (T3) describes a transform from the coordinate system of the camera (226) to the coordinate system of the movable agricultural equipment (227);applying the third calibration matrix to transform coordinate data for objects on the cultivated environment (508); andusing the transformed coordinate data to perform the actions on the objects on the cultivated environment using the movable agricultural equipment.

16. The method according to claim 15, wherein the movable agricultural equipment is a spot spraying apparatus and wherein using the transformed coordinate data perform the actions on the objects on the cultivated environment using the movable agricultural equipment comprises:using the transformed coordinate data to generate a pattern of spot sprays to cover the target objects (510).

17. The method according to claim 16, wherein using the transformed coordinate data to perform the actions on the objects on the cultivated environment using the movable agricultural equipment further comprises:2111826-5 24204007.00019 / ECORO-11spraying the target objects according to the generated pattern of spot sprays (512).

18. A computing device (1000) comprising:a processor (1002); anda memory (1008) arranged to store computer executable instructions that, when executed by the processor, cause the computing device to implement the method of any of the preceding claims.

19. A moveable agricultural equipment comprising a computing device, the computing device comprising a processor (1002); and a memory (1008) arranged to store computer executable instructions that, when executed by the processor, cause the computing device to implement the method of any of claims 15-17.

20. The moveable agricultural equipment according to claim 19, comprising a vision system (106) and a control system (102, 122), wherein the computing device is part of the control system or the vision system.2111826-5 25

Images