Improved camera method and system

Abstract

The present invention relates to methods and systems for monitoring e.g. determining parameter(s) associated with movable objects using cameras. The method of monitoring a parameter more parameter(s) of one or more movable (e.g. freely movable) objects (e.g. animals such as aquatic animals, for example fish), comprises: i) in an on-site camera module which comprises an (e.g. at least one) on-site camera and an (e.g. at least one) on-site ML (machine learning) model, determining, using the on-site ML model, one or more first data element(s) representative of one or more first parameter(s) of a first set of one or more movable object(s) in one or more on-site image frame(s) taken by the on-site camera; ii) in an off-site camera module which comprises an off-site camera, determining one or more second data element(s) representative of one or more second parameter(s) of the first set of one or more movable object(s) in one or more image frame(s) taken by the off-site camera; comparing, or otherwise evaluating, the one or more first data element(s) with respect to the one or more second data element(s); adjusting the on-site ML model using one or more result(s) of the comparison, or of the evaluation (e.g. as an input, for example, a labelled set of training data to re-train the on-site ML model e.g. the training data comprising the one or more second data element(s) associated with the one or more on-site image frames).

Description

[0001] Improved Camera Method and System

[0002] FIELD OF THE INVENTION

[0003] The present invention relates to methods and systems for monitoring e.g. determining parameter(s) associated with movable objects using cameras.

[0004] BACKGROUND OF THE INVENTION

[0005] Fish farming is a type of aquaculture, which is the controlled cultivation and harvesting of aquatic animals such as fish, crustaceans, molluscs and so on, in natural or pseudonatural environments. In many cases, pens, such as cages, are used. These are placed in lakes, ponds, rivers, or oceans to contain and protect fish until these can be harvested. Pens can be constructed of a wide variety of components. Fish are artificially fed and harvested once they reach market size.

[0006] In intensive fish farming, reliable estimation offish biomass is important to monitor yields, feed requirements and the health of the fish. Fish biomass may be defined as a measure of the total number offish counted in a specific area of water multiplied by the average mass offish sampled. Fish biomass maybe defined as the total of the masses of each individual fish. This is likelyto be more accurate but is much harderto measure.

[0007] The average weight offish in ponds or cages can be estimated by manual sampling which can cause physical damage or stress to fish, affecting its welfare and growth. Welfare issues and the fact that sampling is very time consuming and inaccurate have led to the creation of a number of non-invasive techniques. Various methods for the quantification of fish biomass in cages or ponds have been developed and include: machine vision based on visible light or infra-red; fish mass measurement; fish counting; direct fish biomass estimation; active and passive acoustics-based methods; and environmental DNA (eDNA)-based methods.

[0008] Of these, machine vision methods are of particular interest because image processing and machine learning techniques maybe applied to improve the accuracy and efficiency of the methods. Machine vision includes technology and methods which provide imaging-based automatic inspection and analysis of a subject. In the case of measurement offish biomass, one or more camera is used to capture images of the fish in the pen and to process these images to determine an estimate of fish biomass. Such processing remains challenging for several reasons, including the difficulty of developing accurate data processing techniques with poor and limited image data, the potential for stress and damage to the animals, as well as the volume and / or complexity of data.Thus, biomass estimation offish, and in particular accurate biomass estimation in real time in fish pens remains challenging. Much focus has been placed on monitoring of live fish in such environments to assess live weight.

[0009] In W02025 / 056404 PYNE CARTER describes apparatus and methods of biomass estimation using underwater 3D camera modules in a pen arranged to face outwards from a common periphery or centre.

[0010] Co-pending unpublished patent application PCT / EP2024 / 077150 PYNE CARTER filed on 26 September 2024 by the present applicant describes systems and methods for monitoring aquatic animals at harvest, and / or during treatment.

[0011] Co-pending unpublished patent application PCT / EP2024 / 083761 PYNE CARTER filed on 27 November 2024 by the present applicant describes a camera control system for a camera in an aquaculture pen.

[0012] US20190037865 PETTERSEN describes a live fish processing system with a fish handling unit comprising an end effector having an integrated gripper and injection assembly. This does not address imaging prior to harvest.

[0013] US2022071180 CHROBAK describes a system for analysis and sorting in aquaculture based on one or more images of live fish. This does not address imaging at harvest.

[0014] Further, classification of aquatic animals e.g. into grades for market, is currently performed at harvest by eye or by sampling at harvest, or on occasion, prior to harvest by manually extracting samples from the pen with a net. These samples may not be representative of the whole population, and further, this causes severe distress to the aquatic animal in being removed from water into air and handled. For these reasons, pre-harvest sampling is usually smaller and so less accurate. Observations can be carried out on a larger sample at harvest, but this only allows reflective insights into the previous farming process and does not allow any action to betaken, nor any other process improvements.

[0015] In one or more embodiments, the present invention seeks to provide improved predictive parameter monitoring and / or improved predictive classification (also known as categorisation) of freely (or near freely) moveable objects such as crops or animals, for example aquatic animals (e.g. fish) prior to harvest.

[0016] Challenges and problems remain in the art, some of which are outlined above and elsewhere in this document. The present invention seeks to alleviate or improve matters in respect of one or more of these challenges and / or problems.SUMMARY OF THE INVENTION

[0017] In a first aspect of the invention there is provided a method of monitoring (e.g. determining) a (e.g. at least one) parameter (e.g. one or more parameter(s) such as one or more properties, for example a dimension(s) e.g. height, and / or length, and / or distance between key points etc., and / or one or more condition(s), such as maturation and / or grade classification) of (e.g. associated with) one or more movable (e.g. freely movable) objects (e.g. animals such as aquatic animals, for example fish), the method comprising:

[0018] i) in an on-site camera module which comprises an (e.g. at least one) on-site camera and an (e.g. at least one) on-site ML (machine learning) model (e.g. a trained ML model comprising a trained ML algorithm),

[0019] determining, using the on-site ML model, one or more first data element(s) representative of one or more first parameter(s) of a first set of one or more movable object(s) in one or more on-site image frame(s) taken by the on-site camera;

[0020] ii) in an off-site (e.g. harvest or inspection) camera module which comprises an (e.g. at least one) off-site camera, determining one or more second data element(s) representative of one or more second parameter(s) of the first set of one or more movable object(s) in one or more image frame(s) taken by the off-site (e.g. harvest) camera;

[0021] comparing, or otherwise evaluating, the one or more first data element(s) with respect to the one or more second data element(s);

[0022] adjusting (e.g. training and / or re-training) the on-site ML model using one or more result(s) of the comparison, or of the evaluation (e.g. as an input, for example, a labelled set of training data to re-train the on-site ML model e.g. the training data comprising the one or more second data element(s) associated with the one or more on-site image frames).

[0023] In a second aspect there is provided a system for determining a (e.g. at least one) parameter (e.g. a parameter such as a property, for example a dimension e.g. height, and / or length, and / or distance between key points etc., or a condition, such as maturation and / or grade classification) of (e.g. associated with) one or more movable (e.g. freely movable) object(s) (e.g. animals such as aquatic animals, for example fish),

[0024] the system comprising:

[0025] i) an on-site camera module which comprises an (e.g. at least one) on-site camera and an on-site ML (machine learning) model (e.g. a trained ML model comprising a trained ML algorithm); ii) an off-site (e.g. harvest) camera module which comprises an (e.g. at least one) off-site (e.g. harvest) camera and, optionally, also comprises an off-site (e.g. harvest) ML model e.g. comprising a trained ML algorithm;iii) a control module configured to carry out the steps of the method of the first aspect (e.g. as further defined in any of claims 1 to 18 below).

[0026] In a further aspect of the invention there is provided a method of determining a (e.g. at least one) parameter (e.g. a parameter such as a property, for example a dimension e.g. height, length, distance between key points etc., or a condition, such as maturation and / or grade classification) of (e.g. associated with) one or more movable (e.g. freely movable) objects (e.g. animals such as aquatic animals, for example fish),

[0027] comprising:

[0028] using an ML model trained using the method steps as described herein (e.g. of the first aspect or of any of claims 1 to 18 below) to determine the parameter (e.g. to more accurately predict the parameter (e.g. a parameter and / or condition, such as grade classification) prior to harvesting).

[0029] In a further aspect of the invention there is provided a method of producing training data for an ML algorithm comprising:

[0030] i) the method steps as described herein (e.g. any of claims 1 to 18), and / or

[0031] ii) using a system as described herein (e.g. of the second aspect or according to any of claims 19 to 22).

[0032] In a further aspect of the invention there is provided a computer-readable medium comprising computer program instructions for the system as described herein (e.g. of any of claims 19 to 22), to carry out the method as described herein (e.g. of any of claims 1 to 18, or 23 or 24).

[0033] In one or more embodiments, the one or more first parameter(s) and the one or more second parameter(s) share at least one parameter in common. Preferably, the one or more first parameter(s) and the one or more second parameter(s) are the same parameter(s) (or are proxies for one another). For example, optionally, the one or more first parameter(s) and the one or more second parameter(s) may both be referred to as one or more parameter(s).

[0034] In one or more embodiments, the on-site camera module comprises an (e.g. at least one) on-site ML (machine learning) model. Alternatively, the (e.g. at least one) on-site ML model may be referred to as an (e.g. at least one) first ML model. For example, the (e.g. at least one) on-site or first ML model may be in communication with the (e.g. at least one) on-site camera of the on-site camera module. As such, whilst the on-site camera module comprises the (e.g. at least one) on-site or first ML model, the (e.g. at least one) on-site or first ML model may be remote from, but in communication with, the (e.g. at least one) on-site camera. In one or moreembodiments, the same applies mutatis mutandis to the optional off-site ML model introduced below, which may, for example, be referred to as a second ML model.

[0035] In one or more embodiments, the off-site (e.g. harvest or inspection) camera module may comprise an off-site ML model comprising a trained ML algorithm, and the method may comprise: determining, using the off-site ML model, the one or more second data element(s) representative of the one or more second parameter(s) of the first set of one or more movable object(s) in one or more image frame(s) taken by the off-site (e.g. harvest) camera.

[0036] In one or more embodiments, the step of adjusting the on-site ML model maybe repeated. Thus, the on-site ML modeland, for example in particular, the ML algorithm of the model, may be trained or re-trained repeatedly e.g. until a desired level of accuracy is achieved.

[0037] In one or more embodiments, the adjusted on-site ML model may be used to determine (e.g. predict) one or more parameters associated with a first or second set of one or more movable object(s) in the one or more image frame(s) taken by the on-site camera.

[0038] In one or more embodiments, the step of using the adjusted on-site ML model may comprise determining, using the adjusted on-site ML model, one or more first data element(s) representative of one or more first parameters of the first, or a second (e.g. different), set of one or more movable object(s) in one or more image frame(s) taken by the on-site camera.

[0039] In one or more embodiments, adjusting (e.g. training and / or re-training) the off-site ML module using the one or more result(s) of the comparison or of the evaluation may be carried out.

[0040] In one or more embodiments, the on-site ML model may be different to the offsite ML model. In one or more embodiments, the on-site ML model is the same as the off-site ML model.

[0041] In one or more embodiments, the first set of one or more movable object(s) may be non-predictably movable with respect to the on-site camera (e.g. during the capture of image frame(s)). For example, these may be independently movable and untethered to a fixed point, or these may be relatively movable one to another and in their environment e.g. tethered animals, or crops such as kelp or molluscs, tethered to the ground or other structure.

[0042] In one or more embodiments, the first set of one or more objects are predictably movable, or static, with respect to the off-site camera (e.g. during the capture of image frame(s)).

[0043] In one or more embodiments, the capturing the one or more image frames used in determining the one or more first data element(s) may take place when at least one of a) the first set of movable objects, and b) the on-site camera are underwater. Preferably, the on-site camera is located within (e.g. is submerged in) an aquaculture pen (or cage). Preferably, (e.g. where themoveable objects are animals, such as aquatic animals) the capturing the one or more image frames used in determiningthe one or more first data element(s) may take place priorto harvest (e.g. such as during cultivation).

[0044] In other embodiments, the on-site camera and / orthe first set of movable objects may be land-based.

[0045] In one or more embodiments, de-watering the first (and / or any later) sets of one or more movable object(s) may take place priorto capturing the one or more image frame(s) in the off-site camera.

[0046] In one or more embodiments, stunning (e.g. to insensibility and / or immobility for example by electrically stunning in water) the first set of one or more movable object(s) may take place priorto capturing of one or more image frame(s) in the off-site camera. In one or more embodiments, eliciting death of the first set of one or more movable object(s) may take place prior to capturing of one or more image frame(s) in the off-site camera. Generally, it is preferable that, where the objects are animals (such as aquatic animals), the aquatic animals are immobile (e.g. due to death or stunning) at the time of capturing the one or more image frame(s) taken by the offsite camera.

[0047] In some embodiments, the method may comprise bleeding the first (and / or any later) sets of objects after stunning, optionally priorto imaging by the off-site camera.

[0048] In one or more embodiments, one or more image frame(s) used for determining the one or more second data element(s) may be captured when at least one (preferably both) of a) the first set of objects, and b) the off-site camera are in air. Preferably, the off-site camera is not located within (e.g. is not submerged in) an aquaculture pen (or cage). Preferably, (e.g. where the moveable objects are animals, such as aquatic animals) the capturing the one or more image frames used in determiningthe one or more first data element(s) may take place during or immediately after harvesting the first set of objects.

[0049] In one or more embodiments, the off-site camera module comprises a substantially planar platform on which the one or more movable objects (e.g. aquatic animals) travel (e.g. travel out of water). In one or more embodiments, the off-site camera module may comprise a planar and / or horizontal platform on which the one or more moveable objects (e.g. aquatic animals) travel. Thus, preferably, in one or more embodiments, a generally planar, preferably horizontal platform fortravel is provided. Travel may take place by conveyer belt, driven rollers, momentum of ingress etc. In one or more embodiments, the off-site camera module comprises a sloped platform on which aquatic animals travel. Thus, in one or more embodiments, a sloped, planar platform is provided. It may be curved.In one or more embodiments, the distance of the platform and so the object (e.g. the animal) from the camera may be predetermined. Preferably the height and / or orientation of the off-site camera above the platform may be variable to one or more predetermined heights and / or orientations.

[0050] In one or more embodiments, the one or more moveable object(s), in the one or more image frame(s) taken by the off-site camera, are arranged such that they (generally) do not overlap each other and / or such that they (generally) are spaced apart from each other. For example, the off-site camera module is configured to (generally) prevent (or minimise or reduce the occurrence of) overlap of the one or more moveable object(s) at least when the one or more moveable object(s) are in the field of view of the off-site camera. For example, the off-site camera module is configured to (generally) space apart the one or more moveable object(s) at least when the one or more moveable object(s) are in the field of view of the off-site camera, e.g. the spacing apart may be automatic.

[0051] In one or more embodiments, the on-site camera may be movable (e.g. relative to a pen and / or the objects), and the method may comprise (and the system may be configured to):

[0052] controllably moving (move) the movable on-site camera (e.g. in response to instructions from a control module, which instructions may be based on feedback from the comparing and / or evaluating step).

[0053] Thus, the position and / or orientation of the on-site camera may be controllably varied to alter its location in x, y, z e.g. within a pen, and its rotation about one or more axes (e.g. horizontal and / or vertical axes) and / or about a point, so as to control its field of view.

[0054] It will be understood that, whilst the on-site camera of the on-site camera module is to be located on-site e.g. within a pen for animals such as an aquaculture pen for fish, the associated on-site camera ML modeland on-site camera controller within the on-site camera module, may be located elsewhere. For these elements, the term ‘on-site’ refers to the place where data (e.g. image frames) is gathered and / or where a camera is controllably moved. The term ‘on-site’ is thus a label to associate the ‘on-site’ ML model and the ‘on-site’ camera controller with the one or more ‘on-site’ camera(s) which is / are actually located on-site e.g. in a pen. Thus, the ‘on-site’ ML model and camera controller may be remote from the ‘on-site’ actual location of the one or more ‘on-site’ camera(s). Similarly, the ‘off-site' ML model and camera controller may be remote from the actual ‘off-site’ camera(s).

[0055] Nevertheless, the off-site camera may be located close to the on-site camera if the harvesting apparatus is located close to the farming pen(s).It will be understood that a set of objects may comprise one or more, and preferably, comprises several such objects. The set may comprise a sample from within a predetermined population, e.g.from a pen or from a farm, or may be the entirety of the population of one or more pen(s) and / or farm(s). A set may be termed a batch.

[0056] The off-site camera typically comprises a harvest camera which images harvested (e.g. stunned, optionally killed) objects now immovable of themselves, but which may be moved in a predictable manner (e.g. at known distance and / or orientation etc.) with respect to the off-site camera. For example, these harvested objects may be carried by a conveyor such as a conveyor belt as a planar surface at a fixed distance from the off-site camera.

[0057] Several sample sets may be taken from a single pen or single farm to train, re-train (and re-train again) the on-site ML modelforthat particular pen and / or farm. Additional inputs may be used in the (re-)training, for example, data regarding feeding (time, amount, quantity, type, weather, water clarity, predator activity, lice treatment) for example over a period of time such as a two year grow out period.

[0058] In one or more embodiments, the method may comprise training the adjusted onsite ML model for a particular predetermined site or predetermined type of site (e.g. a location, for example a pen, and / or farm, and / or lake, and / or river, and / or ocean location).

[0059] In one or more embodiments, the method (and suitably configured system) may comprise using the adjusted on-site ML model for a plurality of camera modules in one or more pens containing objects and / or farms containing objects. Thus, following the training of one ML model to provide a revised ML model associated with one pen, that revised ML model may be used for camera modules for other pen(s) within the same farm, or other farm(s) in the same or similar location, and / or site conditions.

[0060] In one or more embodiments, the method may comprise taking one or more action(s) to improve a property and / or or condition and / or level of one or more predetermined parameters (e.g. measured dimensions and / or determined grade classifications).

[0061] Thus, the actions may be taken to improve the quality of the objects, such as fish by additional feeding and / or treatment, with a view to improving a particular parameter such as grade classification which is itself based on several quantities.

[0062] In one or more embodiments, the method may comprise identifying a unique individual (e.g. a spot pattern) or may comprise uniquely identifying an aquatic animal in an image (e.g. using spot pattern recognition).

[0063] In one or more embodiments, the method further comprises identifying which one or more first data elements and one or more second data elements correspond to the samemoveable object (e.g. to the same individual) within the first set of one or more movable objects. Preferably, the method further comprises comparing, or otherwise evaluating, the one or more first data element(s) with respect to the one or more second data element(s) that correspond to the same (e.g. to a unique) moveable object (or individual). For example, in the case of aquatic animals, unique spot patterns may be used to distinguish between aquatic animals of a first set. Other techniques may be used to distinguish between individuals of a set of plural movable objects (such as a set of animals).

[0064] In one or more embodiments, the method further comprises using the on-site ML model to assign (or allocate) a unique identification tagto at least one of the one or more movable object(s); for example, a unique characteristic (e.g. spot pattern) may be used to distinguish between individual moveable objects. Preferably, the one or more first data element(s) representative of one or more first parameter(s) correspond with individual moveable objects of the first set of one or more movable object(s), e.g. by using a unique identification tag(s).

[0065] Preferably, the method further comprises using the (optional) off-site ML model to (e.g. using the one or more image frame(s) taken by the off-site camera) re-identify individual moveable object(s) of the first set of one or more movable object(s) that were assigned (or allocated) a unique identification tag by the on-site ML model. For example, this permits comparing, or otherwise evaluating, the one or more first data element(s) with respect to the one or more second data element(s) that correspond to the same (e.g. to a unique) moveable object (or individual) as discussed above.

[0066] In one or more embodiments, the on-site ML model and / or the (optional) off-site ML model directly receive (e.g. as inputs) the one or more (respective on-site or off-site) image frame(s) taken by the respective camera.

[0067] In one or more embodiments, the on-site ML model and / or the (optional) off-site ML model directly receive (e.g. as inputs) the one or more (respective on-site or off-site) image frame(s) taken by the respective camera, and (e.g. using the one or more (respective on-site or offsite) image frame(s)) determine the one or more respective first or second data element(s).

[0068] In one or more embodiments (e.g. including where the off-site camera module does not include an off-site ML model) the one or more second data element(s), representative of one or more second parameter(s) of the first set of one or more movable object(s), are determined directly (and / or automatically) from the one or more image frame(s) taken by the off-site camera. For example, the one or more image frame(s) taken by the off-site camera need not be processed manually in order to determine the one or more second data element(s). In one or more embodiments, this applies mutatis mutandis to determining the one or more first data element(s).The system maybe suitably configured to carryout any of the steps of the method as described herein.

[0069] At least one off-site camera module may comprise an off-site (e.g. harvest) ML model e.g. comprising a trained ML algorithm.

[0070] At least one off-site camera may comprises a 2D and / or 3D camera.

[0071] It will be understood that one or both of the one or more on-site and off-site cameras may comprise a 2D and / or 3D camera unit. It is preferred if the on-site camera comprises at least 2D camera unit and the 3D off-site (e.g. in air) camera comprises at least a 2D and 3D unit.

[0072] In one or more embodiments, the parameter may comprise one or more properties and / or conditions from the following list(s):

[0073] a) property: - key point locations

[0074] - separation of key points

[0075] - length e.g. nose to fin

[0076] - width

[0077] - depth

[0078] - weight

[0079] - colour

[0080] - number and / or area of wounds

[0081] - number and / or area of lice

[0082] b) condition: - maturation

[0083] - grade classification

[0084] In one or more further embodiments of the invention there is provided a system for monitoring objects e.g. aquatic animals (e.g. fish) at harvest (e.g. in real time), and / or for treatment (such as during treatment e.g. vaccination, any occasion where live animals e.g. fish are removed from water, e.g. in real time), further comprising (e.g. optionally in the order stun station, dewatering station and, where provided, a processing station):

[0085] a stun station, preferably an electric stun station (e.g. comprising an inline electric stunner with one or more stun sections each comprising one or more electrodes);

[0086] a dewatering station;

[0087] optionally (e.g. preferably), a processing station (e.g. a post-stun processing station, for example, to elicit death, such as percussive stun and / or bleed, and / or ice treatment, or for other treatment e.g. for lice, and / or a packaging station and so on);

[0088] at least one imaging station (e.g. after the stun station); and,

[0089] at the at least one imaging station, the second or off-site (e.g. harvest) camera module comprising:at least one camera sub-unit (e.g. comprising at least one camera such as (e.g. 2D HD and / or RGB), and / or 3D such as Time of Flight, stereo, lidar etc.) configured for capturing images of objects (e.g. aquatic animals) post stun (e.g. electric stun) as these pass through the imaging station.

[0090] In one or more further embodiments of the invention there is provided a method for monitoring aquatic animals (e.g. fish) at harvest (e.g. in realtime), and / or for treatment (such as during treatment e.g. vaccination or any occasion where live animals e.g. fish are removed from water, optionally in realtime), in a system comprising:

[0091] a stun station (e.g. an electric stun station comprising an electric stunner, such as an inline (e.g. along a pipeline or channel) electric stunner with one or more stun sections each comprising one or more electrodes);

[0092] a dewatering station;

[0093] optionally (e.g. preferably), a processing station (e.g. a post-stun processing station to elicit death, such as percussive stun and / or bleed, and / or ice or vaccine treatment, and / or a packaging station and so on);

[0094] at least one imaging station (e.g. after the stun station);

[0095] at the at least one imaging station, the second or off-site (e.g. harvest) camera module comprising:

[0096] at least one camera sub-unit (e.g. comprising at least one camera such as (e.g. 2D HD and / or RGB), and / or 3D such as Time of Flight, stereo, lidar etc.) configured for capturing images of aquatic animals post (e.g. electric) stun as these pass through the imaging station.

[0097] The imaging station may comprise an image processing unit of, or operably coupled to, the second camera module, the image processing unit comprising:

[0098] at least one microprocessor;

[0099] at least one storage medium comprising one or more programs, the one or more programs configured for execution on the at least one microprocessor;

[0100] the one or more programs comprising instructions for:

[0101] detecting at least one object in at least one image frame;

[0102] determining at least one parameter associated with the object in the at least one image frame.

[0103] The at least one parameter may comprise a first parameter of the object being determined to be an aquatic animal of interest.

[0104] The image processing unit may comprise the second (or off-site) machine learning model. The second machine learning model may be configured to recommend taking an actionassociated with the harvesting, and / or treatment, of aquatic animals (e.g. of the identified aquatic animal, and / or of the harvesting, and / or treatment, system and / or process itself).

[0105] In one or more embodiments, the stun station may comprise an electric stun station. In one or more embodiments, the stun station may comprise an inline electric stun station. In one or more embodiments, the stun station may comprise an inline wet electric stun station (e.g. one through which water flows). In one or more embodiments, the stun station may comprise a dry electric or percussive stun station (optionally with a bleed station). This may be used to increase efficacy of these types of stunning e.g. by helping to adjust the stun station, and / or move fish automatically to a secondary stun area for additional stunning.

[0106] In one or more embodiments, the action may comprise increasing the count of objects (e.g. aquatic animals of interest) by one.

[0107] In one or more embodiments, the parameter (e.g. the at least one parameter) may comprise a dimension of the object (e.g. aquatic animal) (e.g. being determined, for example being determined with sufficient accuracy); and the action may comprise estimating a biomass of the objects (e.g. of the animals, or of a group of animals).

[0108] In one of more embodiments the apparatus may be configured so that determining at least one parameter may comprise determining whether an object (e.g. aquatic animal) can move by comparing a dimension of the object in at least two frames (e.g. from a first frame to a second frame), or in two or more sets of frames (e.g. from an average from a first set of frames, to an average from a second set of frames); and the action comprises indicating a stun state of the object (e.g. aquatic animal).

[0109] In one or more embodiments, the at least one parameter may comprise the aquatic animal not being capable of movement; and the action comprises storing an indication that the aquatic animal is in a stunned state. This storing action maybe done locally e.g. onto a local storage medium, or off site, onto a remote storage medium.

[0110] In one or more embodiments, the at least one parameter may comprise the object (e.g. aquatic animal) being in a stunned (e.g. unconscious) state; and the action may comprise storing an indication that the object (e.g. aquatic animal) is in a stunned (e.g. unconscious) state.

[0111] In one or more embodiments, the at least one parameter may comprise the object (e.g. aquatic animal) being capable of movement (e.g. not in a fully stunned (e.g. unconscious) state); and the action may comprise storing an indication that the object (e.g. aquatic animal) is not in a stunned state.

[0112] In one or more embodiments, the stun station may comprise an electric stun station.In one or more embodiments, the stun station may comprise an inline inlet electric station e.g. capable of submerging objects (e.g. aquatic animals).

[0113] In one or more embodiments, the action may comprise adjusting the stun conditions of the stun station.

[0114] In one or more embodiments, where the stun station is an electric stun station, the action may comprise adjusting the electric stun conditions of the electric stun station (e.g. adjusting the voltage and / or frequency (if varying voltages) and / or current at one or more electrodes).

[0115] The electric stun station may comprise a stun section and, following the stun section, a stun maintenance section. The stun maintenance section can be varied in length or number of electrodes or stun parameters to adjust the length of time the stun is maintained for.

[0116] The stun condition of the electric stun station can be varied in a number of ways, including increasing or decreasing the level of stun (e.g. voltage and / or frequency and / or current) in a stun section, or increasing or decreasingthe level of stun (e.g. voltage and / or frequency and / or current) in a stun maintenance section, where provided.

[0117] Thus, by careful design of the harvesting system and / or by careful control of stun conditions, objects such as aquatic animals can be passed to a post-stun processing station for post-stun treatment, e.g. for treatment and / or to elicit death without recovering from the stun. The careful design typically comprises a time spent post-stun travelling to the processing station, which depends upon speed of travel of the (e.g. aquatic) animals, and length of post-stun line, e.g. the imaging station and the distance to the processing station, as well as that of the processing station itself.

[0118] In one or more embodiments, the imaging station is after the dewatering station and, where provided, before the processing station.

[0119] In one or more embodiments, the imaging station is after a or the processing station.

[0120] In one or more embodiments, the order of the stations may be varied as would be understood by someone skilled in the art e.g. the imaging station may be between percussive stun station and a bleed station.

[0121] In one or more embodiments where a wet e.g. inline electric, stun station is provided, the stun station is before the dewatering station, and the imaging station is provided afterthe dewatering station, e.g. so that imaging maytake place in air. In one or more embodiments, the imaging station is afterthe dewatering station and imaging takes place in air.In one or more embodiments where a dry stun station, and / or percussive stun station, is provided, a dewatering station may be provided before the stun station, and the imaging station may be provided after the stun station e.g. so that stunning and imaging may take place in air. In one or more embodiments, two imaging stations may be provided, a first imaging station between the dewatering station and a or the processing station, and a second imaging station after the processing station. The first and second imaging stations may be the same (e.g. of identical construction) but may not be.

[0122] In one or more embodiments, the at least one second camera module may comprise a 2D camera (e.g. HD and / or RGB). In one or more embodiments, the at least one second camera module may comprise a 2D camera and a 3D camera. In one or more embodiments, the 2D camera may be a high-definition (also known as high resolution) camera. Typically, these have upwards of 12 megapixels). It may be HD and RGB, or HD and monochrome. In one or more embodiments, the 2D camera may be a colour (e.g. RGB) camera. In one or more embodiments, the 3D camera may be a stereo camera (e.g. comprising 2x2D cameras such as 2x2D low resolution, monochrome cameras of resolution of 12 megapixels or lower, typically 8 megapixels or lower). In one or more embodiments, the 3D camera may be a time-of-flight camera. The 3D camera may be a time-of-flight camera e.g. such as that described in GB2539495 PYNE CARTER. In one or more embodiments, two or more cameras within the at least one (e.g. one, two or more, or each) of the second camera module have the same, or substantially the same, focal length and / or same, or substantially same, field of view. In one or more embodiments, at least two cameras are provided within the at least one (e.g. one, two or more, or each) second camera module(s), optionally each camera having a different position in a processing line and / or focal length.

[0123] In one or more embodiments, the imaging station comprises a substantially planar platform on which aquatic animals travel. In one or more embodiments, the imaging station may comprise a planar and / or horizontal platform on which objects (e.g. aquatic animals) travel. Thus, preferably, in one or more embodiments, a generally planar, preferably horizontal platform fortravel is provided. Travel may take place by conveyer belt, driven rollers, momentum of ingress etc. In one or more embodiments, the imaging station comprises a sloped platform on which aquatic animals travel. Thus, in one or more embodiments, a sloped, planar platform is provided. It may be curved.

[0124] In one or more embodiments, the distance of the platform and so the animalfrom the camera may be predetermined. Preferably the height and / or orientation of the imageacquisition unit above the platform may be variable to one or more predetermined heights and / or orientations.

[0125] In one or more embodiments, the system may comprise (e.g. within the image processing unit) the second machine learning model (e.g. in the form of a first trained deep learning network (e.g. a convolutional neural network)) for classifying and / or uniquely identifying an object in an image frame e.g. as an aquatic animal of interest (e.g. a fish and / or a species of fish).

[0126] In one or more embodiments, the image processing unit comprises the same (e.g. the first), or a second, trained deep learning network (e.g. a CNN such as a mask R-CNN e.g. within the image processing unit) to determine a segmentation mask of an aquatic animal in one or more (2D or 3D) image frames.

[0127] In one or more embodiments, the system (e.g. the instructions) may comprise determining a length dimension (e.g. length and / or width and / or height and / or distance between various key points) of an object e.g. aquatic animal and / or determining a weight of an object e.g. aquatic animal.

[0128] In one or more embodiments, the system (e.g. the instructions) may comprise uniquely identifying an aquatic animal in an image e.g. using spot pattern recognition.

[0129] In one or more embodiments, the system (e.g. the instructions) comprise associating a parameter with a uniquely identified aquatic animal.

[0130] In one or more embodiments, the method may comprise determining a dimension of the aquatic animal; and estimating a biomass of the aquatic animal.

[0131] In one or more embodiments, the first and / or second machine learning mode may comprise classifying and / or uniquely identifying an object in an image frame as an aquatic animal of interest (e.g. a fish and / or a species of fish). This may further comprise a unique ID tag in database.

[0132] In one or more embodiments, a stereo camera may be provided as a camera in the first and / or second camera module and the method may further comprise:

[0133] • i) obtaining and storing a pair of digital images captured by, or derived from digital images captured by, the stereo camera;

[0134] • ii) matching pixels between pairs of digital images;

[0135] • iii) producing a disparity map;

[0136] • iv) using the disparity map to determine a depth map (e.g. a point cloud);

[0137] • v) using a trained machine learning model, (e.g. the first or second machine learning model e.g. such as a convolutional neural network), to classify a region of the depth map, (e.g. ascontaining an image of a fish; optionally, cropping the digital image to the image region to generate a cropped image).

[0138] This storing of images may be done locally e.g. onto a local storage medium, or off site, onto a remote storage medium.

[0139] Where a stereo camera is provided, the instructions in the system or the method may comprise the following:

[0140] • based on obtaining and storing a pair of digital images captured by, or derived from digital images, captured by the stereo camera, using a trained ML module, (e.g. convolutional neural network) to classify an image region of a digital image, of the pair of digital images, as contained in an image of an aquatic animal;

[0141] • cropping the digital image to the image region to generate a cropped image;

[0142] • identifying a plurality of landmark points on the aquatic animal within the cropped image;

[0143] • determining a plurality of disparity map values corresponding to at least the plurality of landmark points;

[0144] • based on the plurality of disparity map values, calculating one or more dimensions of the aquatic animal; and

[0145] • estimating a mass of the aquatic animal based on the one or more dimensions.

[0146] In one or more embodiments, the step of detecting objects in 2D may comprise tagging and / or segmenting objects in 2D images from a 2D camera (e.g. using intensity or texture or colour etc.), optionally from a 2D camera separate from the 3D camera.

[0147] In one or more embodiments, the step of detecting objects in 3D may comprise tagging and / or segmenting objects in 3D images (e.g. using depth or indeed using depth and / or using intensity and / or texture and / or colour).

[0148] In one or more embodiments, the method may comprise using machine learning to identify relevant pixel groupings representing the object in a 2D and / or 3D image.

[0149] In one or more embodiments, the system or method may comprise the same (e.g. first), or a second, trained deep learning network (e.g. a CNN such as a mark R-CNN e.g. within the image processing unit) (e.g. for determining a segmentation mask of an aquatic animal in one or more image frames). In some embodiments, the system or the method may comprise determining a segmentation mask of an aquatic animal in one or more image frames.

[0150] In one or more embodiments, the invention may comprise uniquely identifying an aquatic animal in an image. In one or more embodiments, the invention may comprise associating a determined parameter with such a uniquely identified aquatic animal.

[0151] In one or more embodiments, the method may comprise:- identifying a plurality of (i.e. two or more) landmark points and / or one or more body surface lines within the image of the fish on a 2D or 3D (e.g. depth) map;

[0152] - determining one or more body dimensions of the fish using the plurality of landmark points and / or one or more surface lines, and, based on the one or more body dimensions, estimating a mass of the fish.

[0153] In one or more further embodiments of the invention, there may be provided a camera control system for the on-site camera module in an aquaculture pen substantially submerged in a body of water, the system comprising:

[0154] an on-site camera controller configured to position one or more on-site camera(s) in the first or on-site camera module within the pen at at least one (preferably two or more) predetermined position(s) with at least one (preferably two or more) associated field(s) of view;

[0155] A camera image controller may be provided comprising the first or on-site machine learning model which identifies objects of interest in the aquaculture pen which are viewable by one or more on-site camera(s) from the at least one predetermined position.

[0156] In one or more further embodiments of the invention there is provided a method for controlling the one or more on-site camera(s) within a first or on-site camera module in an aquaculture pen which is substantially submerged in a body of water, the method comprising: positioning the camera within the pen at a predetermined position with at least one associated field of view using an on-site camera controller which controls the position of a movable (e.g. laterally and / or rotationally) camera(s) and / or movable camera mount(s) (e.g. to position the one or more camera(s) pointing at one or more field(s) of view);

[0157] identifying on the on-site camera objects of interest in the aquaculture pen which are viewable by the one or more on-site camera(s) from the predetermined position using the on-site machine learning model, optionally using a camera image controller.

[0158] The on-site camera controller may be configured to position one or more camera(s) of the first or on-site camera module within the pen at first and second predetermined positions, each with one or more, preferably two or more, associated fields of view.

[0159] Preferably, the on-site camera controller is configured to control the position of the one or more on-site camera(s) in a z-direction (e.g. in a vertical direction, e.g. within expected tolerances) which is the depth to which a or the camera is submerged in the body of water.

[0160] Preferably, the on-site camera controller is configured to control the position of the one or more on-site camera(s) in an xy plane, e.g. parallel to the top of the aquaculture pen (e.g. to its upper frame). This xy plane is typically horizontal e.g. within expected tolerances.Preferably, the on-site camera controller is programmable in response to instructions from the control module.

[0161] Preferably, the on-site camera controller is programmed to move the one or more on-site camera(s) in the z direction (e.g. vertically) to a plurality of positions separated by one or more predetermined distance intervals.

[0162] Preferably, the on-site camera controller is configured to control (e.g. set, or to determine and set) the time interval for which the one or more on-site camera(s) is / are located at at least one predetermined position.

[0163] Preferably, the on-site camera controller is configured to control the direction in a orthexy plane in which the one or more on-site camera(s) is / are pointingto controlthe on-site camera’s field of view (e.g. about a vertical axis). More preferably, the one or more on-site camera(s) may be rotated (e.g. up to 180°) in either direction (e.g. about an axis in the z direction (e.g. about a vertical axis) and / or about an axis lying in an xy plane (e.g. about a horizontal axis) and / or in a z-direction about a horizontal axis. Rotation (e.g. about a vertical axis in the z direction and / or a horizontal axis in thexy plane) can give afield of view which is 360° around the one or more predetermined positions of the on-site camera.

[0164] The one or more on-site camera(s) may be configured to rotate e.g. in such a manner with respect to its mount, or a mount of the on-site camera(s) may be configured to rotate e.g. in such a manner.

[0165] Preferably, the on-site camera controller is configured to controlthe direction in a or the z plane (e.g. in a vertical (up and down) direction, for example about an axis lying in the typically horizontal xy plane) in which the on-site camera is pointing to control the camera’s field of view (e.g. up and down).

[0166] The camera image controller may be programmable to recognise or note the recognition of an object of interest (e.g. based upon image frames analysed by the on-site ML model).

[0167] Preferably, the camera image controller, once an object of interest has been recognised, may be configured to track movement of the object of interest.

[0168] Preferably, the camera image controller may be configured to create an instruction to the camera controller to change the position of the one or more on-site camera(s) to alter one or more of i) the field of view ii) position along (e.g. along the height of a z axis) iii) direction(s) pointing around a vertical axis (e.g. z direction) and / or horizontal axis (e.g. about an axis lying in an xy plane) (e.g. from one predetermined field of viewto another predetermined fieldof view, for example about a horizontal axis and / or about a vertical axis) in response to the tracked movement of the object of interest and / or in response to environmental conditions.

[0169] Preferably, the camera image controller may be programmable to recognise the absence and / or presence of an object of interest and can create an instruction (SEEK or ROAM) to the on-site camera position controller to change the position of the one or more on-site camera(s) to look for the object of interest.

[0170] Preferably, a movable camera mount may be provided.

[0171] Preferably, the one or more on-site camera(s) may be connected (e.g. rigidly) to the movable camera mount e.g. so that upon movement (e.g. lateral and / or rotating movement) of the camera mount, the one or more camera(s) on it move(s) in a corresponding manner.

[0172] Preferably, the system may comprise a winch. Preferably, the winch may comprise a winch cable, motor, and associated spool. Preferably, the one or more on-site camera(s), and / or its / their associated camera mount(s), is / are connected to one or more winch cable(s) of the winch, wherein the one or more winch cable(s) are mounted on one or more (e.g. respective) spool(s) and wherein rotation of the one or more spool(s) may be actuated by a winch motor which is controlled by the camera controller.

[0173] Preferably, the position of the one or more on-site camera(s) in the z direction (e.g. typically a vertical direction) may be set (e.g. determined) by the length of one or more winch cable(s).

[0174] Preferably, the position of the one or more on-site camera(s) in the xy direction (e.g. typically a horizontal direction) may be set (e.g. determined) by the position of the winch (e.g. on the one or more cable(s)).

[0175] Optionally, the one or more on-site camera(s) may be connected to a cable system which comprises (e.g. at least) a first and a second cable support, the cable supports (which may be in the form of one or more towers or poles) being mountable on or near an aquaculture pen above a top surface of the aquaculture pen, each cable support may be coupled to the one or more on-site camera(s) by means of a cable (e.g. a respective cable, or respective cable portion or a common cable or portion) such that the supports and camera are mutually coupled and the on-site camera(s) is / are positioned below the top surface of the aquaculture pen wherein the distance between the camera(s) and first and second cable supports is determined by adjusting the length of the cable between the on-site camera(s) and respective first and second supports, wherein the cable system allows the on-site camera(s) to be positioned (e.g. in a line) between the cable supports and at a depth below the supports.In one or more embodiments, the system comprises a cable system, and / or the cable system comprising first, or first and second, or first, second and third cable supports (which may have some height e.g. may be in the form of one or more towers or poles) for one or more cables (or cable portions) of the cable system.

[0176] Optionally, the camera may be connected to a cable system which may comprise at least one additional cable support wherein the cable system allows the one or more on-site camera(s) to be positioned between the (respective or common / shared) three or more cable supports and at a depth below the cable supports. Where two or three or more cable supports (which may have some height) are provided, these may be mutually separated, preferably mutually equi-spaced about a periphery (e.g. a circumference) of an upper (e.g. top) of a pen, (e.g. about a rigid upper frame of a pen)).

[0177] In one or more embodiments, the present invention can controllably vary the time interval spent by the one or more camera(s) at one or more particular location(s) and / or can controllably vary the time interval spent looking at one or more particular field(s) of view at each location.

[0178] In one or more embodiments of the invention the on-site or first camera module for estimating biomass of (e.g. freely) moving objects, e.g. fish for example in a pen (e.g. in an aquaculture environment), comprises:

[0179] at least two camera sub-units, a first camera sub-unit for immersion underwater (e.g. in a pen) having at least one 3D camera, and, a second camera sub-unit for immersion underwater (e.g. in a pen), having at least one 3D camera;

[0180] the first camera sub-unit facing outwards in a first lateral direction; the second camera sub-unit facing outwards in a second lateral direction (e.g. different from the first lateral direction, e.g. in the same horizontal plane) whereby at least the first and second camera subunits capture images in different lateral directions.

[0181] The method of estimating the biomass of freely movable objects using a camera module as described herein may comprise the following:

[0182] in a first camera sub-unit, capturing at least one 3D image (also known as a depth map) of the first field of view (e.g. substantially or mostly the same field of view as a separate 2D camera, where provided) from a 3D camera;

[0183] in a second camera sub-unit, capturing at least one 3D image (also known as a depth map) of the second field of view (e.g. substantially or mostly the same field of view as a separate 2D camera, where provided) from a 3D camera;detecting objects (e.g. using machine learning tools such neural networks (e.g. Region Proposal Network (RPN), Faster R-CNN, Mask R-CNN and / or using clustering tools such as DBSCAN and / or using corresponding pixels from a corresponding 2D image, where provided) in 3D images in the first and second field of view (e.g. from the 3D camera, which may be a stereo camera or a time-of-flight camera);

[0184] determining if an object is a fish, and if it is a fish, tracking the object in images across a plurality of frames (e.g. individual images) in one camera sub-unit, or more preferably across two or more camera sub-unit;

[0185] determining a (e.g. at least one) dimension of a tracked object in a plurality of frames in the images;

[0186] estimating a property and / or condition (e.g. mass and / or grade classification) of the tracked object from the determined dimension.

[0187] In one or more embodiments, two or more or each camera sub-unit(s) may face(s) outwards about a common periphery and / or about a common centre, preferably two or more or each camera sub-units may be rigidly held with respect to one another and / or with respect to a main body (e.g. a housing).

[0188] In one or more embodiments, the on-site or first camera module may comprise in the first camera sub-unit at least one 2D camera; and, in the second camera sub-unit, at least one 2D camera; at least the first and second camera sub-units being configured so that the 2D and 3D cameras of that respective camera sub-unit have overlapping fields of view (preferably, generally or substantially overlapping fields of view in one or more directions e.g. horizontally and / or vertically) from that camera module (e.g. preferably from a substantially lateral perspective, in other words, in use, in a substantially horizontal direction about a substantially vertical axis).

[0189] In one or more embodiments, the 2D camera in at least one camera sub-unit may be separate from the 3D camera in that (e.g. the at least one) camera sub-unit. In some preferred embodiments, a separate 2D camera is provided in some or all camera sub-unit. In other embodiments, one of the 2D cameras of a 2x2D stereo (3D) camera may be used, in which case this is not a separate 2D camera, forming instead part of the stereo 3D camera.

[0190] In one or more embodiments the on-site or first camera system may comprise two, or three, or four, or five, or six camera sub-units, each camera sub-unit configured for immersion underwater in a pen and having at least one 2D camera and at least one 3D camera;

[0191] each camera sub-unit preferably being configured so that the 2D and 3D cameras of that camera sub-unit have overlapping fields of view (e.g. in at least one direction or range of directions e.g. horizontal and / or vertical, e.g. from that camera module) (preferably, generally or substantiallyoverlapping fields of viewe.g. preferably from a substantially lateral perspective in other words, in use, in a substantially horizontal direction about a substantially vertical axis);

[0192] each camera sub-unit preferably facing outwards (e.g. in a lateral direction at preferably 90° approximately to a main plane of a front face of a camera module) in a respective lateral direction (e.g. in or about the same horizontal plane) whereby each of the two, or three, or four, or five, or six camera modules capture images in different lateral directions.

[0193] In one or more embodiments, two or more or each of the camera sub-unit(s)face outwards about a common periphery and / or about a common centre.

[0194] In some embodiments a computer program and / or computer readable medium with program instructions for implementing the method of any aspect of the invention may be provided.

[0195] Several embodiments of the invention are described and anyone or more features of any one or more embodiments may be used in any one or more aspects of the invention as described above and elsewhere herein.

[0196] BRIEF DESCRIPTION OF THE DRAWINGS

[0197] The invention will now be described in more detail, without limitation, with reference to Figures illustrating example embodiments in which like reference numerals refer to like features.

[0198] Figure 1 shows a schematic overview of a system for determining (e.g. estimating) one or more parameter(s) (e.g. one or more properties such as a dimension, mass etc. and / or one or more condition(s) such as a classification, e.g. grade (downgrade, ordinary grade and superior grade) and / or age (immature, entered maturation, mature)) of (e.g. freely) movable objects such as animals e.g. aquatic animals such as fish or crustaceans.

[0199] Figure 2 shows an overview of a method of determining one or more parameter(s) (e.g. one or more properties and / or one or more conditions) of (e.g. freely) movable objects such as animals, e.g. aquatic animals such as fish or crustaceans, which can be used with the system of Figure 1.

[0200] Figures 3A, 3B, and 3C show detailed steps for use in the system of Figure 1 and / or the method of Figure 2.

[0201] Figures 4A, 4B, and 4C show the distribution of example data representing the number of objects versus mass.

[0202] Figures 5A, 5B, and 5C show, respectively, in Figure 5A a boundary box locating a freely movable fish in an image frame from an on-site camera module in an on-site pen when underwater, in Figure 5B a segmented image of the fish identified by spot pattern in Figure 5A, inFigure 5C the segmented image of Figure 5B with an additional segmented region identifying the location of lice.

[0203] Figure 6 shows a plan view of an imaging surface (e.g. a conveyor in air) on which fish are transported (e.g. once stunned) in a harvesting line apparatus.

[0204] Figure 7 shows an alternative way of displaying mass data for a batch (e.g. the contents of a pen or group of pens) offish, here as a mass (% of total mass) versus number offish of that mass.

[0205] DETAILED DESCRIPTION OF THE INVENTION

[0206] It will be understood by those skilled in the art that any dimensions and relative orientations such as lower and higher, above and below, and any directions, such as vertical, horizontal, upper, lower, axial, radial, longitudinal, tangential, etc., and any angles etc. referred to in this application, are within expected structural tolerances and limits for the technical field, and for the apparatus and methods described, and these should be interpreted with this in mind.

[0207] A stun station in a stunner may be designed and / or operated to render an (e.g. aquatic) animal unconscious and / or to kill the animal, as would be understood by someone skilled in the art.

[0208] In this application the term ‘pen’ has its ordinary meaning as an enclosure or area of confinement for an animal. Thus, an underwater pen is an area of confinement for an aquatic animal for aquaculture, research or the like. Included in the definition of pen is net, cage, pond and it therefore means any enclosed volume having an portion for containing animals, and it includes for aquatic applications an underwater portion for containing aquatic animals such as fish. The term pen includes land-based enclosures for land-based animals and land-based fish tanks for aquaculture research and the like and open net pens in the ocean, rivers, lakes and so on. Open net pens allow the water to flow from the surroundings into and out of the open net pen.

[0209] In this document reference is made to fish but it will be understood that the invention applies to other freely movable objects such as land-based animals or marine based animals, such as fish, and / or tethered movable objects such as land-based crops or marine based crops such as seaweed, or molluscs on a rope.

[0210] In this application, reference will be made to a parameter of a movable object such as an animal, e.g. an aquatic animal, such as a fish. By parameter is meant a property of such an object or such a group of objects, such as a length (e.g. between two key points on the object) or mass, or average length or mass, or median length or mass. In other words, a property may be a quantity, typically an absolute quantity associated with an object or group of objects. Aparameter may also be a condition, such as the maturation and / or classification grade of an object and / or group of objects. Although assessment of a condition may be more subjective, quantities such as the dimensions orthe properties associated with an object such as the number or area of lice and / or number or area of wounds may contribute to the assessment of the conditions such as classification grade.

[0211] In aquaculture it is well known to establish key points on an outline of an image of a fish or other aquatic animal and use the separation of such key points to estimate dimensions such as length, and so volume, and from which to derive an estimate of mass. For example, the distance between the nose to the tail, and / or from the upper fin to the lower fin, may be established based on key points (particular locations) identified in an image, e.g. from one or more image frames, through image analysis. Whilst these techniques are more easily applied to 2D images, these may also be applied to 3D images. As such, techniques are well known to those skilled in the art, these will not be repeated here.

[0212] The present inventor has appreciated that, whilst PCT / EP2024 / 077150 PYNE CARTER describes improved systems and techniques for providing improved observations and / or measurements of aquatic animals such as fish at harvest, e.g. for measurement of biomass at harvest, this does not address the practical challenges of providing insights into the earlier animal farming process itself. These practical challenges include the monitoring of aquatic animals for size and health conditions prior to harvest, whilst such animals can move, alter their pose, and occlude one another, as well as being subject to on-site conditions pre-harvest such as poor light, poor visibility e.g. due to turbidity, sparkle etc. in an (e.g. aquatic) pen.

[0213] One of the objects of the present invention is to provide improved predictive condition monitoring and / or improved predictive classification into grades of movable objects such as aquatic animals which can be harvested, , for example fish. Thus, providing an improved prediction of the condition and / or classification of such objects before harvesting is useful facilitating the taking of one or more actions to improve the condition and / or classification and / or to indicate relevant market for advance sales. In practice, in aquaculture, in which the welfare of animals raised in water is of considerable importance, this means to improve the prediction of conditions and / or classifications, and / or of parameters such as dimensions on which such predictions are based, of aquatic animals such as fish, from images taken in water. Therefore, an intermediate aim of the present invention is howto improve the determination of one or more parameters such as a condition and / or properties such dimension(s) from underwater images more accurately and / or earlier in the farming process.Currently a small sample of harvested fish are weighed individually to verify the results of ML (machine learning) model(s) (known as machine vision models) used to determine such parameters from underwater images. The difficulty of using such limited data is that this does not provide a sufficiently large and / or sufficiently well-labelled training data set for improving the ML model. Therefore, a further intermediate aim of the present invention is to provide an improved and / or sufficiently large-labelled training data set for training and / or re-training an ML model used to extract data from underwater images.

[0214] In the following description, as an example of objects freely moveable with respect to one another, fish will be used but it is to be understood that the present invention can be applied to other situations in which movable objects can move with respect to one another prior to harvest and which are therefore difficult to accurately measure. For example, the present invention may be applicable to land-based agriculture of crops and / or animals as well as underwater based aquaculture of crops and / or animals.

[0215] In the following description the term ‘a data element’ is used to indicate a value reflective of a physical characteristic of a particular fish. Thus, a data element may be the xy or xyz coordinates in a 2D or 3D image frame (e.g. where z is distance from the camera) of one or more key points on an outline of a fish such as the snout and the start of the tail, fin. In addition or alternatively, a data element may be an intensity or colour of a particular pixel in an image frame or the result of a calculation, for example a dimension such as a length between the coordinates of two key points. Indeed, data elements may comprise any number of things associated with a fish for example, the number or area of wounds and / or lice associated with a particular fish. Thus, the term data element is to be interpreted broadly. Whilst one data element may be used to indicate a parameter of a fish, more typically two or more, more usually several data elements are used.

[0216] Referring now to Figure 1 , a system for determining parameters such as a property, for example a dimension e.g. a height, length etc. and / or a condition such as classification, of a fish is shown. The system 10 for determining the parameter is associated with an on-site pen 20 in a fish farm and an off-site harvesting apparatus 30. System 10 comprises an on-site camera 42, movably mounted within pen 20, here on one or more winch cables located about a periphery of a pen 20. On-site camera 42 is mounted on a rotatable camera mount (not shown) which can rotate about both vertical and horizontal axes (see arrows) so that the field of view captured by on-site camera 42 can be varied as well as its horizontal and vertical position within pen 20. Harvesting apparatus 30 includes a stun station 31, here an inline underwater stun station and, following a de-watering station, imaging platform 32 on which stunned fish pass into the field of view of harvest camera 52, here located above the platform 32.Whilst reference will now be made only to harvesting in this description, more accurately camera 52 is an off-site camera and thus harvesting apparatus 30 may indeed be a treatment or inspection line which temporarily stuns the fish for treatment or inspection and then returns these to pen 20. Alternatively, and as will de described below, apparatus 30 may be a harvesting line which includes one more additional stage post-stun (not shown), optionally preimaging station with camera 52, to facilitate death of the fish in a humane manner.

[0217] It will be noted that, while on-site camera 42 is preferably movable (horizontally and / or vertically and / or rotationally) within pen 20 to achieve improved imaging, typically off-site harvest camera 52 is relatively fixedly located with respect to imaging platform 32 on which fish travel.

[0218] Each camera is provided with a camera controller 44, 54 and an associated ML (machine learning) model 46, 56. Together the on-site camera 42, on-site camera controller 44 and on-site ML model 46 form an on-site camera module 40. Similarly, off-site harvest camera 52, harvest camera controller 54 and harvest ML model 56 form an off-site harvest camera module 50. Whilst use of an off-site ML model 56 is preferred and will be described below, it will be understood that direct measurement of data elements from the one or more image frame(s) collected by the off-site camera 52 may be possible to provide off-site data (element(s) for use on one or more embodiments of the invention.

[0219] Whilst it is essential that on-site camera 42 and off-site harvest camera 52 are in their respective on-site and off-site locations, the camera controllers 44, 54 and the ML models 46, 56 may be centrally located e.g. within a control module 100. Their exact location may be determined by the data transfer and / or data storage requirement for the various imaging activities.

[0220] Camera modules 40 and 50 deliver, respectively, on-site data output 60 and offsite (e.g. harvest) data output 70 which is also referred to as ground truth data 70. Control module 100 uses the on-site data output 60 and harvest data output 70 as input to a comparison module 90, typically within control module 100, for comparing one or more of the on-site versus off-site data element(s) and / or, at a higher level, on-site versus off-site maturation and / or classification determinations from on-site ML model 46 and off-site algorithm(s) such as an ML model 56, may be used. Thus, the comparison module 90 can verify data elements and / or, at a higher level, parameters such as maturation and / or grade classification immediately. Control module 100 comprises a microcontroller 80 having a feedback module 95 and a comparison module 90. Microcontroller 80 is also able to recommend actions as well as verify classifications.

[0221] Suitable storage 92 for a database comprising image frames and / or results of analysis of images and / or identification tags and / or parameters as well as label data and / orinstructions is provided. Storage 92 may also comprise computer program instructions for microcontroller 80.

[0222] Thus, system 10 comprises different camera modules 42 and 52 which can take images of the same set of objects in different situations, here one underwater in a pen 20 where the fish can move freely relative to one another and imaging is more difficult and, secondly, a harvest camera 52 on a harvesting line 30 in which the fish are stationary, or move in a predictable fashion along a conveyer, with respect to camera 52 and are in air, so imaging is less problematic. By combining the data from both cameras in such differing imaging situations, various efficiencies and improvements in accuracy of image data earlier in the farming process is provided, as will be described below.

[0223] Figure 2 shows a method 105 of determining (e.g. estimating) a parameter (e.g. a property and / or condition) of movable objects, here fish, on-site. In step 110 on-site camera 42 captures one or more image frames (2D or 3D, black and white or colour) of one or more fish which are analysed e.g. by camera controller 44, or more usually by ML (machine learning) model 46, to identify the individual object and allocate an identification tag to it. For example, spot pattern of a fish may be used to uniquely identify it. Next, in step 120, on-site ML model 46 determines a property or condition of the object from the one or more image frames delivered by on-site camera 42. Whilst one image frame may be used, multiple e.g. average or the sum of several image frames may be used. On-site camera 42 may be a 2D or a 3D camera but is preferably at least 2D as 2D imaging underwater is easier.

[0224] In step 125, optionally, further image frames are delivered to the on-site ML model 46 from on-site camera 42 over time to track the evolution of the parameter of the same identified object on-site in pen 20. In step 130, the ID tag and associated property or condition of the same objects are stored, optionally as these evolve overtime, e.g. in storage 92. In step 140, ground truth data is received e.g. from harvest camera module 50. Thus, at a later point in time a set of one or more fish which have been previously identified in on-site camera 42 are delivered to offsite apparatus 30, here a harvesting line with a, here, harvest camera 52. 2D or 3D images are captured by off-site (e.g. harvest) camera 52, usually in more favourable imaging conditions e.g. in air and / or with immobile subjects (here fish).

[0225] Whilst direct measurement from image frame may be carried out, preferably image frames from camera 52 are delivered to the off-site (harvest) ML model 56 for analysis. Images obtained from off-site camera 52 are typically clearer and easier to analyse and thus represent, relative to images from on-site camera 42, ground truth data. Thus, the off-site data output 70 comprises the determinations of one or more parameter(s) associated with the imageframes and, in particular, the object in the image frames. These determinations may be directly measured from image frames or may be derived from off-site ML model 56. Thus off-site image frames, which are typically taken in air, provide more accurate data determination of the same parameter (e.g. property and / or condition) of the same object as was previously observed in pen 20.

[0226] Next, microcontroller 80 and, in particular, comparison module 90 and feedback module 95 can be used to compare, and / or indeed verify, the accuracy of the on-site ML model 46. This is especially the case, if the set of images taken by on-site camera 42 and those taken by off-site camera 52 are not spaced too far in time, thus the fish will not have changed in size and / or condition by any significant amount. Thus, the data representing the physical characteristics of the identified fish in off-site camera 52 can be used to label the poorer quality images frames taken by the on-site camera 42. In this way, on-site ML model 46 can be provided with an updated labelled data set to retrain on its original images which have been newly labelled with more accurate data as to the parameters concerned, such as the dimensions or other properties and / or condition of the objects as seen by harvest camera 52, optionally as analysed by harvest ML model 56. Thus, in step 150, on-site ML model 46 is revised using the ground truth data output from harvest camera module 50.

[0227] Figures 3A, 3B, and 3C show this process in more detail. In Figure 3A, method 200 comprises a step 210 of detecting an object in one or more on-site image frames. In step 220, one or more image frames are analysed to segment the individual fish. In step 225, the segmented object is analysed to locate key points in the segmented image such as the snout and the tail. In step 230, optionally and indeed preferably, each individual object is uniquely identified, for example using analysis of a spot pattern somewhat like a barcode. Each individual object is then allocated a unique identity tag with which its data can be tracked through the system. In step 235, optionally, the image frames may be further analysed to segment one or more abnormalities in onsite images, for example, the presence and / or area of lice and / or wounds which extend over the surface area of the fish. In step 245, one or more parameters such as length, height, thickness, volume, mass, presence and / or area of abnormalities of an object is measured or determined from one or more image frames and the resulting one or more data element(s) are stored along with the unique ID tag.

[0228] It will, of course, be understood that groups of fish may be analysed so that an average parameter such as average length from nose to tail or average number of lice or average area of abnormalities may be associated with a group of fish and, thus, any later comparisons or provision of ground truth data may be carried out with respect to the same group offish, ratherthan with respect to individual fish. Thus, the data associated with fish from, for example, one pen may be compared in both on-site and harvest camera modules to produce an average comparison rather than an individual comparison. This reduces the quantity of data to be analyzed, transferred and stored.

[0229] In step 255, one or more conditions of the fish e.g. maturation and / or classification such as superior, ordinary, downgrade, are measured or determined. Preferably this data is stored with the ID tag (for an individualor a set offish). Preferably, both steps 245 and 255 are carried out but in one or more embodiments only one of these may be carried out and the other may be optional. This may reduce the amount of data that is required to be analysed, particularly if only one condition of the object e.g. grade classification is stored with the ID tag. Thus, whilst it is preferred if both steps 245 and 255 are carried out, in some example embodiments only one is carried out.

[0230] In step 265, optionally one or more improvement actions such as feeding or treating fish, may be recommended by microcontroller 80. Alternatively or in addition, one or more on-site camera actions may be recommended by the microcontroller, for example, moving to a new location, length of image capture, rotating its orientation about a vertical or horizontal axis, taking more or fewer images, changing focal length etc.

[0231] Process 200 continues in Figure 3B in step 270, an object is detected in an off-site (e.g. inspection or harvest) image frame(s). In step 280, individual objects such as fish are segmented in the captured image frames. In step 290, individual objects are identified e.g. using spot pattern analysis to determine the relevant ID tag and associated ID tag record. The ID tag record can be examined to establish the features which have previously been measured. In step 300, one or more of the image frames captured off-site is analysed e.g. by camera controller 54, or, more preferably, by ML model 56, to determine the same or a selected number of features, e.g. key points, abnormalities, e.g. lice and / or wounds and measurements of properties such as dimensions for example height, width, thickness, volume, weight as well as condition such as maturation and / or classification.

[0232] In step 310, comparison is made of the same features between the on-site data and the harvest data, typically by comparison module 90 within micro controller 100.

[0233] In step 320, the tag record may be updated with the additional data in the database kept in the storage 92. In step 330, the updated tag record with the additional data is used to verify and / or re-train the on-site machine learning model 46. For example, ground truth data in the form of original on-site image frame newly labelled with updated measurement data from the off-site imaging process, may be provided to the on-site ML model 46.In step 340, the classification label associated with the ID tag may be updated to use the new classification derived from the off-site ML model 56. In step 345, updated classification and / or other information, for example quality score information, may be stored and optionally displayed.

[0234] Figures 4A, 4B, and 4C show a histogram distribution of the determined biomass of a set of objects, here the fish population of a single pen. Histogram 48 shows the determined mass distribution using image frames captured on-site and an on-site ML model at a first point in time, say on Monday morning. Histogram 58 shows the mass distribution offish a week later following an intervention in the form of an action taken to provide additional feed. The change in average mass is +10%. Histogram 68 shows the change in the number of objects within various mass categories between the on-site measurements 48 and the measurements determined in the harvest camera module 50 a week later. Histogram 68 shows that the number of objects with lower mass has decreased whereas the number of objects with higher mass has increased. In this case, on-site ML model 46 was previously adjusted to more accurately predict the biomass of the fish, so the change in average mass of +10% in a week represents a more accurate reflection of the change in biomass in the pen.

[0235] Of course, similar data may have been collected with minimal time difference (say half a day) between on-site observations and off-site data collection at harvest, which data can then be used to provide a newly labeled training data set to update the on-site ML model 46. In such a case, the ML model 46 has underestimated the mass of the fish in the pen by 10% and can thus be commensurately adjusted. The now adjusted ML model may then be used pre harvest for on-site biomass measurements of e.g. neighbouring pens on the same farm, or for similar species of fish on the same or differing farms.

[0236] Figure 5A shows a fish 400 in an on-site image frame 410 taken offish 400 whilst moving underwater by on-site camera 42. Image frame 410 has been divided into boundary boxes 412 within which individual fish can be seen. Identifying boundary boxes of objects in image frames is well known and any suitable technique may be used, for example, colour and / or intensity analysis. Here, an identifying spot pattern 414 of fish 400 can be seen. Further, a louse 416 located on a lower portion offish 400 is also seen.

[0237] In Figure 5B, a segmented outline 418 offish 400 has been determined by any suitable technique, for example colour and / or intensity matching to determine key points and draw lines between these, as would be understood by someone skilled in the art. Identifying spot pattern 414 can also be seen. Thus, image frames 410 can be analysed to identify the outline and so area and / or length or other dimension of a fish.In Figure 5C, an outline 420 of louse 416 has been identified again by any suitable segmentation techniques, as would be understood by someone skilled in the art. Thus, in Figures 5B and 5C, the outline 418 is defined as the line between various key points such as an upper fin, a snout, a lower fin and a tail fin, again as well understood in the art.

[0238] Figure 6 shows fish 500 in inspection or harvesting apparatus 30. Here, the fish 500 are on conveyor or platform 32 having been stunned in a stun station 31. The fish 500 are now in air, and noware rendered immobile of themselves, and imaging via off-site (here harvest) camera 52 is capable of more accuracy. Thus, the harvest image frame 510 can be analysed to identify a smoother fish outline 518 e.g. using segmentation and shape buildingas well understood in the art of object recognition. Since the outline 518 more accurately represents the shape of the fish, the data determined from analysis of the outline will be more accurate, for example the length and / or width and / or determined biomass of each fish. Indeed, both 2D and 3D imaging may be used to determine the volume of the fish and so its mass, especially in this off-site camera module. The identifying spot pattern 514 is determined by the harvest camera module 50 and the data derived from analysis of image frames 510 is allocated to the tag record of the fish in the database.

[0239] Figure 7 shows a further example histogram of the mass distribution, here as a percentage of the total mass of a set (here a batch from a single pen) offish. Thus, the median mass is 20.64.

[0240] In one or more embodiments, each set offish can be classified according to the proportion of individuals classified as a particular grade within that group. So, for example, if 70% of the fish within a particular group e.g. a set offish from a particular group of pens ora particular fish farm are classified as superior grade and the remainder as ordinary grade, the entire set may be classified as superior grade. Alternatively, if 10% of a set offish analysed by the system of the invention are determined to be of the low grade and 80% are of ordinary grade, the entire batch may be classified as of ordinary grade.

[0241] By using an improved on-site ML model 46 which has been verified against multiple, typically hundreds or thousands of parameters determined in an off-site (e.g. harvest or inspection) line 30 by on off-site camera module 50, for example from that particular fish farm or that particular pen, more accurate prediction of the classification may be provided earlier for future sets of fish from the same fish farm or pen within a fish farm where the improved ML model 46 is used. Thus, in one or more embodiments, the invention provides earlier and / or better determination of one or more parameters and / or conditions e.g. grade classification offish andbetter recommendation for action which may be taken prior to harvest. This represents a significant improvement in management offish prior to harvest.

[0242] Fish farmers typically categorise their fish into three classes: superior, ordinary and downgrades. This invention proposes a feedback loop between two Al (e.g. ML) camera modules: an on-site underwater biomass camera, and an off-site in-air (e.g. inspection or at harvest) biomass camera. The feedback loop allows farmers to understand what quantity of fish they are likely to have in each of the three classification buckets before the fish reach the harvest station, and to verify those quantities in each classification bucket as the fish reach the harvest station (typically around 24 hours before final processing data is gained). This allows more efficient allocation to sales channels earlier to correctly sell what they have in advance of fish entering packaging. This makes for more efficient processing and distribution and reduces waste.

[0243] The off-site (e.g. harvest) camera module takes many thousands of images offish, detecting length, height, mass, and in doing so captures huge amounts of ground truth data about the measurements offish. Using this huge data stream the mass detection MLalgorithm(s) (e.g. in the on-site camera and / or in the harvest camera itself can be honed and improved, allowing the underwater on-site biomass camera to determine biomass weights and other parameters, such as classification, more accurately.

[0244] Fish farmers typically categorize harvested fish into ordinary, superior, and downgraded (or reject) grades based on a combination of physical quality, health status, and market requirements. Here’s a breakdown of how these categories are generally defined:

[0245] Superior Grade (Premium / Grade A): These are the best-quality fish, meeting all ideal conditions for export or high-end markets.

[0246] Criteria:

[0247] • Uniform size and weight (within optimal range)

[0248] • No physical damage (no wounds, deformities, or fin damage)

[0249] • Bright skin color and good slime layer (indicative of health)

[0250] • Clear eyes and firm body

[0251] • Active behavior prior to harvest (for live markets)

[0252] • No disease or parasites

[0253] Used for:

[0254] High-end retailers, sushi / sashimi markets, premium export orders.Ordinary Grade (Standard / Grade B): These fish are healthy and suitable for consumption but may have minor imperfections.

[0255] Criteria:

[0256] • Acceptable size but possibly some variation

[0257] • Minor handling damage (small scratches, slight bruising)

[0258] • Slight discoloration or scale loss

[0259] • Mild signs of stress but still healthy

[0260] Used for:

[0261] Mass market retail, frozen fillets, cooked / processed products.

[0262] Downgrade / Rejects (Grade C / Trash Fish): Fish that don’t meet the minimum standards due to health, size, or appearance issues.

[0263] Criteria:

[0264] • Underweight or stunted growth

[0265] • Visible deformities (e.g., bent spines, jaw deformities)

[0266] • Severe injuries or disease signs

[0267] • Parasite infestations

[0268] • Excessive scale loss or skin damage

[0269] Used for:

[0270] Petfood, fishmeal, compost — or discarded depending on regulations.

[0271] In one or more embodiments, this invention allows farmers to assess the three classes initially on-site (and later harvest) via machine learning (ML) algorithms by assessing parameters such as wounds, fin damage, maturation, and lice. By assessing the severity of the issues, the ML algorithms in the ML models determine which of the three classes the fish would be placed in once harvested. Actions may be recommended to the farmer (e.g. feed that is good to heal wounds, or lice removal treatment), and performance improvement can be measured over time by repeating the assessment of which of the three classes the fish would be placed in. Once taken to harvest the harvest camera module and its associated ML module performs the same checks, and validates the actualfigures. The measurements in the harvest station provide ground truth data about one or more parameters e.g. the length, height, weight and classification offish, and allows the underwater camera ML algorithms to be validated and honed to provide a revised (e.g. re-trained) ML model. The same three classes can then be assessed earlier, for example 24 hours before the fish go to packaging.In one example, the previously gathered on-site data can be re-analysed by the revised on-site ML model. In another example, new on-site data (e.g. for a different group e.g. batch) of fish in the same or a different pen can be analysed for the first time by the revised on-site ML model.

[0272] One issue facing farmers is that they do not know what they are selling until it is in packaging, resulting in waste if a buyerforthat quality class is not found. Also, biomass on-site camera operators need to ground truth their weight algorithms, and to do this they currently hand weigh fish and adjust their on-site algorithms based on measured length and height and confirmed weights. The proposed approach provides much higher quantities of ground truth data (in other words much higher quantities of (real life) training data) which is typically of higher quality (e.g. accuracy) which can be fed into the underwater on-site camera algorithm to revise (or re-train) the on-site ML model. The harvest camera data and associated determined parameters also provide a second check on quality of the fish before packaging.

[0273] In preferred embodiments, a feedback loop between data from a harvest camera module and an underwater camera module is provided. This improves class offish identification and prediction. Thus, for fish still in pen(s), an improved predictive classification is provided (as feed forward data) at an earlier stage in the process. This allows farmers to anticipate what market and / or price may be appropriate for fish well in advance of the harvest and packaging point. Farmers may also adjust feed or handling etc. to improve the quality issues.

[0274] In practice, a harvest ML camera module is installed after a stunner where it can see the fish post-stun and typically also post-bleeding. An on-site ML camera module 40 is installed in the cages e.g. over the preceding two-year grow out period. The control module 100 takes data from the two camera modules and uses spot pattern detection to identify the same fish from grow out period to the harvest station. The control module 100 typically also controls and provides interaction with a user interface in the form of a portal.

[0275] The process followed by respective camera module(s) is typically as follows: 1. Fish detected by on-site Al camera module (e.g. within ML module)

[0276] 2. Fish segmented and key points tagged.

[0277] 3. Fish wounds segmented.

[0278] 4. Maturation detected via length of jaw to fin.

[0279] 5. Lice detected with segmentation

[0280] 6. Spot pattern identified and checked with vectors in database. Individual tagged as new or same fish.

[0281] 7. Fish allocated tag of ordinary, superior, downgrade,8. Advice for improvements flagged (e.g. change feed due to high wounds) 9. Off-site Al camera module detects fish in harvest station.

[0282] 10. Prior knowledge offish origin flagged in portal (control module)

[0283] 11. Off-site ML detection algorithm detects same features as above for on-site underwater camera.

[0284] 12. Fish ID updated with superior, ordinary, downgrade.

[0285] 13. On-site ML algorithm updated.

[0286] It will be understood that one or more image frames from still images or moving video may be used e.g. averaged or combined to produce clearer image frames. Information regarding such use of image frames, e.g. combinations thereof, may be used as an input to the training of the on-site ML model. Similarly, weather, season, date, time of day, water turbidity, time since feeding etc. location (in xyz) of on-site camera and / or orientation of an on-site camera may be used as inputs in training and / or re-training the on-site ML algorithm / model 46. The (re-) training cycle maybe repeated intermittently every few months or continuously (quasi-continuously) e.g. after every set of objects have been processed, to provide continuous improvement.

[0287] Whilst the invention finds particular application to aquatic animals such as fish, it may also be applicable to land-based animals. Indeed, it may also be applicable to any harvestable crop (e.g. underwater - such as kelp) and in-air such as grain.

[0288] Whilst the term ‘on-site’ is used to indicate apparatus and software associated with on-site imaging, the software modules such as the camera controller 44, 54 and ML model 46, 56 need not be located on-site but may be within the control module 100 , which may be at any convenient location.

[0289] The invention has been described with reference to a preferred embodiment. However, it will be appreciated that variations and modifications can be effected by a person of ordinary skill in the artwithout departing from the scope of the invention.

[0290] The present application refers to the following co-pending patent applications PCT / EP2024 / 07150 and PCT / EP2024 / 083761 filed by the present applicant and includes the content thereof by reference, particularly in reference to those details referred to explicitly int his present application.

[0291] LIST OF REFERENCE NUMERALS

[0292] 10 system for determining a property and / or condition of freely movable object(s) e.g. aquatic animals such as fish20 pen e.g. on-site enclosure for (freely) movable objects

[0293] 30 off-site (e.g. harvesting or inspection) apparatus e.g. a harvesting or inspection line for stunning freely movable objects for harvest and / or treatment and / or inspection, may be close to the on-site location, and typically provides different imaging conditions.

[0294] 31 stun station

[0295] 32 imaging platform (e.g. conveyor e.g. in air)

[0296] 40 on-site camera module

[0297] 42 on-site camera, 2D, or 3D, or 2D and 3D

[0298] 44 on-site camera controller

[0299] 46 machine learning (ML) model for on-site camera

[0300] 48 determined on-site mass distribution of on-site pen population

[0301] 50 off-site (e.g. harvest) camera module

[0302] 52 off-site (e.g. harvest) camera

[0303] 54 off-site (e.g. harvest) camera controller

[0304] 56 machine learning (ML) model for off-site (e.g. harvest) camera

[0305] 58 determined harvest (or inspection) mass distribution of same pen population when harvested in harvesting apparatus 30

[0306] 68 relative change in numbers of mass distribution between on-site and off-site (e.g. harvest) for example, after time elapses and / or action taken (e.g. feeding)).

[0307] 70 off-site (e.g. harvest) data output (data stream)

[0308] 80 Controller e.g. microcontroller

[0309] 90 comparison and control module

[0310] 92 storage for database and / or instructions

[0311] 94 display

[0312] 95 Feedback module

[0313] 100 Control Module

[0314] 105 a method of determining (e.g. estimating) a parameter (e.g. property and / or condition) of movable objects e.g. freely movable objects such as aquatic animals e.g. fish

[0315] 110 identify object(s)

[0316] 120 estimate, preferably determine parameter (e.g. property and / or condition)

[0317] 125 track evolution of parameter (e.g. property and / or condition)

[0318] 130 Store ID TAG and parameter (e.g. property and / or condition)

[0319] 140 Receive ground truth data from an alternative data source e.g. an off-site data output of an offsite camera module

[0320] 150 Update on-site machine learning

[0321] 200 Method of determining (e.g. estimating) a parameter of movable objects illustrating detailed steps

[0322] 210 detect object in on-site image frame(s)

[0323] 220 segment individual object in on-site image frame(s)

[0324] 225 optionally locate key points

[0325] 230 preferably identify individual objects using identifying features

[0326] 235 optionally segment abnormality(ies)

[0327] 245 measure (one or more) parameter(s)

[0328] 255 determine (one or more condition(s)

[0329] 265 optionally recommend action

[0330] 270 detect object(s) in harvest image frame(s)

[0331] 280 segment individual object(s) in image frame(s)

[0332] 290 identify individual object(s) and determine its ID tag

[0333] 300 determine one or more of same parameter(s) e.g. property(ies) and / or condition(s) as for onsite

[0334] 310 compare one or more of same features320 update ID tag record in the database with the additional data (from step 300)

[0335] 330 use one or more updated ID tag record(s) with the additional data to verify and / or re-train the on-site ML model

[0336] 340 update a classification label associated with the ID tag record

[0337] 345 store, optionally display, updated classification with the ID tag record

[0338] 400 fish

[0339] 410 on-site image frame (e.g. taken of moving fish underwater)

[0340] 412 on-site boundary box

[0341] 414 identifying spot pattern

[0342] 416 lice

[0343] 418 fish outline (e.g. rough fish outline) e.g. defined by lines between key points

[0344] 420 segmented lice outline

[0345] 500 fish

[0346] 510 harvest image frame

[0347] 514 identifying spot pattern

[0348] 518 fish outline (e.g. more well defined and accurate fish outline, e.g. identified by segmenting image taken in air)

Claims

CLAIMS:

1. A method of monitoring a parameter of one or more movable objects, the method comprising:i) in an on-site camera module which comprises an on-site camera and an on-site ML (machine learning) model,determining, using the on-site ML model, one or more first data element(s) representative of one or more first parameter(s) of a first set of one or more movable object(s) in one or more on-site image frame(s) taken by the on-site camera;ii) in an off-site camera module which comprises an off-site camera, determining one or more second data element(s) representative of one or more second parameter(s) of the first set of one or more movable object(s) in one or more image frame(s) taken by the off-site camera;comparing, or otherwise evaluating, the one or more first data element(s) with respect to the one or more second data element(s);adjustingthe on-site ML model using one or more result(s) of the comparison, or of the evaluation.

2. A method according to claim 1 in which the off-site camera module comprises an off-site ML model comprising a trained ML algorithm, and the method comprises:determining, using the off-site ML model, the one or more second data element(s) representative of the one or more second parameter(s) of the first set of one or more movable object(s) in one or more image frame(s) taken by the off-site camera.

3. A method accordingto claim 1 or 2 comprising repeatingthe step of adjustingthe on-site ML model.

4. A method according to any preceding claim:- using the adjusted on-site ML model to determine one or more parameters associated with a first or second set of one or more movable object(s) in the one or more image frame(s) taken by the onsite camera.

5. A method accordingto claim 4 in which the step of using the adjusted on-site ML model comprises determining, using the adjusted on-site ML model, one or more first data element(s) representative of one or more first parameters of the first, or a second, set of one or more movable object(s) in one or more image frame(s) taken by the on-site camera.- 38-6. A method according to any preceding claim, when dependent on claim 2, comprising: - adjusting the off-site ML module usingthe one or more result(s) of the comparison or of the evaluation.

7. A method according to any preceding claim, when dependent on claim 2, in which the onsite ML model is different to the off-site ML model.

8. A method according to any preceding claim in which the first set of one or more movable object(s) are non-predictably movable with respect to the on-site camera.

9. A method according to any preceding claim in which the first set of one or more objects are predictably movable, or static, with respect to the off-site camera.

10. A method according to any preceding claim comprising: capturing the one or more image frames used in determiningthe one or more first data element(s) when at least one of a) the first set of movable objects, and b) the on-site camera are underwater.

11. A method according to claim 10 comprising:de-watering the first, and / or any later, sets of one or more movable object(s) prior to capturing the one or more image frame(s) in the off-site camera.

12. A method according to any preceding claim comprisingStunning the first set of one or more movable object(s) prior to capturing of one or more image frame(s) in the off-site camera.13 A method according to any preceding claim in which the one or more image frame(s) used for determiningthe one or more second data element(s) are captured when at least one of a) the first set of objects, and b) the off-site camera are in air.

14. A method according to any preceding claim in which the on-site camera is movable, the method comprising:controllably moving the movable on-site camera.- 39-15. Amethod according to any preceding claim comprisingtrainingthe adjusted on-site ML model for a particular type of site.

16. A method according to any preceding claim comprising using the adjusted on-site ML model for a plurality of camera modules in one or more pens containing objects and / or farms containing objects.

17. Amethod according to any preceding claim comprising taking one or more action(s) to improve property and / or or condition and / or level of one or more predetermined parameters.

18. Amethod according to any of claims 1 to 17 comprising identifying a unique individual.

19. A system for determining a parameter of one or more movable object(s),the system comprising:i) an on-site camera module which comprises an on-site camera and an on-site ML (machine learning) model;ii) an off-site camera module which comprises an off-site camera and, optionally, also comprises an off-site ML model;iii) a control module configured to carry out the steps of the method of any of claims 1 to 18.

20. A system according to claim 19 in which at least one off-site camera module comprises an off-site ML model e.g. comprising a trained ML algorithm.

21. A system according to claim 19 or 20 in which at least one off-site camera comprises a 2D and / or 3D camera.

22. Amethod according to any of claims 1 to 18 or a system according to any of claims 19 to 21 in which the parameter comprises one or more properties and / or conditions from the following list(s):a) property: - key point locations- separation of key points- length e.g. nose to fin- width- depth- weight-40-- colour- number and / or area of wounds- number and / or area of liceb) condition: - maturation- grade classification23. A method of determining a parameter of one or more movable objects, comprising:using an ML model trained using the method steps of claims 1 to 19 to determine the parameter prior to harvesting.

24. A method of producing training data for an ML algorithm comprising:i) the method steps of any of claims 1 to 18, and / orii) using a system according to any of claims 19 to 22.

25. A computer-readable medium comprising computer program instructions for the system of any of claims 19 to 22 to carry out the method of any one or more of claims 1 to 18, 23 or 24.

Images