Automated fish processing system and method

Abstract

The method for processing live fish may include determining a fish attribute set for the fish, optionally sorting the fish based on the fish attribute set, euthanizing the fish, bleeding the fish, and / or any other suitable steps. In variations, the method may additionally or alternatively include discarding the fish after sorting, further processing the fish, and / or tracking the fish. However, the method may additionally and / or alternatively include any other suitable elements.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application No. 63 / 446,699, filed February 17, 2023, which is incorporated by reference in its entirety.

[0002] The present invention relates generally to the seafood processing field, and more particularly to a novel and useful system and method for ikijime in the seafood processing field. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 is a schematic diagram of a variation of the method. [Figure 2] FIG. 2 is a schematic diagram of a variation of the system. [Figure 3] FIG. 3 is a schematic diagram of a variation of the system. [Figure 4] Figure 4 is a schematic diagram of a variation of brain localization. [Figure 5] FIG. 5 is an exemplary representation of variations for determining euthanasia trajectories. [Figure 6] FIG. 6 is an exemplary representation of variations for determining euthanasia trajectories. [Figure 7] FIG. 7 is an exemplary representation of variations for determining euthanasia trajectories. [Figure 8] FIG. 8 is an exemplary representation showing variations in fish sorting. [Figure 9] FIG. 9 is a schematic diagram of a variation of the method. [Figure 10] FIG. 10 is an exemplary representation of mass estimation using an imaging system. [Figure 11] FIG. 11 is an exemplary representation of selecting a subset of sample images of fish. [Figure 12]12A, 12B, and 12C are exemplary representations of a variation for determining a bleeding trajectory, a front view showing a bleeding trajectory for a fish, and a top view showing a bleeding trajectory for a fish, respectively. [Figure 13] FIG. 13 is an exemplary representation of a variation for determining a bloodletting trajectory. [Figure 14] FIG. 14 is a schematic diagram of a species-specific variation of the method. [Figure 15] 15A and 15B show variations of the restraint subsystem. [Figure 16] FIG. 16 is an exemplary representation of a variation for determining a set of trajectories. [Figure 17] FIG. 17 is an exemplary representation of a variation for determining a set of trajectories. DETAILED DESCRIPTION OF THE INVENTION

[0004] The following description of embodiments of the invention is not intended to limit the invention to those embodiments, but rather to enable any person skilled in the art to make and use the invention.

[0005] 1. Overview 1 , the method for processing live fish may include step S100 of determining a fish attribute set for the fish, step S150 of optionally sorting the fish based on the fish attribute set, step S200 of euthanizing the fish, step S300 of bleeding the fish, and / or any other suitable steps. In variations, the method may additionally or alternatively include steps of discarding the fish after sorting, step S400 of further processing the fish, and / or step S500 of tracking the fish. However, the method may additionally and / or alternatively include any other suitable elements.

[0006] In an example embodiment, the method may include receiving fish, sampling a first set of images of the fish, using a first set of trained models to identify a set of fish attributes (e.g., size, species, etc.) for the fish based on the images S100, optionally sorting the fish based on the attributes S150, and either processing the fish or discarding the fish (e.g., because the fish species is out of season or the fish is too small).

[0007] In an exemplary embodiment, processing the fish may include some or all of the following steps: sampling a second set of images of the fish (e.g., images encompassing the brain region, images of the restrained fish, etc.); determining a euthanasia trajectory based on the first and / or second set of images (e.g., the euthanasia trajectory is approximately perpendicular to the surface of the fish's head and intersects the fish's brain); controlling a set of euthanasia tools to euthanize the fish based on the euthanasia trajectory S200; confirming euthanasia (e.g., via additional imaging and / or other sensor measurements); sampling a third set of images (e.g., exsanguination images); determining a exsanguination trajectory (e.g., exsanguination parameters, one or more cuts along the length of the fish (e.g., cranial cut, tail cut, etc.), one or more cuts along the height of the fish, one or more cuts along the width of the fish, etc.) for the first, second, and / or third set of images; controlling a set of exsanguination tools to exsanguinate the fish based on the exsanguination trajectory; and further processing the fish. In an optional variation, the method may further include using one or more learned anatomy models to identify the location of one or more anatomical features of the fish based on any set of images, wherein any trajectory may optionally further be planned using the locations output by the anatomy models.

[0008] In an optional variation, the fish and / or final fish containers may be identified and associated with fish attributes, fish measurements, and / or other fish information (e.g., farming information, harvest information, etc.), and any relevant information may be stored in an enterprise resource planning (ERP) system.

[0009] 2.Technical advantages The variations in live fish processing techniques offer several advantages over conventional systems and methods.

[0010] First, fish can exhibit large variations in size, shape, pattern, and other anatomical characteristics, both between species and within the same species. A variation of this technology integrates computer vision into the fish processing pipeline (e.g., analyzing the anatomy and / or attributes of individual fish) to process fish of various species, sizes, and anatomical characteristics with consistent results. This technology can determine processing parameters for precise cuts, blows, and / or other processing. This allows fish to be strangled more quickly, ensuring a more humane death than traditional methods.

[0011] Second, variations of this technique can reduce or prevent stress to fish during processing and before euthanasia, resulting in higher-quality fish products (e.g., meat, skin, organs, bones, tails, etc.). This technique can reduce the time fish release stress hormones, place live fish in a stress-free environment (e.g., dark, humid, swimming against the current), or reduce fish stress. Reducing fish stress levels can reduce suffering before ikijime, reduce or completely prevent the fish from releasing stress hormones that can reduce meat quality (e.g., toughness), or cause the fish to struggle to escape the system (which can inadvertently damage system hardware), and / or provide other benefits. Variations of this technique can employ the techniques, benefits, and / or goals of ikijime, one of the most humane methods of ikijime (killing fish), and / or its variations, which involves euthanizing the fish with one or more sharp blows to the head. While this process requires great precision, it minimizes stress to the fish and improves food quality.

[0012] Third, by automating fish processing, variations of this technology can reduce the need for manual and highly skilled labor, making the overall process safer and increasing processing capacity. Automating ikejime and / or its variations can increase its accuracy and success rate without requiring more fishermen to be trained in ikejime.

[0013] Fourth, variations of this technology could improve compliance in fisheries by performing computer vision analysis to identify the attributes of each fish (e.g., species and size) and culling fish that do not meet a specified set of criteria. However, this technology may offer other relevant benefits.

[0014] 3. System As shown in FIG. 2 , the system can include and / or interface with an imaging system, a processing system, a toolset (e.g., euthanasia tools, exsanguination tools, etc.), a model set, and / or any other suitable components. Preferably, the system and method can include any of the systems, methods, and / or components described in U.S. Provisional Application No. 63 / 451,508, filed March 10, 2023, which is incorporated herein by reference in its entirety. The system (e.g., fish processing equipment) can be located on a vessel (e.g., an ocean-going vessel, fishing vessel, ship, boat, etc.), on land (e.g., at or near a fish farm, in a fish processing plant, etc.), and / or elsewhere. In any variation (e.g., when fish are caught on a vessel), locating the system directly on the vessel has the advantage of allowing live fish to be killed and processed immediately after capture rather than being stored in crowded tanks on the vessel for extended periods until transported to a fish processing facility, thereby reducing stress and other potential injury to the fish. Processing fish directly on board the vessel further saves money and energy (oil, gas, manpower, etc.) by eliminating the costs and energy required for transporting the fish from the vessel to the seafood processing facility.

[0015] Preferably, the system includes and / or interfaces with a single fish processing device (e.g., an integrated fish processing device), where at least fish sorting, euthanasia, and bleeding are performed within the single fish processing device. Additionally or alternatively, one or more further processing steps (e.g., filleting), all method steps, a subset of method steps (e.g., only euthanasia and bleeding), and / or any other suitable steps may be performed within the single fish processing device. Additionally or alternatively, the system can include multiple fish processing devices (e.g., sorting is performed in a separate device from euthanasia and bleeding, euthanasia is performed in a separate device from bleeding, S100-S300 are performed in a separate device from S400, etc.), and fish can be automatically and / or manually transferred between fish processing devices. In variations, each fish processing unit may be configured to process one fish at a time, multiple fish at a time (e.g., 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, more than 12, more than 20, etc.), e.g., in parallel (e.g., in parallel stations), and / or any other quantity of fish.

[0016] Preferably, all or a portion of the method is performed on board a vessel by a local processing system in conjunction with the fish processing system. Alternatively, all or a portion of the method may be performed by a local processing system attached to a land-based fish processing system, a remote system, a third-party system, and / or other manner. The processing system may be operable to perform one or more processing functions, such as running a model, sending control signals to a set of tools, and / or other functions.

[0017] The system may include one or more models, which may function to determine fish attributes (e.g., fish attribute models), determine the location of anatomical features of interest (e.g., fish body part models), determine optimal tool trajectories and / or other parameters (e.g., trajectory models), verify the execution of method steps (e.g., verify euthanasia), and / or perform other functions. At least a subset of the models (e.g., one or more of fish attribute models, fish body part models, trajectory models, verification models, etc.) is preferably specialized to at least one of fish species, fish size parameters (e.g., length, grade, weight, etc.), season, operating situation, other attributes, and / or other parameters. Specific models may optionally be obtained based on each combination of parameters (e.g., fish species, size parameters, attributes, etc.). However, any model may be generic (e.g., applicable to all fish species, seasons, sizes, etc.), applicable to a subset of possible parameters, and / or other applications.

[0018] The model may include one or more attribute models (eg, equivalently referred to herein as fish attribute models), which may function to determine a set of fish attributes.

[0019] In examples, the fish attribute model may include one or more of a classifier (e.g., the model classifies the fish species), an object detector (e.g., the model does or does not detect fish of a given species), a segmentation model (e.g., exceeding a threshold number of pixels labeled as belonging to a particular attribute class), and / or any other suitable model, and / or any combination of models. Examples of classifiers include binary (e.g., "fish present" vs. "no fish present," "in season" vs. "out of season"), multi-class (e.g., "salmon," "trout," "tilapia," etc., "fish species: salmon," "length: 20 inches," "width: 5 inches," etc.). The system may include a different fish attribute model for each attribute, a single fish attribute model, a combination or cascade of fish attribute models, and / or any other suitable number of fish attribute models.

[0020] The model may optionally include one or more body part models (e.g., equivalently referred to herein as fish body part models), which may function to identify (e.g., identify, locate, etc.) a region and / or set of points (e.g., a point, a plurality of points forming a line, a plurality of points defining a boundary, etc.) that include an anatomical feature and / or coincide with an anatomical feature (e.g., a centerline of an anatomical feature of interest). Anatomical features of a fish may include the brain, brain cavity region, gills, tail, body, body circumference, belly, heart, intestines, liver, spines, fins, spines, eyes, ridges, patterns (e.g., stripes, spots, etc.), blood vessels (e.g., blood vessels, arteries, etc.), and / or other anatomical features and / or regions of the fish. Preferably, the anatomy model includes a keypoint detector that identifies a single point (e.g., in 2D space, 3D space) and / or region where an anatomical feature of interest (e.g., its center of gravity) is most likely located, although the anatomy model may additionally or alternatively include an object detector, a neural network, a segmentation model, and / or any other suitable model. The output of the anatomy model can be used as input to another model (e.g., a trajectory model), as image labeling for training another model (e.g., a fish attribute model, a trajectory model, etc.), to verify regulatory compliance, to verify successful execution of method steps, and / or for other uses.

[0021] The anatomy model can be used to find anatomical features of a fish that are visible within measurements of the fish (e.g., images, such as red-green-blue (RGB) camera data, infrared data, etc.). In variations, the same measurements can be used to determine the location of multiple anatomical features (e.g., locating both the tail and gills in a single image), or multiple measurements can be used to determine the location of different anatomical features (e.g., an image of the tail, an image of the head, etc.). Optionally, the images include exterior images, and the anatomical features are exterior (e.g., defined by and / or located on the exterior surface of the fish, visible to the human eye, etc.). Additionally or alternatively, the images include interior (synonymous with "inside" herein) images (e.g., X-ray, ultrasound, etc.), and the anatomical features are internal and / or external. Additionally or alternatively, the anatomy model can be used to identify anatomical features of a fish that are not visible within the images (e.g., because the images are exterior images of the fish, such as RGB, infrared, etc.). In a first example, the anatomy model directly outputs a set of invisible anatomical features based on the measurements. Optionally, in a first example, the anatomy models can implicitly infer key visible anatomical features (e.g., eyes, face, mouth, palate, etc.) and implicitly calculate a best estimate of the target's non-visible features (e.g., brain) based on relationships learned during model training. In a second example, a first set of anatomy models is used to identify a set of visible anatomical features, and a second set of anatomy models is used to identify a set of non-visible anatomical features based on the output of the first set of anatomy models (e.g., the example shown in FIG. 4). In certain examples, for example, the images can include external images that detect external features (e.g., landmarks, anatomical structures, etc.) that can be used to infer internal features (e.g., arteries, brain, brain cavities, spinal cord, etc.) based on predetermined associations, learned associations, and / or any other association or combination of associations. The anatomy models can predict location based on the fish image and / or a set of features extracted by a prior anatomy model.In an illustrative example, the location of the brain is identified as being a specified distance from the intersection of two particular stripes (e.g., the stripes are detected using a priori fish anatomy models). In another illustrative example, the location of the brain is identified as being a specified distance from features on the eyes and head contour (e.g., the eyes and head contour are detected using a priori fish anatomy models).

[0022] The model may include one or more trajectory models. In examples, the trajectory models may include a euthanasia trajectory model, an exsanguination trajectory model, a post-processing trajectory model (e.g., a fillet trajectory model, etc.), and / or any other suitable trajectory model.

[0023] In variations, the trajectory model may function to determine a set of tool parameters that operate the tool to perform a process and / or step (e.g., euthanasia, exsanguination, processing, etc.). In examples, the tool parameters (e.g., euthanasia tool parameters, exsanguination tool parameters, processing tool parameters, etc.) may include the tool trajectory and / or portions thereof (e.g., position, velocity, acceleration, angle, etc.), orientation, feed rate, speed, tool type, force, revolutions per minute (e.g., of a cutting blade, of a drill, etc.), contact parameters between the tool and the target fish (contact point, contact angle, etc.), and / or other parameters related to control of the robotic tool (e.g., cutting, etc.) elements. The trajectory and / or any other tool parameters may be subject to predefined, measurement set, and / or otherwise determined constraints (e.g., minimum and / or maximum cutting depth and / or length, etc.) and / or goals (e.g., maximizing the amount of harvestable meat, minimizing tool load, etc.) based on the fish species, fish size, user determination, machine and / or tool specifications. The trajectory may optionally intersect one or more target anatomical features (e.g., gills, tail, belly, brain, arteries, etc.). In a variation, the trajectory model determines a set of tool parameters (e.g., euthanasia trajectory, exsanguination trajectory, etc.) for one or more operations based on a set of measurements (e.g., images) of the fish, as shown in the example of FIG. 16. Additionally or alternatively, the trajectory model can determine a set of tool parameters (e.g., euthanasia trajectory, exsanguination trajectory, etc.) for one or more operations based on the output of one or more other models (e.g., attribute model, anatomy model, etc.), as shown in the example of FIG. 17. Measurements can be sampled in a known reference frame (e.g., workstation coordinate system), and the locations of key anatomical features (e.g., determined by the anatomy model) can be transformed to the tool reference frame (e.g., euthanasia tool reference frame, exsanguination tool reference frame, etc.).In certain examples, the trajectory model can find an optimal toolpath (i.e., a set of points), which can optionally coincide with the centerline of a target anatomical feature (e.g., detected by the anatomy model, implicitly inferred by the trajectory model, etc.).

[0024] Examples of models include computer vision models, neural networks (e.g., CNN, DNN, etc.), models based on convolutional neural networks (e.g., region-based convolutional neural networks (R-CNN), Faster R-CNN, Faster R-CNN, region-based fully convolutional networks (R-FCN), etc.), object detectors (e.g., CNN-based algorithms, You Only Look Once (YOLO) algorithm, Single Shot Detector (SSD) algorithm, Single Shot Multibox Detector algorithm, Histogram of Oriented Gradients (HOG), etc.), and transformer-based methods (Vision Transformers (ViT), etc.), segmentation models (thresholding algorithms, clustering algorithms, instance-based segmentation, semantic segmentation, etc.), comparison models (vector comparison, image comparison, etc.), keypoint detectors, clustering, selection and / or search (e.g., from a database and / or library), equation-based techniques (e.g., weighted equations), regression (e.g., leveraged regression), rules or heuristics, classical approaches (e.g., SIFT, HOG, edge detectors, etc.), classification models and / or algorithms (e.g., binary classifiers, multi-class classifiers, semantic segmentation models, instance-based segmentation models, etc.), instance-based techniques (e.g., nearest neighbor), regularization techniques (e.g., ridge regression), decision trees, Bayesian techniques (e.g., naive Bayes, Markov, etc.), kernel techniques, statistical techniques (e.g., probability), determinism, support vectors, other machine learning models, other models, and / or any combination of models, algorithms, and / or tools.

[0025] In examples, any of the models (e.g., all or a subset thereof) can be trained with training data including a set of labeled measurements (e.g., multiple images showing multiple fish). In variations, the labels may include the location of the desired cut and / or any other cut parameters (e.g., depth of the cut, location of the cut along the length of the fish, angle of the cut, trajectory parameters, etc.), location of anatomical features of interest (e.g., features visible on the outside of the fish such as eyes, features not visible on the outside of the fish such as the brain, etc.), trajectories (e.g., including tool parameters, tool target points, etc.), attributes (e.g., attribute values), and / or any other training data.

[0026] Labels may be manually generated, computer-generated (e.g., using a trained model), and / or otherwise generated. In variations, labels may be determined based on one or more secondary measurements. In examples, secondary measurements may include weighing the fish, physically measuring the fish (e.g., length, thickness, width, etc.), sampling secondary images using the same or different image modality as the label image, sectioning the fish, and / or any other suitable technique. Optionally, measurements of the fish before and after processing can be sampled (e.g., the fish is held in the same position and / or visual keypoints are preserved for subsequent image alignment).

[0027] In examples, any of the models (e.g., all or a subset thereof) may be trained with training data including measurements of a single type of attribute and / or set of attributes (e.g., in the case of models specialized in a set of one or more fish attributes) and / or multiple fish attributes. In one particular set of examples (e.g., shown in FIG. 14 ), the system may include one or more models specialized in a fish species. Some or all of the models may be trained using measurements (e.g., images) of a particular fish species and / or of a particular fish species and one or more other fish species (e.g., those sharing a common anatomical pattern). Additionally or alternatively, some or all of the models may be trained with measurements of a particular size attribute (such as grade, length, weight, etc.) and / or any other suitable attribute.

[0028] Optionally, some or all of the models can be trained using transfer learning techniques (e.g., pre-training, fine-tuning, knowledge transfer, adaptation, evaluation, refinement, weight modification, pre-training weight modification, etc.). In a first example, a first set of models can be trained with measurements including fish having a first combination of fish attributes, such as fish species, size attributes, and / or any other suitable attributes, to perform a first task (e.g., identifying a first combination of fish attributes). A second set of models (e.g., the same as or different from the first set) can then be trained for a second task on a second set of measurements including fish having a second combination of fish attributes, using knowledge gained during training of the first set of models to perform a second task (e.g., identifying a second combination of fish attributes). In a second example, a first set of models can be trained with measurements of a first measurement parameter (e.g., measurement type, image angle, measurement quality, etc.) for a first task, and a second set of models (e.g., the same or different from the first set) can then be trained for a second task (e.g., the same or different from the first task) using knowledge gained during training of the first set of models. In a particular variation of the second example, the first set of models is trained with images of a first measurement type (e.g., images in a color scale such as red-blue-green, x-rays, etc.), and transfer learning techniques are applied so that the second set of models can make decisions based on images of a second measurement type (e.g., grayscale, infrared, etc.).

[0029] Optionally, transfer learning can be used to extend the trained model to one or more of: another fish attribute (e.g., fish species, size, grade, etc.), another animal (e.g., pig, cow, etc.), another measurement type, another machine configuration (e.g., tool type, tool configuration, sensor type, sensor configuration, etc.), another target trajectory, another euthanasia method (e.g., striking, killing, decapitation, etc.), any other processing method (e.g., bleeding, filleting, etc.), and / or any other suitable parameters.

[0030] Optionally, some or all of the model can be updated over time based on new data (e.g., user-generated labels, customer reviews, etc.) Optionally, new data can be added to the data repository used for training.

[0031] The model can be trained using self-supervised learning, semi-supervised learning, supervised learning, unsupervised learning, reinforcement learning, transfer learning, Bayesian optimization, positive label-free learning, backpropagation, and / or other learning methods. The model can be learned or trained on labeled data (e.g., data labeled with a target label), unlabeled data, a positive training set (e.g., a dataset with true positive labels), a negative training set (e.g., a dataset with true negative labels), and / or any other suitable dataset. In a first variation, different models can be trained for different fish species, sizes, and / or other parameter values. In a second variation, a general model can be trained on a generic dataset (e.g., trained on a dataset including different fish species, sizes, and / or other parameter values, in the same or a different domain, trained on a first set of measurement types, etc.) and then tailored (e.g., using transfer learning) to create a specific model for a particular fish species, fish size, method step, system configuration, second set of measurement types, and / or any other parameters. In a third variation, different models can be trained on the same fish parameter set and model outputs can be weighted or selected (e.g., based on model accuracy), although models can be trained or configured in other ways. Model inputs can include one or more of the measurements described above, anatomical features or attributes extracted by other models, and / or other inputs.

[0032] However, the system may include any additional or alternative elements and / or be otherwise configured.

[0033] 4. Method 1 , the method may include steps S100 of determining a fish attribute set for the fish, S150 of optionally sorting the fish, S200 of euthanizing the fish, and S300 of bleeding the fish. In variations, the method may additionally or alternatively include steps of discarding the fish after sorting, S400 of further processing the fish, S500 of tracking the fish, and / or any other suitable elements. The method may also have the capability to process live fish, other marine animals, other animals (e.g., chickens, pigs, cows, etc.), and / or other functions. In examples, fish and / or other marine animals may include salmon, trout (e.g., rainbow trout, brown trout), tilapia, tuna (e.g., yellowfin, albacore, bluefin), carp, cod, haddock, mackerel, sardines, anchovies, sea bass (e.g., striped jack, largemouth bass, etc.), perch, flounder, swordfish, barracuda, whitefish, sturgeon, eel, swordfish, flounder, catfish, pollock, red sea bream, grouper, mahi-mahi (dolphinfish), halibut, herring, clams, lobster, crab, octopus, shrimp, squid, mollusks, and / or other suitable species.

[0034] One or more embodiments of the method may be performed on one or more fish, one or more species of fish, one or more sizes of fish, and / or other types of fish. Different embodiments of the method may be performed on multiple fish simultaneously (e.g., in parallel), may be performed on multiple fish sequentially, and / or may be performed in any other suitable order.

[0035] All or part of the method may be performed in response to a request (e.g., a user selection), automatically (e.g., upon identifying the presence of fish in the fish processing system), and / or at any other suitable time.

[0036] Preferably, each step of the method (equivalently referred to herein as a process) is performed in a station (e.g., a chamber) configured for that step (e.g., where each station is located within a single fish processing device, where two or more stations are located in separate, stand-alone fish processing devices, etc.), and the method is performed simultaneously (sequentially) on multiple fish passed from station to station. Additionally or alternatively, one fish may be processed at a time, in which case stations before and after the current step may remain empty. Additionally or alternatively, one or more steps may occur within the same station. In a variation, the system may additionally or alternatively be configured to process fish in parallel (e.g., multiple stations of the same type arranged adjacent to each other within a single fish processing device).

[0037] All or part of this method can be performed using measurements of the fish. The measurements can include images (e.g., images, RGB images, stereo images, video, 2D image data, 3D image data, etc.), depth measurements (e.g., point clouds, etc.), forces (e.g., weight), pressure, vibration, strain, displacement, acceleration, tactile measurements, audio, electrical signals, and / or any other suitable measurements. In a variation, all or part of the images sampled by the system can include images that do not require an illumination source (e.g., X-ray imaging, infrared imaging, thermal imaging, microwave imaging, radar imaging, ultrasound imaging, radio frequency imaging, computed tomography imaging, etc.), which can allow live fish to be imaged in a closed, dark container, thereby reducing the stress level of the fish upon entering the system, reducing the complexity and / or cost of the system (e.g., no need to integrate or replace illumination sources, etc.), and / or providing other suitable advantages. In variations, internal imaging (e.g., X-ray imaging, ultrasound imaging, computed tomography imaging, etc.) can provide the advantage of providing a more detailed view of internal anatomical features (e.g., brain, arteries, etc.) (e.g., compared to external imaging), which can allow for more accurate location of internal anatomical features and / or reduce model complexity. In a specific example, for example, internal imaging can target the specific anatomical structure where cutting is desired (e.g., location of an artery) rather than targeting the anatomical structure indirectly through detection of an external feature (e.g., gills), thereby increasing the precision of cutting, maximizing the amount of available, undamaged fish meat, and / or optimizing the results of the process.

[0038] The method may include sampling one or more measurements using a measurement system (e.g., an imaging system as shown in FIG. 2 ) including a set of sensors. Types of sensors may include optical sensors such as cameras, tactile sensors (e.g., embedded in straps), weigh scales, force sensors, displacement sensors, tensiometers, pressure sensors, vibration sensors, current sensors, voltage sensors, ECGs, and / or other sensors. Each processing stage (e.g., step) of the method may include one or more sensors (e.g., of the same type and / or different types). The sensors may be associated with a fish processing workstation that houses fish having a front, back, top, bottom, left side, and right side. The sensors may be positioned along any face of the workstation, at intersections of the faces of the workstation, at any angle relative to the workstation, within the workstation, external to the workstation, and / or otherwise associated with the workstation.

[0039] Preferably, measurements, which can optionally match the environment for the training data set, are sampled in a controlled environment (e.g., controlled lighting, unobstructed field of view, controlled position and / or orientation relative to the reference frame, etc.), but can alternatively be acquired in an uncontrolled or partially controlled environment. The controlled environment can optionally be specific to the sampling station (e.g., associated with a particular step of the method), specific to a set of fish attributes (e.g., fish species), and / or specified in other ways. Preferably, the sensors are calibrated in a common coordinate frame (i.e., sensor coordinate frame calibrated to the relative geometry of the fish processing system) at a fixed and / or predetermined location relative to the (joint) coordinate frame of the robotic assembly module, but other suitable configurations are possible.

[0040] Measurements can be sampled on live fish, dead fish, or both pre- and post-mortem fish. A different set of measurements can be sampled for each step (e.g., the example shown in FIG. 9), or, additionally or alternatively, the same set of measurements and / or information extracted therefrom can be used in different steps. Measurements can be sampled while the fish is held (e.g., held in a fixture, equivalently referred to herein as restrained), immobilized, and / or unrestrained. In a variation, unrestrained live fish can be out of the water or swimming in a controlled water current. Measurements can be sampled from the side, bottom, top, front, back, isometric, and / or other suitable angles relative to the fish.

[0041] In one example, images of a fish restrained in a fixture at a method station are sampled from a camera pose that is fixed relative to the fixture configuration. In another example, tactile measurements of the restrained fish can be sampled to determine the size of the fish. Sensors can be attached to the fixture (straps, rotating arms, etc.) to measure displacement (angular, axial, etc.) from a reference position to determine the girth of the fish.

[0042] The method may optionally include selecting a subset of the obtained measurements to perform an analytical task (e.g., determine parameter values) for some or all steps of the method (e.g., an example shown in FIG. 11 ). Selecting the subset of measurements can determine a set of measurements that meet a set of specifications (e.g., resolution, quality, anatomical features included, captured in a well-controlled environment, etc.) to ensure uniformity in the analysis and / or consistency with a set of training data. The selection of the subset of measurements can be based on the visibility of key fish anatomical features (e.g., eyes, fins, head, tail, gills, overall contour, etc., where each anatomical feature in the expected set of anatomical features is detected in a single image), the visibility of the fish (e.g., occlusion by obstacles, perspective, etc.), the quality of the obtained measurements (e.g., resolution, maximum noise threshold, whether water has accumulated in the restraining mechanism and is occluding part or all of the fish, etc.), and / or a set of other specifications. In a first variation, a model is used to identify a single measurement from the set of measurements (e.g., the measurement that is best suited for a particular analytical task). In a second variation, the method may include using a model to identify multiple measurements (e.g., all suitable measurements) from the set of measurements, and then determining the parameter by aggregating (e.g., averaging) the selected measurements or selecting the best subset of measurements from the selected measurements. In a third variation, one or more measurements may be successively sampled and evaluated using the model until a suitable measurement is sampled. However, in alternatives, all sampled measurements may be used for the analysis (e.g., a single measurement may be sampled to determine a single parameter, multiple measurements may be sampled to determine a single parameter, and all measurements may be aggregated to determine the parameter value, etc.).

[0043] Optionally, one or more measurements can be sampled from a particular measurement configuration that ensures visibility of one or more target anatomical features (e.g., to determine trajectory, to determine fish attributes, etc.). In one example, a first set of images is sampled from a first view (e.g., an underside profile) of the fish from a first configuration (e.g., below the base of the workstation) in which a first set of anatomical features (e.g., gills, underbelly, etc.) are clearly visible. The first set of images can be processed with a first set of models (e.g., attribute model, trajectory model, first set of anatomy models, etc.) to generate a first set of outputs. Optionally, a second set of images is sampled from a second view (e.g., a side profile) of the fish from a second configuration (e.g., the side of the workstation) in which a second set of anatomical features (e.g., tail, fins, jaw, etc.) are clearly visible. The second set of images can be processed with a second set of models to generate a second set of outputs. Additionally, images can optionally be sampled with different parameters (e.g., camera angle, measurement modality, etc.) to further detect features of interest. However, the images can be taken from any other suitable direction.

[0044] In another variation, images can be taken from a single orientation of the fish (e.g., side view, top view, front view, bottom view, etc.), such as when restrained in a translucent fixture.

[0045] One or more models (and / or their functionality described herein) may be run simultaneously, in parallel, asynchronously, and / or in any other suitable order. One or more models may be run on the same fish, different fish, at the same processing stage, and / or at different processing stages.

[0046] Additionally or alternatively, the method may be suitably carried out in other ways.

[0047] 4.1 Step S100: Identifying a Set of Fish Attributes Identifying a set of fish attributes step S100 may function to identify a set of fish attributes that can be used by the automated fish processing system to inform decision making in subsequent steps of the method.

[0048] Determining a set of fish attributes for the fish may include receiving a live fish, determining a set of data for the fish (e.g., equivalently referred to herein as determining a set of measurements for the fish), and determining the set of fish attributes based on the set of data. Determining a set of measurements for the fish may include sampling measurements, receiving measurements (e.g., from an imaging system), and / or otherwise determining the measurements.

[0049] At one or more times during the method (e.g., before sorting the fish), measurements can be sampled without immobilizing (e.g., equivalently referred to herein as "restraint") the fish. As an example, measurements (e.g., images) are sampled of unimmobilized (e.g., unrestrained) fish in an enclosure filled with water. The water can be still, moving (e.g., selectively flowing along an axis so that the fish swims along the axis), misted, sprayed, or otherwise configured. Additionally or alternatively, at one or more times during the method (e.g., during sorting, before and / or after euthanasia, exsanguination, further processing, etc.), measurements of the fish are sampled while they are immobilized or otherwise restrained.

[0050] The method may include collecting a first set of measurements (e.g., images) simultaneously with restraining the live fish. Preferably, measurements of the live fish are sampled one or more times during the method (e.g., before culling, before euthanasia, etc.). Additionally or alternatively, measurements of the fish may be sampled after euthanasia (e.g., before bleeding, before further processing, etc.). Additionally or alternatively, measurements of objects that are not live fish (e.g., received dead fish, debris, etc.) may be sampled (e.g., the objects are subsequently sorted and discarded in step S150). However, measurements may be sampled at any other time and / or for any other items.

[0051] Identifying a set of fish attributes based on the set of measurements may be performed using one or more fish attribute models to determine the set of fish attributes (e.g., equivalently referred to herein as attributes). Examples of identifiable fish attributes include fish species, size, length, shape, weight, seasonality, quality indicators, and / or other fish attributes. In examples, quality indicators include color, skin gloss, gill quality (moisture, color, shine, etc.), eye quality (color, shine, cloudiness, eye sockets, etc.), firmness, scale loss, parasites, blood clots, visual defects, and / or other quality indicators. In examples, the fish attributes may include desirable (e.g., target) or undesirable fish attributes. In examples, the fish attributes may be categorized as a continuous quantity (e.g., percentage, continuous value, etc.), a discrete quantity (e.g., binary above / below threshold, buckets, quantiles, etc.), and / or other manners.

[0052] The attributes of the fish may additionally or alternatively include the relative orientation of the fish to the system, such as heading, position, blockage, and / or any other relative attributes that indicate how suitable the fish is for further processing (in its current state) and / or whether the fish should be excluded from further processing, reoriented before further processing, and / or manipulated in other ways.

[0053] S100 may be performed using a fish attribute model, a combination of models, and / or other suitable models. The output of S100 may optionally be used to sort fish (e.g., to determine whether to process the fish in the system or release them), euthanize the fish, bleed the fish, perform further processing of the fish, and / or other uses.

[0054] In a variation, fish attributes can be inferred from images of the fish using a set of trained fish attribute models. In a first example, fish attributes (e.g., fish species, class, etc.) can be identified using a classifier (e.g., an image classifier). In a second example, the fish attribute model can determine a bounding box surrounding the fish, identify the fish dimensions based on the bounding box dimensions, and / or estimate the fish weight based on the fish dimensions (e.g., based on the fish species, using a lookup table, etc.). In a further variation, fish attributes can be inferred based on measurements (e.g., fish weight inferred from an estimated volume of the fish identified from stereo images).

[0055] Optionally, S100 may include using the set of fish attributes output by the first set of models (e.g., fish attribute models) as parameters (e.g., model weights, model parameters, etc.) to inform the selection of the second set of models and / or influence the output of the second set of models. In a variation, S100 may include identifying the fish species prior to the subsequent step of identifying the fish attributes, which may advantageously account for significant variations in fish between fish species. In an additional or alternative variation, S100 may include identifying the fish species and additional fish attributes (e.g., size, grade, etc.), which may advantageously account for significant variations within a fish species.

[0056] In one example, S100 may include using the first set of models to identify a first set of attributes of the fish, where the attributes of the fish include at least the fish species, and optionally further include the fish size and / or any other suitable attributes. A second set of models may be selected to identify a second set of attributes of the fish (e.g., fish size, quality metrics, etc.) based on the fish species and / or other suitable attributes. In the example shown in FIG. 14 , after classifying the fish species with the first model, subsequent models (e.g., culling, euthanasia, bleeding, etc.) specific to the fish species are obtained. Additionally or alternatively, the fish species and / or other suitable attributes may be provided as a set of input parameters to the second set of models.

[0057] In a first variation, a first model is used to identify multiple fish attributes (e.g., fish species and size) from the same set of measurements (e.g., to perform multi-label classification). Optionally, the output of the first model may further inform the selection and / or output of a second set of models. In one example, the first set of models identifies the fish species and size, and any or all of the subsequent models (e.g., euthanasia model, cutting trajectory model, etc.) utilize both the fish species and size to determine an optimal trajectory. For example, complex information about how a certain type of fish grows, along with the size of the fish, can be used to optimally euthanize or butcher the fish (e.g., in a manner that does not simply reduce the size of larger fish of that species).

[0058] In a second variation, multiple sets of models are used to identify multiple fish attributes from different sets of measurements. For example, different measurement types and / or sampling methods (e.g., camera viewpoints) can be used to identify different fish attributes. In a first example, a first set of models is used to identify fish species from a first measurement type (e.g., a single image, an RGB image, an infrared image, etc.), and a second set of models is used to identify a second attribute (e.g., weight, density, etc.) from a second measurement type (e.g., two or more images using stereo vision technology, 3D imaging, scale readings, etc.). In a second example, a first set of models is used to identify fish species from an image showing the fish's outline, and a second set of models is used to identify fish mass from an image showing the fish's underside (e.g., the example shown in FIG. 10). Optionally, the output of the first models may inform the selection and / or output of the second set of models.

[0059] In a third variation, sequential models are used to identify attributes and filter out fish (e.g., from the same and / or different sets of measurements) that do not meet processing requirements. As an example, a first model is used to identify the fish species, and a second model is used to identify the fish size only if the fish species meets a set of processing requirements. Optionally, additional models may be used to identify additional attributes if the fish species and / or size meet the requirements.

[0060] However, the step of identifying the set of attributes of the fish can be performed in other ways.

[0061] 4.2 Step S150 of Sorting Fish The method may optionally include a step S150 of sorting the fish, which may function to determine whether the fish meets a set of processing requirements. S150 preferably occurs after S100, but may additionally or alternatively occur at other suitable times. In an example, the method may optionally include a step of verifying a set of one or more processing requirements (e.g., system processing requirements) before performing each subsequent step of the method.

[0062] The step of sorting the fish may be based on a set of taken measurements (e.g., images), which are preferably sampled before the live fish are restrained, but may additionally or alternatively be sampled after the live fish are restrained. The step of sorting the fish is preferably performed in a collection station (e.g., an aquaculture subsystem), where fish are allowed to pass to the station where S200 is performed only if they meet a set of processing criteria, but may additionally or alternatively be performed in the same station as S200 and / or in other manners.

[0063] Processing requirements may include regulatory requirements, system processing requirements, quality requirements, market requirements (e.g., what types of fish a system user is willing / unwilling to process), safety requirements (e.g., ensuring that no non-fish objects (e.g., human hands, fingers, debris, etc.) are present before performing additional processing), and / or other suitable requirements. Processing requirements may be general, dependent on a set of one or more fish attributes (e.g., fish species, size, etc.), specific to a context (e.g., the region, vessel, and / or season in which the method is performed), specified by a user (e.g., via a user interface), specified by the system (e.g., default settings), and / or specified in other ways.

[0064] Examples of regulatory requirements include fishing seasons (e.g., by species), size limits for target species (e.g., body length, weight, total length, fork length, standard length, curved fork length, eye fork length, head length, minimum size threshold that must be exceeded, etc.), catch limits (e.g., by species), and / or other appropriate regulatory requirements. Optionally, the system can interface (e.g., satellite, API, radio, internet, etc.) with external computing systems and / or databases to obtain regulatory requirements based on the location where the system is being used.

[0065] Examples of system processing requirements may include detection of the presence of fish, a set of one or more types of fish (e.g., fish species) that the system can or cannot process, a minimum fish size, a maximum fish size, a range of fish sizes, a required size, symmetry (e.g., placement of the spine on either side of the sagittal plane), position, or orientation of one or more anatomical features, a required placement of the fish relative to the system (e.g., the fish is upright, the fish is facing forward in the fish processing unit, the fish is properly secured by a restrainer, etc., which can optionally be specified with a certain tolerance, for example, the nose of the fish is within a pre-specified number of degrees, such as less than 90°, less than 75°, less than 60°, less than 45°, less than 30°, less than 15°, less than 5°, less than 1°, etc., from the target angle), desired quality indicators and / or absence of defects (e.g., abnormal mass, parasites, disease, etc.), and / or any other combination of fish attributes that the system can or cannot process.

[0066] Examples of quality requirements may include the fish being alive, the fish being healthy (e.g., as indicated by the quality indicators identified in S100), conforming to a set of desired quality indicators, being free of a set of undesirable quality indicators (e.g., parasites, visual defects, loss of scales, etc.), and / or any other suitable quality requirement.

[0067] Sorting the fish preferably includes sorting for regulatory compliance. In a specific example, sorting for regulatory compliance may include, based on the fish species (e.g., determined in S100), identifying a set of relevant regulatory requirements that apply to the fish species, comparing one or more additional fish attributes (e.g., size requirements) to the set of relevant regulatory requirements, comparing the one or more additional fish attributes to the set of relevant regulatory requirements, and retaining or discarding the fish based on the comparison (e.g., the example shown in FIG. 8).

[0068] Sorting the fish may additionally or alternatively include sorting for compliance with other processing requirements (e.g., a set of criteria). In an example, S150 may include comparing the set of fish attributes identified in S100 with a set of target attributes (e.g., processing requirements), and retaining or discarding the fish based on the comparison. Sorting the fish may include determining whether the fish attributes meet threshold processing requirements (e.g., percentages, categories, ratings, etc.). Comparing the fish attributes with the set of target attributes may be performed using value-to-value comparisons, comparing a weighted sum of one or more fish attributes with a target rating, modeling techniques, and / or any other suitable method.

[0069] If (e.g., only if) the fish meets the processing requirements, subsequent steps of the method may be executed (e.g., S200, S300, S400, etc.). However, if the fish do not meet the requirements, they may be discarded (e.g., removed from the system, stopping further method processes, triggering a request for manual intervention, etc.). Discarding the fish may optionally include automatically releasing the fish from the fish processing device, for example, by sending the fish back to its source (e.g., a body of water, a fish farm, etc.), or discarding the fish (e.g., if the fish is dead, if the received object is not a fish, etc.). Additionally or alternatively, discarding the fish may include reprocessing the fish (e.g., as shown in FIG. 9). In a first example, the fish may be reprocessed if it meets certain processing requirements (e.g., regulatory requirements, quality requirements, etc.) but does not meet certain system processing requirements (e.g., if the fish is out of alignment within the system) and / or safety requirements (e.g., if the fish is harvested with other non-fish objects, if multiple fish are harvested, etc.). In a second example, if one or more measurements are inconclusive (e.g., the lens is fogged, the measurements are noisy, etc.), reprocessing of the fish (e.g., transporting the fish back to the inlet of the processing equipment) can be performed.

[0070] S150 may optionally include triggering a safety protocol (e.g., triggering an alarm, powering off the system, initiating a cleaning cycle, etc.) if an unrecognized object, an unsafe object (e.g., a human body part, a non-fish object, etc.), a contaminated object (e.g., a diseased fish, a parasite-containing fish, etc.), and / or any other inappropriate item is detected in the system. Triggering a safety protocol may optionally be performed instead of ejecting the object from the system (e.g., if automatically ejecting the object could damage the object or the system) or in addition to ejecting the object from the system.

[0071] However, the step of sorting the fish can be performed in other ways.

[0072] 4.3 Step S200 of Euthanizing the Fish The step S200 of euthanizing the fish functions to kill the fish quickly, accurately, and / or in an optimal manner. Preferably, S200 is performed after the fish have been automatically sorted in S150. However, in alternative variations, S150 may be bypassed and S200 may be performed as the first step of the method (e.g., assuming the received fish meet any processing requirements) and / or S200 may be performed at any other suitable time. While S200 can be performed in the same station as the step S150 of sorting the fish, the step S300 of exsanguination, the step S400 of further processing, and / or other steps, preferably, S200 is performed in a separate station configured for euthanasia. Euthanasia is preferably performed by cutting (e.g., cutting with a drill, cutting with a spike, cutting with a water jet, etc.), but may additionally or alternatively be performed by blow to the head region (e.g., percussion, pneumatic blow, etc.), decapitation, partial decapitation, electric shock, and / or any other suitable method. S200 is preferably performed quickly while the fish is conscious (e.g., without first putting the fish to sleep), which provides the most humane euthanasia procedure, although the fish can alternatively be anesthetized and then euthanized.

[0073] The fish is preferably at least partially restrained before euthanasia, although it does not have to be restrained. The fish (e.g., live fish) can optionally be imaged while restrained by a restraint subsystem (e.g., a euthanasia restraint subsystem). The restraint subsystem preferably holds the fish upright (e.g., relative to a major plane), but may additionally or alternatively hold the fish on its side relative to a major plane and / or hold the fish in any other suitable configuration. The restraint subsystem preferably includes a material for restraining the fish that is flexible (e.g., inelastic, elastic, compliant, non-compliant, etc., so as not to change dimensions). However, the material may additionally or alternatively be semi-flexible, rigid, malleable, resilient, pliable, ductile, and / or other configurations. The material is preferably transparent (e.g., equivalently referred to herein as translucent or transparent), allowing the imaging subsystem to sample images of the fish (e.g., live fish) through the transparent material. However, the material can alternatively be translucent, opaque, and / or of other configurations, and optionally, the imaging subsystem can sample an image of the fish through the material (e.g., using X-rays, etc.). The material (e.g., a transparent material) can include any combination of plastic (acrylic, cellulose, cellulose acetate, nylon, silicone, PVC, polycarbonate, rubber, vinyl, etc.), glass, and / or other suitable materials. The material is preferably configured into one, two, or more sheets that enclose the fish when the restraint subsystem closure mechanism is activated (e.g., the example shown in Figures 15A and 15B), although the material may be in other configurations (e.g., a set of claws, clamps, etc.). While the restraint subsystem is described in the context of euthanasia, similar restraint subsystems can be employed to restrain the fish in other steps of the method.

[0074] Alternatively, the restraint subsystem may be rigid, such as including a set of rigid and translucent (e.g., transparent, translucent, etc.) panels that orient the fish in a particular configuration (e.g., upright, sideways, etc.).

[0075] Euthanasia is preferably performed using a euthanasia tool (e.g., equivalently referred to herein as a killing tool) including some or all of a drill, pneumatic spike, spike, water jet, air jet, blade, electrode set, fishing priest, fishing bat, and / or other suitable tool.

[0076] Euthanizing the fish may include determining a set of euthanasia parameters (e.g., equivalently referred to herein as kill parameters) based on a set of measurements of the fish and / or based on a set of attributes of the fish (e.g., fish species, size, etc.). Determining the euthanasia parameters may optionally include locating one or more anatomical features of the fish (e.g., the brain) based on the set of measurements (e.g., using a regional model of the fish). Alternatively, a trained model may be used to analyze only the collected data without explicitly identifying a set of intermediate anatomical measurements or identifying information. Preferably, the measurements include images, but additionally or alternatively, other measurement modalities may be included. The set of measurements may be the same as or different from the set of measurements used in S150. Preferably, the set of measurements is indicative of at least the head region of the fish. Additionally or alternatively, the set of measurements may depict the entire fish and / or other parts of the fish. In a variation, the set of measurements may include images depicting a side view of the fish, a front view of the fish, a top view of the fish, a bottom view of the fish, and / or any other suitable perspective. The step of determining the euthanasia parameters may be performed using a set of trajectory models, such as euthanasia trajectory models, that may function to determine a euthanasia trajectory for one or more euthanasia tools and / or any other suitable euthanasia parameters.

[0077] Determining the set of euthanasia parameters may include determining a euthanasia tool trajectory (e.g., equivalently referred to herein as a killing tool trajectory), a euthanasia tool entry point, and / or other suitable parameters, including a euthanasia tool angle (e.g., an angle relative to any suitable plane, an angle relative to an axis as defined in some or all of Figures 12A-12C, etc.). Preferably, the euthanasia trajectory is perpendicular to the fish's head at the point where the tool first contacts the head (e.g., a trajectory perpendicular to the fish's head and / or contour), as the inventors have discovered that the euthanasia tool encounters less slippage and resistance when entering the fish's head and brain at a nearly orthogonal (e.g., vertical) entry (e.g., entry at a substantially 90-degree angle in one or more planes) relative to a plane defined by the fish (e.g., the top of the fish's head). Additionally or alternatively, the tool trajectory can follow an angle that is not perpendicular to the fish's head. A non-perpendicular angle may be beneficial, for example, when the shortest path from the outside of the fish's head (e.g., along the sagittal plane) to the brain is not a right angle, when a perpendicular path intersects hard tissue (e.g., skull, bone, etc.), when a non-perpendicular angle is optimal for hitting the fish's brain and / or spinal cord based on the shape of the brain and / or the position of the spinal cord relative to the brain, and / or under other appropriate circumstances (e.g., to minimize tool recalibration between different fish and / or different fish species). In variations, the tool entry angle (e.g., the angle relative to an axis defined by the intersection of the transverse plane and the sagittal plane) can be defined as 90 degrees, less than 90 degrees (e.g., less than 85 degrees, less than 80 degrees, less than 75 degrees, less than 70 degrees, less than 65 degrees, less than 60 degrees, etc.), greater than 90 degrees (e.g., greater than 95 degrees, greater than 100 degrees, greater than 105 degrees, greater than 110 degrees, greater than 120 degrees, greater than 125 degrees, etc.), within a range (e.g., 85-95 degrees, 80-100 degrees, 75-105 degrees, 70-110 degrees, etc.), and / or otherwise.

[0078] Preferably, the trajectory intersects the brain location (e.g., the example shown in FIG. 5), but alternatively, it may not intersect the brain location (e.g., when the euthanasia method is decapitation). In the example, the trajectory intersects the fish's forehead along the sagittal plane, but may additionally or alternatively intersect the side of the fish's head and / or intersect the fish in other ways. The trajectory can be determined using measurements of the fish (e.g., images), brain location, and / or other inputs. The trajectory may be determined using a euthanasia trajectory model (e.g., a classical ML model, a scoring model, etc.), and / or other models.

[0079] Determining the set of euthanasia parameters may additionally or alternatively include determining euthanasia / killing tool parameters (e.g., tool type, speed, force, etc.) to accompany the trajectory. In examples, the tool parameters may be determined using a model (e.g., a euthanasia trajectory model, a planning module, etc.), determined based on a set of measurements, predetermined (e.g., obtained) based on fish species, size, and / or any other attributes, and / or determined in other ways.

[0080] Preferably, the euthanasia trajectory model determines the trajectory directly based on measurements of the fish (e.g., a set of images including the head region of the fish). In an example, given a set of images of a fish (e.g., a restrained fish), the euthanasia trajectory model can output one or more euthanasia parameters from a set of euthanasia parameters.

[0081] Additionally or alternatively, the euthanasia trajectory model can determine a trajectory based on a set of one or more outputs (e.g., brain location, stripe pattern, etc.) of the regional model. In a specific example, the fish regional model can optionally include a fish regional model (e.g., a brain location model) that can identify (e.g., measure, predict, etc.) the location of the fish's brain and / or brain cavity. Optionally, the output of the regional model can also be used to apply trajectory constraints. In one example, if the euthanasia trajectory includes a path that is perpendicular or substantially perpendicular (e.g., within an angular range encompassing 90 degrees) to the slope of the fish's head that intersects with the brain location, the method can include identifying a contour of the fish's head (e.g., a mask of the fish's head, a projection of the fish's head, etc.), optionally determining a set of surface normals for each of the points on the contour, selecting a point having a surface normal that coincides with the brain location, and optionally determining a line or vector connecting the point to the brain location.

[0082] In a first variation, a classical method is used to determine the trajectory. In an example, one or more candidate tool entry points and / or tool entry angles (e.g., each satisfying a set of criteria) along the contour of the fish's head (e.g., as detected in the image) may be proposed by a euthanasia trajectory model. If multiple candidates are proposed by the model, the optimal point and / or angle may be selected (e.g., based on maximizing a set of criteria). In a specific example, a surface normal is determined at each member of the head points (e.g., near the killing tool location), and the head point whose surface normal substantially coincides with the brain location is selected. The surface normal may be determined from the head contour, head projection, depth measurement, and / or other measurements.

[0083] In a second variation, a region model is additionally used to determine the trajectory. The region model may be a separate model from the euthanasia trajectory model, or may be a component of the euthanasia trajectory model (e.g., an intermediate layer of a multi-layer neural network). In a first example, the same trajectory model includes the region model and is trained to determine both the location of an anatomical feature (e.g., the location of the brain) and the trajectory (e.g., the example shown in Figure 7). In a second example, a first model (e.g., the region model) is used to determine the location of an anatomical feature (e.g., the location of the brain), and a second model (e.g., the euthanasia trajectory model) determines the trajectory based on the location of the anatomical feature (e.g., the example shown in Figure 6).

[0084] However, the euthanasia trajectory model may be determined in other ways.

[0085] After determining the killing parameters, the killing parameters (e.g., tool control commands) are preferably executed by the robotic fish processing system to euthanize the fish.

[0086] S200 can optionally include a step of confirming euthanasia, which functions to determine that the fish is not alive when S300 is executed. The criterion for confirming euthanasia is preferably brain death, but other humane criteria may also be used. Euthanasia can be confirmed by detecting the absence of movement in the fish (e.g., based on video of the fish, other sensor readings, etc.), detecting splayed fins in an image of the fish, indicating death, determining whether the fish moves in response to a stimulus (e.g., applied force, electrical stimulus), using sensors (e.g., electrocardiogram, ECG, electrodes, force sensors, accelerometers, etc.), and / or other methods. The method used to confirm euthanasia can be species-specific or general across one or more species. Euthanasia can be confirmed using a confirmation model (e.g., a neural network, a motion detection model, etc.) and / or any other suitable model.

[0087] However, other methods of euthanasia may also be used.

[0088] 4.4 Step S300 of bleeding the fish The step S300 of bleeding the fish functions to remove blood from the fish's body after the step S200 of euthanizing the fish. Preferably, S300 is performed in the same fish processing equipment as S200, but may alternatively be performed in a different fish processing equipment (e.g., fish may optionally be automatically transported between fish processing equipment via a conveyor, chute, etc., or may be transported within the same location, such as within the same fixation mechanism). If S300 and S200 are performed within the same fish processing equipment, S300 is preferably performed in a different workstation from S200 within the fish processing equipment, but alternatively may be performed in the same workstation (e.g., when fixation devices are changed between S200 and S300, when fixation devices are not changed between S200 and S300, etc.). S300 preferably includes determining a bleeding instruction (e.g., a bleeding trajectory). S300 may optionally include locating anatomical features (e.g., gills, tail, belly, etc.) and determining a bleeding instruction based on the anatomical features. Preferably, fish are exsanguinated through one or more incisions (e.g., one proximal to the gills and one proximal to the tail) that penetrate a major artery. A single incision reduces system complexity, while two or more incisions increase the speed of exsanguination and improve system throughput. Exsanguination tools may include blades, rotating blades, spikes, water jet cutters, air jet cutters, hot wires, and / or other suitable tools. However, additional or alternative exsanguination methods may be employed.

[0089] S300 is preferably performed separately from S200 (e.g., sequentially, with different tools, with different tool parameters such as angle and / or position), or may be performed as part of S200, in parallel with or partially overlapping with S200, and / or at any other time.

[0090] In a variation, the fish is at least partially restrained within the workstation (e.g., by a fixation system, equivalently referred to herein as a restraint subsystem) to ensure it remains stationary and / or otherwise maintains a particular orientation (e.g., upright, not tilted, etc.) during dissection. The fish may be positioned (e.g., restrained) relative to a major plane of the workstation (e.g., a surface from which and / or to which the cutting tool can actuate). Optionally, the fish is positioned within the workstation (e.g., against a corner, a stopper / bumper, etc.) before restraint, during restraint (e.g., as shown in FIG. 15A ), and / or at any other suitable time. The fish may be positioned transversely relative to the base of the workstation (e.g., the base defines the major plane), upright relative to the base of the workstation, and / or in other positions. Regardless of how the fish is positioned relative to the workstation, the bleeding tool may intersect (e.g., perpendicularly and / or at any other angle) with a plane offset (e.g., >= om) from the longitudinal, sagittal, transverse, and / or any other plane of the fish.

[0091] Preferably, the exsanguination is performed by a tool (e.g., equivalently referred to herein as a "dissection tool" and a "cutting tool") constrained in operational space. In examples, the cutting tool can have up to six degrees of freedom relative to a major plane (e.g., one or two degrees of freedom along the plane, one, two, or three degrees of angular freedom relative to the plane, and / or one degree of angular freedom relative to the plane), but may alternatively have more than six degrees of freedom. The exsanguination trajectory may be perpendicular to the fish (e.g., relative to its underbelly, relative to the side of its body, as shown in FIGS. 12B and 12C ), angled relative to the fish (e.g., angled relative to the plane of symmetry of the fish as shown in FIG. 12A , at a non-zero angle relative to an axis shown in FIG. 12B , at a non-zero angle relative to an axis shown in FIG. 12C , etc.), a combination of angled and perpendicular (e.g., depending on which plane and / or axis is considered), and / or other orientations. The tool preferably acts along a single axis, but may additionally or alternatively act along multiple dimensions (e.g., the tool moves and / or rotates along one or more non-axial degrees of freedom during actuation, the tool changes direction mid-incision, the tool makes sweep cuts in two or three dimensions, etc.), and / or be constrained in other ways. The trajectory can be determined based on parameters of one or more anatomical features (e.g., location, area, etc.), measurements of the fish (e.g., images), and / or any other suitable information. The trajectory can be determined using a bloodletting trajectory model (e.g., a path planner, a rule set, etc.) and / or any other suitable model such that the cutting tool intersects a target anatomical feature (e.g., gills, tail, artery, heart, etc.) along its trajectory. The target anatomical feature (e.g., artery) may be detected by a site model or may be implicitly intersected by the trajectory without considering the location of the anatomical feature (e.g., by a trained trajectory model that outputs an ideal trajectory based on sampled measurements).

[0092] The bleeding trajectory model functions to determine the bleeding trajectory of one or more bleeding tools (e.g., including bleeding tool angle, bleeding tool entry point, etc.). Optionally, the bleeding trajectory can be determined by a single model without using an intermediate site model. Optionally, the bleeding trajectory model may determine the trajectory separately from the site model based on parameters related to anatomical features (e.g., location, extent, etc.) output by the site model and images or other measurements of the fish (e.g., using classical methods, neural networks, etc.). Optionally, the site model (and / or trajectory model) can determine the location of anatomical features that define a stopping point, and the bleeding trajectory model can determine a cutting trajectory that ends before and / or at the stopping point. The stopping point can be the extent of the anatomical feature (e.g., gill length), a secondary anatomical feature (e.g., spine), and / or other anatomical feature. In a specific example, the bleeding trajectory extends from the cutting point of the fish tool (e.g., the bottom of the fish) toward the spine and ends before reaching the spine. However, the bleeding trajectory model may have other configurations.

[0093] In a variation, the bleeding trajectory may be configured to target or incise near any or all of the following: the gills, gill plates, heart, tail, base of the head, under the jaw, fins (e.g., pectoral fins, base of the dorsal fin, etc.), and / or other suitable areas (e.g., areas with a high volume and / or size of blood vessels). Preferably, the bleeding trajectory is determined using one or more trained models (e.g., species-specific trajectory models and / or size-specific trajectory models) based on measurement data. Optionally, a region model can be used to identify anatomical features (e.g., gills, tail, arteries, etc.) based on measurements that can be sampled from the fish prior to bleeding (e.g., examples shown in Figures 12 and 13) to identify anatomical features using one of the variations described in S200. The region model can optionally include models specialized for anatomical features (e.g., gill / brain / tail / eye detectors) and / or the ability to label multiple anatomical features. However, in a preferred variation, detection of intermediate regions is not required.

[0094] In a first variation, the step of exsanguining the fish may include making a first incision (equivalently referred to herein as a cranial and / or anterior incision, indicating, in examples, closer to the head of the fish than the tail of the fish). The cranial incision may include an incision proximal to the gills of the fish (e.g., through the gills, between the gills and the heart, behind the gill plates, etc.) and / or through the heart. The incision is preferably located between the gills and the heart (e.g., to hit an artery and / or increase the rate of blood outflow), but may be located elsewhere (e.g., through the gills, through the heart, etc.). Exemplary incisions may include incisions oriented in the longitudinal plane of the fish (e.g., where the line originates on the lower abdomen of the fish), the transverse plane (e.g., where the line originates on the dorsal surface of the fish), the sagittal plane (e.g., where the line originates on the left and / or right side of the fish), and / or any other direction. Preferably, the incision made by the incision tool is substantially parallel to the sagittal plane and extends from the lower abdomen of the fish toward the front of the fish (e.g., bypassing larger portions of harvestable flesh). In a first example, the gills are lifted from the abdomen, and the incision is located below the lifted gills. In a second example, the incision trajectory begins at the lower abdomen of the fish, extends from the bottom to the top of the fish, and terminates at a predetermined incision depth (e.g., a predetermined distance before the top boundary of the fish, a predetermined distance from the detected spine region of the fish, a predetermined incision distance, etc.) to intersect the target region (e.g., the gill region) from multiple directions depending on which side of the body the fish is located on. Optionally, a region model can be used to identify which side of the body the fish is located on. In a fourth example, the incision is perpendicular to the long axis of the fish body and the lower abdomen of the fish, but may be at another angle.

[0095] In a second variation, the step of exsanguining the fish includes making a second incision (equivalently referred to herein as a tail incision and / or a posterior incision, which may, in examples, indicate being closer to the tail of the fish than the head of the fish), which may include making an incision through and / or near (e.g., as close as possible without directly cutting) the tail (e.g., the example shown in FIG. 13 ), and / or at any other suitable point. The incision may move directly below the restrained position of the fish, laterally relative to the fish, or along any other suitable path. Relative to the fish, the incision may extend from below to above the fish, from above to below the fish, to the side of the fish, and / or in other directions. A region model (e.g., a tail detector) can optionally be used to identify the location of the tail (e.g., the center of mass of the tail, the area surrounding the tail, etc.), and optionally, the orientation and / or thickness of the tail. Based on the detected tail position, a cutting trajectory model can be used to determine an incision tool path (e.g., perpendicular to the spine). In a first example, the tail is cut with a maximum cutting depth that severs the dorsal aorta (e.g., determined based on the thickness of the tail) and leaves the tail partially attached. In a second example, the tail is completely cut off. Alternatively, the incision trajectory may be determined without identifying the location of the tail, without identifying the locations of other anatomical sites, and / or by other methods.

[0096] A first incision and a second incision may be made on one or more fish, a single incision may be made on one or more fish, or other incisions (e.g., two or more cuts, different cuts, etc.) may be made on one or more fish, and the number of incisions made may depend on one or more fish attributes and / or the results of the first incision, and / or any other cuts may be made. Preferably, the cranial incision is located anterior to the caudal incision, although in another example, the caudal incision may be located anterior to the cranial incision. Still alternatively, multiple incisions may be made at the same location along the length of the fish but at different heights (e.g., dorsal and ventral) and / or depths / widths (e.g., right and left sides).

[0097] In a third variation, the step of exsanguining the fish includes making multiple incisions (e.g., first and second incisions, first and second vascular incisions, first, second and third incisions, etc.). In specific examples, the multiple incisions can include incisions through and / or near the gills and tail. Additionally or alternatively, any other location can be cut and / or a proximal incision can be made. Any of the methods described in the first two variations can be used.

[0098] In a fourth variation, the step of exsanguining the fish includes incising the belly of the fish along its longitudinal axis (eg, along the entire extent of the belly, along a subset of the length of the belly, etc.).

[0099] Optionally, the step of bleeding the fish further includes assisted bleeding, where pressurized fluid (e.g., water, saline, compressed air, etc.) is passed through the fish's blood vessels after incision to promote blood flow. Assisted bleeding can be sensor-assisted (e.g., to identify a target insertion point for pressurized fluid application).

[0100] Optionally, the step of bleeding the fish (and / or euthanizing, further processing, etc.) includes determining a trajectory that maximizes and / or otherwise preserves the amount of harvestable meat (e.g., viable meat, edible meat, meat, etc.) remaining after bleeding. The anatomy model can output the location (e.g., defining the region) of harvestable meat within the fish. Based on the output of the anatomy model, the trajectory model can determine (e.g., using optimization techniques) a tool path to a target anatomical feature (e.g., artery, blood vessel, heart, etc.) that bypasses the harvestable meat (e.g., via gills, etc.). Optionally, the system can employ techniques used to achieve medical precision (e.g., robotic surgery, minimally invasive surgical procedures, etc.) to minimize damage to the harvestable meat. Constraints for maximizing the output of harvestable meat may include the number of degrees of freedom and / or orientation of the tool (e.g., exsanguination tool). To achieve a trajectory that can circumvent the harvestable meat, the system (e.g., the bleeding station) may include a cutting tool with multiple degrees of freedom (DOF) (e.g., having one or more DOFs along the workstation principal plane and / or one or more angular DOFs). Optionally, more complex trajectories (e.g., trajectories more closely optimized to maximize harvestable meat) can be achieved by moving the cutting tool along one or more of its degrees of freedom during tool actuation. In the example shown in FIG. 12A , for example, the angle θ can be varied from 0 degrees (as shown) to any other suitable value to optimize harvestable meat parameters.

[0101] However, the step of bleeding the fish may be carried out in other ways.

[0102] 4.5 Further Processing of Fish Step S400 The method may optionally include a step S400 of further processing the fish. The further processing may include any of the techniques described herein for detecting key anatomical features, determining processing tool instructions, and executing the instructions. Further processing may include filleting, skinning, descaling, cutting, finning, despinning, packaging, and / or other processing.

[0103] Optionally, S400 may include repeating S100 and S150 for the fish and then determining what processing to do.

[0104] In a first set of examples, after a fish has been skinned, scaled, cut, or filleted, subsequent images of the fish can be processed by one or more models (e.g., models described herein) to reveal potential quality indicators (e.g., defects such as parasites, blood clots, etc.) that would result in the fish being rejected from further processing.

[0105] In a second set of examples, after euthanasia and exsanguination, processing of subsequent images of the fish with one or more models (e.g., models described herein) can reveal that a handling error occurred during at least one of the euthanasia and exsanguination steps (e.g., the fish being excessively mangled). Processing the error can also suggest further manual processing (e.g., filleting) of the fish rather than automatically processing it.

[0106] However, the step of further processing the fish may be carried out in other ways.

[0107] 4.6 Fish Tracking Step S500 The method may optionally include a fish tracking step S500, which may function to track data for individual fish and / or sets of fish. The data may include sensor measurements collected at any time during the method, fish attributes as determined in S150, locations of key anatomical features (e.g., brain, gills, tail, etc.) as determined in S200-S400, trajectory and / or other tool parameters as determined in S200-S400, differences between stages of the method and / or between the start and end of processing (e.g., pre-processing and post-processing weights, etc.), fish source (e.g., farm ID, vessel ID, pond ID, etc.), fish farming information (e.g., capture location, feed type, feeding frequency, pond density, medical history, etc.), and / or other information.

[0108] S500 may include identifying an identifier for each fish and associating the data collected about the fish with the identifier. The identifier may be determined from measurements of the fish (e.g., feature vectors of anatomical features remaining after processing or remaining intact during processing), may be a tag attached to the fish (e.g., a visual identifier, an NFC tag, etc.), may be an identifier for a container holding the processed fish, and / or may be any other identifier.

[0109] Optionally, tracking the fish may include sending any of the data collected about the fish to a data store (e.g., a local data storage system, a cloud storage system, a database, an example shown in FIG. 3, etc.), sending any of the collected data to an ERP system or platform, printing or otherwise rendering the data, and / or otherwise storing the data.

[0110] Additionally or alternatively, tracking the fish may include determining environmental information related to the context in which the system is being used (e.g., information about the marine environment, performing marine domain recognition, etc.). For example, the environmental information may include the location in which the system is being used, the date, season, time of day the method is being performed, regulatory requirements, and / or any other suitable information. Environmental information related to the fish being tracked may optionally be recorded and stored (e.g., associated with the fish, fish batch, etc.).

[0111] By way of example, the location information may be determined using GPS, INS, RF, ECS, ECDIS, radar, AIS, GNSS, depth sounders and echo sounders, compass systems, lidar and sonar systems, satellite communication systems, data connections to the vessel and / or factory where the system is installed, and / or any other suitable location technology. In examples, the location information may be used to obtain processing requirements (e.g., regulatory requirements) to inform step S150 of sorting the fish and / or other relevant information to inform other steps of the method.

[0112] However, the step of tracking fish may be performed in other manners. Alternative embodiments may implement the above methods and / or processing modules on a non-transitory computer-readable medium storing computer-readable instructions that, when executed by a processing system, cause the processing system to perform the methods described herein. The instructions may be executed by a computer-readable medium and / or a computer-executable component integrated with the processing system. The computer-readable medium may include any suitable computer-readable medium, such as RAM, ROM, flash memory, EEPROM, an optical device (CD or DVD), a hard drive, a floppy drive, a non-transitory computer-readable medium, or any suitable device. The computer-executable component may include a computing system and / or processing system (e.g., including one or more co-located or distributed, remote or local processors) connected to a non-transitory computer-readable medium, such as a CPU, GPU, TPUS, microprocessor, or ASIC, although the instructions may alternatively or additionally be executed by any suitable dedicated hardware device.

[0113] System and / or method embodiments may include any combination and permutation of the various system components and various method processes, and / or variations or examples thereof, and one or more instances of the methods and / or processes described herein may be performed asynchronously (e.g., serially), simultaneously (e.g., in parallel), or in any other suitable order by and / or using one or more instances of the systems, elements, and / or entities described herein.

[0114] Those skilled in the art will recognize from the foregoing detailed description and drawings, and the appended claims, that modifications and variations can be made to the embodiments of the invention without departing from the scope of the invention, which is defined in the appended claims.

Claims

1. In the fish processing unit, a) receiving live fish; b) restraining the live fish in a restraining subsystem; c) collecting a set of images simultaneously with the restraining of the live fish, the set of images including imaging the fish through the restraint subsystem; d) planning a euthanasia trajectory based on said set of images using a set of euthanasia models; e) controlling a set of euthanasia tools to euthanize fish based on the euthanasia trajectory.

2. Furthermore, in the fish processing device, collecting a set of intake images prior to the step of restraining the live fish; using a fish attribute model to identify attributes of fish based on the set of captured images, wherein steps b) to e) are performed only if the attributes of the fish satisfy a set of criteria; and automatically releasing the fish from the fish processing equipment if the set of attributes of the fish does not meet a criterion.

3. The method of claim 2 , further comprising the step of deriving the set of euthanasia models based on a set of attributes of the fish.

4. The method of claim 3 , wherein the set of fish attributes includes fish species.

5. The method of claim 3 , wherein the set of fish attributes includes at least one of a size parameter or a weight parameter.

6. The method of claim 1 , wherein the euthanasia trajectory comprises a set of euthanasia parameters including a euthanasia tool angle.

7. Furthermore, in the fish processing device, collecting a second set of images after euthanasia of the fish; automatically planning a bloodletting trajectory based on the second set of images using a set of bloodletting trajectory models; and controlling a set of bleeding tools to bleed the fish based on the bleeding trajectory.

8. The method of claim 7 , wherein the bleeding trajectory comprises a set of bleeding parameters including a bleeding tool angle, and determining the bleeding tool angle comprises optimizing a harvestable meat parameter.

9. The method of claim 7 , further comprising deriving the set of bleeding trajectory models based on fish species.

10. The method of claim 1 , wherein the fish processing equipment is onboard a vessel.

11. The method of claim 1 , wherein the restraint subsystem includes a translucent material, and imaging the fish through the restraint subsystem includes imaging the fish through the translucent material.

12. collecting a set of images of live fish; automatically planning a euthanasia trajectory based on the set of images using a first set of trained models; controlling a euthanasia tool of a fish processing device to euthanize live fish according to the euthanasia trajectory; automatically planning a blood exsanguination trajectory based on the set of images using a second set of trained models; and controlling a bleeding tool of the fish processing device based on the bleeding trajectory to bleed the euthanized fish.

13. The method of claim 12 , wherein the euthanasia trajectory comprises a set of euthanasia parameters including a tool angle and an entry point.

14. 14. The method of claim 13, wherein the euthanasia tool enters the head of the fish at the entry point, the entry point being located along the sagittal plane of the live fish.

15. 13. The method of claim 12, wherein the fish processing device is configured to perform the method for each of a set of fish of varying sizes and varying species.

16. The method of claim 12 , wherein the euthanasia tool comprises a drill, and the euthanasia trajectory comprises a set of euthanasia parameters comprising a euthanasia tool position and a euthanasia tool depth.

17. 13. The method of claim 12, wherein the method does not include startling the fish, the automatically planning a bleeding trajectory includes determining a set of multiple locations along a length of the fish, and the bleeding the euthanized fish based on the bleeding trajectory includes performing a partial incision in a ventral-to-dorsal direction at each of the set of multiple locations.

18. The method of claim 12 , wherein the set of images includes at least one of an infrared image, an RGB image, an X-ray image, or a CT scan.

19. Prior to the step of collecting the set of images, further collecting a set of measurements of live fish; identifying a set of fish attributes based on the set of measurements using a set of trained fish attribute models; 13. The method of claim 12, including automatically ejecting fish from the fish processing device or performing a subsequent step of the method based on the set of attributes of the fish.

20. The method of claim 19 , wherein the set of fish attributes includes fish quality parameters.

Images