System and method for describing and counting external structures of fish

Abstract

A system (1) for in situ monitoring and recording of fish health status of fish in a fish cage (9), the system (1) comprising at least one camera housing (3), the camera housing (3) comprising a camera group (2) including at least two cameras (21) configured to take synchronized photographs for close-range digital photogrammetry, the system (1) comprising a central data processing unit (7) configured to calculate a three-dimensional model (6, 60, 69) of an object (5) synchronously photographed by the at least two cameras (21), the data processing unit (7) configured to report the number of structures (53, 61-63) in the three-dimensional model (6, 60) that deviate from the smooth surface of the object (5).

Description

[Technical field]

[0001] The present invention relates to a system for depicting underwater structures. More particularly, the present invention relates to a system including a camera configured for close-up digital photogrammetry and a data processing unit for calculating a three-dimensional model of an object photographed by the camera. Even more particularly, the present invention relates to a data processing unit configured to record and count structures that deviate from the smooth surface of the object. The object may be a live fish, which is swimming freely past the camera and is not in a channel or other guiding device. And more particularly, the recorded structures may be external crustacean parasites, such as salmon lice. Alternatively, the recorded structures may consist of damage to the skin of the fish. The camera and the data processing unit may be configured to report the dimensions of the structures or parts of the structures. The system provides for continuous monitoring and recording of the number of external crustacean parasites in the fish cage. [Background technology]

[0002] Fish farming may take place in a facility that floats on the water surface. The facility may include multiple enclosures, each of which is closed in the sense that fish cannot swim out of the enclosure or jump out of the enclosure. The enclosures may include open walls made of netting and an open bottom made of netting. Such enclosures are called open fish cages. Alternatively, the enclosures may include tight walls and a tight bottom. Such enclosures are called closed fish cages. In the following, the term fish cage is used for both open and closed fish cages, unless it is clear from the context that it is either an open or closed fish cage.

[0003] Salmon (Salmo salar) and rainbow trout (Oncorhynchus mykiss) have a spotted skin surface and spotted scales, especially on the dorsal side. External fish parasites, such as salmon lice (Lepeophtheirus salmonis), are flat. The sessile carymus I and carymus II stages of salmon lice are about 1 mm to about 2.3 mm long. These stages are difficult to recognize and identify by image recognition when attached to the fish skin. At the mobile pre-adult I stage, salmon lice are about 3.6 mm long. These are also difficult to recognize and identify by image recognition. The pre-adult II stage is about 4.3 to 5.2 mm long. The adult stages are easy to recognize and identify by image recognition, as adult males are about 5 to 6 mm long and adult females are about 8 to 12 mm long. Males and females are very large and more conspicuous than the spotted pattern of salmon and rainbow trout. Additionally, adult salmon lice are a different color than the skin of the fish.

[0004] External fish parasites are a problem in cage farming.

[0005] The '691 patent discloses a channel through which fish swim. The fish health scanner includes two imaging devices in the form of cameras. Fish health and fish well-being are recorded for the fish swimming through the channel. Data acquired from the cameras is subjected to image analysis to identify features of the fish and characterize their health.

[0006] US Patent No. 5,999,333 discloses a system that includes a camera connected to an image recognition system. A control system directs laser pulses to areas of the fish that show contrast differences, which are typical of parasites on the surface of the fish. The camera can capture images of the fish swimming through the pipe. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] International Publication No. 2018011745 Brochure [Patent Document 2] European Patent No. 2531022 Summary of the Invention [Problem to be solved by the invention]

[0008] The present invention aims to ameliorate or mitigate at least one of the disadvantages of the prior art, or at least to provide a useful alternative to the prior art. [Means for solving the problem]

[0009] This object is achieved through the features specified in the following description and the claims that follow.

[0010] The present invention relates to the use of close-proximity digital photogrammetry. More particularly, the present invention relates to the use of a system including a plurality of cameras. The system may also include one or more light sources. The cameras and light sources are adapted for use underwater. The system may further be connected to an underwater vehicle, which is capable of autonomously moving around the entire enclosed volume of the fish cage. The system may further include a charging station submerged in the fish cage. The charging station receives a supply of electrical energy from an energy source outside the fish cage. The underwater vehicle searches for the charging station to which the electrical energy is transmitted. The underwater vehicle remains at the charging station even when it is not moving around in the fish cage. The charging station may be configured to transmit data from the system to a central data processing unit.

[0011] The system may include one camera group. Each camera group includes at least two cameras. Each camera of the camera group takes synchronized pictures of objects in the fish cage. The at least two cameras of the camera group have the same optics and resolution. The synchronized images from the same camera are processed by a central data processing unit, which creates a three-dimensional model of the photographed object. The camera may be a video camera. Frames of the video camera recordings are processed by the central data processing unit, which creates a three-dimensional model of the photographed object. The data processing unit may process data from the synchronized frames of the two video cameras.

[0012] The system may include multiple camera groups, each of which may point in a different direction relative to another camera group.

[0013] One of the cameras in each camera group may be a video surveillance camera. The video surveillance camera may capture a wider viewing angle than another camera in the camera group. Data from the video surveillance camera is not used to compute the three-dimensional model of the object.

[0014] The light source is adapted to provide a light spectrum suitable for imaging underwater objects. The light intensity may be adjustable. The light may be a flashlight.

[0015] The system may include orientation sensors, such as an inclinometer, a compass, and a gyrocompass. The system may further include other sensors for measuring physical properties of the water and surroundings, such as an oxygen sensor, a temperature sensor, a depth sensor, a photo sensor, a salinity meter.

[0016] The system may include means for wireless communication as the underwater vehicle moves around inside the fish cage. Video signals from the video surveillance camera may be transmitted wirelessly in real time. Images transmitted from the video surveillance camera may monitor the feeding of feed pellets. Images transmitted from the video surveillance camera may be used to inspect the net walls of the fish cage.

[0017] The present invention also relates to placing multiple underwater vehicles, each equipped with one or more camera groups, in the same fish cage. The underwater vehicles may be configured to communicate with each other. There may be multiple charging stations in one fish cage.

[0018] The system may be configured to position an underwater vehicle within the fish cage relative to where the fish are resting within the fish cage and record the swimming direction of most of the fish in the fish cage.

[0019] The underwater vehicle may be configured to be connected to a cable, which may be advantageous if the system is used to inspect the mesh walls of the fish cage and also to inspect anchor lines and other elements of the fixation of the fish cage.

[0020] The underwater vehicle may be equipped with a means to float to the surface, which may be advantageous in the event of a fault or if the underwater vehicle cannot find its way to the charging station.

[0021] The invention is defined by the independent claims. The dependent claims define advantageous embodiments of the invention.

[0022] In a first aspect, the invention more particularly relates to a system for in situ monitoring and recording of fish health of fish in a fish cage, the system comprising at least one camera housing with a camera group including at least two cameras configured to take synchronous photographs for close-up digital photogrammetry, the system comprising a central data processing unit configured to calculate a three-dimensional model of an object synchronously photographed by the at least two cameras, the data processing unit configured to report the number of structures in the three-dimensional model that deviate from a smooth surface of the object.

[0023] The camera may be a video camera.

[0024] The camera group may include a video surveillance camera. The camera housing may include two camera groups. The camera housing may be mounted to an underwater vehicle. The underwater vehicle may be an autonomous underwater vehicle. The system may include at least two camera housings mounted to the underwater vehicle.

[0025] The system may include communication means for wireless transmission of acquired data from the at least two cameras to a central data processing unit.

[0026] The fish cage may include a system as described above. The fish cage may include a charging station for the underwater vehicles. The fish cage may include two underwater vehicles, each of which may include at least one camera housing.

[0027] In a second aspect, the present invention more particularly relates to a method for in situ monitoring and recording of fish health status of fish in a fish cage, the method comprising: Providing a system as previously described; synchronously photographing an object in the fish cage using the cameras of the camera group; processing data obtained from the system in a central data processing unit; calculating a three-dimensional model of the photographed object within the fish cage; Analysing the three-dimensional model for structures that deviate from a smooth surface of the object and counting the number of such structures; and Reporting the number of such structures.

[0028] The method may further include calculating and reporting the size of the structure. [Brief description of the drawings]

[0029] In the following, an example of a preferred embodiment is described, which is visualized in the accompanying drawings, in which: [Figure 1] FIG. 2 shows a schematic view of the camera group, showing the camera group from the front; [Diagram 2] FIG. 2 shows the camera group from the rear, on the same scale as FIG. 1. [Diagram 3] FIG. 2 shows, on a slightly smaller scale than FIG. 1, two camera groups pointing in different directions. [Figure 4] FIG. 1 shows a photograph of a salmon, with markers of various sizes placed on the salmon's surface. [Diagram 5] FIG. 5 shows a three-dimensional representation of the salmon shown in FIG. 4 based on close-range digital photogrammetry. [Figure 6] FIG. 5 shows the same photograph of salmon as in FIG. 4, but showing simulated damage being made to the fish's skin. [Figure 7] FIG. 7 shows a three-dimensional representation of the salmon shown in FIG. 6 based on close-range digital photogrammetry. [Figure 8] FIG. 7 shows a processed 3D representation of the lesion shown in FIG. 6, looking towards the head of the fish. [Figure 9] FIG. 1 shows a schematic diagram of a system according to the invention; [Figure 10] FIG. 10 shows the same as FIG. 9 in an alternative embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] In the figure, reference number 1 denotes a system according to the invention. The system 1 comprises a camera group 2 arranged in a camera housing 3. The camera group 2 comprises at least two identical cameras 21 adapted to image an object 5 based on close-up digital photogrammetry. The cameras 21 may be video cameras. The camera group 2 is shown together with a video surveillance camera 23. In the figure, four identical cameras 21 are shown in the camera group 2. The camera group 2 may comprise three identical cameras 21. The camera group 2 may comprise five identical cameras 21. The system 1 may comprise multiple camera housings 3, each camera housing 3 being equipped with one camera group 2.

[0031] The camera housing 3 is waterproof. The camera housing 3 is provided with a window 31 to admit light for the camera 21 and the video camera 23. The camera housing 3 may have an outer shape adapted to the number of cameras 21, 23 in the camera group 2.

[0032] Video surveillance camera 23 is shown positioned centrally in camera group 2. Video surveillance camera 23 may have a different position relative to other identical cameras 21 than that shown in the figure.

[0033] Figure 3 shows diagrammatically two camera groups 2 in one camera housing 3. The camera groups 2 point in different directions.

[0034] The system 1 further comprises a central data processing unit 7. Data from the cameras 21 are transmitted to the central data processing unit 7. The central data processing unit 7 is configured to form a three-dimensional model 6 of the object 5 synchronously photographed by the cameras 21 of the camera group 2. In an alternative embodiment, the central data processing unit 7 may be configured to form the three-dimensional model 6 of the object 5 photographed by the cameras 21 of the camera group 2 by using synchronized frames from a video camera.

[0035] The system 1 may also include an underwater vehicle 8, which is disposed inside a fish cage 9 (not shown) for cultivating fish. The fish cage 9 floats on the water surface 99, as shown in Figures 9 and 10. The underwater vehicle 8 may be an autonomous underwater vehicle 8. The fish cage 9 may be provided with a charging station 81, which is supplied with electrical energy from an energy source (not shown) outside the fish cage 9. The charging station 81 may be configured to transmit data from the cameras 21 of the camera group 2 to the central data processing unit 7.

[0036] In an alternative embodiment, the underwater vehicles 8 are configured to wirelessly transmit data from the cameras 21 of the camera group 2 to the central data processing unit 7. In a further alternative embodiment, multiple underwater vehicles 8 may be arranged in the same fish cage 9, as shown in Figure 10. The fish cage 9 may be provided with multiple charging stations 81. In a further alternative embodiment, each underwater vehicle 8 may comprise multiple camera housings 3, as shown in Figure 10. There may be one underwater vehicle 8 with multiple camera housings 3 in one fish cage 9.

[0037] Three different markers 4, namely 41-43, were placed on the object 5. The object 5 was a dead salmon (S. salar) 51, see FIG. 4. The first marker 41 was circular with a diameter of 12.0 mm and a height of 5.7 mm. The second marker 42 was circular with a diameter of 7.0 mm and a height of 3.1 mm. The third marker 43 was circular with a diameter of 5.1 mm and a height of 2.2 mm. One first marker 41, five second markers 42, and two third markers 43 were used. The object 5 with the markers 4 was shown with a distance of 60 cm between the object 5 and the camera group 2.

[0038] The result based on close-range digital photogrammetry is shown as a three-dimensional model 6 in Fig. 5. The markers 41-43 appear clearly on the fish skin 55 as raised structures 61-63. Fig. 5 shows that the three-dimensional modeling based on close-range digital photogrammetry is suitable for observing and counting external fish parasites. The method may further be combined with image recognition. Such a combination allows even smaller objects to be detected and identified. A high-resolution camera 21 allows such raised structures to be observed at a greater distance.

[0039] The simulated lesion 53 was inflicted on the dead salmon 51 by making two perpendicular incisions in the fish skin 55, see FIG. 6. Each incision was approximately 2 cm long and 3-4 mm deep. The results based on close-up digital photogrammetry are shown as a 3D model 60 in FIG. 7. The simulated lesion 53 clearly appears as a structure in the 3D model 60. The base data can also be used to form a 3D model 69 viewed from another viewing angle, as shown in FIG. 8. As the distance between the cameras 21 is known, the size of the lesion 53 can be shown in the 3D model 69 as shown.

[0040] The central data processing unit may be configured to present the calculated three-dimensional model 6, 60 to an observer simultaneously together with a photographic image representation of the object 5. The calculated three-dimensional model 6, 60 may be presented together with markings of structures 61-63 that deviate from a smooth surface. The observer may compare the three-dimensional model 6, 60 with the photograph to determine whether the data processing unit has correctly marked them. The method may also be used for machine learning.

[0041] It should be noted that all the above embodiments illustrate the present invention but do not limit it, and that a person skilled in the art can construct many alternative embodiments without departing from the scope of the appended claims. In the claims, the reference numbers in parentheses should not be considered limiting.

[0042] The use of the verb "to comprise" and its different forms does not exclude the presence of elements or steps not mentioned in a claim. The indefinite article "a" or "an" preceding an element does not exclude the presence of several such elements.

[0043] The fact that certain features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be used to advantage.

Claims

1. 1. A system (1) for in situ monitoring and recording of fish health status of fish in a fish cage (9), comprising at least one camera housing (3), the camera housing (3) comprising a camera group (2) including at least two cameras (21) configured to take synchronized photographs for close-range digital photogrammetry, the system (1) comprising a central data processing unit (7) configured to calculate a three-dimensional model (6, 60, 69) of an object (5) synchronously photographed by the at least two cameras (21), and the data processing unit (7) configured to report a number of structures (53, 61-63) in the three-dimensional model (6, 60) that deviate from a smooth surface of the object (5).

2. The system (1) of claim 1, wherein the camera group (2) comprises a video surveillance camera (23).

3. The system (1) according to any one of claims 1 to 2, wherein the camera housing (3) comprises two camera groups (2).

4. The system (1) according to any one of claims 1 to 3, wherein the camera housing (3) is mounted on an underwater vehicle (8).

5. The system (1) of claim 4, wherein the underwater vehicle (8) is an autonomous underwater vehicle (8).

6. A system (1) according to any one of claims 1 to 5, comprising at least two camera housings (3) mounted on an underwater vehicle (8).

7. The system (1) according to any one of the preceding claims, comprising communication means for wireless transmission of acquired data from said at least two cameras (21) to said central data processing unit (7).

8. A fish cage (9) equipped with a system (1) according to any one of claims 1 to 7, the fish cage (9) being provided with a charging station (81) for an underwater vehicle (8).

9. A fish cage (9) provided in the system (1) according to any one of claims 1 to 8, wherein two underwater vehicles (8) are provided, each underwater vehicle (8) being provided with at least one camera housing (3).

10. A method for in situ monitoring and recording fish health status of fish in a fish cage (9), comprising: Providing a system (1) according to any one of claims 1 to 9, synchronously photographing an object (5) in the fish cage (9) using the cameras (21) of the camera group (2); processing said data obtained from said system (1) in said central data processing unit (7); - calculating a three-dimensional model (6, 60, 69) of said photographed object (5), - analyzing the three-dimensional model (6, 60) for structures (53, 61-63) of the object (5) that deviate from the smooth surface and counting the number of such structures (53, 61-63); and reporting said number of such structures (53, 61-63).

11. The method of claim 10, further comprising calculating and reporting the size of said structures (53, 61-63).

Images