Automatic feeding system, image analysis device, automatic feeding adjusting device, automatic feeding method, image analysis program, automatic feeding adjusting program, and recording medium
The system addresses uneven fish distribution by using network cameras and AI to optimize feeding and water flow, achieving uniform growth and reducing cannibalism in farmed fish.
Patent Information
- Application Number
- JP2023087408
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-29
- Publication Date
- 2024-12-11
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing automatic feeding systems for farmed fish may incorrectly determine the activity levels of fish due to uneven distribution within the cage, leading to variations in growth and potential cannibalism.
The system employs multiple network cameras to capture non-overlapping views of the fish pen, a video analysis device to create comprehensive fish distribution data, and an automatic feeding adjustment device to optimize feeding and water flow based on fish density, using AI and machine learning to adjust feeding amounts and water flow according to the distribution state.
This approach reduces variations in feeding activity among fish, promotes uniform growth, and prevents cannibalism by ensuring balanced feeding and water flow across the cage.
Smart Images

Figure 2024170732000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an automatic feeding system that automatically feeds farmed fish kept in fish cages. [Background technology]
[0002] In the field of aquaculture, Patent Document 1 discloses an automatic feeding method and system for farmed fish. Patent document 1 shows an automatic feeding system that includes an automatic feeder, a control device that controls the operation of the automatic feeder, a network camera that captures the behavior of the farmed fish, particularly their feeding status, in real time, an environmental sensor, a cloud server that is connected to the control device via a communication network (Internet) and has built-in artificial intelligence (AI), and a user communication terminal that is connected to the cloud server via the communication network.
[0003] In the automatic feeding system shown in Patent Document 1, the cloud server uses artificial intelligence and machine learning to automatically determine (calculate) the daily feeding amount and feeding time (timing), and determines the activity of the farmed fish when they are feeding, that is, whether the activity of the farmed fish when they are feeding is high (eating food) or low (not eating food) by analyzing live footage (video) taken in real time by a network camera, and based on the results of this determination, controls (manages) the control device of the automatic feeder and automatically adjusts the amount and time (timing) of feed administered to the farmed fish from the automatic feeder. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2020-78278 A Summary of the Invention [Problem to be solved by the invention]
[0005] In the automatic feeding system and method disclosed in Patent Document 1, the amount and time of feed administered to the cultured fish are automatically adjusted based on whether the activity of the cultured fish at the time of feeding is high or low. However, if there is an imbalance in the distribution of the cultured fish raised in the fish cages at the time of feeding, this can result in an erroneous judgment of the activity of the cultured fish at the time of feeding. As a result, it may not be possible to grow the farmed fish in the cages to a uniform size.
[0006] The present disclosure has been made in consideration of the above-mentioned points, and aims to provide an automatic feeding system that reduces variation in predation among farmed fish raised in fish pens, and enables the farmed fish raised in fish pens to grow while suppressing bias in their growth. [Means for solving the problem]
[0007] The automatic feeding system disclosed herein comprises a plurality of network cameras that each capture an image of a different area of the fish pen and capture the entire pen, a video analysis device that creates farmed fish distribution image data by superimposing distribution information indicating the overall distribution of the farmed fish raised in the pen on image data of the entire pen that is created by synthesizing video data based on images captured by each of the plurality of network cameras, and an automatic feeding adjustment device that creates automatic feeding adjustment information using the distribution information indicating the overall distribution of the farmed fish from the video analysis device to create the set automatic feeding information. Effect of the Invention
[0008] According to the present disclosure, it is possible to reduce bias in the growth of farmed fish kept in fish cages. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 is a configuration diagram showing an automatic feeding system according to a first embodiment. [Diagram 2] FIG. 4 is a diagram showing a horizontal area in an aquarium used in the automatic feeding system according to the first embodiment; [Diagram 3]FIG. 2 is a block diagram showing a video analysis device in the automatic feeding system according to the first embodiment. [Figure 4] FIG. 2 is a diagram for explaining an example of the distribution of all the farmed fish raised in an aquarium in the automatic feeding system according to embodiment 1. [Diagram 5] FIG. 1 is a block diagram showing an automatic feeding adjustment device in the automatic feeding system according to the first embodiment. [Figure 6] FIG. 2 is a diagram for explaining the control of water flow in a tank by a pump using the automatic feeding adjustment device in the automatic feeding system according to embodiment 1. [Figure 7] FIG. 2 is a diagram for explaining control of the feed amount by the automatic feeder using the automatic feed adjustment device in the automatic feeding system according to embodiment 1. [Figure 8] 4 is a flowchart illustrating the operation of the video analysis device in the automatic feeding system according to the first embodiment. [Figure 9] 4 is a flowchart illustrating the operation of the automatic feeding adjustment device in the automatic feeding system according to the first embodiment. [Figure 10] FIG. 2 is a configuration diagram showing the hardware configuration of a video analysis device and an automatic feeding adjustment device in the automatic feeding system according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Embodiment 1 The automatic feeding system according to the first embodiment will be described with reference to Figs. The automatic feeding system according to the first embodiment is a system that automatically feeds a large number of farmed fish kept in an aquarium 100, which is a fish preserve. Although the fish preserve is not limited to an aquarium, the following description will be given assuming that the aquarium is an aquarium 100. As shown in Figure 1, the automatic feeding system of embodiment 1 comprises a plurality of network cameras 10, a data distribution switch 20, a video analysis device 30, a display and operation terminal 40, an automatic feeding adjustment device 50, a control device 60, a plurality of automatic feeders 70, and a plurality of pumps 80.
[0011] Each of the multiple network cameras 10 captures images of farmed fish being raised in an aquarium 100, which is a fish preserve, in real time. The multiple network cameras 10 are installed so that their angles of view do not overlap and capture different areas of the aquarium, and the entire aquarium 100 is captured by all of the multiple network cameras 10.
[0012] Note that installation so that the angles of view do not overlap does not mean that the angles of view do not completely overlap, but includes an installation state in which the angles of view overlap around adjacent network cameras 10 within the overlapping areas required to capture the entire aquarium 100 and the design tolerance.
[0013] For example, as shown in FIG. 2, the multiple network cameras 10 are arranged by dividing the horizontal area of the aquarium 100 into multiple rows and columns (M rows and N columns) according to the shooting range that one network camera 10 can shoot, and each network camera 10 is arranged corresponding to each area.
[0014] In the example shown in FIG. 2, the aquarium 100 is divided into 5 rows and 5 columns on the horizontal plane, and the area R 11 From region R 55 Compatible with network cameras 10 11 Network Camera 10 55 Network cameras are installed. 11 Network Camera 10 55 The area R of the tank 100 that corresponds to 11 From region R 55 is shown as an example for facilitating understanding, however, the plane of the aquarium 100 need not be round, but may be any shape, such as rectangular. Furthermore, it is sufficient that the entire aquarium 100 is photographed by the multiple network cameras 10, and the multiple network cameras 10 are positioned arbitrarily in accordance with the aquarium.
[0015] Video data based on the images captured by each of the multiple network cameras 10 is transmitted to a video analysis device 30 via a data distribution switch 20 . The data distribution switch 20 is a commonly known data distribution switch such as an L2 switch or an L3 switch that is connected to a network.
[0016] At a set timing, the video analysis device 30 creates farmed fish distribution image data by overlaying distribution information of farmed fish throughout the entire aquarium 100 on video data (hereinafter referred to as image data) of the entire aquarium 100, which is generated by combining video data (hereinafter simply referred to as multiple video data) based on images captured by each of the multiple network cameras 10 received via the data distribution switch 20. The set timing is, for example, the timing for feeding the farmed fish raised in the aquarium 100 by the multiple automatic feeders 70.
[0017] The video analysis device 30 creates image data of the entire aquarium 100 by combining multiple video data, calculates distribution information of farmed fish throughout the entire aquarium 100 based on the detection results of the farmed fish obtained from the multiple video data, and creates farmed fish distribution image data by overlaying the distribution information of farmed fish throughout the entire aquarium 100 on the created image data of the entire aquarium 100. In this example, the video analysis device 30 is a video analysis server.
[0018] As shown in Figure 3, the video analysis device 30 includes a video receiving unit 31, an image synthesis unit 32, a synthetic image memory unit 33, a farmed fish detection unit 34, a fish count measurement unit 35, a fish count measurement result memory unit 36, a distribution calculation unit 37, a distribution memory unit 38, a farmed fish distribution image data creation unit 39, and a display unit 30A. The video receiving unit 31 receives video data based on the video captured by each of the plurality of network cameras 10 and distributed from each of the plurality of network cameras 10 via the data distribution switch 20 .
[0019] The image synthesis unit 32 synthesizes the video data from the multiple network cameras 10 received by the video receiving unit 31, and creates video data (image data) of the entire aquarium 100. The image synthesis unit 32 performs distortion correction on the image data from each of the multiple network cameras 10 based on the lens information of each network camera 10, and synthesizes an area R 11 ~R 55 is converted into video data viewed from above (hereinafter referred to as distortion-corrected video data).
[0020] The image synthesis unit 32 converts the coordinate information of the fixed position of each network camera 10 at the time of installation into world coordinates (common coordinates) based on the coordinate information, and associates each distortion-corrected image data with the world coordinates. A specific coordinate in the world coordinates of one of the multiple network cameras 10 is set as a synthesis reference point for synthesis, and distortion-corrected video data for the multiple network cameras 10 is used to synthesize the images based on the synthesis reference point and the world coordinates of each network camera 10, thereby obtaining image data of the entire aquarium 100 viewed from above the aquarium 100. The composite image storage unit 33 stores the image data from the image composition unit 32 .
[0021] The farmed fish detection unit 34 detects farmed fish being raised in the aquarium 100 from the video data captured by each of the multiple network cameras 10 received by the video receiving unit 31. Detection by the farmed fish detection unit 34 is detection using object detection technology based on artificial intelligence (AI) using a learning model. The learning model was created using a known method known as "supervised learning," in which video data based on multiple images showing farmed fish is used as learning data, and then farmed fish are detected using artificial intelligence object detection technology.
[0022] The fish number measurement unit 35 counts the farmed fish detected by the farmed fish detection unit 34 for each video data from each of the multiple network cameras 10, i.e., adds up the detected farmed fish, and counts the number of fish appearing in the image shown by the video data for each video data. The fish number measuring unit 35 counts the number of farmed fish within the field of view of each network camera 10 by accumulating the results of farmed fish detection by the farmed fish detection unit 34.
[0023] The fish number measuring unit 35 adds up the number of farmed fish captured by each network camera 10 and calculates the total number of farmed fish being raised in the aquarium 100. The fish counting unit 35 performs masking processing on the video data from each of the multiple network cameras 10 for areas where the angles of view overlap around adjacent network cameras 10 .
[0024] The fish number counting unit 35 performs mask processing on the overlapping areas of the angles of view, thereby preventing duplicate counting of farmed fish. The fish number measurement result memory unit 36 stores the number of farmed fish measured by the fish number measurement unit 35 for each network camera 10 and the total number of farmed fish kept in the aquarium 100 .
[0025] The distribution calculation unit 37 calculates distribution information indicating the overall distribution of the farmed fish being raised in the aquarium 100 based on the number of fish for each video data from the multiple network cameras 10 calculated by the fish number measurement unit 35. The distribution calculation unit 37 reads out the number of fish for each video data from the multiple network cameras 10 stored in the fish count measurement result memory unit 36 and the total number of farmed fish raised in the aquarium 100, and calculates distribution information indicating the distribution state of all the farmed fish raised in the aquarium 100 using world coordinates linked to the fixed camera positions of each network camera 10 at the time of installation.
[0026] The distribution calculation unit 37 has, for example, four thresholds, a first threshold Th1 to a fourth threshold Th4, stored in the memory unit, and calculates distribution information indicating the degree of density of farmed fish, i.e., the distribution state, which is classified into five levels: “low”, “slightly low”, “normal”, “slightly high”, and “high”. The first threshold value Th1 to the fourth threshold value Th4 each indicate the number of fish. "Low" is less than the first threshold Th1, "slightly low" is greater than or equal to the first threshold Th1 and less than the second threshold Th2, "normal" is greater than or equal to the second threshold Th2 and less than the third threshold Th3, "slightly high" is greater than or equal to the third threshold Th3 and less than the fourth threshold Th4, and "high" is greater than or equal to the fourth threshold Th4.
[0027] The distribution calculation unit 37 compares the number of fish for each network camera 10 stored in the fish number measurement result memory unit 36 with the first threshold value Th1 to the fourth threshold value Th4, and obtains a distribution state divided according to the degree of density of farmed fish in the area photographed by each network camera 10 according to the first threshold value Th1 to the fourth threshold value Th4, and obtains distribution information that indicates the overall distribution state of the farmed fish raised in the aquarium 100.
[0028] An example of the distribution state of farmed fish obtained by the distribution calculation unit 37 is shown in FIG. For example, in Figure 4, the distribution of farmed fish is classified into three groups, with the first group G1 showing a distribution state indicating densities of “high,” “slightly high,” “normal,” and “slightly low,” the second group G2 showing a distribution state indicating densities of “high” and “slightly high,” and the third group G3 showing a distribution state indicating densities of “normal” and “slightly low.”
[0029] The threshold values Th1 to Th4 in the distribution calculation unit 37 are determined based on the size of the aquarium 100 and the total number of farmed fish raised in the aquarium 100, and the number of threshold values is determined based on the size of the aquarium 100, and the user inputs the threshold values Th1 to Th4 from the user interface of the display / operation terminal 40, and the threshold values are stored in the memory unit in the distribution calculation unit 37. The distribution storage unit 38 stores distribution information indicating the distribution state obtained by the distribution calculation unit 37 .
[0030] The farmed fish distribution image data creation unit 39 creates farmed fish distribution image data by superimposing the distribution information calculated by the distribution calculation unit 37 on the image data obtained by the image synthesis unit 32. The farmed fish distribution image data creation unit 39 creates farmed fish distribution image data by overlaying distribution information, which is the distribution state of all the farmed fish within the aquarium 100 that indicates the density state of the farmed fish being raised in the aquarium 100, stored in the distribution memory unit 38, onto image data of the entire aquarium 100 viewed from above, stored in the composite image memory unit 33.
[0031] The display unit 30A receives the farmed fish distribution image data from the farmed fish distribution image data creation unit 39 and displays a farmed fish distribution image based on the input farmed fish distribution image data. The cultured fish distribution image data from the cultured fish distribution image data creation unit 39 is transmitted to the display / operation terminal 40 and the automatic feeding adjustment device. The display unit 30A is a display. The display unit 30A may be a display device of the display / operation terminal 40.
[0032] The display / operation terminal 40 is used to acquire video data based on images captured by multiple network cameras 10 and display it on the display unit to check the images from the multiple network cameras 10, acquire image data of the entire aquarium 100 created by the image synthesis unit 32 and display it on the display unit to check the image of the entire aquarium 100, set thresholds Th1 to Th4 in the distribution calculation unit 37 which are input from the user interface, and acquire farmed fish distribution image data from the farmed fish distribution image data creation unit 39 and display it on the display unit to check the farmed fish distribution image, in other words, to check the analysis results of the video analysis device 30.
[0033] The automatic feeding adjustment device 50 receives automatic feeding information (set automatic feeding information) entered from the user interface of the display / operation terminal 40 via the data allocation switch 20, and creates automatic feeding adjustment information by adjusting the set automatic feeding information using the analysis results of the video analysis device 30, i.e., the distribution information from the distribution calculation unit 37 which indicates the distribution state of all the farmed fish being raised in the aquarium 100.
[0034] In addition, as distribution information indicating the distribution state of all the farmed fish raised in the aquarium 100, distribution information indicating the distribution state of all the farmed fish extracted from the farmed fish distribution image data from the farmed fish distribution image data creation unit 39 may be used. The automatic feeding information inputted from the user interface of the display / operation terminal 40 includes feeding information and pump control information.
[0035] The feeding information is information indicating the feeding of the aquarium 100, such as the amount of feeding per day or the amount of feeding per session and the number of feedings per day for each of the multiple automatic feeders 70, the timing of feeding, the feeding time from the start of feeding to the stop of feeding, and the amount of remaining food. Each of the multiple automatic feeders 70 is placed at a different position relative to the aquarium 100, and each feeds the farmed fish in the aquarium 100. The initial settings of the feeding information for each of the multiple automatic feeders 70 are the same.
[0036] The pump control information is control information that indicates, for example, control of the rotation speed of each of the multiple pumps 80. Each of the multiple pumps 80 is disposed at a different position relative to the aquarium 100, and each generates a different water flow within the aquarium 100. The initial settings of the pump control information for each of the multiple pumps 80 are the same, and in the initial settings, the force with which each pump 80 generates a water current in the aquarium 100 is the same. do.
[0037] As shown in FIG. 5, the automatic feeding adjusting device 50 includes an automatic feeding information storage unit 51, a pump control information adjustment unit 52, a pump control information storage unit 53, a feeding information adjustment unit 54, and a feeding information storage unit 55. The automatic feeding information storage unit 51 stores the automatic feeding information inputted from the user interface of the display / operation terminal 40.
[0038] The pump control information adjustment unit 52 receives pump control information in the automatic feeding information stored in the automatic feeding information memory unit 51 and distribution information from the distribution calculation unit 37 in the video analysis device 30, and creates adjusted pump control information for multiple pumps 80 using the distribution information. The pump control information adjustment unit 52 creates adjusted pump control adjustment information for each of the multiple pumps 80 so that the water flow is stronger in areas of the aquarium 100 where the density of cultured fish is high and weaker in areas where the density of cultured fish is low.
[0039] Now, assuming that multiple pumps 80 are arranged at the four corners of the aquarium 100, as shown in Figure 6 as an example, and that the distribution of farmed fish in the aquarium 100 obtained from the distribution information from the distribution calculation unit 37 is the distribution state shown in Figure 4, the adjusted pump control information for the multiple pumps 801-804 created by the pump control information adjustment unit 52 is, for example, as shown in Figure 6, pump control information for weakening the water flow for the first pump 801, pump control information for strengthening the water flow for the second pump 802, pump control information for weakening the water flow for the third pump 803, and pump control information for strengthening the water flow for the fourth pump 804.
[0040] That is, in terms of the distribution of farmed fish in the aquarium 100, the first pump 801 indicates a low density, so the water flow is weakened, the second pump 802 indicates a "normal" density in the third group G3, so the water flow is strengthened, the third pump 803 indicates a "slightly low" density in the first group G1, so the water flow is weakened, and the fourth pump 804 indicates a "high" density in the second group G2, so the water flow is strengthened.
[0041] The first pump 801 to the fourth pump 804 create stronger water currents in areas of the aquarium 100 where the density of farmed fish is high and weaker water currents in areas where the density of farmed fish is low, so that the farmed fish move from the areas with stronger water currents to the areas with weaker currents, reducing the crowding of the farmed fish. The first pump 801 to the fourth pump 804 are controlled, and as an example, the water flow in the aquarium 100 caused by the first pump 801 to the fourth pump 804 becomes stable and the movement of the farmed fish becomes less. FIG. 7 shows the distribution of the farmed fish in the aquarium 100, for example, after 5 minutes.
[0042] For example, in FIG. 11 indicates the distribution state showing the density of "normal" and "slightly low", and the second group G 21 indicates the distribution state showing the density of "high", "slightly high", and "normal", and the third group G 31 indicates a distribution state showing a "slightly low" density.
[0043] Compared to the distribution of farmed fish shown in Figure 4, the distribution of farmed fish shown in Figure 7 has fewer areas of "high" density and more areas of "normal" and "slightly low" density. By controlling the first pump 801 to the fourth pump 804 using distribution information from the distribution calculation unit 37, it becomes possible to optimize the water flow conditions in the aquarium 100 by the first pump 801 to the fourth pump 804 according to the distribution state of the farmed fish in the aquarium 100. In this way, by feeding the farmed fish using the automatic feeder 70 in a distribution state where the density of the farmed fish is improved, the variation in predation among the farmed fish kept in the aquarium 100 is reduced, bias in the growth of the farmed fish is suppressed, and cannibalism is also prevented.
[0044] It should be noted that, although the water flow is strong in areas where the density of farmed fish is high and weak in areas where the density of farmed fish is low, the pump control information adjustment unit 52 may create pump control adjustment information for controlling multiple pumps 80 so that the strength of the water flow is divided into multiple stages according to the classification of the density of farmed fish. In addition, the pump control information adjustment unit 52 may use the initially set pump control information as reference information and create pump control adjustment information for continuously increasing or decreasing the strength of the water flow based on the reference information depending on the distribution state of the farmed fish, i.e., the degree of density of the farmed fish relative to each pump 80.
[0045] Furthermore, the number of pumps 80 is not limited to four, and any number of pumps may be installed depending on the size of the water tank 100. The pump control information storage unit 53 stores the adjusted pump control adjustment information for the plurality of pumps 801-804 created by the pump control information adjustment unit 52 in association with the pumps 801-804.
[0046] The feeding information adjustment unit 54 receives feeding information (set feeding information) for farmed fish in the automatic feeding information stored in the automatic feeding information memory unit 51 and distribution information from the distribution calculation unit 37 in the video analysis device 30, and creates feeding adjustment information for multiple automatic feeders 70 using the distribution information. The feeding information adjustment unit 54 creates feeding adjustment information for each of the multiple automatic feeders 70 so that in areas of the aquarium 100 where the density of cultured fish is high, the amount of feeding per time is greater than the value of the set feeding information, and in areas where the density of cultured fish is low, the amount of feeding per time is less than the value of the set feeding information.
[0047] The amount of food fed per feeding can be adjusted by keeping the feeding time from the start to the stop of feeding the same as the initial setting feeding time and increasing or decreasing the amount of food compared to the initial setting feeding amount, by keeping the amount of food the same as the initial setting feeding amount and increasing or decreasing the feeding time compared to the initial setting feeding time, or by increasing or decreasing the feeding time and amount of food compared to the initial setting feeding time and amount of food.
[0048] Assume that multiple automatic feeders 70 are arranged at the four corners of an aquarium 100 as shown in FIG. 7, and the feeding adjustment information for the multiple automatic feeders 701-704 created by the feeding information adjustment unit 54 is, for example, the distribution state of the cultured fish in the aquarium 100 obtained from the distribution information from the distribution calculation unit 37 is, for example, the distribution state of the cultured fish in the aquarium 100 after 5 minutes, when the first pump 801 to the fourth pump 804 are controlled, the water flow in the aquarium 100 becomes stable, and the movement of the cultured fish becomes less. Then, as shown in Figure 7, feeding information is set to the first automatic feeder 701 to feed an amount per time that is less than the set amount per time, feeding information is set to the second automatic feeder 702 to feed an amount per time that is more than the set amount per time, feeding information is set to the third automatic feeder 703 to feed an amount per time that is less than the set amount per time, and feeding information is set to the fourth automatic feeder 704 to feed an amount per time that is more than the set amount per time.
[0049] In other words, the distribution of the farmed fish in the aquarium 100 indicates that the density is slightly low compared to the first automatic feeder 701, so the amount of food fed per feeding is reduced, the density is slightly low to slightly high compared to the second automatic feeder 702, so the amount of food fed per feeding is increased, the density is slightly low to normal compared to the third automatic feeder 703, so the amount of food fed per feeding is reduced, and the density is high compared to the fourth automatic feeder 704, so the amount of food fed per feeding is increased.
[0050] The second automatic feeder 702 and the fourth automatic feeder 704 feed more food per time in areas where the density of farmed fish is high, and the first automatic feeder 701 and the third automatic feeder 703 feed less food per time in areas where the density of farmed fish is low, thereby reducing the variation in the amount of food consumed by each farmed fish. By controlling the first automatic feeder 701 to the fourth automatic feeder 704 using distribution information from the distribution calculation unit 37, it becomes possible to optimize the amount of food fed to the cultured fish in the aquarium 100 by the first automatic feeder 701 to the fourth automatic feeder 704 according to the distribution state of the cultured fish in the aquarium 100.
[0051] In this manner, by controlling the first automatic feeder 701 to the fourth automatic feeder 704 using the distribution information from the distribution calculation unit 37, the amount of food fed per time from the automatic feeder 70 is adjusted taking into account the distribution state of the farmed fish in the aquarium 100, thereby reducing the variation in predation among the farmed fish kept in the aquarium 100, suppressing bias in the growth of the farmed fish, and preventing cannibalism.
[0052] In addition, larger amounts of feeding are provided per time in areas with a high density of farmed fish and smaller amounts are provided per time in areas with a low density of farmed fish. However, the feeding information adjustment unit 54 may create feeding adjustment information to control multiple automatic feeders 70 so that the feeding amount per time can be adjusted in multiple stages depending on the density of farmed fish. In addition, the feeding information adjustment unit 54 may use the feeding information indicating the initially set amount of feeding per time as reference information, and create feeding adjustment information for increasing or decreasing the amount of feeding per time in a stepless manner based on the reference information depending on the distribution state of the farmed fish, i.e., the degree of density of the farmed fish relative to each automatic feeder 70.
[0053] Furthermore, the number of automatic feeders 70 is not limited to four, but may be any number depending on the size of the aquarium 100 . The feeding information storage unit 55 stores feeding adjustment information that has been adjusted for the plurality of automatic feeders 701-704 and that is created by the feeding information adjustment unit 54, in association with the automatic feeders 701-704.
[0054] The display / operation terminal 40 is used to set the automatic feeding information input from the user interface and stored in the automatic feeding information memory unit 51, to check the pump control adjustment information stored in the pump control information memory unit 53, and to check the feeding adjustment information stored in the feeding information memory unit 55, i.e., to check the analysis results of the automatic feeding adjustment device 50.
[0055] The control device 60 receives automatic feeding adjustment information from the automatic feeding adjustment device 50, converts the input automatic feeding adjustment information into an automatic feeding adjustment signal, and outputs it to the automatic feeder 70 and pump 80 that constitute the automatic feeding device. The control device 60 includes a pump control unit 61 and an automatic feeder control unit 62 . The pump control unit 61 reads out the pump control adjustment information stored in the pump control information storage unit 53 , and outputs pump control signals for controlling each of the plurality of pumps 80 to each of the plurality of pumps 80 . Each pump 80 operates under the control of a corresponding pump control signal from controller 60 .
[0056] The automatic feeder control unit 62 reads out the feeding adjustment information stored in the feeding information storage unit 55 , and outputs feeding signals to each of the multiple automatic feeders 70 to control each of the multiple automatic feeders 70 . Each automatic feeder 70 is controlled by a corresponding feeding signal from the control device 60, and feeds the animals according to the amount of food per feeding instructed by the feeding signal.
[0057] Next, the operation of the automatic feeding system according to the first embodiment will be described with reference to FIGS. First, the operation of the video analysis device 30 will be described with reference to FIG. In step ST10, the video receiving unit 31 receives video data based on videos captured by each of the plurality of network cameras 10 at the same time and distributed from each of the plurality of network cameras 10 via the data distribution switch 20. Each received image data is linked to the coordinate information of the fixed position of each network camera 10 when the camera was installed.
[0058] In step ST11, the image synthesis unit 32 performs distortion correction on the received video data on the basis of the lens information of each network camera 10. In step ST12, the image synthesis unit 32 converts each of the distortion-corrected video data into video data viewed from above (distortion-corrected video data). In step ST12, the image synthesis unit 32 converts the coordinate information associated with each image data received into world coordinates, and associates the world coordinates with each image data to be distortion-corrected.
[0059] In step ST12, the image synthesis unit 32 sets a synthesis reference point, synthesizes each distortion-corrected image data based on the synthesis reference point, and obtains image data that is an overhead image of the entire aquarium 100 viewed from above the aquarium 100. Steps ST11 and ST12 are steps in which the image synthesis unit 32 synthesizes image data based on the images captured by each of the multiple network cameras 10 to obtain image data of the entire aquarium 100. The image data is linked to the time of shooting and stored in the composite image storage unit 33 (step ST13).
[0060] In step ST21, the farmed fish detection unit 34 detects the farmed fish being raised in the aquarium 100 from each video data received by the video receiving unit 31 in step ST10 using an AI-based object detection technology that uses a learning model. Step ST21 is a step in which the farmed fish detection unit 34 detects farmed fish being raised in the aquarium 100 from the video data based on the images captured by each of the multiple network cameras 10.
[0061] In step ST22, the fish number measuring unit 35 measures the farmed fish detected for each video data, and counts the number of fish appearing in the video represented by the video data for each video data. In step ST22, the fish number measuring unit 35 adds up the number of farmed fish obtained for each video data, and calculates the total number of farmed fish raised in the aquarium 100. Step ST22 is a step in which the fish number measuring unit 35 measures the farmed fish detected for each piece of video data captured by each of the multiple network cameras 10, and counts the number of fish for each piece of video data.
[0062] In step ST23, the number of farmed fish obtained for each image data is linked to the world coordinates corresponding to each image data and the time when the image was captured, and stored in the fish number measurement result memory unit 36. In step ST23, the total number of farmed fish is linked to the time the image was taken and stored in the fish number measurement result memory unit 36.
[0063] In step ST24, the distribution calculation unit 37 compares the number of farmed fish obtained for each video data with the set threshold values Th1 to Th4 to obtain a distribution state classified according to the degree of density of the farmed fish, and obtains distribution information indicating the distribution state of all the farmed fish kept in the aquarium 100 based on the world coordinates linked to each video data. An example of the distribution state of the farmed fish obtained by the distribution calculation unit 37 is shown in Figure 4. Step ST24 is a step in which the distribution calculation unit 37 calculates distribution information indicating the overall distribution state of the farmed fish being raised in the aquarium 100 based on the number of fish in each video data captured by each of the multiple network cameras 10.
[0064] In step ST25, distribution information indicating the distribution state of all the farmed fish being raised in the aquarium 100 is linked to the time the image was photographed and stored in the distribution memory unit 38, while also being transmitted to the automatic feeding adjustment device 50 via the network.
[0065] In step ST30, the farmed fish distribution image data creation unit 39 creates farmed fish distribution image data by overlaying distribution information indicating the density state of farmed fish in the aquarium 100 for all the farmed fish being raised in the aquarium 100, calculated by the distribution calculation unit 37 in step ST24, onto image data of the entire aquarium 100 viewed from above the aquarium 100, obtained by the image synthesis unit 32 in step ST12. The farmed fish distribution image data is input to the display unit 30A and is displayed on the display unit 30A as a farmed fish distribution image. In addition, the farmed fish distribution image data is transmitted to the display / operation terminal 40 via the network.
[0066] The video analysis device 30 is realized by a hardware configuration of a server-type computer, and as shown in FIG. 10, includes a CPU (Central Processing Unit) 101, a large-capacity semiconductor memory (RAM: Random Access Memory) 102, a storage device (ROM: Read only memory) 103 such as a hard disk device or a non-volatile recording device such as an SSD device, an input interface unit 104, an output interface unit 105, and a signal path (bus) 106.
[0067] The functional blocks of the video analysis device 30, namely, the image synthesis unit 32, the farmed fish detection unit 34, the fish count measurement unit 35, the distribution calculation unit 37, and the farmed fish distribution image data creation unit 39, are components that function when the CPU 101 loads a program stored in ROM 103 into RAM 102 and the CPU 101 executes processing based on the program loaded into RAM 102. The composite image storage unit 33 , the fish count result storage unit 36 , and the distribution storage unit 38 are components configured by the RAM 102 . The video receiving section 31, the plurality of network cameras 10, and the display / operation terminal 40 correspond to an input section 200, and the display section 30A, the display / operation terminal 40, and the automatic feeding adjustment device 50 correspond to an output section.
[0068] The program stored in ROM 103 is a video analysis program for an automatic feeding system that includes the steps of: synthesizing video data based on images taken by each of the multiple network cameras to obtain image data of the entire fish pen; detecting farmed fish raised in the fish pen from the video data based on images taken by each of the multiple network cameras; measuring the farmed fish detected in each of the video data from each of the multiple network cameras and counting the number of fish for each of the video data; calculating distribution information indicating the overall distribution of the farmed fish raised in the fish pen based on the number of fish in each of the video data from each of the multiple network cameras; and creating farmed fish distribution image data by overlaying the distribution information on the image data.
[0069] Next, the operation of the automatic feeding adjusting device 50 will be described with reference to FIGS. In step ST40, which is a preliminary step, the automatic feeding information storage unit 51 stores the automatic feeding information inputted from the user interface of the display / operation terminal 40. First, the pump control information adjustment section 52 in the automatic feeding adjustment device 50 operates.
[0070] In step ST41, pump control information adjustment unit 52 receives pump control information in the automatic feeding information stored in automatic feeding information storage unit 51 and distribution information from distribution calculation unit 37 in video analysis device 30, and creates adjusted pump control information for multiple pumps 80 using the distribution information so that the water flow is strong in areas with a high density of cultured fish and weak in areas with a low density of cultured fish. Figure 7 shows the distribution of cultured fish in aquarium 100, for example, 5 minutes later, when the water flow in aquarium 100 caused by pump 80 has stabilized and the movement of the cultured fish has decreased.
[0071] Step ST41 is a step in which the pump control information adjustment unit 52 creates pump control adjustment information for the plurality of pumps 80 from the pump control information set for the plurality of pumps 80 by using the distribution information. In step ST42, the adjusted pump control adjustment information for the multiple pumps 80 is associated with each of the multiple pumps 80 and stored in the pump control information memory unit 53, while being transmitted to the control device 60 and the display / operation terminal 40 via the network.
[0072] The operation of the pump control information adjustment unit 52 ends, and the feeding information adjustment unit 54 starts operating, for example, five minutes later. In step ST51, the feeding information adjustment unit 54 receives feeding information for farmed fish (set feeding information) in the automatic feeding information stored in the automatic feeding information memory unit 51 and distribution information from the distribution calculation unit 37 in the video analysis device 30, and uses the distribution information to create feeding adjustment information for the multiple automatic feeders 70 so that in areas with a high density of farmed fish, the amount of feeding per time is greater than the value of the set feeding information, and in areas with a low density of farmed fish, the amount of feeding per time is less than the value of the set feeding information.
[0073] Step ST51 is a step in which the feeding information adjustment unit 54 creates feeding adjustment information for the multiple automatic feeders 70 using distribution information, which is feeding information set for the cultured fish for the multiple automatic feeders 70. In step ST52, the feeding adjustment information adjusted for the multiple automatic feeders 70 is associated with each of the multiple automatic feeders 70 and stored in the feeding information memory unit 55, while being transmitted to the control device 60 and the display / operation terminal 40 via the network.
[0074] The automatic feeding adjustment device 50 is realized by a hardware configuration of a server-type computer, and as shown in Figure 10, comprises a CPU (Central Processing Unit) 101, a large-capacity semiconductor memory (RAM: Random Access Memory) 102, a storage device (ROM: Read only memory) 103 such as a hard disk device or a non-volatile recording device such as an SSD device, an input interface unit 104, an output interface unit 105, and a signal path (bus) 106.
[0075] The pump control information adjustment unit 52 and the feeding information adjustment unit 54, which are functional blocks possessed by the automatic feeding adjustment device 50, are components that function when the CPU 101 loads a program stored in ROM 103 into RAM 102 and the CPU 101 executes processing based on the program loaded into RAM 102. The automatic feeding information storage unit 51, the pump control information storage unit 53, and the feeding information storage unit 55 are components configured by the RAM 102. The distribution calculation section and the display / operation terminal 40 in the video analysis device 30 correspond to the input section 200 , and the control device 60 and the display / operation terminal 40 correspond to the output section 300 .
[0076] The program stored in ROM 103 is an automatic feeding adjustment program for an automatic feeding system that causes a computer to execute the following steps: creating feeding adjustment information for multiple automatic feeders using distribution information indicating the overall distribution of farmed fish raised in the fish pen based on images captured by multiple network cameras each capturing a different area of the fish pen; and creating pump control adjustment information for multiple pumps using the distribution information, which is set for multiple pumps each positioned at a different position relative to the fish pen.
[0077] Following operation of the video analysis device 30 and the automatic feeding adjustment device 50, the pump control unit 61 in the control device 60 reads out pump control adjustment information in the automatic feeding adjustment information from the automatic feeding adjustment device 50 stored in the pump control information memory unit 53, and outputs a pump control signal that controls each of the multiple pumps 80 to each of the multiple pumps 80. Each of the multiple pumps 80 operates in response to a pump control signal from a pump control unit 61 according to the density of the cultured fish, and changes the water flow within the aquarium 100 to improve the density of the cultured fish.
[0078] Next, the automatic feeder control unit 62 in the control device 60 reads out the feeding adjustment information from the automatic feeding adjustment device 50 stored in the feeding information storage unit 55 at a time when the water flow in the aquarium 100 is stable and the movement of the farmed fish is expected to decrease, and outputs a feeding signal to each of the multiple automatic feeders 70 to control each of the multiple automatic feeders 70.
[0079] As described above, the automatic feeding system of embodiment 1 is equipped with a video analysis device 30 which creates farmed fish distribution image data by overlaying distribution information indicating the overall distribution of the farmed fish raised in the aquarium 100 on image data of the entire aquarium 100 which is a combination of video data based on images taken by each of the multiple network cameras 10, and an automatic feeding adjustment device 50 which creates automatic feeding adjustment information using the distribution information indicating the overall distribution of the farmed fish from the video analysis device 30, based on the set automatic feeding information.Therefore, variation in predation among the farmed fish raised in the aquarium 100 can be reduced and bias in the growth of the farmed fish can be suppressed.
[0080] In addition, the video analysis device 30 in the automatic feeding system of embodiment 1 creates image data of the entire aquarium 100 by synthesizing video data based on images captured by each of the multiple network cameras 10, calculates distribution information of the farmed fish throughout the entire aquarium 100 based on the detection results of the farmed fish being raised in the aquarium 100 obtained from the video data from the multiple network cameras 10, and creates farmed fish distribution image data by overlaying the distribution information on the image data of the entire aquarium 100, so that the distribution of the farmed fish within the aquarium 100 can be obtained and feeding of the farmed fish taking into account the variation within the aquarium 100 can be achieved.
[0081] In addition, the automatic feeding adjustment device in the automatic feeding system of embodiment 1 is equipped with a pump control information adjustment unit that creates pump control adjustment information for the multiple pumps 80 using distribution information that indicates the overall distribution state of the farmed fish, and therefore the strength of the water flow in the aquarium 100 can be adjusted to improve the congestion of farmed fish in the aquarium 100, the variation in predation among the farmed fish can be reduced, and bias in the growth of the farmed fish can be suppressed.
[0082] In addition, the automatic feeding adjustment device in the automatic feeding system of embodiment 1 is equipped with a feeding information adjustment unit that creates feeding adjustment information for the multiple automatic feeders 70 using distribution information that indicates the overall distribution state of the farmed fish, based on the feeding information set for the farmed fish for the multiple automatic feeders 70.Therefore, the feeding amount is adjusted according to the density of the farmed fish, which reduces variation in predation among the farmed fish and suppresses bias in the growth of the farmed fish. [Explanation of symbols]
[0083] 10 network camera, 20 data distribution switch, 30 video analysis device, 31 video receiving unit, 32 image synthesis unit, 33 synthesized image recording unit, 34 farmed fish detection unit, 35 fish number measurement unit, 36 fish number measurement result recording unit, 37 distribution calculation unit, 38 distribution recording unit, 39 farmed fish distribution image data creation unit, 40 display / operation terminal, 50 automatic feeding adjustment device, 51 automatic feeding information storage unit, 52 pump control information adjustment unit, 53 pump control information storage unit, 54 feeding information adjustment unit, 55 feeding information storage unit, 60 control device, 61 pump control unit, 62 automatic feeder control unit, 70, 701-704 automatic feeders, 80, 801-804 pumps, 100 aquarium.
Claims
1. Multiple network cameras, each capturing different areas of the fish pen, capture the entire pen, a video analysis device that creates farmed fish distribution image data by superimposing distribution information showing the distribution state of all the farmed fish raised in the fish pen on image data of the entire fish pen, the image data being obtained by synthesizing image data based on the images taken by each of the plurality of network cameras; an automatic feeding adjustment device to which set automatic feeding information is input and which creates automatic feeding adjustment information based on the input automatic feeding information and distribution information indicating the distribution state of the entire farmed fish from the image analysis device; An automatic feeding system comprising:
2. The automatic feeding system of claim 1, wherein the video analysis device creates image data of the entire fish pen by synthesizing video data based on video footage taken by each of the multiple network cameras, calculates distribution information of the farmed fish in the entire fish pen based on detection results of the farmed fish being raised in the fish pen obtained from video data based on video footage taken by each of the multiple network cameras, and creates farmed fish distribution image data by overlaying the calculated distribution information of the farmed fish in the entire fish pen on the created image data of the entire fish pen.
3. The video analysis device includes an image synthesis unit that synthesizes video data based on the videos captured by the plurality of network cameras to obtain image data of the entire fish cage; a farmed fish detection unit that detects farmed fish raised in the fish cage from video data based on the videos captured by each of the plurality of network cameras; a fish number measuring unit that measures the farmed fish detected for each piece of video data captured by each of the plurality of network cameras by the farmed fish detection unit and counts the number of fish for each piece of video data; a distribution calculation unit that calculates distribution information indicating a distribution state of all the farmed fish raised in the fish cage based on the number of fish for each video data captured by the plurality of network cameras calculated by the fish number measuring unit; a farmed fish distribution image data creation unit that creates farmed fish distribution image data by superimposing the distribution information calculated by the distribution calculation unit on the image data obtained by the image synthesis unit; The automatic feeding system of claim 1 .
4. A plurality of automatic feeders are provided, each of which is disposed at a different position relative to the fish cage, and each of which feeds the cultured fish in the fish cage; The automatic feeding adjustment device includes a feeding information adjustment unit that creates feeding adjustment information for the plurality of automatic feeders using the feeding information for the farmed fish, which is the set automatic feeding information, based on distribution information from the image analysis device. An automatic feeding system according to any one of claims 1 to 3.
5. a plurality of pumps, each of which is disposed at a different position relative to the cage, each of which generates a different water flow within the cage; The automatic feeding adjustment device includes a pump control information adjustment unit that creates pump control adjustment information for the plurality of pumps using the pump control information, which is the set automatic feeding information, from the distribution information from the video analysis device. An automatic feeding system according to any one of claims 1 to 3.
6. an image synthesis unit that synthesizes image data based on images captured by each of the plurality of network cameras to obtain image data of the entire fish pen; a farmed fish detection unit that detects farmed fish raised in the fish cage from video data based on the videos captured by each of the plurality of network cameras; a fish number measuring unit that measures the farmed fish detected for each piece of video data captured by each of the plurality of network cameras by the farmed fish detection unit and counts the number of fish for each piece of video data; a distribution calculation unit that calculates distribution information indicating a distribution state of all the farmed fish raised in the fish cage based on the number of fish for each video data captured by the plurality of network cameras calculated by the fish number measuring unit; a farmed fish distribution image data creation unit that creates farmed fish distribution image data by superimposing the distribution information calculated by the distribution calculation unit on the image data obtained by the image synthesis unit; A video analysis device for an automatic feeding system equipped with the device.
7. a feeding information adjustment unit that creates feeding adjustment information for a plurality of automatic feeders using distribution information indicating the overall distribution of the cultured fish raised in the fish pen, the distribution information being based on images captured by a plurality of network cameras each capturing an image of a different area of the fish pen; and a pump control information adjustment unit that uses the distribution information to create pump control adjustment information for a plurality of pumps, each of which is disposed at a different position relative to the fish cage; An automatic feeding adjustment device for an automatic feeding system comprising:
8. In the video analysis device, An image synthesis unit synthesizes image data based on images captured by each of the plurality of network cameras to obtain image data of the entire fish pen; a farmed fish detection unit detecting farmed fish raised in the fish cage from video data based on the videos captured by each of the plurality of network cameras; a fish number measuring unit measuring the number of farmed fish detected for each piece of video data captured by each of the plurality of network cameras and measuring the number of fish for each piece of video data; a distribution calculation unit calculating distribution information indicating a distribution state of all the farmed fish raised in the fish pen based on the number of fish in each video data captured by the plurality of network cameras; a step of generating farmed fish distribution image data by overlaying the distribution information on the image data, In the automatic feeding adjustment device, a pump control information adjustment unit creating pump control adjustment information for a plurality of pumps, each of which is disposed at a different position relative to the fish cage, using the distribution information; A feeding information adjustment unit creates feeding information set for the cultured fish for a plurality of automatic feeders, each of which is disposed at a different position relative to the fish cage, by using the distribution information to create feeding adjustment information for the plurality of automatic feeders. Automatic feeding method.
9. A step of synthesizing image data based on images captured by each of the plurality of network cameras to obtain image data of the entire fish pen; detecting farmed fish raised in the fish cage from video data based on the videos captured by each of the plurality of network cameras; measuring the farmed fish detected for each piece of video data captured by each of the plurality of network cameras and counting the number of fish for each piece of video data; calculating distribution information indicating the overall distribution of the farmed fish raised in the fish cage based on the number of fish in each video data captured by the plurality of network cameras; A step of creating farmed fish distribution image data by overlaying the distribution information on the image data; A video analysis program for an automatic feeding system that runs on a computer.
10. a step of creating feeding adjustment information for a plurality of automatic feeders, each of which is disposed at a different position relative to a fish pen, by using distribution information indicating the overall distribution of the cultured fish raised in the fish pen, based on images captured by a plurality of network cameras each capturing an image of a different area of the fish pen; creating pump control adjustment information for a plurality of pumps, each of which is disposed at a different position relative to the fish pen, using the distribution information; An automatic feeding adjustment program for an automatic feeding system that causes a computer to execute the above.
11. A step of synthesizing image data based on images captured by each of the plurality of network cameras to obtain image data of the entire fish pen; detecting farmed fish raised in the fish cage from video data based on the videos captured by each of the plurality of network cameras; measuring the farmed fish detected for each piece of video data captured by each of the plurality of network cameras and counting the number of fish for each piece of video data; calculating distribution information indicating the overall distribution of the farmed fish raised in the fish cage based on the number of fish in each video data captured by the plurality of network cameras; A step of creating farmed fish distribution image data by overlaying the distribution information on the image data; A recording medium storing a video analysis program for an automatic feeding system that causes a computer to execute the above.
12. creating pump control adjustment information for a plurality of pumps, each of which is disposed at a different position relative to the fish pen, by using distribution information indicating the overall distribution of the farmed fish raised in the fish pen, based on images captured by a plurality of network cameras each capturing an image of a different area of the fish pen; a step of creating feeding adjustment information for a plurality of automatic feeders, each of which is disposed at a different position relative to the fish pen, by using the distribution information; A recording medium storing an automatic feeding adjustment program for an automatic feeding system that causes a computer to execute the above.
Citation Information
Patent Citations
Automatic feeding method and automatic feeding system of farmed fish
JP2020078278A
