Fish monitoring system, fish monitoring method and program

Abstract

To provide a fish monitoring system, fish monitoring method and program capable of detecting signs that fish may be affected by disease.SOLUTION: A fish monitoring system includes: cameras 5A and 5B for capturing images of a periphery of a water tank, including images of fish; an observation data acquisition unit 22 for acquiring observation data representing whether the fish is healthy or not as determined by a manager; a learning unit 241 for, on the basis of the images of the fish and the observation data, machine learning behavior of the fish when the fish is healthy by using the images when the fish is healthy as input; and a determination unit 242 for acquiring images of the fish swimming in the water tank 11 and determining signs of disease of the fish by using results of the machine learning.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a fish monitoring system, a fish monitoring method, and a program for detecting signs of disease in fish. [Background technology]

[0002] Non-Patent Document 1 discloses that the production volume of fish farming is increasing worldwide. In fish farming, farmed fish can become infected with viruses, bacteria, parasites, etc. In such cases, many fish can die at once, causing great damage to the aquaculture industry. For this reason, it is necessary to detect fish diseases as early as possible and, if a disease is determined, to take prompt measures.

[0003] Non-Patent Document 2 discloses measuring the swimming vector of a fish swimming in water and detecting whether the fish is diseased based on changes in the swimming vector. That is, it is empirically known that diseased fish (hereinafter referred to as "disordered fish") have changes in their swimming trajectories compared to healthy fish. Non-Patent Document 2 discloses using a system capable of measuring the underwater three-dimensional position coordinates of fish to determine the swimming vectors of healthy fish and diseased fish, and to detect whether the fish is diseased early based on changes in the swimming vector that appear in the diseased fish. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] https: / / www.jfa.maff.go.jp / j / kikaku / wpaper / h29_h / trend / 1 / t1_2_3_1.html "Global Fisheries and Aquaculture Production; Fisheries Agency" [Non-patent document 2] https: / / ipsj.ixsq.nii.ac.jp / ej / ?action=pages_view_main&active_action=repository_view_main_item_detail&item_id=221863&item_no=1&page_id=13&block_id=8 "Early detection of fish disease infection outbreaks based on swimming vectors in recreational fishing" Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the above-mentioned Non-Patent Document 2, since malfunctioning fish are detected based on swimming vectors, it is possible to detect disease in fish after the disease has developed, but it is difficult to detect abnormalities before the disease develops.

[0006] The present disclosure has been made in consideration of the above circumstances, and its purpose is to provide a fish monitoring system, a fish monitoring method, and a program capable of detecting signs of fish becoming infected. [Means for solving the problem]

[0007] A fish monitoring system according to one embodiment of the present disclosure is a fish monitoring system for monitoring fish in an aquarium, and includes an imaging unit that captures images of the area around the aquarium including images of the fish; an observation data acquisition unit that acquires observation data based on which an administrator has determined whether the fish are healthy; a learning unit that uses images of the fish when healthy as input and performs machine learning to learn the behavior of the fish when healthy based on the images of the fish and the observation data; and a judgment unit that acquires images of the fish in the aquarium and uses the results of the machine learning to judge whether the fish are showing signs of illness.

[0008] One aspect of the fish monitoring method of the present disclosure is a fish monitoring method for monitoring fish in an aquarium, wherein an imaging unit captures images of the area around the aquarium including images of the fish, an observation data acquisition unit acquires observation data in which an administrator has determined whether the fish are healthy, a learning unit machine-learns the behavior of the fish when healthy based on the images of the fish and the observation data, using the images of the fish when healthy as input, and a judgment unit acquires images of the fish in the aquarium and uses the results of the machine learning to judge the signs of disease in the fish.

[0009] One aspect of the present disclosure is a program for causing a computer to function as a processing unit of the fish monitoring system. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to detect signs of disease in fish. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing the configuration of a fish monitoring system according to an embodiment and its peripheral devices. [Figure 2] FIG. 2 is an explanatory diagram showing each piece of data used when determining signs of disease in fish and a plurality of parameters set for each piece of data. [Figure 3] FIG. 3 is an explanatory diagram showing an example of an observation log in which a caretaker observes fish swimming in an aquarium. [Figure 4] FIG. 4 is an explanatory diagram showing the state of the fish over time as observed by the manager. [Figure 5] FIG. 5 is an explanatory diagram showing two-dimensional data of a feature space projected using the kernel trick of the "One Class SVM" method in the learning section. [Figure 6] FIG. 6 is a flowchart showing the processing procedures of machine learning in the conversion unit, data creation unit, and learning unit according to the first embodiment. [Figure 7] FIG. 7 is a flowchart showing the processing procedures of the determination process in the conversion unit, data creation unit, and determination unit according to the first embodiment. [Figure 8] FIG. 8 is an explanatory diagram showing two-dimensional data of a feature space projected using the kernel trick of the "One Class SVM" method in the determination unit. [Figure 9] FIG. 9 is a graph showing the state of the fish over time as observed by the manager and the results of the judgment made by the judgment unit. [Figure 10] FIG. 10 is a flowchart showing the processing steps of the machine learning process in the conversion unit, the data creation unit, and the learning unit according to the second embodiment. [Figure 11] FIG. 11 is an explanatory diagram showing two-dimensional data of a feature space projected using the kernel trick of the "One Class SVM" method in the learning unit according to the second embodiment. [Figure 12] FIG. 12 is an explanatory diagram showing an observation method, observation items, and targets for acquiring normal data according to the third embodiment. [Figure 13] FIG. 13 is a block diagram showing the hardware configuration of this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] [First embodiment] Hereinafter, an embodiment will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a fish monitoring system and its peripheral devices according to an embodiment. A fish monitoring system 100 according to this embodiment monitors fish (e.g., farmed fish) swimming in an aquarium 11 and determines whether there are signs of disease in the fish. The aquarium 11 is filled with water (seawater or freshwater) for farming the fish. The aquarium 11 is connected to a sedimentation tank 12 and a filtration tank 13. The filtration tank 13 is connected to a water purification treatment unit 14.

[0013] The water in the water tank 11 is sent to the sedimentation tank 12 where foreign matter settles, and the supernatant is filtered in the filtration tank 13. It is further purified in the water purification treatment unit 14 and returned to the water tank 11. In other words, the water in the water tank 11 is circulated through a route that passes through the sedimentation tank 12, the filtration tank 13, and the water purification treatment unit 14 before being returned to the water tank 11, and is controlled so that good water quality is always maintained.

[0014] As shown in FIG. 1, a fish monitoring system 100 according to the embodiment includes a monitoring unit 101 and a processing unit 102.

[0015] The monitoring unit 101 includes a water quality data acquisition unit 1, a biological data acquisition unit 2, an environmental data acquisition unit 3, microphones 4A and 4B, cameras 5A and 5B (imaging units), a tank data acquisition unit 6, an automatic feeder 7, an amplifier 8, and a conversion unit 9.

[0016] The water quality data acquisition unit 1 acquires water quality data for the water (seawater or freshwater) filled in the aquarium 11 into which fish F1 and F2 are released. As shown in Figure 2, the water quality data includes data on water pH, water temperature, dissolved oxygen, salinity, turbidity, residual chlorine, water pressure, suspended solids, biochemical oxygen demand (BOD), chemical oxygen demand (COD), electrical conductivity, phosphorus, nitrogen, and silicon concentrations, and iron content. The water quality data acquisition unit 1 transmits the acquired water quality data to the conversion unit 9.

[0017] The biometric data acquisition unit 2 is, for example, a biologging device attached to the dorsal fin or tail fin of the fish F1 and F2. The biometric data acquisition unit 2 acquires biometric data of the fish F1 and F2. As shown in FIG. 2, the biometric data includes acceleration data on three axes (X, Y, Z) of the fish F1 and F2, their speed in water, their body temperature, the amount of light received by the fish, geomagnetism at the swimming position of the fish, pressure, myoelectric potential, and heart rate. Furthermore, when biologging is used as the biometric data acquisition unit 2, it is also possible to measure the distance traveled and swimming speed of the fish. The biometric data acquisition unit 2 transmits the acquired biometric data to the conversion unit 9 wirelessly.

[0018] The environmental data acquisition unit 3 acquires environmental data around the aquarium 11. As shown in Figure 2, the environmental data includes the amount of solar radiation, temperature, air pressure, and humidity. The environmental data acquisition unit 3 transmits the acquired environmental data to the conversion unit 9.

[0019] The microphone 4A is installed outside the aquarium 11 and acquires acoustic data outside the aquarium 11. The microphone 4A detects, for example, the sound of splashes caused by the swimming of the fish F1 and F2. The microphone 4B is installed inside the aquarium 11 and acquires acoustic data from within the water. The microphone 4B detects, for example, the swimming sounds, cries, and feeding sounds of the fish F1 and F2 swimming inside the aquarium 11. The microphones 4A and 4B transmit the detected acoustic data to the amplifier 8.

[0020] The amplifier 8 amplifies the acoustic data acquired by the microphones 4A and 4B and transmits the amplified data to the conversion unit 9.

[0021] Camera 5A is installed around aquarium 11. Camera 5A is installed in a position where the entire aquarium 11 is within its angle of view, and captures an overhead image of aquarium 11. Camera 5A captures images of food released from automatic feeder 7. By analyzing the image data captured by camera 5A, it is possible to detect the state of splashes occurring on the water surface of aquarium 11, the wave height, the time when food was released from automatic feeder 7, and the amount of food released.

[0022] The camera 5B is an underwater camera installed in the aquarium 11, and captures images of the fish F1 and F2 swimming in the aquarium 11. By analyzing the image data captured by the camera 5B, the behavioral patterns of the fish F1 and F2 can be detected. Specifically, as shown in FIG. 2, the number of up and down movements of the fish F1 and F2, the amount of food intake, weight (estimated value), length, width, density, the number of times the mouth opens and closes per unit time, the distance traveled, etc. are detected. Each of the cameras 5A and 5B transmits the captured image data to the conversion unit 9. An RGB camera, a CCD camera, etc. can be used as the cameras 5A and 5B. The cameras 5A and 5B are an example of an imaging unit that captures images of the surroundings of the aquarium 11, including the fish swimming therein.

[0023] The tank data acquisition unit 6 measures the substances that have settled in the settling tank 12 and the substances that have been filtered in the filtration tank 13. The tank data acquisition unit 6 is equipped with a residual food sensor, which detects the residual food contained in the settling tank 12 as shown in Figure 2. The residual food sensor can be, for example, an optical sensor, an acoustic sensor, or an electrochemical sensor. The optical sensor detects the amount of residual food that falls into the settling tank 12 by using the reflection and transmission of light. A high-resolution camera and a light source are used to monitor the presence and amount of sediment and estimate the amount of residual food. The optical sensor can be installed at the bottom of the aquarium 11 or the settling tank 12, allowing you to visually see how much of the food particles have sunk.

[0024] The acoustic sensor detects remaining bait by analyzing the acoustic characteristics of the water. The amount of remaining bait is estimated by detecting and analyzing the acoustic signals emitted by the falling of bait particles in the water and the accumulation of bait particles in the sedimentation tank 12. The use of a low-frequency acoustic sensor is particularly effective in detecting the movement of bait particles in the water and the accumulation of sediment.

[0025] The electrochemical sensor monitors changes in water quality and indirectly estimates the amount of remaining food. Unconsumed food and excrement affect water quality, causing fluctuations in certain chemical parameters (such as dissolved oxygen, ammonia concentration, and pH value). By detecting these changes, the amount of remaining food can be indirectly estimated. The tank data acquisition unit 6 transmits the acquired data on the amount of remaining food to the conversion unit 9.

[0026] The automatic feeder 7 automatically releases food for the fish F1 and F2 into the aquarium 11 at a preset time or at predetermined time intervals. The automatic feeder 7 transmits data on the time and amount of food released to the conversion unit 9.

[0027] The converter 9 acquires the image data, water quality data, biological data, environmental data, acoustic data, and tank data, synchronizes the time at which each piece of data was acquired, and outputs the synchronized data to the processor 102.

[0028] The arithmetic processing unit 102 includes a database 21 (denoted as “DB” in the drawing), an observation data acquisition unit 22, a data creation unit 23, and an AI processing unit 24. The arithmetic processing unit 102 is connected to a user PC 25.

[0029] The database 21 stores each piece of data transmitted from the conversion unit 9. The database 21 stores the learning data and the determination data (details of which will be described later) created by the data creation unit 23.

[0030] The observation data acquisition unit 22 acquires data from a management journal created by the manager. The management journal contains data from when the manager visually observed the fish in the aquarium 11. The manager may be, for example, an expert who is well versed in visually judging the health status of fish. The observation data acquisition unit 22 acquires observation data from the manager's judgment of whether the fish are healthy or not.

[0031] FIG. 3 is a diagram showing an example of data included in the management diary. As shown in FIG. 3, the management diary includes data on date, time, event, feeding, and observation results. An event indicates an action taken by the manager in the aquarium 11, such as administering medicine. Feeding is data related to the type and amount of food released into the aquarium 11. Observation data is data showing the results of the manager's visual observation of the inside of the aquarium 11. Examples of observation data include data such as "the water has become a little cloudy," "there is something strange about the condition of the fish," and "a disease has been discovered."

[0032] Based on each data transmitted from the conversion unit 9 and the observation data acquired by the observation data acquisition unit 22, the data creation unit 23 creates learning data to be used when performing machine learning in the AI ​​processing unit 24 described below, and judgment data to be used when judging signs of disease in fish.

[0033] Specifically, the data creation unit 23 sets each piece of data transmitted from the conversion unit 9 when the manager has evaluated the fish as healthy based on the observation data as learning data. Figure 4 is an explanatory diagram showing the time series of the manager's evaluation, drug administration, and disease detection shown in Figure 3.

[0034] Period D1 shown in FIG. 4 indicates the period during which the caretaker determined that the fish in the aquarium 11 were normal, and period D2 indicates the period during which the caretaker determined that "something was wrong." Period D3 indicates the period during which the fish were found to be ill, and period D4 indicates the period during which medication was administered. For example, during period D1, from dates "2023 / 10 / 2" to "2023 / 10 / 20," the caretaker determined that "something was wrong." During period D2, from dates "2023 / 10 / 21" to "2023 / 10 / 26," the caretaker determined that "something was wrong." During period D3, from date "2023 / 10 / 27," the caretaker discovered the fish were ill, and during period D4, from date "2023 / 10 / 29," medication was administered.

[0035] 4, the fish is judged to be healthy during period D1, and judged to be ill or a sign of illness during other periods D2 to D4. Based on period D1 during which the fish is judged to be healthy, data creation unit 23 sets each piece of data acquired from conversion unit 9 during this period D1 as learning data.

[0036] When determining whether a fish has a disease sign, the data creation unit 23 sets each piece of data transmitted from the conversion unit 9 within a predetermined period (for example, one week) before this time as data for determination. The data creation unit 23 outputs the learning data and determination data set as described above to the AI ​​processing unit 24.

[0037] The AI ​​processing unit 24 includes a learning unit 241 and a determination unit 242 .

[0038] The learning unit 241 uses the learning data created by the data creation unit 23 to perform machine learning, for example, using the "One Class SVM (Support Vector Machine)" method. In the "One Class SVM" method, normal value data set as learning data is projected onto a feature space. For example, the projection is performed using the kernel trick (nonlinear variable transformation). In this case, the normal value data is projected as far away from the origin as possible, and the area that most closely surrounds the normal data is set as the discrimination boundary. The "discrimination boundary" refers to the boundary at which a fish is determined to be a sign of disease.

[0039] By employing the "One Class SVM" method in the learning unit 241, for example, two-dimensional data as shown in Figure 5 can be obtained. That is, Figure 5 is an explanatory diagram showing two-dimensional data in a feature space projected using the kernel trick method. In Figure 5(a), the point (white circle) indicated by symbol d1 indicates data of fish that the administrator has determined to be healthy, i.e., data of normal values. Figure 5(a) shows the data for the first day generated based on data of only normal values, and it can be seen that two data groups have been formed.

[0040] Figure 5(b) shows data from the fourth day. The dots (black circles) indicated by symbol d2 represent data on fish that the administrator judged to be abnormal, and the dots (x points) indicated by symbol d3 represent data on fish that are predicted to be abnormal. Figure 5(b) shows that variation is beginning to appear between the two data groups shown in Figure 5(a). Figure 5(c) shows data from the eighth day. The variation between the two data groups has increased, indicating that the administrator is beginning to suspect something is wrong. The learning unit 241 sets the classification boundary using only the normal value data shown in Figure 5(a). That is, the learning unit 241 performs machine learning to learn the behavior of healthy fish based on images of fish and observation data, using images of healthy fish as input.

[0041] The determination unit 242 uses the determination data created by the data creation unit 23 to perform a determination process to determine whether or not the fish is at risk of contracting a disease, for example, using the "One Class SVM" method. At this time, the determination process is performed based on the discrimination boundary set by the learning unit 241. That is, the determination unit 242 acquires images of fish swimming in the aquarium 11 and determines whether or not the fish is at risk of contracting a disease, using the results of machine learning.

[0042] Next, first to fourth examples will be described as specific examples of determining signs of disease in fish.

[0043] [First Example] 6 is a flow diagram showing the machine learning processing steps in the conversion unit 9, data creation unit 23, and learning unit 241 according to the first embodiment. First, the conversion unit 9 shown in FIG. 1 acquires various types of data. Specifically, the conversion unit 9 acquires water quality data (s11) acquired by the water quality data acquisition unit 1, environmental data (s12) acquired by the environmental data acquisition unit 3, image data (s13) captured by the cameras 5A and 5B, acoustic data (s14) measured by the microphones 4A and 4B, biological data (s15) acquired by the biological data acquisition unit 2, and tank data (s16) acquired by the tank data acquisition unit 6.

[0044] The conversion unit 9 generates data after time synchronization processing of each of the above data (s17). The data creation unit 23 generates data of normal values ​​(s19) based on the data after time synchronization processing and the management log data acquired by the observation data acquisition unit 22 (s18). The data creation unit 23 outputs the data of normal values ​​to the learning unit 241 as learning data.

[0045] The learning unit 241 receives the above-described training data (s20). The learning unit 241 projects the training data in a data space (s21) using the kernel trick to generate data in a high-dimensional feature space (s22). The learning unit 241 sets a decision boundary in the high-dimensional feature space (s23). As a result, for example, decision boundaries Q1 and Q2 shown in FIG. 5(b) are set.

[0046] 7 is a flow diagram showing the processing steps of the determination process in the conversion unit 9, data creation unit 23, and determination unit 242 according to the first embodiment. The conversion unit 9 shown in FIG. 1 acquires various types of data. Specifically, the conversion unit 9 acquires water quality data (s31) acquired by the water quality data acquisition unit 1, environmental data (s32) acquired by the environmental data acquisition unit 3, image data (s33) captured by the cameras 5A and 5B, acoustic data (s34) measured by the microphones 4A and 4B, biological data (s35) acquired by the biological data acquisition unit 2, and tank data (s36) acquired by the tank data acquisition unit 6.

[0047] The conversion unit 9 generates data after time synchronization processing of each of the above data (s37). The data creation unit 23 sets each of the data transmitted from the conversion unit 9 as determination data within a predetermined period before the predicted time of the fish's signs of disease, based on the data in the management log (s38) (s39). The data creation unit 23 outputs the set determination data to the determination unit 242.

[0048] The determination unit 242 receives the above-mentioned determination data (s40). The determination unit 242 projects the determination data in a data space (s41) using the kernel trick to generate data in a high-dimensional feature space (s42). The determination unit 242 determines whether the fish is at risk of contracting a disease based on the high-dimensional feature space and the above-mentioned decision boundary (s43). The determination unit 242 outputs the determination result, whether it is normal (s44) or abnormal (s45), to the user PC 25 shown in FIG. 1.

[0049] Fig. 8 is an explanatory diagram showing two-dimensional data in a feature space projected using the "One Class SVM" method in the determination unit 242. Fig. 8(a) shows data from the first day, and the point (white circle) indicated by symbol d1 shows data when only healthy fish were swimming in the aquarium 11.

[0050] Figure 8(b) shows the data for the fourth day, and the point (black circle) indicated by symbol d2 occurs outside the classification boundaries Q1 and Q2, indicating that some abnormal values ​​have occurred, but that the data is not noticeable to a person. In other words, the data shows that the fish in the aquarium 11 are showing signs of illness, but that the signs are not noticeable to a person with low experience.

[0051] Figure 8(c) shows the data from the eighth day, and indicates that the variance between the two data groups shown in Figure 8(a) has increased, indicating that the administrator is beginning to sense that something is wrong.

[0052] FIG. 9 is a graph showing the results of the determination made by the determination unit 242 of the AI ​​processing unit 24. The curve S1 shown in FIG. 9 indicates the AI-calculated value of the degree of abnormality of the fish. In FIG. 9, the threshold value is set to 0.8, and the determination unit 242 determines that a sign of disease is present when the AI-calculated value exceeds the threshold value. As can be seen from FIG. 8, the AI-calculated value S1 exceeds the threshold value (0.8) before the period D2 when the manager realizes that "something is wrong." In other words, the AI-calculated value S1 exceeds the threshold value at the times indicated by symbols p1, p2, p3, and p4 in FIG. 9. It can be seen that a sign of disease can be recognized before the manager visually notices anything abnormal in the fish.

[0053] As described above, the fish monitoring system 100 of the first embodiment is a system for monitoring fish in an aquarium 11, and includes cameras 5A, 5B (imaging units) that capture images of the area around the aquarium including images of the fish, an observation data acquisition unit 22 that acquires observation data used by an administrator to determine whether the fish are healthy, a learning unit 241 that uses images of healthy fish as input and performs machine learning to learn the behavior of healthy fish based on the images of the fish and the observation data, and a judgment unit 242 that acquires images of the fish in the aquarium 11 and uses the results of machine learning to judge whether the fish are showing signs of illness.

[0054] In the first embodiment, the behavior of the fish is learned to determine signs of illness, so even if an unskilled manager is monitoring the fish, the manager can reliably recognize signs of illness and respond to fish diseases. This makes it possible to prevent problems such as mass deaths of the fish F1 and F2 swimming in the aquarium 11, and avoid major losses.

[0055] In addition to the images captured by cameras 5A and 5B, the system also includes a water quality data acquisition unit 1 that acquires water quality data, microphones 4A and 4B that measure sounds around the aquarium, an environmental data acquisition unit 3 that acquires the environment surrounding aquarium 11, a biological data acquisition unit 2 that acquires biological data about the fish, and a tank data acquisition unit 6 that acquires tank data about the settling tank 12 or filtration tank 13 that purifies the water in aquarium 11. Signs of disease in fish are determined using at least one of the related data consisting of water quality data, acoustic data, environmental data, biological data, and tank data. This allows for more accurate detection of signs of disease in fish.

[0056] The observation data acquired by the observation data acquisition unit 22 includes data on fish visually observed by the manager and judged to be healthy, and data on fish judged to be unhealthy. The learning unit 241 performs machine learning using image data when the manager judges the fish to be healthy. This allows for highly accurate machine learning, making it possible to more accurately determine signs of disease in fish.

[0057] [Second Example] Next, a second embodiment will be described. In the first embodiment described above, it was determined whether fish F1 and F2 were healthy based on water quality data, environmental data, image data, acoustic data, biological data, and tank data, and machine learning was performed using the data when the fish were healthy.

[0058] Each piece of data used in machine learning contains multiple parameters, as shown in Figure 2. Specifically, multiple parameters such as "pH" and "water temperature" are set as water quality data. Among the multiple parameters mentioned above, there may be some parameters that do not contribute to predicting disease in fish.

[0059] In the second embodiment, unnecessary calculations are omitted and the calculation load of the entire system is reduced by excluding parameters that are not necessary for machine learning based on the decision boundary generated by the processing of the learning unit 241 described above. This will be explained in detail below.

[0060] Fig. 10 is a flow diagram showing the processing procedure of the second embodiment. As shown in Fig. 10, a plurality of parameters, such as pH, water temperature, oxygen concentration, salinity concentration, light intensity, and turbidity, are set in the water quality data. Of these, for example, machine learning is performed using water quality data including "pH," and the discrimination boundary is determined by the process shown in Fig. 6 described above, and the result obtained is set as the first judgment result.

[0061] Furthermore, machine learning is performed in the same manner as above, excluding "pH" from the water quality data, to determine the classification boundary. The result obtained is set as the second judgment result. The learning unit 241 compares the first judgment result with the second judgment result, and determines whether there is a change in both results (s51 in FIG. 10). For example, the similarity between both graphs is calculated.

[0062] The change in the judgment result can be determined by the similarity evaluation index between images obtained by the above-mentioned "One Class SVM" method. For example, suppose the graph shown in Figure 11(a) is obtained as the first judgment result, and the graph shown in Figure 11(b) is obtained as the second judgment result. In this case, for each graph in Figure 11(a) and (b), "AKAZE," "FCN score (pIxel-level-accuracy score)," "Dice count," "Jacard count," "Simpson count," etc. are used to determine whether the similarity is 0.99 or higher, assuming that perfect similarity is "1." If the similarity is 0.99 or higher, this parameter is excluded from the data used for machine learning. Conversely, parameters with a value less than 0.99 are determined to affect the results and are not excluded from the data used for machine learning.

[0063] That is, for "pH," if the similarity is 0.99 or more, this parameter is excluded, and if it is less than 0.99, this parameter is used to determine the signs of disease in fish.

[0064] That is, the learning unit 241 calculates a first determination result when machine learning is performed using the image and at least one of the associated data, calculates a second determination result when machine learning is performed after deleting at least one parameter included in the associated data from the image and at least one of the associated data, and if the similarity between the first determination result and the second determination result is equal to or greater than a predetermined threshold, performs machine learning using the image and the associated data from which the parameter has been deleted. The determination unit 242 determines a sign of morbidity using the image and the associated data from which the parameter has been deleted.

[0065] In this way, in the second embodiment, the signs of disease are estimated by excluding parameters that are determined to be unnecessary for estimating the disease in fish, thereby reducing the computational load required for estimating the signs of disease.

[0066] [Third Example] Next, a third example will be described. In the third embodiment, the data used to estimate signs of disease in fish are obtained from image data acquired by the monitoring unit 101, and include splashes in the aquarium 11, the state of the water surface, bubbles, and the number of feedings supplied from the automatic feeder 7. Furthermore, the sounds of water and feeding made by the fish while they are swimming are measured by the microphone 4B. The amount of remaining food accumulated in the sedimentation tank 12 is acquired from the tank data acquisition unit 6. The turbidity of the water in the aquarium 11 is acquired from the water quality data acquisition unit 1.

[0067] Specifically, as shown in Figure 12, cameras 5A and 5B are set to a shutter speed of 1 / 1000 sec or faster to measure the number of splashes when the fish is healthy and when it is in a diseased state, and the difference is analyzed to determine the parameters. Microphones 4A and 4B measure the sound of water splashing in tank 11. Based on the measured acoustic data, the feeding sounds when the fish is healthy and when it is in a diseased state are measured, and the difference is analyzed to determine parameters such as whether a sound level below 50 dB or 80 dB is considered to be a diseased state.

[0068] Based on the images captured by cameras 5A and 5B, the water settles when the fish is healthy and when there is a problem, and the time it takes to reach a steady state (when the water surface settles) is compared to determine the parameters.

[0069] A wave height meter (not shown) measures the amount of water splashing in the tank 11 when the fish are healthy and when there is a problem, and determines parameters such as whether a wave height of 5 cm or less is a problem.

[0070] The amount of remaining food when the fish is healthy and when it is in an unhealthy state is measured by a remaining food sensor, and the difference is analyzed to determine the parameters.

[0071] The data creation unit 23 generates learning data based on the above-mentioned parameters. The learning unit 241 performs machine learning using this learning data. The determination unit 242 acquires the above-mentioned data from the conversion unit 9 and performs a determination process of whether or not the data is a sign of illness using this data.

[0072] That is, it is equipped with a water quality data acquisition unit 1 that acquires water quality data in aquarium 11, and microphones 4A and 4B that measure sounds around the aquarium, and a learning unit 241 performs machine learning based on the water condition in aquarium 11 and the number of feedings into aquarium 11 acquired from images taken by cameras 5A and 5B, the turbidity of the water in aquarium 11 acquired by water quality data acquisition unit 1, and the splashing sounds and feeding sounds measured by microphones 4A and 4B. A determination unit 242 determines signs of disease in the fish using the water condition, the number of feedings, the turbidity, the splashing sounds, and the feeding sounds.

[0073] In this way, in the third embodiment, it is possible to quantitatively determine the extent to which the fish F1, F2 swimming in the aquarium 11 are biting on food, and to estimate signs of disease in the fish.

[0074] [Fourth Example] Next, a fourth example will be described. In the fourth example, signs of illness are determined based on the normal behavior of fish F1 and F2. In the fourth embodiment, the data used to determine signs of illness in fish is obtained from image data acquired by the monitoring unit 101, including the number of tail fin oscillations, swimming direction, number of up and down movements, number of mouth openings and closings, and fin position information of fish F1 and F2. Furthermore, acceleration data in the three axes X, Y, and Z of fish F1 and F2, as well as their speed through the water, are measured from the biological data acquisition unit 2.

[0075] The data creation unit 23 generates learning data based on the above-mentioned data, the management log data, and data when the fish is healthy. The learning unit 241 performs machine learning using this learning data. Furthermore, the determination unit 242 acquires the above-mentioned data from the conversion unit 9 and performs a process of determining signs of illness using this data.

[0076] That is, the system is provided with a biological data acquisition unit 2 that acquires biological data of the fish, and a learning unit 241 performs machine learning based on at least one of the behavioral data acquired from images captured by cameras 5A and 5B, such as the number of times the fish's tail fin vibrates, the number of times it moves up and down, the number of times its mouth opens and closes, its swimming direction, and its fin position, and the swimming speed and acceleration of the fish acquired from the biological data acquisition unit 2. A determination unit 242 determines whether the fish has signs of disease using the at least one of the behavioral data and the swimming speed and acceleration of the fish.

[0077] In this way, in the fourth embodiment, it is possible to quantitatively determine the extent to which the swimming behavior of the fish F1, F2 swimming in the aquarium 11 has changed compared to normal, and to determine signs of illness.

[0078] In this embodiment, various types of detection data and the intuition and experience of skilled managers are quantified, making it possible to detect signs of disease in fish at an earlier stage, thereby making it possible to prevent problems such as mass deaths of fish due to virus infection.

[0079] It also enables medication to be administered at the appropriate time, reducing unnecessary medication and contributing to cost reduction.

[0080] The monitoring unit 101 can automatically grasp the status of the fish F1 and F2, thereby reducing manual labor. In this embodiment, even a beginner who is not familiar with monitoring work can observe the farmed fish almost as well as an experienced manager.

[0081] The processing unit 102 of the fish monitoring system 100 of this embodiment described above can be, for example, a general-purpose computer system including a CPU (Central Processing Unit, processor) 901, a memory 902, a storage 903 (HDD: Hard Disk Drive, SSD: Solid State Drive), a communication device 904, an input device 905, and an output device 906, as shown in Fig. 13. The memory 902 and the storage 903 are storage devices. In this computer system, the CPU 901 executes a predetermined program loaded onto the memory 902, thereby realizing each function of the processing unit 102.

[0082] The arithmetic processing unit 102 may be implemented in one computer or in multiple computers, or may be a virtual machine implemented in a computer.

[0083] The program for the arithmetic processing unit 102 can be stored in a computer-readable recording medium such as an HDD, an SSD, a USB (Universal Serial Bus) memory, a CD (Compact Disc), or a DVD (Digital Versatile Disc), or can be distributed via a network. The computer-readable recording medium is, for example, a non-transitory recording medium.

[0084] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure. [Explanation of symbols]

[0085] 1. Water quality data acquisition section 2 Biometric data acquisition unit 3 Environmental data acquisition section 4A, 4B microphones 5A, 5B Camera (imaging unit) 6 Tank data acquisition unit 7 Automatic feeder 8 Amplifiers 9 Conversion section 11 Aquarium 12 Settling tank 13 Filtration tank 14 Water purification treatment unit 21 Databases 22 Observation data acquisition unit 23 Data Creation Department 24 AI processing section 100 Fish Monitoring System 101 Monitoring Department 102 Processing unit 241 Learning Department 242 Judgment section F1, F2 fish

Claims

1. A fish monitoring system for monitoring fish in an aquarium, an imaging unit that captures an image of the surroundings of the aquarium including an image of the fish; an observation data acquisition unit that acquires observation data that indicates whether the fish is healthy or not, determined by an administrator; a learning unit that performs machine learning to learn the behavior of the fish when it is healthy, based on the image of the fish and the observation data, using the image of the fish when it is healthy as an input; a determination unit that acquires images of the fish in the aquarium and determines signs of disease in the fish using the results of the machine learning; A fish monitoring system with

2. The apparatus further includes at least one of a water quality data acquisition unit that acquires water quality data in the aquarium, a microphone that measures sounds around the aquarium, an environmental data acquisition unit that acquires the surrounding environment of the aquarium, a biological data acquisition unit that acquires biological data of the fish, and a tank data acquisition unit that acquires tank data of a sedimentation tank or a filtration tank that purifies the water in the aquarium, the learning unit performs machine learning using, in addition to the image of the fish, at least one of the associated data consisting of the water quality data, sounds around the aquarium, the surrounding environment, biological data, and tank data; The determination unit determines a sign of disease in the fish by using at least one of the related data in addition to the image of the fish. The fish monitoring system of claim 1 .

3. The observation data includes data indicating that the fish are healthy and data indicating that the fish are not healthy, based on visual observation by the manager, The learning unit performs the machine learning using image data when the administrator determines that the person is healthy.

3. A fish monitoring system according to claim 1 or 2.

4. The learning unit calculating a first determination result when machine learning is performed using the image and at least one of the associated data; calculating a second determination result when machine learning is performed after deleting at least one parameter included in the associated data from the image and at least one associated data; If the similarity between the first determination result and the second determination result is equal to or greater than a predetermined threshold, the machine learning is performed using the image and related data from which the parameters have been deleted; The determination unit determines the sign of morbidity using the image and related data from which the parameters have been deleted.

3. A fish monitoring system according to claim 1 or 2.

5. The water quality data acquisition unit acquires water quality data in the aquarium, and a microphone measures sounds around the aquarium. The learning unit Machine learning is performed based on the state of the water in the aquarium and the number of feedings into the aquarium acquired from the images captured by the imaging unit, the turbidity of the water in the aquarium acquired by the water quality data acquisition unit, and the splashing sounds and feeding sounds measured by the microphone; The determination unit The water condition, feeding frequency, turbidity, splashing sound, and feeding sound are used to determine the signs of disease in the fish. The fish monitoring system of claim 1 .

6. Further, a biological data acquisition unit is provided to acquire biological data of the fish, The learning unit At least one of the following motion data is acquired from the image captured by the imaging unit: the number of times the tail fin of the fish oscillates, the number of times it moves up and down, the number of times its mouth opens and closes within a certain period of time; the swimming direction of the fish; and the position of its fins. The swimming speed and acceleration of the fish acquired from the biological data acquisition unit; Based on this, machine learning is performed. The determination unit At least one of the motion data and the swimming speed and acceleration of the fish are used to determine a sign of illness in the fish. The fish monitoring system of claim 1 .

7. A fish monitoring method for monitoring fish swimming in an aquarium, comprising: an imaging unit that captures an image of the surroundings of the aquarium including an image of the fish; an observation data acquisition unit acquires observation data in which the manager determines whether the fish is healthy; a learning unit, based on the image of the fish and the observation data, using the image of the fish when healthy as an input, and performing machine learning to learn the behavior of the fish when healthy; A determination unit acquires images of fish swimming in the aquarium and determines signs of disease in the fish using the results of the machine learning. Fish monitoring methods.

8. A program that causes a computer to function as the learning unit and the determination unit of the fish monitoring system according to claim 1 or 2.

Images