Behavior determination device, behavior determination method, and behavior determination program

Abstract

An action determination device (10) is provided with: an action determination unit (27) that determines the action of a subject creature, i.e. an aquatic life, using an action feature and a reference action model (31), said action feature being a feature corresponding to the action of the subject creature and calculated on the basis of the time transition of at least one of the position and the flexion of each joint of the subject creature, said reference action model comprising a model for indicating the action category of the subject creature on the basis of the action feature; and a rearing condition control unit (28) that, according to the determined action of the subject creature, generates data for determining whether to cause a device for feeding the subject creature to feed the subject creature.

Description

[Technical field]

[0001] The present disclosure relates to a behavior determination device, a behavior determination method, and a behavior determination program. [Background technology]

[0002] In recent years, land-based aquaculture, which can stably produce aquatic organisms, has been attracting attention. In conventional land-based aquaculture, it was necessary for experienced workers to grasp the health of the aquatic organisms, adjust the water quality, and operate equipment related to feeding, etc., while observing the aquatic organisms, relying on intuition to carry out such tasks. Patent Document 1 discloses technology for automating systems in land-based aquaculture with the aim of reducing the labor required for these manual tasks. The technology disclosed in Patent Document 1 determines the three-dimensional position of each aquatic organism, such as fish, in an aquarium from a captured image, as well as individual characteristic data of each organism, such as its size and body shape, and tracks the organism based on the determined position and characteristic data, and monitors the organism's growth status based on the tracking results. Furthermore, the technology is characterized by having a means for controlling the amount of feed and growth environment conditions using statistically managed data to achieve optimal growth status. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-250382 A Summary of the Invention [Problem to be solved by the invention]

[0004] The technology disclosed in Patent Document 1 does not take into consideration the timing of feeding. Therefore, this technology has a problem in that feeding is not always performed at an appropriate timing, and therefore efficient feeding may not be possible.

[0005] The present disclosure aims to provide data that enables an automatic feeding device to feed the aquatic organisms being cultivated at appropriate times. [Means for solving the problem]

[0006] The behavior determination device according to the present disclosure comprises: A feature amount calculated based on time transition of at least one of the position and bending of each joint of a target organism, which is an aquatic organism, and a behavior feature amount corresponding to the behavior of the target organism and a reference behavior model including a model indicating a classification of the behavior of the target organism according to the behavior feature amount are used to calculate the behavior of the target organism. Classification A behavior determination unit that determines The behavior of the target organism is determined based on the data for determining whether or not the apparatus for feeding the target organism should feed the target organism. Classification A growth condition control unit that generates a growth condition according to the Equipped with. Effect of the Invention

[0007] According to the present disclosure, the behavior determination unit determines the behavior of the target organism according to the behavior feature, and the growth condition control unit generates data for a device that feeds the target organism to decide whether or not to feed the target organism according to the behavior of the target organism. Here, the device corresponds to an automatic feeding device, and the automatic feeding device decides whether or not to feed the target organism based on the data, so that the automatic feeding device can feed the target organism at an appropriate timing. Therefore, according to the present disclosure, data can be provided that allows the automatic feeding device to feed the cultivated aquatic organism at an appropriate timing. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing an example of the configuration of a behavior determination system 9 according to the first embodiment. [Diagram 2] 5 is a diagram for explaining the processing of a skeleton extraction unit 23 according to the first embodiment. FIG. [Diagram 3] 1A and 1B are diagrams for explaining the processing of the behavior determination unit 27 in embodiment 1, where (a) is a diagram showing a specific example when the target organism is a shrimp, and (b) is a diagram showing a specific example when the target organism is a catfish. [Figure 4] FIG. 2 is a diagram for explaining a feeding behavior model according to the first embodiment. [Diagram 5] FIG. 2 is a diagram for explaining a non-feeding behavior model according to the first embodiment. [Figure 6] FIG. 2 is a diagram for explaining a normal behavior model according to the first embodiment. [Figure 7] FIG. 2 is a diagram for explaining an abnormal behavior model according to the first embodiment. [Figure 8] 1 is a diagram showing an example of a hardware configuration of a behavior determining device 10 according to a first embodiment. [Figure 9] 3 is a flowchart showing the operation of the behavior determining device 10 according to the first embodiment. [Figure 10] FIG. 13 is a diagram showing an example of a hardware configuration of a behavior determining device 10 according to a modification of the first embodiment. [Figure 11] FIG. 13 is a diagram showing an example of the configuration of a behavior determining system 9 according to a second embodiment. [Figure 12] 13A to 13C are diagrams for explaining the processing of the learning unit 301 according to the second embodiment, where (a) is a diagram showing an example of creating a feeding behavior model, (b) is a diagram showing an example of creating a non-feeding behavior model, and (c) is a diagram showing an example of creating an abnormal behavior model. [Figure 13] 10 is a flowchart showing the operation of a remaining feed amount obtaining section 200 and a learning section 301 according to the second embodiment. [Figure 14] 10 is a flowchart showing the operation of the corpse number acquisition unit 400 and the learning unit 301 according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] In the description of the embodiments and the drawings, the same elements and corresponding elements are given the same reference numerals. Descriptions of elements given the same reference numerals are omitted or simplified as appropriate. Arrows in the drawings primarily indicate data flow or processing flow. In addition, "part" may be read as "circuit," "step," "procedure," "processing," or "circuitry" as appropriate.

[0010] Embodiment 1 Hereinafter, the present embodiment will be described in detail with reference to the drawings.

[0011] ***Configuration Description*** 1 shows an example of the configuration of a behavior determination system 9 according to the present embodiment. As shown in FIG. 1, the behavior determination system 9 includes a camera 1, a behavior determination device 10, an automatic feeding device 60, and a growth environment control device 70. The behavior determination device 10 has a function of determining the behavior of an aquatic organism. In this application, an aquatic organism typically refers to an aquatic organism cultivated in an aquarium. As shown in FIG. 1, the behavior determination device 10 includes an image acquisition unit 21, a target organism detection unit 22, a skeleton extraction unit 23, a target organism tracking unit 24, a motion feature amount calculation unit 25, a bending feature amount calculation unit 26, a behavior determination unit 27, and a growth condition control unit 28, and stores a reference behavior model 31. In addition, the behavior determination device 10 is connected to a camera 1, an automatic feeding device 60, and a growth environment control device 70. The camera 1 is a device for photographing aquatic organisms, and there may be multiple cameras. The camera 1 outputs video data showing the image photographed by the camera 1. The video data is data showing multiple frames, and as a specific example, is video data. The automatic feeding device 60 is a device that feeds target organisms, and is a device that automatically feeds aquatic organisms. The target organisms are aquatic organisms having joints, and specific examples thereof include fish and crustaceans. The growth environment control device 70 is a device that controls the growth environment of aquatic organisms.

[0012] The video acquisition unit 21 acquires the video data output by the camera 1 .

[0013] The target organism detection unit 22 detects a target organism in each frame indicated by the video data acquired by the video acquisition unit 21. The target organism detection unit 22 may detect the location of the target organism in each frame by using a trained model based on AI (Artificial Intelligence) technology. As a specific example, the trained model is a model that receives data indicating a frame as input and outputs the result of detecting the target organism. When there are multiple frames in which a target organism is detected, target organism detection section 22 generates, as detection information, information indicating the target organism detected in each frame in which the target organism is detected.

[0014] The skeleton extraction unit 23 extracts the skeleton of the target organism indicated by the detection information generated by the target organism detection unit 22, and generates skeleton information indicating the extracted skeleton. At this time, typically, the skeleton extraction unit 23 extracts each joint of the target organism in each frame indicated by the video data, and generates information indicating each extracted joint as skeleton information. At this time, the video data indicates a plurality of frames in which the target organism is shown. The skeleton information may be information indicating the position and bending of each joint, or may be information indicating parts such as the eyes, each fin, the tail, and the legs. The skeleton extraction unit 23 may change the skeleton to be extracted depending on the type of the target organism. In addition, when the target organism is a crustacean, the target organism tracking unit 24 extracts the joints of the target organism as the skeleton. Fig. 2 is a diagram for explaining the processing of the skeleton extraction unit 23. When the target organism is a fish, as a specific example, the skeleton extraction unit 23 extracts each part of the skeleton of the target organism, that is, the left eye, the right eye, the left pectoral fin, the right pectoral fin, the pelvic fin, the center of the dorsal fin, the anal fin, the base of the caudal fin, and the tip of the caudal fin, as shown in Fig. 2.

[0015] The target organism tracking unit 24 tracks the target organism based on the skeletal information, and generates information indicating the tracking results as tracking information. Specifically, the target organism tracking unit 24 tracks the target organism based on multiple frames indicated by the video data acquired by the video acquisition unit 21 and the skeletal information generated by the skeletal extraction unit 23, and generates tracking information indicating the tracking results. At this time, the target organism tracking unit 24 sets the tracking reference point, as a specific example, to a certain point in the head of the target organism. As a specific example, the certain point is a point corresponding to the mouth, eyes, etc. The tracking information, as a specific example, indicates the moving distance and moving time of the target organism.

[0016] The motion feature amount calculation unit 25 calculates the motion feature amount of the target organism based on the tracking information generated by the target organism tracking unit 24. The motion feature amount is a feature amount related to the motion of the target organism, and corresponds to at least one of the movement, position, and posture of the target organism. As a specific example, the motion feature amount is composed of at least one of a value indicating the position of the target organism, a value indicating the posture of the target organism, a value indicating the swimming distance of the target organism, a value indicating the swimming speed of the target organism, and a value indicating the acceleration of the target organism while swimming. The motion feature amount calculation unit 25 may change the feature amount calculated according to the item of the reference behavior model 31.

[0017] The bending feature amount calculation unit 26 calculates the bending feature amount of the target organism based on the skeleton information generated by the skeleton extraction unit 23. The bending feature amount calculation unit 26 may use the tracking information generated by the target organism tracking unit 24 when calculating the bending feature amount. The bending feature amount is a feature amount related to the bending of the joints of the target organism, and is a feature amount corresponding to at least one of the bending of the joints of the target organism and the joints of the target organism that have moved with the bending of the target joint. As a specific example, the bending feature amount is composed of at least one of a value indicating each of the bending direction and degree of each of at least some joints of the target organism, a value indicating each of the angular velocity and angular acceleration at the time of bending of each of the bent joints of the joints of the target organism, and a value indicating each of the movement amount, movement speed, and movement direction of each of the joints that have moved with the bending of the joints of the target organism. The bending feature amount calculation unit 26 may change the feature amount calculated according to the item of the reference behavior model 31. As a specific example, the bend feature amount calculation unit 26 calculates the movement trajectory of each joint of the target organism, and calculates the movement distance and movement direction of each joint based on the calculated movement trajectory. Note that the nature of the movement trajectory calculated by the bend feature amount calculation unit 26 basically differs depending on the type, age, etc. of the target organism.

[0018] The behavior determination unit 27 determines the behavior of the target organism using the behavior feature amount and the reference behavior model 31. The behavior feature amount is a feature amount calculated based on the time progression of at least one of the position and bending of each joint of the target organism, and corresponds to the behavior of the target organism. As a specific example, the behavior feature amount is composed of at least one of a movement feature amount and a bending feature amount. Specifically, the behavior determination unit 27 applies the calculated movement feature amount and the displacement feature amount to the reference behavior model 31 to determine what kind of behavior the target organism is. As a specific example, behavior determination unit 27 determines the behavior of the target organism as feeding behavior when the swimming speed of the target organism is faster than a predetermined speed, the orientation of the target organism's head is within a predetermined angle from the vertical upward direction, and the target organism wags its tail at intervals shorter than a predetermined interval. Feeding behavior is the behavior of the target organism when it desires food. As another specific example, the behavior determination unit 27 determines the behavior of the target organism as abnormal when the swimming speed of the target organism is slower than a predetermined speed and the abdomen of the target organism is vertically above the dorsal fin of the target organism. Abnormal behavior is behavior that is not normal, and specific examples include behavior of the target organism when the target organism is in a weakened state, a high stress state, or an ill state. Normal behavior is behavior of the target organism when the target organism is in good health.

[0019] FIG. 3 is a diagram for explaining the processing of the behavior determining unit 27. As shown in FIG. Fig. 3(a) shows a specific example in which the target organism is a shrimp. When the shrimp floats to the water surface and the shrimp's legs move a lot, the behavior determination unit 27 determines the shrimp's behavior as a feeding behavior. On the other hand, when the shrimp stays on the water bottom and the shrimp's movement amount is small, the behavior determination unit 27 determines the shrimp's behavior as a non-feeding behavior. Non-feeding behavior is behavior that is not a feeding behavior. 3(b) shows a specific example in which the target organism is a catfish. When the catfish frequently moves its tail, the behavior determination unit 27 determines that the behavior of the catfish is normal. On the other hand, when the catfish hardly moves its tail, the behavior determination unit 27 determines that the behavior of the catfish is abnormal.

[0020] The growth condition control unit 28 generates data for the device that feeds the target organism to determine whether or not to feed the target organism, according to the behavior of the target organism determined by the behavior determination unit 27. The data may be data indicating whether or not to feed the target organism, data indicating the amount of feeding, data indicating whether or not there is a target organism that is performing a behavior determined to be a feeding behavior, data indicating each of the total number of target organisms performing a behavior determined to be a feeding behavior and the total number of target organisms in the aquarium, or data indicating the processing results of each unit included in the behavior determination device 10. When the data indicates the amount of feeding, the automatic feeding device 60 determines not to feed the target organism when the amount of feeding is 0, and determines to feed the target organism when the amount of feeding is not 0. Furthermore, the growth condition control unit 28 may generate data indicating the amount of adjustment of each of the automatic feeding device 60 and the growth environment control device 70 based on the determination result of the behavior determination unit 27, and output the generated data. As a specific example, the growth condition control unit 28 calculates at least one of the feeding timing and feeding amount for the target organism according to the determined behavior of the target organism, and also generates data used to adjust the growth environment of the target organism according to the determined behavior of the target organism.

[0021] The reference behavior model 31 is a model that indicates the classification of the behavior of the target organism according to the behavior feature amount, and is a model used to classify the behavior of aquatic organisms. The reference behavior model 31 may be a model expressed in a table format, or may be a classifier based on machine learning. 4 to 7 show specific examples of the reference behavior model 31. Below, the reference behavior model 31 will be explained using Figs. 4 to 7. Note that the feeding behavior model, the non-feeding behavior model, the normal behavior model, and the abnormal behavior model are each a specific example of a model included in the reference behavior model 31.

[0022] FIG. 4 shows a specific example of the feeding behavior model. The feeding behavior model is a model used to determine whether the behavior of a target organism is feeding behavior or not, and as a specific example, is a model showing an index for classifying the behavior of the target organism into feeding behavior according to behavioral features. By using the feeding behavior model shown in FIG. 4, the behavior determination unit 27 can determine whether the behavior of the target organism is feeding behavior or not based on the swimming speed, head angle, swimming depth, and caudal fin angle change amount of the target organism. Here, the swimming speed, head angle, and swimming depth correspond to the movement features, and the caudal fin angle change amount corresponds to the bending features.

[0023] FIG. 5 shows a specific example of the non-feeding behavior model. The non-feeding behavior model is a model used to determine whether the behavior of the target organism is non-feeding behavior or not, and as a specific example, is a model showing an index for classifying the behavior of the target organism into non-feeding behavior according to the behavior feature amount. By using the non-feeding behavior model shown in FIG. 5, the behavior determination unit 27 can determine whether the behavior of the target organism is non-feeding behavior or not based on the swimming speed, head angle, swimming depth, and caudal fin angle change amount of the target organism. Non-feeding behavior is behavior that is not feeding behavior, and as a specific example, is behavior that the target organism takes when full.

[0024] FIG. 6 shows a specific example of the normal behavior model. The normal behavior model is a model used to determine whether the behavior of a target organism is normal or not, and as a specific example, is a model showing an index for classifying the behavior of the target organism as normal behavior according to the behavior feature amount. By using the normal behavior model shown in FIG. 6, the behavior determination unit 27 can determine whether the behavior of the target organism is normal or not based on the swimming speed, pectoral fin position, and number of changes in the caudal fin angle of the target organism. Here, the pectoral fin position corresponds to the movement feature amount, and the number of changes in the caudal fin angle corresponds to the bending feature amount.

[0025] Fig. 7 shows a specific example of the abnormal behavior model. The abnormal behavior model is a model used to determine whether the behavior of the target organism is abnormal or not, and as a specific example, is a model showing an index for classifying the behavior of the target organism into abnormal behavior according to the behavior feature amount. By using the abnormal behavior model shown in Fig. 7, the behavior determination unit 27 can determine whether the behavior of the target organism is abnormal or not based on the swimming speed, pectoral fin position, and number of changes in the caudal fin angle of the target organism.

[0026] 8 shows an example of a hardware configuration of the behavior determination device 10 according to this embodiment. The behavior determination device 10 is composed of a computer, and includes hardware such as a processor 20, a storage device 30, a communication device 40, and an input / output interface 50. The behavior determination device 10 may be composed of multiple computers.

[0027] The processor 20 is an integrated circuit (IC) that performs arithmetic processing and controls the hardware of the computer. Specific examples of the processor 20 include a central processing unit (CPU), a digital signal processor (DSP), or a graphics processing unit (GPU). The behavior determination device 10 may include a plurality of processors that replace the processor 20. The plurality of processors share the role of the processor 20.

[0028] The storage device 30 may be a volatile storage device, a non-volatile storage device, or a combination thereof. A specific example of the volatile storage device is a random access memory (RAM). A specific example of the non-volatile storage device is a read only memory (ROM), a hard disk drive (HDD), or a flash memory.

[0029] The communication device 40 is a receiver and a transmitter. A specific example of the communication device 40 is a communication chip or a NIC (Network Interface Card).

[0030] The input / output interface 50 is a port to which an input device and an output device are connected. A specific example of the input / output interface 50 is a Universal Serial Bus (USB) terminal. Specific examples of the input device are a keyboard and a mouse. A specific example of the output device is a display.

[0031] Each unit of the behavior determination device 10 may appropriately use the input / output interface 50 and the communication device 40 when communicating with other devices, etc. The communication device 40 and the input / output interface 50 acquire signals output by the camera 1, etc.

[0032] The storage device 30 stores a reference behavior model 31 and a behavior determination program. The behavior determination program is a program that causes a computer to realize the functions of each unit of the behavior determination device 10. The processor 20 reads out and executes the behavior determination program stored in the storage device 30 to operate as each unit of the behavior determination device 10. The functions of each unit of the behavior determination device 10 are realized by software.

[0033] Data used when executing the behavior determination program and data obtained by executing the behavior determination program are appropriately stored in the storage device 30. Each unit of the behavior determination device 10 appropriately uses the storage device 30. Note that the terms "data" and "information" may have the same meaning. The storage device 30 may be independent of the computer.

[0034] The behavior determination program may be recorded in a computer-readable non-volatile recording medium. Specific examples of the non-volatile recording medium include an optical disk and a flash memory. The behavior determination program may be provided as a program product.

[0035] ***Explanation of Operation*** The operation procedure of the behavior determination device 10 corresponds to a behavior determination method, and the program that realizes the operation of the behavior determination device 10 corresponds to a behavior determination program.

[0036] 9 is a flowchart showing an example of the operation of the behavior determining device 10. The operation of the behavior determining device 10 will be described with reference to FIG.

[0037] (Step S1) The video acquisition unit 21 acquires, as target video data, the video data output by the camera 1. The video acquisition unit 21 may acquire, as target video data, only data within a certain time range from among the video data output by the camera 1.

[0038] (Step S2) The target organism detection section 22 detects a target organism from each frame represented by the target video data. The target organism detection section 22 may use only a portion of the frames represented by the target video data.

[0039] (Step S3) If a target living thing is detected from the image represented by the target image data, the behavior determination apparatus 10 proceeds to step S4, otherwise the behavior determination apparatus 10 returns to step S1. In addition, when a target organism is detected from the image represented by the target image data, it is deemed that one or more target organisms have been detected.

[0040] (Step S4) The behavior determination device 10 selects, as a selected target organism, one of the one or more target organisms that has not yet been selected in the repeated processing from step S4 to step S10. Note that the selected target organism is assumed to have been detected in multiple frames.

[0041] (Step S5) The skeleton extraction unit 23 extracts the skeleton of the selected target organism in each frame in which the selected target organism is detected among the frames indicated by the target video data, and generates information indicating the extracted skeleton as target skeleton information.

[0042] (Step S6) The target organism tracking unit 24 tracks the selected target organism based on the target skeletal information, and generates information indicating the tracking result as target tracking information.

[0043] (Step S7) The motion feature amount calculation unit 25 calculates the motion feature amount of the selected target organism based on the target tracking information. As a specific example, the motion feature amount calculation unit 25 calculates the swimming distance and swimming time of the target organism based on the target tracking information, and sets the calculated results as the motion feature amount. In this step, the curvature feature amount calculation section 26 may calculate the curvature feature amount of the selected target organism based on the target skeletal information.

[0044] (Step S8) The behavior determination unit 27 compares the feature amount calculated in step S7 with a reference behavior model 31 prepared in advance.

[0045] (Step S9) The behavior determination section 27 determines the behavior of the selected target creature based on the comparison result in step S8.

[0046] (Step S10) The behavior determination device 10 repeatedly executes steps S5 to S9 the number of times corresponding to the number of target living things detected from the target video data.

[0047] (Step S11) The growth condition control unit 28 generates data indicating the amount of adjustment of each of the automatic feeding device 60 and the growth environment control device 70 based on the judgment result of the behavior judgment unit 27 in step S9, and outputs the generated data to each of the automatic feeding device 60 and the growth environment control device 70. As a specific example, when a target organism exhibiting a behavior determined to be a feeding behavior in step S9 is found, the automatic feeding device 60 determines that it is time to feed and executes feeding. At this time, the automatic feeding device 60 may change the amount of feeding depending on the proportion of target organisms exhibiting the behavior determined to be a feeding behavior in step S9. As a specific example, the automatic feeding device 60 has a reference feeding amount, which is a reference feeding amount for each target organism, in a database (not shown), obtains the number of target organisms exhibiting the behavior determined to be a feeding behavior in step S9 and the number of target organisms detected in step S3, calculates the proportion of target organisms exhibiting the behavior determined to be a feeding behavior by dividing the number of target organisms exhibiting the behavior determined to be a feeding behavior by the number of detected target organisms, and sets the feeding amount to 1.2 times the reference feeding amount when the calculated proportion is 80% or more. As another specific example, the automatic feeder 60 has a feeding amount index that indicates a higher value than the value indicated by the feeding behavior index, and when the feature value corresponding to the target organism exhibiting behavior determined to be feeding is equal to or greater than the value indicated by the feeding amount index, the feeding amount is set to 1.2 times the reference feeding amount. As a specific example, the feeding behavior index indicates a swimming speed of 15 cm / s, and the feeding amount index indicates a swimming speed of 25 cm / s. The feeding behavior index is an index used to determine whether the behavior of the target organism is feeding behavior. The feeding amount index is an index used to determine the feeding amount. As a specific example, when there is one or more target organisms whose behavior is determined to be abnormal in step S9, the growth environment control device 70 first sets the water temperature or oxygen supply amount in the aquarium high. After that, the behavior determination device 10 executes the behavior determination flow shown in FIG. 9 again after a certain time has elapsed, and when there is no target organism whose behavior is determined to be abnormal in step S9, the growth environment control device 70 sets the water temperature or oxygen supply amount in the aquarium low. The reason for first setting the water temperature or oxygen supply amount in the aquarium high is that, in general, the higher the water temperature and the higher the oxygen concentration, the more active (actively moving) the fishes are. The growth environment control device 70 may first set the water temperature or oxygen supply amount in the aquarium low, and then set the water temperature or oxygen supply amount in the aquarium high.

[0048] ***Explanation of the Effects of the First Embodiment*** As described above, according to this embodiment, the behavior of the target organism can be understood in more detail than by only tracking the target organism by determining the type of behavior of the target organism based on the swimming behavior and leg movements of the target organism, etc. Also, according to this embodiment, the skeleton of the target organism is extracted according to the type of the target organism, and the behavior of the target organism is determined based on the behavior of the target organism tracked based on the extracted skeleton and a model for classifying the behavior of the target organism, thereby making it possible to understand the behavior according to the ecology of the target organism. Furthermore, according to this embodiment, efficient feeding can be performed based on the results of understanding the behavior of the target organism. That is, according to this embodiment, since feeding can be performed at an appropriate timing, it is possible to prevent the water quality from deteriorating by feeding the target organism when it is full, and to prevent not feeding the target organism when it is hungry, and it is possible to adjust the amount of feeding according to the health condition of the target organism.

[0049] ***Other configurations*** <Variation 1> FIG. 10 shows an example of the hardware configuration of the behavior determining device 10 according to this modified example. The behavior determination device 10 includes a processing circuit 19 instead of the processor 20 or instead of the processor 20 and the storage device 30 . The processing circuitry 19 is hardware that realizes at least a part of each unit included in the behavior determination device 10. The processing circuitry 19 may be dedicated hardware, or may be a processor that executes a program stored in a storage device 30 .

[0050] When processing circuitry 19 is dedicated hardware, processing circuitry 19 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The behavior determination device 10 may include a plurality of processing circuits that replace the processing circuit 19. The plurality of processing circuits share the role of the processing circuit 19.

[0051] In the behavior determination device 10, some of the functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

[0052] The processing circuitry 19 may illustratively be implemented in hardware, software, firmware, or a combination thereof. The processor 20, the storage device 30, and the processing circuit 19 are collectively referred to as the “processing circuitry.” In other words, the functions of the functional components of the behavior determination device 10 are realized by the processing circuitry. The behavior determining device 10 according to other embodiments may also have a similar configuration to this modified example.

[0053] Embodiment 2 The following mainly describes the differences from the above-described embodiment with reference to the drawings.

[0054] ***Configuration Description*** 11 shows an example of the configuration of a behavior determination system 9 according to this embodiment. As shown in FIG. 11, the behavior determination device 10 according to this embodiment further includes a remaining food amount acquisition unit 200, a learning unit 301, and a corpse number acquisition unit 400. The remaining food amount acquisition section 200 comprises a video acquisition section 201 , a remaining food detection section 202 , and a remaining food amount calculation section 203 . The corpse number acquisition unit 400 includes a video acquisition unit 401 , a corpse detection unit 402 , and a corpse number calculation unit 403 .

[0055] The image acquisition unit 201 is similar to the image acquisition unit 21 .

[0056] The remaining food detection unit 202 detects remaining food in the tank after feeding from each frame shown in the video data acquired by the video acquisition unit 201. After feeding is after the target organism has eaten the provided food, and as a specific example, after the time normally required for the target organism to finish eating the food has elapsed from the timing of feeding. The remaining food detection unit 202 may determine remaining food using a background subtraction method, or may determine where remaining food is present in each frame using a trained model based on AI technology. As a specific example, the trained model is a model that receives data showing a frame as input and outputs the result of detecting remaining food.

[0057] The remaining food amount calculation unit 203 calculates the amount of food detected by the remaining food detection unit 202.

[0058] The image acquisition unit 401 is similar to the image acquisition unit 21 .

[0059] The corpse detection unit 402 detects a corpse in the tank from each frame of the video data acquired by the video acquisition unit 401. The corpse detection unit 402 may determine the presence of a corpse using motion features and a background subtraction method, or may determine where in each frame the corpse is present using a trained model based on AI technology. As a specific example, the trained model is a model that receives data indicating a frame as input and outputs the result of detecting a corpse.

[0060] Corpse number calculation unit 403 calculates the number of corpses detected by corpse detection unit 402. As a specific example, corpse number calculation unit 403 calculates the number of corpses based on the total number of pixels estimated to reflect corpses when corpses are detected using the background subtraction method, and calculates the number of corpses based on the number of detection frames when corpses are detected using AI technology.

[0061] The learning unit 301 generates a model based on the movement feature amount and the curvature feature amount, and data indicating the results of detection by the remaining food detection unit 202 or the carcass detection unit 402. As a specific example, the learning unit 301 uses the non-feeding behavior data and the feeding behavior data to learn a model that classifies the behavior of the target organism into feeding behavior and non-feeding behavior according to the behavior feature of the target organism. The non-feeding behavior data is data when the amount of remaining food is equal to or greater than a reference remaining food amount, data showing the behavior feature of the learning aquatic organism before feeding, and data labeled as data corresponding to non-feeding behavior. The remaining food amount is the amount of food remaining after the feeding time has elapsed from the feeding time. The feeding time is a time set as the time required for the learning aquatic organism to eat the fed food. The learning aquatic organism is an aquatic organism corresponding to the target organism. The learning aquatic organism may be the target organism itself, or may be an aquatic organism having the same or similar properties as the target organism. The feeding behavior data is data when the amount of remaining food is less than a reference remaining food amount, data showing the behavior feature of the learning aquatic organism before feeding, and data labeled as data corresponding to feeding behavior. In this example, when the amount of food calculated by the remaining food amount calculation unit 203 is less than a certain amount, the learning unit 301 learns the feeding behavior of the target organism based on the movement features and curvature features before feeding and the amount of remaining food after feeding, and may use a model indicating the movement features and curvature features corresponding to the feeding behavior as at least a part of the reference behavior model 31. As another specific example, the learning unit 301 uses the abnormal behavior data and the normal behavior data to learn a model that classifies the behavior of the target organism into either normal behavior or abnormal behavior according to the behavior feature of the target organism. Here, in this example, it is assumed that an aquarium is used to raise a group of aquatic organisms composed of a plurality of aquatic organisms corresponding to the target organism. Each of the plurality of aquatic organisms corresponding to the target organism may be an aquatic organism having the same or similar properties as the target organism. The abnormal behavior data is data when an abnormal number or more of dead aquatic organisms constituting the aquatic organism group are detected in the aquarium, and is data indicating the behavior feature of each aquatic organism constituting the aquatic organism group before the abnormal number or more of dead aquatic organisms constituting the aquatic organism group are detected in the aquarium, and is labeled as data corresponding to abnormal behavior. The normal behavior data is data when the corpses of aquatic organisms constituting the aquatic organism group less than an abnormal number are detected in the aquarium, and is data indicating behavioral features of each aquatic organism constituting the aquatic organism group before the corpses of the aquatic organisms constituting the aquatic organism group less than an abnormal number are detected in the aquarium, and is data labeled as data corresponding to normal behavior. In this example, when the number of corpses calculated by the corpse number calculation unit 403 is less than a certain number, the learning unit 301 may learn abnormal behavior of the target organism based on the movement features and curvature features before the detection of corpses and the number of corpses, and may use a model indicating the movement features and curvature features corresponding to abnormal behavior as at least a part of the reference behavior model 31.

[0062] FIG. 12 is a diagram for explaining the processing of the learning unit 301. As shown in FIG. 12(a) shows a specific example of creating a feeding behavior model by the learning unit 301. When the amount of remaining food detected after feeding is less than a certain amount, i.e., when the target product has eaten a sufficient amount of the provided food, the learning unit 301 creates a feeding behavior model in which the behavior of the target organism before feeding is classified as feeding behavior. 12(b) shows a specific example of creating a non-feeding behavior model by the learning unit 301. When the amount of remaining food detected after feeding is greater than a certain amount, i.e., when the target product has not eaten enough of the provided food, the learning unit 301 creates a non-feeding behavior model in which the behavior of the target organism before feeding is classified as non-feeding behavior. 12(c) shows a specific example of creating an abnormal behavior model by the learning unit 301. When the number of detected target organism corpses is greater than a certain number, the learning unit 301 creates an abnormal behavior model in which the behavior of the target organism before the detection of more than a certain number of corpses is classified as abnormal behavior. The learning unit 301 may generate a normal behavior model when the number of detected target creature corpses is less than a certain number.

[0063] ***Explanation of Operation*** 13 is a flow chart showing an example of the operation of the remaining feed amount obtaining section 200 and the learning section 301. The operation of the remaining feed amount obtaining section 200 and the learning section 301 will be described with reference to FIG.

[0064] (Step S21) The image acquisition unit 201 acquires the image data output by the camera 1 as target image data. This step is the same as step S1. Note that the target image data indicates an image captured inside an aquarium in which a target organism is kept.

[0065] (Step S22) If the target video data is data after feeding, the behavior determination apparatus 10 proceeds to step S23. Otherwise, the behavior determination apparatus 10 returns to step S21. Note that, in cases where the behavior determination apparatus 10 knows that the target video data is data after feeding, the behavior determination apparatus 10 may skip this step.

[0066] (Step S23) The remaining food detection unit 202 detects remaining food from each frame represented by the target video data.

[0067] (Step S24) If remaining food is detected in step S23, the behavior determination apparatus 10 proceeds to step S25, otherwise the behavior determination apparatus 10 proceeds to step S26.

[0068] (Step S25) The remaining food amount calculation unit 203 calculates the amount of remaining food detected in step S24.

[0069] (Step S26) The learning unit 301 acquires information indicating each of the movement amount feature amount and the curvature feature amount before feeding.

[0070] (Step S27, Step S28, Step S29) If the amount of remaining food calculated in step S25 is equal to or greater than a certain amount, the learning unit 301 labels the information acquired in step S26 as a non-feeding behavior. Otherwise, the learning unit 301 labels the information acquired in step S26 as a feeding behavior. The learning unit 301 may label each piece of information about an individual. The data labeled in steps S28 and S29 corresponds to learning data. The data labeled as non-feeding behavior corresponds to non-feeding behavior data. The data labeled as feeding behavior corresponds to feeding behavior data.

[0071] (Step S30) If the amount of learning data is sufficient, the behavior determination apparatus 10 proceeds to step S31. Otherwise, the behavior determination apparatus 10 returns to step S21.

[0072] (Step S31) The learning unit 301 uses the learning data labeled in steps S28 and S29 to learn the relationship between the behavior of the target organism and the label.

[0073] (Step S32) Based on the result of learning in step S31, the learning unit 301 creates a feeding behavior model as a model to be included in the reference behavior model 31. Note that the result of learning in step S31 may be the feeding behavior model itself.

[0074] 14 is a flowchart showing an example of the operation of the corpse number obtaining unit 400 and the learning unit 301. The operation of the corpse number obtaining unit 400 and the learning unit 301 will be described with reference to FIG.

[0075] (Step S41) The video acquisition unit 401 acquires, as target video data, the video data output by the camera 1. This step is similar to step S21.

[0076] (Step S42) The corpse detection unit 402 detects the corpse of a target living thing from each frame represented by the target video data.

[0077] (Step S43) If a corpse is detected in step S42, the behavior determination apparatus 10 proceeds to step S44. Otherwise, the behavior determination apparatus 10 proceeds to step S45.

[0078] (Step S44) Corpse number calculation unit 403 calculates the number of corpses detected in step S43.

[0079] (Step S45) The learning unit 301 acquires information indicating each of the motion feature amount and the curvature feature amount before the detection of the corpse. The learning unit 301 may acquire only information regarding the individual estimated to be a corpse. Specific examples of the individual estimated to be a corpse include an individual floating on the water surface, or an individual whose surface color has changed.

[0080] (Step S46, Step S47, Step S48) If the number of corpses calculated in step S45 is equal to or greater than a certain number, the learning unit 301 labels the information acquired in step S46 as abnormal behavior. Otherwise, the learning unit 301 labels the information acquired in step S46 as normal behavior. The learning unit 301 may label each piece of information about an individual. The data labeled in steps S47 and S48 corresponds to learning data, the data labeled as abnormal behavior corresponds to abnormal behavior data, and the data labeled as normal behavior corresponds to normal behavior data.

[0081] (Step S49) If the amount of learning data is sufficient, the behavior determination device 10 proceeds to step S50, otherwise, the behavior determination device 10 returns to step S41.

[0082] (Step S50) The learning unit 301 uses the learning data labeled in steps S47 and S48 to learn the relationship between the behavior of the target organism and the label.

[0083] (Step S51) Based on the result of learning in step S50, the learning unit 301 creates an abnormal behavior model as a model to be included in the reference behavior model 31. Note that the result of learning in step S50 may be the abnormal behavior model itself.

[0084] ***Explanation of the effect of the second embodiment*** As described above, according to this embodiment, a model used to classify the behavior of a target organism can be generated based on the behavior of the target organism and the state inside the aquarium.

[0085] ***Other embodiments*** The above-described embodiments may be freely combined, or any of the components in each embodiment may be modified, or any of the components in each embodiment may be omitted. Moreover, the embodiments are not limited to those shown in the first and second embodiments, and various modifications are possible as necessary. The procedures explained using the flowcharts and the like may be modified as appropriate. [Explanation of symbols]

[0086] 1 camera, 9 behavior determination system, 10 behavior determination device, 19 processing circuit, 20 processor, 21 image acquisition unit, 22 target organism detection unit, 23 skeleton extraction unit, 24 target organism tracking unit, 25 movement feature calculation unit, 26 bending feature calculation unit, 27 behavior determination unit, 28 growth condition control unit, 30 storage device, 31 reference behavior model, 40 communication device, 50 input / output interface, 60 automatic feeding device, 70 growth environment control device, 200 remaining food amount acquisition unit, 201 image acquisition unit, 202 remaining food detection unit, 203 remaining food amount calculation unit, 301 learning unit, 400 corpse number acquisition unit, 401 image acquisition unit, 402 corpse detection unit, 403 corpse number calculation unit.

Claims

1. a behavior determination unit that determines whether the behavior of the target organism is a feeding behavior using a behavior feature that is a feature calculated based on time transitions of at least one of the position and bending of each joint of the target organism, the behavior feature corresponding to the behavior of the target organism, and a reference behavior model that is a model that indicates a classification of the behavior of the target organism according to the behavior feature; a growth condition control unit that generates data for determining whether or not the device that performs feeding on the target organism performs feeding on the target organism, based on a result of determining whether or not the behavior of the target organism is a feeding behavior; A behavior determination device comprising:

2. The behavior feature amount is A motion feature corresponding to at least one of the movement, position, and posture of the target living thing; The behavior determination device according to claim 1 , comprising at least one of a bending feature amount which is a feature amount corresponding to at least one of the bending of the joint of the target organism and a joint among the joints of the target organism which moves in conjunction with the bending of the target joint.

3. The behavior determination device further includes: a skeleton extraction unit that extracts each joint of the target organism in each frame represented by video data representing a plurality of frames in which the target organism is captured, and generates information representing each of the extracted joints as skeleton information; a target organism tracking unit that tracks the target organism based on the skeletal information and generates information indicating a tracking result as tracking information; a motion feature amount calculation unit that calculates a motion feature amount of the target living thing based on the tracking information; a curvature feature amount calculation unit that calculates a curvature feature amount of the target organism based on the skeletal information; The behavior determination device according to claim 2 .

4. The behavior determination device according to any one of claims 1 to 3, wherein the growth condition control unit generates data used to adjust the growth environment of the target organism depending on the result of determining whether the behavior of the target organism is feeding behavior or not.

5. The behavior determination device according to any one of claims 1 to 3, wherein the reference behavior model includes at least one of a feeding behavior model indicating an index for classifying the behavior of the target organism into feeding behavior, which is the behavior of the target organism when the target organism desires food, according to the behavioral features, and a non-feeding behavior model indicating an index for classifying the behavior of the target organism into non-feeding behavior, which is behavior other than the feeding behavior, according to the behavioral features.

6. The behavior determination device further includes: a learning unit that learns a model that classifies the behavior of the target organism into the feeding behavior or the non-feeding behavior according to the behavioral feature of the target organism, using non-feeding behavior data, which is data when a remaining feed amount, which is the amount of food remaining after a feeding time, which is a time set as the time required for the learning aquatic organism corresponding to the target organism, to eat the fed food, has elapsed from the feeding time, and which indicates a behavioral feature of the learning aquatic organism before feeding, and which is labeled as data corresponding to the non-feeding behavior, and feeding behavior data, which is data when the remaining feed amount is less than the standard remaining feed amount, and which indicates a behavioral feature of the learning aquatic organism before feeding, and which is labeled as data corresponding to the feeding behavior The behavior determination device according to claim 5 .

7. The behavior determination device according to any one of claims 1 to 3, wherein the reference behavior model includes at least one of a normal behavior model indicating an index for classifying the behavior of the target organism into normal behavior, which is the behavior of the target organism when the health condition of the target organism is good, according to the behavioral features, and an abnormal behavior model indicating an index for classifying the behavior of the target organism into abnormal behavior, which is behavior other than the normal behavior, according to the behavioral features.

8. The behavior determination device further includes: When a group of aquatic organisms consisting of a plurality of aquatic organisms corresponding to the target organisms is kept in an aquarium, a learning unit that learns a model that classifies the behavior of the target organism into either the normal behavior or the abnormal behavior according to the behavior feature of the target organism using abnormal behavior data, which is data when an abnormal number or more of dead aquatic organisms constituting the aquatic organism group are detected in the aquarium, the ... The behavior determination device according to claim 7 .

9. a computer determines whether or not the behavior of the target organism is a feeding behavior using a behavior feature calculated based on time transitions of at least one of the position and bending of each joint of the target organism, the behavior feature being a feature corresponding to the behavior of the target organism, and a reference behavior model including a model indicating a classification of the behavior of the target organism according to the behavior feature; A behavior determination method in which the computer generates data for determining whether or not an apparatus that feeds the target organism will feed the target organism based on the result of determining whether or not the behavior of the target organism is feeding behavior.

10. a behavior determination process for determining whether or not the behavior of the target organism is a feeding behavior using behavior features calculated based on time transitions of at least one of the positions and flexions of each joint of the target organism, the behavior features corresponding to the behavior of the target organism, and a reference behavior model formed of a model showing a classification of the behavior of the target organism according to the behavior features; a growth condition control process for generating data for determining whether or not a device for feeding the target organism should feed the target organism in accordance with a result of determining whether or not the behavior of the target organism is a feeding behavior; A behavior determination program that causes a behavior determination device, which is a computer, to execute the above.

Images