Precision feeding method, system and device based on quantification of fish feeding activity

Abstract

The application discloses a kind of precision feeding method, system and device based on fish feeding activity quantification, it is related to image analysis processing and animal behavior analysis field.The precision feeding method includes: obtaining fish activity video library, including fish non-feeding video set and fish feeding video set;With fish non-feeding video set, based on deep unsupervised learning, construct fish feeding activity quantification model, continuously quantize analysis is carried out to fish feeding activity;Whether the current fish is feeding according to the continuous fish feeding activity index and detection threshold value is judged, and the feeding amount is adaptively regulated according to the judgment result.The present application can realize the continuous quantification of fish feeding activity without data labeling by introducing deep unsupervised learning, and realizes the intelligent precision feeding of fish based on continuous quantification value, which can effectively guarantee the fish feeding welfare and avoid bait waste.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image analysis processing and animal behavior analysis, and particularly relates to a precision feeding method, system and device based on fish feeding activity quantification. BACKGROUND

[0002] In intensive aquaculture, accurate assessment of fish feeding level and formulation of precision feeding plan can not only improve the growth rate of fish, but also save labor and material costs. Traditional feeding mode is mainly based on manual labor, which has multiple problems such as low breeding efficiency, high labor intensity and strong subjectivity, and cannot meet the production requirements. At present, the fish precision feeding system combined with Internet of Things, big data and computer vision technology is expected to solve the above problems. However, to realize the fish precision feeding system, the prerequisite is to accurately assess the fish feeding level.

[0003] As a fast, non-destructive and convenient measurement and analysis method, computer vision technology provides an effective means for fish feeding activity analysis. The existing fish feeding activity analysis methods based on computer vision mainly include traditional fish feeding activity analysis method and deep learning fish feeding activity analysis method. The former can quantify each instant of fish feeding process, but background segmentation and target feature extraction are required before quantification, which is only suitable for specific scenarios and has poor robustness and practicability. The latter avoids the tedious learning process, is suitable for various scenarios, and has better accuracy and robustness, but this method

[0004] There are some limitations: 1) they are all strongly supervised methods that need label guidance for learning; 2) they can only qualitatively assess the feeding stage of fish, and cannot realize continuous quantification. SUMMARY

[0005] The purpose of the present application is to provide a precision feeding method, system and device based on fish feeding activity quantification, which can not only avoid complex manual annotation problems, improve the stability and accuracy of fish feeding activity quantification, but also realize intelligent precision feeding according to the feeding needs of fish school, and has good applicability.

[0006] To achieve the above purpose, the present application provides the following solutions:

[0007] A precision feeding method based on fish feeding activity quantification, the precision feeding method comprising the following steps:

[0008] S1, acquiring a fish activity video library including a fish non-feeding video set and a fish feeding video set;

[0009] S2, based on deep unsupervised learning, a fish feeding activity quantification model is constructed using a fish non-feeding video set for continuous quantitative analysis of fish feeding activity, specifically including:

[0010] Based on the appearance autoencoder network and the motion autoencoder network, a fish feeding activity quantification model is constructed, the appearance autoencoder network includes an appearance encoder module, an attention module and an appearance decoder module, and the motion autoencoder network includes a motion encoder module, a memory module and a motion decoder module;

[0011] Using a fish non-feeding video set, a video sample dataset is obtained, the video sample dataset is a plurality of time-continuous video samples;

[0012] A fixed number of frames of video sequences are extracted from the video sample dataset, and the video sequences are converted to obtain corresponding grayscale image sequences and difference frame sequences;

[0013] The grayscale image sequence is input into the appearance autoencoder network, and the difference frame sequence is input into the motion autoencoder network, and the reconstructed grayscale image sequence and the reconstructed difference frame sequence are output;

[0014] According to the grayscale image sequence and the reconstructed grayscale image sequence, an appearance loss function is determined; according to the difference frame sequence and the reconstructed difference frame sequence, a motion loss function is determined; according to the appearance loss function and the motion loss function, a total loss function is determined;

[0015] According to the total loss function, the parameters in the appearance autoencoder network and the motion autoencoder network are iteratively optimized to train the fish feeding activity quantification model, and a trained fish feeding activity quantification model is obtained;

[0016] The trained fish feeding activity quantification model is used to quantify the fish feeding video set to be detected in the fish feeding video set, and a continuous fish feeding activity index is obtained.

[0017] S3, according to the continuous fish feeding activity index and the detection threshold value, it is judged whether the current fish school is feeding or not, and the feeding amount is adaptively controlled according to the judgment result; the detection threshold value is calculated as follows:

[0018] The trained fish feeding activity quantification model is used to quantify the video in the fish non-feeding video set, and the fish feeding activity index corresponding to the non-feeding video is obtained;

[0019] According to the fish feeding activity index corresponding to the non-feeding video, a threshold learning sample set is constructed;

[0020] determining a detection threshold according to the threshold learning sample set, the detection threshold comprising: a sample center of the threshold learning sample set c a mean of distances from each sample point in the threshold learning sample set to the sample center and a standard deviation s .

[0021] The application also provides a precision feeding system based on fish feeding activity quantification, which utilizes the aforementioned precision feeding method based on fish feeding activity quantification, and comprises:

[0022] a fish activity video library acquisition unit configured to acquire a fish activity video library, including a fish non-feeding video set and a fish feeding video set;

[0023] a quantitative analysis unit configured to utilize the fish non-feeding video set, construct a fish feeding activity quantification model based on deep unsupervised learning, and continuously quantitatively analyze fish feeding activity, specifically comprising:

[0024] constructing a fish feeding activity quantification model based on an appearance autoencoder network and a motion autoencoder network, the appearance autoencoder network comprising an appearance encoder module, an attention module, and an appearance decoder module, and the motion autoencoder network comprising a motion encoder module, a memory module, and a motion decoder module;

[0025] acquiring a video sample dataset from the fish non-feeding video set, the video sample dataset being a plurality of time-continuous video samples;

[0026] extracting a fixed number of video sequences from the video sample dataset, converting the video sequences to obtain corresponding grayscale image sequences and difference frame image sequences;

[0027] inputting the grayscale image sequences into the appearance autoencoder network and inputting the difference frame image sequences into the motion autoencoder network to output reconstructed grayscale image sequences and reconstructed difference frame image sequences;

[0028] determining an appearance loss function according to the grayscale image sequences and the reconstructed grayscale image sequences, determining a motion loss function according to the difference frame image sequences and the reconstructed difference frame image sequences, and determining a total loss function according to the appearance loss function and the motion loss function;

[0029] iteratively optimizing parameters in the appearance autoencoder network and the motion autoencoder network according to the total loss function to train the fish feeding activity quantification model, and obtaining a trained fish feeding activity quantification model;

[0030] The fish school feeding activity quantification model after the training is used for quantifying fish school feeding videos to be detected in a fish school feeding video set, and a continuous fish school feeding activity index is obtained.

[0031] A feeding decision unit is configured to determine whether the current fish school is feeding according to the continuous fish school feeding activity index and a detection threshold, and to adaptively control the feeding amount according to the determination result.

[0032] The fish school feeding activity quantification model after the training is used for quantifying videos in a fish school non-feeding video set, and a fish school feeding activity index corresponding to the non-feeding video is obtained.

[0033] A threshold learning sample set is constructed according to the fish school feeding activity index corresponding to the non-feeding video.

[0034] A detection threshold is determined according to the threshold learning sample set, and the detection threshold includes a sample center of the threshold learning sample set, a mean value of distances from each sample point in the threshold learning sample set to the sample center, and a standard deviation. c s .

[0035] The application further provides a precise feeding device based on fish school feeding activity quantification, which utilizes the precise feeding method based on fish school feeding activity quantification.

[0036] The industrial camera is connected to the core processor and fixed on the top of the recirculating aquaculture pond, faces the water surface, and is configured to acquire fish school activity videos in the aquaculture pond and transmit the acquired videos to the core processor.

[0037] The core processor is configured to calculate a fish school feeding activity index and a detection threshold according to the fish school activity videos acquired by the industrial camera, determine whether the fish school is feeding according to the fish school feeding activity index and the detection threshold, and obtain a feeding decision result.

[0038] The feeding system includes a communication controller, an electromagnetic valve, and a bait bin.

[0039] ​​The display platform is connected with the core processor, and is used for displaying the real-time acquired fish school activity video and displaying the real-time calculated fish school feeding activity index and feeding decision result.

[0040] According to the specific embodiments of the present application, the following technical effects are disclosed.

[0041] The application discloses a precise feeding method, system and device based on fish school feeding activity quantification, which realizes continuous quantification of fish school feeding activity through a fish school feeding activity quantification model, and realizes intelligent precise feeding of fish schools based on the quantification results, effectively saving manpower and material resources, guaranteeing fish school feeding welfare, and reducing production cost.

[0042] Compared with the prior art, the present application has the following beneficial effects: 1) the fish school feeding activity quantification model fully utilizes non-feeding data of fish schools without labeling, avoids complex and tedious manual labeling problems, and greatly saves labeling cost; 2) the fish school feeding activity quantification model fully considers the characteristics of appearance and movement changes of fish schools in the feeding process, quantifies the appearance change information and movement change information of fish schools in the feeding process through the model, fully learns the spatio-temporal variation law of fish school feeding activity, and improves the accuracy and reliability of fish school feeding activity quantification; 3) the precise feeding system has strong migration, low data collection and labeling cost, and can be conveniently and quickly extended to other aquaculture environments. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0044] Figure 1 A flowchart of a precise feeding method based on fish school feeding activity quantification provided for the present application embodiment 1 is provided.

[0045] Figure 2 A training and quantification flowchart of the fish school feeding activity quantification model of step S2 in the precise feeding method based on fish school feeding activity quantification provided for the present application embodiment 1 is provided.

[0046] Figure 3 A general schematic diagram of the fish school feeding activity quantification model is provided.

[0047] Figure 4 A structural schematic diagram of the autoencoder of the fish school feeding activity quantification model is provided.

[0048] Figure 5aA result diagram of the quantitative model of fish feeding activity provided by the present invention;

[0049] Figure 5b Another result diagram of the quantitative model of fish feeding activity provided by the present invention;

[0050] Figure 6 This is a block diagram of a precision feeding system based on quantification of fish feeding activity provided in Example 2 of the present invention;

[0051] Figure 7 This is a structural diagram of a precision feeding device based on quantification of fish feeding activity provided in Example 3 of the present invention;

[0052] Explanation of the symbols in the accompanying drawings: industrial camera-1, core processor-2, communication controller-3, solenoid valve-4, bait bin-5, display platform-6. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0054] Quantifying fish feeding activity is a prerequisite for precise feeding. Existing computer vision-based methods for analyzing fish feeding activity are mainly divided into traditional methods and deep learning-based methods. The former can quantify every moment of the fish feeding process, but requires background segmentation and target feature extraction before quantification. This makes it applicable only to specific scenarios and has poor robustness and practicality. The latter avoids the tedious learning process, is applicable to a variety of scenarios, and has better accuracy and robustness. However, these methods are supervised and require large manually annotated datasets (i.e., existing technologies require annotating the feeding status of fish to obtain labels for different feeding states, such as strong, medium, weak, and absent). Furthermore, current deep learning-based methods for assessing fish feeding activity can only qualitatively assess the feeding phase of a fish and cannot achieve continuous quantitative characterization.

[0055] In order to solve the above problems, the present invention provides a precise feeding method, system and device based on the quantification of fish feeding activities. Through an extremely low-cost unsupervised method, the continuous quantification of fish feeding activities is achieved without manual labeling, which solves the problem of manual labeling in the automatic evaluation of fish feeding activities and improves the stability and accuracy of the quantification of feeding activities. Furthermore, feeding decisions are made according to the quantified feeding activities, and intelligent and precise feeding can be performed according to the feeding needs of the fish themselves.

[0056] In order to make the above objectives, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0057] Embodiment 1:

[0058] Please refer to Figure 1 The present application provides a precision feeding method based on fish feeding activity quantification, which comprises the following steps:

[0059] S1: Obtain a fish activity video library, including a fish non-feeding video set and a fish feeding video set;

[0060] S2: Use the fish non-feeding video set to construct a fish feeding activity quantification model based on deep unsupervised learning, and continuously quantitatively analyze the fish feeding activity;

[0061] S3: Determine whether the current fish is feeding according to the continuous fish feeding activity index and the detection threshold value, and adaptively control the feeding amount according to the determination result, wherein the detection threshold value is calculated as follows:

[0062] The trained fish feeding activity quantification model is used to quantize the videos in the fish non-feeding video set to obtain the fish feeding activity index corresponding to the non-feeding video;

[0063] According to the fish feeding activity index corresponding to the non-feeding video, a threshold learning sample set is constructed ;

[0064] The threshold learning sample set S 0 The mean value of all samples is obtained, and the sample center is obtained c ;

[0065] The threshold learning sample set S 0 Each sample point x i to the sample center c The distance d i , i =1… N , N is a positive integer greater than 1, d i The calculation formula of

[0066] ;

[0067] According to the distance of each sample point in the threshold learning sample set to the sample center di determining the mean value of the distance of each sample point in the threshold learning sample set to the sample center and the standard deviation s .

[0068] Further, as shown in Figure 2 , the training and quantification of the fish feeding activity quantification model of step S2 specifically includes:

[0069] S21: constructing a fish feeding activity quantification model based on an appearance autoencoder network and a motion autoencoder network, the appearance autoencoder network including an appearance encoder module, an attention module, and an appearance decoder module, and the motion autoencoder network including a motion encoder module, a memory module, and a motion decoder module; in this embodiment, the attention module is composed of a position attention mechanism and a channel attention mechanism of a DANet (Dual Attention Network);

[0070] S22: obtaining a video sample data set using a fish non-feeding video set, the video sample data set being a plurality of time-continuous video samples;

[0071] S23: extracting a fixed number of frame video sequences from the video sample data set, and converting the video sequences to obtain corresponding grayscale image sequences and difference frame image sequences;

[0072] S24: inputting the grayscale image sequences into the appearance autoencoder network and inputting the difference frame image sequences into the motion autoencoder network, and outputting reconstructed grayscale image sequences and reconstructed difference frame image sequences;

[0073] S25: determining an appearance loss function according to the grayscale image sequences and the reconstructed grayscale image sequences, determining a motion loss function according to the difference frame image sequences and the reconstructed difference frame image sequences, and determining a total loss function according to the appearance loss function and the motion loss function;

[0074] S26: iteratively optimizing parameters in the appearance autoencoder network and the motion autoencoder network according to the total loss function to train the fish feeding activity quantification model, and obtaining a trained fish feeding activity quantification model;

[0075] S27: using the trained fish feeding activity quantification model to quantify fish feeding videos to be detected in a fish feeding video set, and obtaining continuous fish feeding activity indexes.

[0076] Further, after inputting the grayscale image sequences into the appearance autoencoder network and before determining the appearance loss function, it specifically further includes:

[0077] input each frame of the gray scale sequence into the appearance encoder module in turn for feature extraction to obtain an appearance feature representation of each frame of the gray scale;

[0078] The attention module is used to focus on the fish school self information in the appearance feature representation of each frame of the gray scale, and ignore the external light and background noise information, to obtain an appearance attention feature of each frame of the gray scale;

[0079] The appearance attention feature of each frame of the gray scale is input into the appearance decoder module in turn for reconstruction to obtain each frame of the reconstructed gray scale;

[0080] The reconstructed gray scale sequence is obtained by superimposing each frame of the reconstructed gray scale.

[0081] Further, after the difference frame sequence is input into the motion autoencoder network, before the motion loss function is determined, specifically, the method further comprises:

[0082] The difference frame sequence is input into the motion encoder module for feature extraction to obtain a motion feature representation;

[0083] The memory module is used to determine a typical motion feature of the fish school in a non-feeding state in the motion feature representation;

[0084] The typical motion feature of the fish school in the non-feeding state is input into the motion decoder module for reconstruction to obtain a reconstructed difference frame sequence.

[0085] As shown in Figure 3 and Figure 4 The fish school feeding activity quantification model is composed of an appearance autoencoder network and a motion autoencoder network; the appearance autoencoder network is used to reconstruct the gray scale sequence and learn the appearance distribution law of the fish school in a non-feeding state, and is composed of an appearance encoder module, an attention module and an appearance decoder module; the motion autoencoder network is used to reconstruct the difference frame sequence and learn the motion change law of the fish school in a non-feeding state, and is composed of a motion encoder module, a memory module and a motion decoder module.

[0086] For the appearance autoencoder network, each frame of the gray scale sequence is input into the appearance encoder module in turn, the appearance encoder module includes four convolutional layers, each convolutional layer is followed by a batch normalization layer and a ReLU activation function layer, the convolutional layer uses a convolutional kernel with a size of 3x3 and a step of 2, the appearance encoder module is used to extract features of the input gray scale to obtain an appearance feature representation, and the feature map size output after each convolutional layer of the appearance encoder module is [H / 2xW / 2x64, H / 4xW / 4x128, H / 8xW / 8x128, H / 16xW / 16x256].

[0087] In order to reduce the influence of light and background noise in the breeding environment, make the appearance self-encoder network learn more useful appearance information, and ignore irrelevant background information, the application adds an attention module to the appearance self-encoder network, wherein the attention module uses a double attention mechanism, including a position attention branch and a channel attention branch; specifically, the appearance feature representation output by the appearance encoder module is input into the position attention branch and the channel attention branch respectively, the position attention branch is used to process the effective features of the appearance feature representation in the spatial dimension, and the channel attention branch is used to process the effective features of the appearance feature representation in the channel dimension, and then the result output by the position attention branch is fused with the result output by the channel attention branch to obtain the appearance attention feature.

[0088] Then, the appearance attention feature is input into the appearance decoder module, the appearance decoder module includes four deconvolution layers, and except for the last deconvolution layer, there is a batch normalization layer and a ReLU activation function layer after each deconvolution layer, the deconvolution layer adopts a convolution kernel with a size of 3*3 and a step of 2, and the appearance decoder module is used to reconstruct each frame of gray image according to the appearance attention feature, superimposes the gray images to obtain a reconstructed gray image sequence, and the feature map size output by each deconvolution layer of the appearance decoder module is [H / 8*W / 8*256, H / 4*W / 4*128, H / 2*W / 2*128, H*W*1].

[0089] For the motion self-encoder network, first, the motion encoder module is used to extract features from the input difference frame image sequence to obtain a motion feature representation, the motion encoder module is composed of four convolution layers, and a batch normalization layer and a ReLU activation function layer are arranged after each convolution layer, the convolution layer adopts a convolution kernel with a size of 3*3 and a step of 2, and the feature map size output after each convolution layer of the motion encoder module is [H / 2*W / 2*64, H / 4*W / 4*128, H / 8*W / 8*256, H / 16*W / 16*256].

[0090] In the motion self-encoder network, the application adds a memory module to capture the typical motion features of the fish school in the non-feeding state, the memory module can make the motion feature reconstructed by the motion self-encoder network closer to the sample in the non-feeding state of the fish school, so that the motion distinction degree between the non-feeding sample and the feeding sample of the fish school is higher in the quantization process; the memory module includes a memory for recording a typical encoding template and an addressing operator w for accessing the memory, wherein N is the number of memory units, Cis the dimension of the motion feature representation; specifically, the motion feature representation output by the motion encoder module is used as a query, and the addressing operator w is used to retrieve the closest typical motion feature from the memory:

[0091]

[0092] in, Typical movement characteristics. The first i values, m i is the first i The cosine similarity is used to evaluate the similarity between the motion feature representation and each memory unit in the memory M, and the addressing operator w is obtained.

[0093] The typical motion features output by the memory module are input into the motion decoder module. The motion decoder module reconstructs the image based on the typical motion features to obtain a reconstructed differential frame sequence. The motion decoder module includes four deconvolution layers. Except for the last deconvolution layer, the other deconvolution layers are followed by a batching layer and a ReLU activation function layer. The deconvolution layer uses a 3×3 convolution kernel with a step size of 2. The feature map output by each deconvolution layer of the motion decoder module has a size of [H / 8×W / 8×256, H / 4×W / 4×128, H / 2×W / 2×64, H×W×15].

[0094] Furthermore, in step S22, obtaining a video sample dataset specifically includes:

[0095] Sequentially extract videos of fish in a non-feeding state from the fish non-feeding video set;

[0096] Randomly extract video samples from the video of the fish school in a non-feeding state, each video sample has a length of 200 frames, and the video samples do not overlap with each other;

[0097] All video samples were from videos in the non-eating state and did not require further labeling.

[0098] Furthermore, in step S23, a video sequence with a fixed number of frames is extracted from the video sample data set using a sliding window method, and the video sequence is converted to obtain a corresponding grayscale image sequence and a difference frame image sequence.

[0099] Further, in step S26, an optimization function, an iteration number, a batch size, and a learning rate for training of the fish feeding activity quantification model are preset; the fish feeding activity quantification model is iteratively trained according to the preset, and when the loss value converges, the training is stopped and the optimal parameters of the network are determined; the fish feeding activity quantification model loaded with the optimal parameters is determined as the trained fish feeding activity quantification model; and a total loss function of the fish feeding activity quantification model is composed of an appearance loss function and a motion loss function:

[0100]

[0101] wherein, L is the total loss function, L a is the appearance loss function, used to constrain the reconstructed gray image sequence obtained by the appearance autoencoder network with the input gray image sequence to be as similar as possible to the input gray image sequence, L m is the motion loss function, used to constrain the reconstructed difference frame sequence obtained by the motion autoencoder network with the input difference frame sequence to be as similar as possible to the input difference frame sequence, x a is the input gray image sequence, is the reconstructed gray image sequence, x m is the input difference frame sequence, is the reconstructed difference frame sequence.

[0102] Further, in step S27, specifically comprising:

[0103] selecting a video to be quantified from a fish feeding video set, and sequentially reading a video sequence of a fixed frame number from the video; converting the video sequence to obtain corresponding gray image sequences and difference frame sequences;

[0104] inputting the gray image sequences and the difference frame sequences into the trained fish feeding activity quantification model to obtain reconstructed gray image sequences and reconstructed difference frame sequences, respectively;

[0105] calculating an appearance reconstruction error through the gray image sequences and the reconstructed gray image sequences, and calculating a motion reconstruction error through the difference frame sequences and the reconstructed difference frame sequences, wherein the appearance reconstruction error can reflect changes in the aggregation degree of the fish school, and the motion reconstruction error can reflect changes in the food grabbing speed of the fish school; combining the appearance reconstruction error and the motion reconstruction error to obtain a comprehensive fish feeding activity index, which can be used to quantify the fish feeding activity, and a higher score indicates more intense fish feeding; and a calculation formula of the fish feeding activity index is as follows:

[0106]

[0107] Among them, FAI is the fish feeding activity index, x b is the input grayscale image sequence, is the reconstructed grayscale image sequence, x n is the input difference frame sequence, is the reconstructed difference frame sequence, λ It is the balance fraction, usually set to 0.5.

[0108] Furthermore, in step S3, whether the current fish school is feeding is determined based on the continuous fish school feeding activity index and the detection threshold, and the feeding amount is adaptively regulated based on the determination result, specifically including:

[0109] judge Is it established? Get the first judgment result; df Fish feeding activity index to the sample center c distance, is the mean of the distance from each sample point to the sample center in the threshold learning sample set, s The standard deviation of the distance from each sample point to the sample center in the threshold learning sample set;

[0110] If the first judgment result is yes, it means that the fish school's feeding activity index is within the feeding state space, indicating that the fish school is feeding in the current video sequence of the fish school, which is a feeding event;

[0111] If the first judgment result is no, it means that the fish school's feeding activity index is in the non-feeding state space, indicating that the fish school has completed feeding in the current video sequence of the fish school, which is a non-feeding event;

[0112] Determining whether the number of the non-feeding events exceeds a preset threshold, and obtaining a second determination result;

[0113] If the second judgment result is no, keeping the bait throwing machine turned on;

[0114] If the second judgment result is yes, the feeding machine is turned off and the feeding is finished;

[0115] Figure 5a and Figure 5b The results of the continuous quantification of fish feeding activity in recirculating aquaculture using the precise feeding method based on the quantification of fish feeding activity are shown. The horizontal axis represents the number of video frames, the vertical axis represents the fish feeding activity index, the curve represents the fish feeding activity index of each video sequence of the feeding video, and the shaded area represents the process from the bait feeder being turned on to the bait feeder being turned off. Figure 5a and Figure 5bIt can be seen from the two fish population feeding activity quantification results that the FAI value remains stable and fluctuates slightly before feeding because the fish population freely swims normally; when feeding starts, the FAI value rapidly increases, which is caused by the rapid gathering of the fish population to compete for feed; as the feeding process continues, the fish population becomes increasingly full, and the decrease in the gathering degree and the speed of feeding makes the FAI value gradually decrease; when the fish population is fully fed, the FAI value decreases to the level before feeding, and feeding stops; after feeding ends, the FAI value is slightly lower than the level before feeding, which is caused by the decrease in the swimming speed of the fish population after feeding is completed.

[0116] The method can quantize the feeding activity of the fish population continuously, and realize intelligent feeding decision-making by combining with a detection threshold, so that the fish population can be fed according to specific feeding needs, the growth rate of the fish population is ensured, and waste of feed is avoided.

[0117] Embodiment 2:

[0118] Please refer to Figure 6 The application provides a precise feeding system based on fish population feeding activity quantification, which utilizes the precise feeding method based on fish population feeding activity quantification in embodiment 1, and comprises the following components:

[0119] A fish population activity video library acquisition unit A1 is configured to acquire a fish population activity video library, including a fish population non-feeding video set and a fish population feeding video set;

[0120] A quantification analysis unit A2 is configured to utilize the fish population non-feeding video set, construct a fish population feeding activity quantification model based on deep unsupervised learning, and continuously quantitatively analyze the fish population feeding activity, specifically including the following steps:

[0121] The fish population feeding activity quantification model is constructed based on an appearance autoencoder network and a motion autoencoder network, the appearance autoencoder network comprises an appearance encoder module, an attention module and an appearance decoder module, and the motion autoencoder network comprises a motion encoder module, a memory module and a motion decoder module;

[0122] The fish population non-feeding video set is utilized to acquire a video sample data set, and the video sample data set is a plurality of time-continuous video samples;

[0123] A fixed number of frame video sequences are extracted from the video sample data set, and the video sequences are converted to obtain corresponding gray image sequences and difference frame image sequences;

[0124] The gray image sequences are input into the appearance autoencoder network, and the difference frame image sequences are input into the motion autoencoder network, to output reconstructed gray image sequences and reconstructed difference frame image sequences;

[0125] determine an appearance loss function according to the gray image sequence and the reconstructed gray image sequence, determine a motion loss function according to the difference frame sequence and the reconstructed difference frame sequence, and determine a total loss function according to the appearance loss function and the motion loss function;

[0126] According to the total loss function, the parameters in the appearance autoencoder network and the motion autoencoder network are iteratively optimized to train the fish feeding activity quantification model, and a trained fish feeding activity quantification model is obtained.

[0127] The trained fish feeding activity quantification model is used to quantify the fish feeding video to be detected in the fish feeding video set, and a continuous fish feeding activity index is obtained.

[0128] The feeding decision unit A3 is used to determine whether the current fish group is feeding according to the continuous fish feeding activity index and the detection threshold, and to adaptively control the feeding amount according to the determination result.

[0129] Embodiment 3:

[0130] Please refer to Figure 7 The present application provides a precise feeding device based on fish feeding activity quantification, which uses the precise feeding method based on fish feeding activity quantification of embodiment 1. The feeding device comprises an industrial camera 1, a core processor 2, a feeding system and a display platform 6.

[0131] The industrial camera 1 is connected to the core processor 2 and is fixed on the top of the recirculating aquaculture pond. The industrial camera 1 faces the water surface and is used to obtain the fish activity video in the aquaculture pond and transmit the obtained video to the core processor 2.

[0132] The core processor 2 is used to calculate the fish feeding activity index and the detection threshold according to the fish activity video in the aquaculture pond obtained by the industrial camera 1, to determine whether the fish group is feeding according to the fish feeding activity index and the detection threshold, and to obtain a feeding decision result.

[0133] The feeding system comprises a communication controller 3, an electromagnetic valve 4 and a bait bin 5. The communication controller 3 is connected to the core processor 2 and is used to receive the feeding decision result and drive the electromagnetic valve 4 according to the feeding decision result. The bait bin 5 is used to hold bait, and a baffle is arranged at the bait bin discharge port. The electromagnetic valve 4 is used to control the opening or closing of the baffle to control the feeding amount.

[0134] The display platform 6 is connected with the core processor 2, and is used for displaying the real-time acquired fish activity video and displaying the real-time calculated fish feeding activity index and the feeding decision result.

[0135] It should be noted that the core processor is used for constructing a fish feeding activity quantification model, and after receiving the fish activity video in the aquaculture pond acquired by the industrial camera, the trained fish feeding activity quantification model is used to acquire the fish feeding activity index; according to the fish feeding activity index and the detection threshold, it is judged whether the fish is feeding, and the feeding decision result is obtained; the trained fish feeding activity quantification model is a model obtained according to the precise feeding method based on fish feeding activity quantification in embodiment 1.

[0136] In the specification, each embodiment is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0137] The principles and implementation manners of the present application are described by applying specific examples in the specification. The above description of the embodiments is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range can be changed. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A precision feeding method based on quantification of fish feeding activity, characterized in that: The method comprises: S1, obtain the fish activity video library, including the fish non-feeding video set and the fish feeding video set; S2 uses a video collection of fish not feeding to build a quantitative model of fish feeding activity based on deep unsupervised learning, and conducts continuous quantitative analysis of fish feeding activity. Specifically, A fish feeding activity quantification model is constructed based on an appearance autoencoder network and a motion autoencoder network. The appearance autoencoder network includes an appearance encoder module, an attention module, and an appearance decoder module. The motion autoencoder network includes a motion encoder module, a memory module, and a motion decoder module. Using a non-feeding video set of a school of fish, a video sample dataset is obtained, wherein the video sample dataset is a plurality of temporally continuous video samples; Extracting a video sequence with a fixed number of frames from the video sample data set, converting the video sequence to obtain a corresponding grayscale image sequence and a difference frame image sequence; Inputting the grayscale image sequence into the appearance autoencoder network, inputting the difference frame image sequence into the motion autoencoder network, and outputting a reconstructed grayscale image sequence and a reconstructed difference frame image sequence; Determine an appearance loss function based on the grayscale image sequence and the reconstructed grayscale image sequence; determine a motion loss function based on the difference frame image sequence and the reconstructed difference frame image sequence; determine a total loss function based on the appearance loss function and the motion loss function; Iteratively optimizing parameters in the appearance autoencoder network and the motion autoencoder network according to the total loss function to train the quantization model of the school of fish feeding activity, thereby obtaining a trained quantization model of the school of fish feeding activity; quantifying the fish feeding activity quantification model trained to obtain a continuous fish feeding activity index by quantifying the fish feeding video to be detected in the fish feeding video set; S3, judging whether the current fish school is feeding according to the continuous fish school feeding activity index and the detection threshold, and adaptively adjusting the feeding amount according to the judgment result.

2. The method for accurate feeding based on quantification of fish feeding activity according to claim 1, wherein: The detection threshold is specifically calculated as follows: quantifying the videos in the fish non-feeding video set using the trained fish feeding activity quantification model to obtain a fish feeding activity index corresponding to the non-feeding video; Constructing a threshold learning sample set based on the fish feeding activity index corresponding to the non-feeding video; Determine the detection threshold based on the threshold learning sample set, the detection threshold includes: the sample center c of the threshold learning sample set, the mean of the distance from each sample point to the sample center in the threshold learning sample set and standard deviation s.

3. The method for accurate feeding based on quantification of fish feeding activity according to claim 1, wherein: After inputting the grayscale image sequence into the appearance autoencoder network and before determining the appearance loss function, the method further includes: Inputting each frame of the grayscale image sequence into the appearance encoder module in sequence to extract features, thereby obtaining an appearance feature representation of each frame of the grayscale image; Using the attention module to focus on the fish school's own information in the appearance feature representation of each frame of the grayscale image, ignoring external light and background noise information, to obtain the appearance attention feature of each frame of the grayscale image; Inputting the appearance attention features of each frame of the grayscale image into the appearance decoder module in sequence for reconstruction, thereby obtaining a reconstructed grayscale image of each frame; The reconstructed grayscale images of each frame are superimposed to obtain a reconstructed grayscale image sequence.

4. The method for accurate feeding based on quantification of fish feeding activity according to claim 1, wherein: After inputting the difference frame image sequence into the motion autoencoder network and before determining the motion loss function, the method further includes: Inputting the difference frame image sequence into the motion encoder module for feature extraction to obtain motion feature representation; Determining, using the memory module, typical movement characteristics of the school of fish in a non-feeding state in the movement characteristic representation; The typical motion features of the fish school in the non-feeding state are input into the motion decoder module for reconstruction to obtain a reconstructed difference frame sequence.

5. The method for accurate feeding based on quantification of fish feeding activity according to claim 1, wherein: The calculation formula of the total loss function is as follows: Among them, L is the total loss function, L a is the appearance loss function, L m is the motion loss function, x a is the input grayscale image sequence, is the reconstructed grayscale image sequence, x m is the input difference frame sequence, is the reconstructed difference frame sequence.

6. The method for accurate feeding based on quantification of fish feeding activity according to claim 1, wherein: The trained fish feeding activity quantification model is used to quantify the fish feeding videos to be detected in the fish feeding video set to obtain a continuous fish feeding activity index, specifically including: Selecting a video to be quantized from a video set of fish feeding, and sequentially reading a video sequence with a fixed number of frames from the video; converting the video sequence to obtain a corresponding grayscale image sequence and a difference frame image sequence; Inputting the grayscale image sequence and the difference frame image sequence into a trained fish feeding activity quantification model to obtain a reconstructed grayscale image sequence and a reconstructed difference frame image sequence respectively; The fish school feeding activity index is determined based on the grayscale image sequence and the difference frame image sequence and the reconstructed grayscale image sequence and the reconstructed difference frame image sequence. The calculation formula is as follows: Among them, FAI is the fish feeding activity index, x b is the input grayscale image sequence, is the reconstructed grayscale image sequence, x n is the input difference frame sequence, is the reconstructed difference frame sequence, and λ is the balance score.

7. The method for accurate feeding based on quantification of fish feeding activity according to claim 1, wherein: The method of determining whether the current fish school is feeding based on the continuous fish school feeding activity index and the detection threshold, and adaptively adjusting the feeding amount based on the determination result, specifically includes: judge Is it true? The first judgment result is obtained; where df is the distance from the fish feeding activity index to the sample center c, is the mean of the distance from each sample point to the sample center in the threshold learning sample set, and s is the standard deviation of the distance from each sample point to the sample center in the threshold learning sample set; If the first judgment result is yes, it means that the fish feeding activity index corresponds to a feeding event; If the first judgment result is no, it means that the fish feeding activity index corresponds to a non-feeding event; Determining whether the number of the non-feeding events exceeds a preset threshold, and obtaining a second determination result; If the second judgment result is no, keeping the bait throwing machine turned on; If the second judgment result is yes, the bait throwing machine is turned off.

8. A precision feeding system based on quantification of fish feeding activity, the system utilizing the precision feeding method based on quantification of fish feeding activity according to any one of claims 1 to 7, the system comprising: A fish school activity video library acquisition unit is used to acquire a fish school activity video library, including a fish school non-feeding video set and a fish school feeding video set; The quantitative analysis unit is used to build a quantitative model of fish feeding activity based on deep unsupervised learning using the video collection of fish non-feeding, and conduct continuous quantitative analysis of fish feeding activity. Specifically, it includes: A fish feeding activity quantification model is constructed based on an appearance autoencoder network and a motion autoencoder network. The appearance autoencoder network includes an appearance encoder module, an attention module, and an appearance decoder module. The motion autoencoder network includes a motion encoder module, a memory module, and a motion decoder module. Using a non-feeding video set of a school of fish, a video sample dataset is obtained, wherein the video sample dataset is a plurality of temporally continuous video samples; Extracting a video sequence with a fixed number of frames from the video sample data set, converting the video sequence to obtain a corresponding grayscale image sequence and a difference frame image sequence; Inputting the grayscale image sequence into the appearance autoencoder network, inputting the difference frame image sequence into the motion autoencoder network, and outputting a reconstructed grayscale image sequence and a reconstructed difference frame image sequence; Determine an appearance loss function based on the grayscale image sequence and the reconstructed grayscale image sequence; determine a motion loss function based on the difference frame image sequence and the reconstructed difference frame image sequence; determine a total loss function based on the appearance loss function and the motion loss function; Iteratively optimizing parameters in the appearance autoencoder network and the motion autoencoder network according to the total loss function to train the quantization model of the school of fish feeding activity, thereby obtaining a trained quantization model of the school of fish feeding activity; quantifying the fish feeding activity quantification model trained to obtain a continuous fish feeding activity index by quantifying the fish feeding video to be detected in the fish feeding video set; The feeding decision unit is used to determine whether the current fish school is feeding based on the continuous fish school feeding activity index and the detection threshold, and to adaptively adjust the feeding amount based on the judgment result.

9. A precision feeding device based on quantification of fish feeding activity, the device utilizing the precision feeding method based on quantification of fish feeding activity according to any one of claims 1 to 7, the feeding device comprising: Industrial cameras, core processors, feeding systems and display platforms; The industrial camera is connected to the core processor and fixed on the top of the circulating water aquaculture pond. The industrial camera faces the water surface and is used to obtain videos of fish activities in the aquaculture pond and transmit the obtained videos to the core processor. The core processor is used to calculate a fish feeding activity index and a detection threshold based on the fish activity video in the aquaculture pond acquired by the industrial camera, and determine whether the fish are feeding based on the fish feeding activity index and the detection threshold, thereby obtaining a feeding decision result; The feeding system includes a communication controller, a solenoid valve, and a bait hopper; the communication controller is connected to the core processor and is used to receive the feeding decision result and drive the solenoid valve according to the feeding decision result; the bait hopper is used to hold bait, and a baffle is provided at the outlet of the bait hopper; the solenoid valve is used to control the opening or closing of the baffle to regulate the feeding amount; The display platform is connected to the core processor and is used to display the real-time acquired fish activity video, the real-time calculated fish feeding activity index, and the feeding decision result.

Images