Fish feeding method and device based on multi-task active learning framework

Abstract

The present invention discloses a fish feeding method and device based on a multi-task active learning framework. Based on a multi-task learning model, an active learning framework is integrated, and information entropy and approximate Bayesian inference are combined to calculate the uncertainty scores of the feeding intensity classification task and the residual bait counting task respectively. The total uncertainty score is calculated by a weighted method to select the samples to be labeled in the active learning process. Through the interactive training between the annotation expert and the model, not only can the performance approximate to that of full supervised learning be achieved with fewer training samples, but also the model can often correct the errors in manual annotation during the interaction process, so as to achieve the effect of reducing the data annotation cost and improving the model recognition accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of fishery aquaculture, and particularly relates to a fish feeding method and device based on a multi-task active learning framework. Background Art

[0002] As an innovative aquaculture model, the marine ranch aims to meet the growing market demand for aquatic products and reduce the over-reliance on wild fishery resources by implementing large-scale and intensive aquaculture strategies in the marine environment. Its efficient operation depends on cutting-edge intelligent feeding technology, which integrates various advanced means such as sensor monitoring, computer vision recognition, and artificial intelligence algorithms to achieve real-time monitoring of the aquaculture environment and precise management of fish status.

[0003] Although the aquaculture feeding method based on artificial intelligence has significant advantages in terms of scientificity and accuracy and demonstrates excellent performance in areas such as large-scale aquaculture requirements, its application is also accompanied by a series of technical challenges. First, the cost of model training is high. Not only does it require the classification and annotation of feeding intensity for a large amount of image data, but also the precise annotation of the position of residual bait. This process involves a large amount of data preprocessing and manual intervention, thus significantly increasing the training cost of the model. Second, due to the lack of obvious classification features of feeding intensity in different feeding scenarios and the difficulty in positioning caused by highly dense residual bait, annotation errors are likely to occur during the classification process, such as multiple annotation, less annotation, or misannotation of the position of residual bait. These problems will directly affect the accuracy and generalization ability of the trained model. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a fish feeding method and device based on a multi-task active learning framework.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A fish feeding method based on a multi-task active learning framework includes:

[0007] Step S1, constructing a fish school feeding behavior data set and a multi-task active learning framework; wherein, the multi-task active learning framework includes: an annotation expert, an unlabeled sample pool , a labeled sample pool , a multi-task learning model, and a multi-task inference network;

[0008] Step S2, randomly selecting in the unlabeled sample pool Use [[[number]]] image samples as the initial samples, which are labeled by annotation experts. After labeling, they are put into the labeled sample pool. Based on the labeled initial samples, the multi-task learning model is initially trained to obtain an initial model. The multi-task learning model includes: a feature sharing network, a feeding intensity classification network, and a residual bait counting network. The feeding intensity classification network and the residual bait counting network are respectively connected to the feature sharing network;

[0009] Step S3: Use the multi-task inference network to perform sample inference on the initial model in the unlabeled sample pool, calculate the feeding intensity classification uncertainty score and the residual bait counting regression uncertainty score for each image sample in the unlabeled sample pool respectively, and calculate the total uncertainty score by a weighted method. Select the top [[[number]]] image samples with the highest total uncertainty score as the samples to be labeled, and continue with expert annotation. After annotation, they are put into the labeled sample pool, and the model is iteratively trained on the basis of the initial model until the threshold of the evaluation index is reached or the maximum number of iterations is reached, to obtain the final multi-task learning model; Use [[[number]]] image samples as the samples to be labeled, and continue with expert annotation. After annotation, they are put into the labeled sample pool, and the model is iteratively trained on the basis of the initial model until the threshold of the evaluation index is reached or the maximum number of iterations is reached, to obtain the final multi-task learning model;

[0010] Step S4: Input the video of the fish school feeding behavior obtained in real time into the final multi-task learning model to predict the feeding intensity and the number of residual baits of the fish school, and at the same time output the corresponding feeding strategy for application.

[0011] Preferably, the unlabeled sample pool contains: all initial fish school feeding behavior image samples and the remaining fish school feeding behavior image samples after each iterative training of the multi-task learning model; the labeled sample pool is used to store the fish school feeding behavior image sample dataset that has been expertly annotated and the labeled samples that are continuously iteratively updated during the training process of the multi-task learning model.

[0012] Preferably, in step S3, the samples to be labeled during the iteration are determined through the query strategy function for unlabeled sample selection. Among them, the query strategy function includes: a classification query strategy and a regression query strategy. The classification query strategy uses the category information entropy to calculate the feeding intensity classification uncertainty of each image; the regression query strategy uses an approximate Bayesian inference network to calculate the residual bait counting regression uncertainty of each sample image; the total uncertainty score of each image is determined by a weighted summation method, and the samples to be labeled during the iteration are determined according to the total uncertainty score.

[0013] Preferably, the multi-task inference network includes: an approximate Bayesian network and the multi-task learning model obtained from the current round of training; the approximate Bayesian network is placed after the residual bait counting network in the multi-task learning model obtained from the current round of training; the approximate Bayesian inference network includes a second average pooling layer, a first Dropout layer, a first linear layer, a second Dropout layer, and a second linear layer connected in sequence.

[0014] The present invention also provides a fish feeding device based on a multi-task active learning framework, comprising:

[0015] A first processing module, configured to construct a fish school feeding behavior data set and a multi-task active learning framework; wherein; the multi-task active learning framework includes: an annotation expert, an unlabeled sample pool , a labeled sample pool , a multi-task learning model and a multi-task inference network;

[0016] A second processing module, configured to randomly select image samples in the unlabeled sample pool as initial samples, and have them annotated by the annotation expert. After annotation, they will be put into the labeled sample pool, and the multi-task learning model will be initially trained according to the annotated initial samples to obtain an initial model; the multi-task learning model includes: a feature sharing network, a feeding intensity classification network, and a residual bait counting network. The feeding intensity classification network and the residual bait counting network are respectively connected to the feature sharing network;

[0017] A third processing module, configured to perform sample inference on the initial model in the unlabeled sample pool through the multi-task inference network, calculate the feeding intensity classification uncertainty score and the residual bait counting regression uncertainty score of each image sample in the unlabeled sample pool respectively, and calculate the total uncertainty score by a weighted method, and select the top image samples with the highest total uncertainty score as samples to be labeled, and continue with expert annotation. After annotation, they will be put into the labeled sample pool, and the model will be iteratively trained on the basis of the initial model until the threshold of the evaluation index is reached or the maximum number of iterations is reached, to obtain a final multi-task learning model;

[0018] A fourth processing module, configured to input the real-time acquired fish school feeding behavior video into the final multi-task learning model to predict the fish school feeding intensity and the number of residual baits, and at the same time output the corresponding feeding strategy for application.

[0019] Preferably, the unlabeled sample pool contains: all initial fish school feeding behavior image samples and the fish school feeding behavior image samples remaining after each iterative training of the multi-task learning model; the labeled sample pool is used to store the fish school feeding behavior image sample data set that has been expert-annotated and the annotated samples that are continuously iteratively updated during the training process of the multi-task learning model.

[0020] Preferably, the third processing device determines the samples to be labeled during the iteration process through a query strategy function for unlabeled sample selection; wherein, the query strategy function includes: a classification query strategy and a regression query strategy. The classification query strategy calculates the classification uncertainty of the feeding intensity of each image using the category information entropy; the regression query strategy calculates the regression uncertainty of the remaining bait count of each sample image using an approximate Bayesian inference network; the total uncertainty score of each image is determined by means of weighted summation, and the samples to be labeled during the iteration process are determined according to the total uncertainty score.

[0021] Preferably, the multi-task inference network includes: an approximate Bayesian network and a multi-task learning model obtained from the current round of training; the approximate Bayesian network is placed after the remaining bait count network in the multi-task learning model obtained from the current round of training; the approximate Bayesian inference network includes a second average pooling layer, a first Dropout layer, a first linear layer, a second Dropout layer, and a second linear layer connected in sequence.

[0022] Based on the multi-task learning model, the present invention integrates an active learning framework, and combines information entropy and approximate Bayesian inference to select the uncertainty scores for the feeding intensity classification task and the remaining bait count task in the samples to be labeled. It can not only achieve performance similar to that of fully supervised learning with fewer training samples, but also enable the model to often correct errors in manual annotation during the interaction process, so as to reduce the data annotation cost and improve the model recognition accuracy, providing a feasible technical solution for solving the method of precise fish feeding. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0024] Figure 1 It is a flowchart of the fish feeding method based on the multi-task active learning framework in the embodiment of the present invention;

[0025] Figure 2 It is a schematic diagram of the multi-task active learning framework in the embodiment of the present invention;

[0026] Figure 3 It is a schematic diagram of the structure of the feeding intensity classification network;

[0027] Figure 4 It is a schematic diagram of the structure of the remaining bait count network;

[0028] Figure 5It is a schematic structural diagram of an approximate Bayesian inference network. Specific implementation manners

[0029] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0030] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0031] Embodiment 1:

[0032] As Figure 1 , 2 shown, the embodiment of the present invention provides a fish feeding method based on a multi-task active learning framework, including:

[0033] Step S1, constructing a fish school feeding behavior data set and a multi-task active learning framework; wherein; the multi-task active learning framework includes: a labeling expert, an unlabeled sample pool , a labeled sample pool , a multi-task learning model, and a multi-task inference network; dividing the fish school feeding behavior data set into a training set and a test set; annotating the feeding intensity classification label and the residual bait position label for the test set, and not annotating the feeding intensity classification label and the residual bait position label for the training set; storing the training set in the unlabeled sample pool ;

[0034] Step S2, randomly selecting image samples in the unlabeled sample pool as initial samples, and annotating them by a labeling expert. The annotation content includes the feeding intensity category and the number of residual baits. After annotation, they will be put into the labeled sample pool. The multi-task learning model is initially trained according to the annotated initial samples to obtain an initial model;

[0035] Step S3, performing sample inference on the initial model in the unlabeled sample pool through the multi-task inference network, respectively calculating the feeding intensity classification uncertainty score and the residual bait count regression uncertainty score of each image sample in the unlabeled sample pool, and calculating the total uncertainty score by a weighted method, and selecting the top Use

[0036] Step S4: Input the video of the fish school feeding behavior obtained in real time into the final multi-task learning model to predict the feeding intensity and the number of remaining baits of the fish school, and at the same time output the corresponding feeding strategy for application.

[0037] As an implementation manner of the embodiment of the present invention, in step S1, constructing the fish school feeding behavior data set includes:

[0038] Step 11: Use the constructed experimental environment to obtain the video of the fish school feeding behavior; wherein, the experimental environment includes: a circulating water aquaculture system, a video acquisition system and a data transmission system;

[0039] Step 12: Obtain the fish school feeding images according to the video of the fish school feeding behavior; wherein, adopt the method of extracting one frame every 5 s for image screening to obtain the fish school feeding images;

[0040] Step 13: Crop the fish school feeding images to cut off the irrelevant background in the fish school feeding images, and the resolution of the cropped images is 1024 ×1024, and then scale the cropped fish school feeding images, and the final resolution size of the scaled images is 512 ×512;

[0041] Step 14: Use the cropped and scaled fish school feeding images as the fish school feeding behavior data set; wherein, divide the fish school feeding behavior data set into a training set, a validation set and a test set according to the ratio of 7:1:2.

[0042] Furthermore, the multi-task active learning framework includes: an annotation expert ( ), an unlabeled sample pool and a labeled sample pool and a multi-task learning model . Connect them in series through the unlabeled data stream, the labeled data stream and human intervention, and the next round of training can be carried out only after the sample selection and expert annotation of each active learning algorithm.

[0043] Annotation expert ( ): Use the labelme software to annotate the test set, and the image label content includes two parts: the feeding intensity classification label and the remaining bait position label.

[0044] The feeding image intensity is divided into two types of labels - "Active" and "Not Active", as shown in Table 1, and the labeled classification labels are saved in txt format.

[0045] Annotation classification criteria: First, two experienced experts conduct a classification evaluation of the feeding intensity of the images. If the evaluation results are the same, the label of the image is determined. If they are different, a third expert is invited to conduct an annotation evaluation until the classification evaluation results of each annotating expert reach consistency.

[0046] Table 1

[0047]

[0048] For the annotation of the residual bait position, control points are used to annotate the residual bait position in the image, and the position coordinates of the residual bait are saved in json format.

[0049] Unlabeled sample pool Contains: all initial fish feeding behavior image samples and the fish feeding behavior image samples remaining after each iteration training of the multi-task learning model.

[0050] Labeled sample pool Used to store the fish feeding behavior image sample dataset that has been expert-annotated and the annotated samples continuously updated through iteration during the training process of the multi-task learning model.

[0051] As Figures 3 to 5 shown, the multi-task learning model Includes: a feature sharing network, a feeding intensity classification network, and a residual bait counting network. The feeding intensity classification network and the residual bait counting network are respectively connected to the feature sharing network. Among them, the feature sharing network designs the corresponding network structure according to different application scenarios; for example, lightweight networks such as MobileNet and ShuffleNet can be selected to reduce the model parameters. The feeding intensity classification network includes: a first convolutional layer, a second convolutional layer, a first max pooling layer, a third convolutional layer, a fourth convolutional layer, a second max pooling layer, a first average pooling layer (AvgPool), and a fully connected layer (FC) connected in sequence; among them, the convolutional kernel sizes of the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer are 3*3, and the stride is 1. Through the first max pooling layer and the second max pooling layer, the feature map output by the feature sharing network can be downsampled twice, and the feeding intensity category (Active or NotActive) of each image can be output through the average pooling operation. The residual bait counting network includes: 5 fifth convolutional layers connected in sequence, and the convolutional kernel size of the fifth convolutional layer is 3 3. The expansion rate d is 2. Among them, the inflated convolution can effectively count the residual baits in the dense area, and the number of residual baits in the whole image is obtained by summing the finally output density map.

[0052] As an implementation manner of the embodiment of the present invention, in step S2, the loss function for training the multi-task active model includes:

[0053] Using binary cross-entropy loss as the classification regression loss of feeding intensity, that is,

[0054]

[0055] where, is the number of samples in the training set, is the predicted classification category, is the probability belonging to this category.

[0056] The regression output of the residual bait counting network is the number of residual baits in each image, and the loss function used is the mean absolute error loss, that is

[0057]

[0058] where, is the number of test images, is the actual number of residual baits in the image, is the number of residual baits predicted by the model.

[0059] The loss function for training the multi-task active learning model is as follows:

[0060]

[0061] where, and respectively represent the variances of the classification loss and the regression loss following their respective distributions, is the regularization term.

[0062] As an implementation manner of the embodiment of the present invention, in step S3, the unlabeled samples to be marked in the iterative process are determined through the query strategy function; among them, the query strategy function includes: classification query strategy and regression query strategy , the classification query strategy uses the category information entropy to calculate the classification uncertainty of the feeding intensity of each image ; the regression query strategy uses the approximate Bayesian inference network to calculate the regression uncertainty of the residual bait count of each sample image ; the total uncertainty score of each image is determined according to the weighted summation method, and the unlabeled samples to be marked in the iterative process are determined according to the total uncertainty score.

[0063] Further, the multi-task inference network includes: an approximate Bayesian network and a multi-task learning model obtained from the current round of training; the approximate Bayesian network is placed after the residual bait counting network in the multi-task learning model obtained from the current round of training; the approximate Bayesian inference network includes a second average pooling layer, a first Dropout layer, a first linear layer, a second Dropout layer, and a second linear layer connected in sequence; wherein, the number of neurons in the first linear layer and the second linear layer are 1024 and 512 respectively, and the neuron deletion ratios of the first Dropout layer and the second Dropout layer are 0.3 and 0.15 respectively.

[0064] Further, the feeding intensity classification uncertainty of each image is calculated using the category information entropy, specifically: given an image data in the unlabeled pool , the information entropy of each category output by the multi-task model for it is:

[0065]

[0066] represents the classification uncertainty, and respectively represent the conditional probabilities that the image belongs to the feeding intensities of "Not Active" and "Active". Since there are only two categories, therefore, the conditional probabilities of the two are 1. When the conditional probabilities of the two are closer, it means that the classification performance of the model for this image is worse, and the information amount shown is larger, that is, the uncertainty score is higher.

[0067] Further, the residual bait counting regression uncertainty of each sample image is calculated using the approximate Bayesian inference network. Specifically: in the deep Bayesian neural network, the parameters of the model have a prior probability distribution, which can be used to quantify the prediction uncertainty in the regression task. By using the sampling method, that is, Monte Carlo Dropout regularization, it is used to predict the regression uncertainty. In the actual experimental process, the regression uncertainty is determined by calculating the variance according to the results output each time in the inference stage. The formula is as follows:

[0068]

[0069] wherein, represents the regression uncertainty, represents the regression result output each time in the inference, is the average value of the output results in the inference stage. Among them, is the number of Monte Carlo samplings based on sampling. By performing times of Monte Carlo sampling and taking the average, the final regression uncertainty is obtained.

[0070] By weighting the above classification uncertainty and regression uncertainty, the final total uncertainty score for each image is obtained , that is:

[0071]

[0072] Among them, is the weight factor, which is used to balance the uncertainties of the two tasks.

[0073] The training process of the multi-task active learning framework implemented in the present invention is as follows: Randomly select image samples from the unlabeled sample pool as the initial samples, and manually annotate them by experts. After annotation, they will be put into the labeled sample pool. The multi-task learning model will be initially trained according to the annotated initial samples to obtain the initial model;

[0074] Infer the samples of the initial model in the unlabeled sample pool, calculate the classification uncertainty score of the feeding intensity and the regression uncertainty score of the residual bait count for each image sample in the unlabeled sample pool respectively, and calculate the total uncertainty score by weighting. Select the top image samples with the highest uncertainty score as the samples to be labeled, and continue with expert annotation. After annotation, put them into the labeled sample pool and continue to train the model on the basis of the initial model. Repeat the above operations until the threshold of the evaluation index is reached or the maximum number of iterations is reached, and the algorithm ends to obtain the final trained model. Here, the number of initial samples selected , and the number of samples selected in subsequent iterations ; The detailed process is shown in Algorithm 1.

[0075]

[0076] As an implementation manner of the embodiment of the present invention, in step S4, different feeding strategies are formulated according to the feeding intensity of the fish group and the density level of the residual bait. The feeding strategies are divided into four types: continuous feeding, feeding stagnation, feeding suspension, and feeding end. The following is a specific introduction to these four feeding strategies respectively:

[0077] 1. Continuous feeding: The fish group shows a very active feeding state. Most fish actively stick their heads out of the water to look for bait, and the amount of bait in the pool is small and cannot meet the feeding needs of the fish group;

[0078] 2. Feeding stagnation: The fish group shows a relatively active feeding state, but the bait in the pool can meet the feeding needs of the fish group. Most fish only eat the nearby bait and do not actively look for bait. At this time, stop feeding in the short term;

[0079] 3. Feeding suspension: The fish school shows a less active feeding state. The fish school basically does not actively eat the bait nearby, and feeding is stopped for a period of time.

[0080] 4. Feeding end: The fish school shows a very inactive feeding state. Most of the fish school will not eat the bait, and there is still a large amount of residual bait in the pool. At this time, feeding stops and no more feeding is carried out.

[0081] Embodiment 2:

[0082] The embodiment of the present invention also provides a fish feeding device based on a multi-task active learning framework, including:

[0083] The first processing module is used to construct a fish school feeding behavior dataset and a multi-task active learning framework; wherein; the multi-task active learning framework includes: an annotation expert, an unlabeled sample pool , a labeled sample pool , a multi-task learning model and a multi-task inference network;

[0084] The second processing module is used to randomly select image samples in the unlabeled sample pool as initial samples, and the annotation expert will annotate them. After annotation, they will be put into the labeled sample pool. The multi-task learning model will be initially trained according to the annotated initial samples to obtain an initial model; the multi-task learning model includes: a feature sharing network, a feeding intensity classification network, and a residual bait counting network. The feeding intensity classification network and the residual bait counting network are respectively connected to the feature sharing network;

[0085] The third processing module is used to perform sample inference on the initial model in the unlabeled sample pool through the multi-task inference network, calculate the feeding intensity classification uncertainty score and the residual bait counting regression uncertainty score of each image sample in the unlabeled sample pool respectively, and calculate the total uncertainty score by a weighted method. Select the top image samples with the highest total uncertainty score as the samples to be labeled, and continue with expert annotation. After annotation, they will be put into the labeled sample pool. Iteratively train the model on the basis of the initial model until the threshold of the evaluation index is reached or the maximum number of iterations is reached to obtain the final multi-task learning model;

[0086] The fourth processing module is used to input the video of the real-time fish school feeding behavior into the final multi-task learning model to predict the feeding intensity and the number of residual baits of the fish school, and at the same time output the corresponding feeding strategy.

[0087] As an implementation manner of the embodiment of the present invention, the unlabeled sample pool contains: all initial fish school feeding behavior image samples and the fish school feeding behavior image samples remaining after each iterative training of the multi-task learning model; the labeled sample pool For storing the dataset of fish feeding behavior image samples that have been marked by experts and the marked samples that are continuously iteratively updated during the training process of the multi-task learning model.

[0088] As an implementation manner of an embodiment of the present invention, the third processing device determines the samples to be marked during the iteration process through the query policy function selected by the unmarked samples; wherein, the query policy function includes: a classification query policy and a regression query policy. The classification query policy calculates the classification uncertainty of the feeding intensity of each image using the category information entropy; the regression query policy calculates the regression uncertainty of the residual bait count of each sample image using an approximate Bayesian inference network; the total uncertainty score of each image is determined according to the weighted summation method, and the samples to be marked during the iteration process are determined according to the total uncertainty score.

[0089] As an implementation manner of an embodiment of the present invention, the multi-task inference network includes: an approximate Bayesian network and the multi-task learning model obtained by training in the current round; the approximate Bayesian network is placed after the residual bait count network in the multi-task learning model obtained by training in the current round; the approximate Bayesian inference network includes a second average pooling layer, a first Dropout layer, a first linear layer, a second Dropout layer, and a second linear layer connected in sequence.

[0090] The above-described embodiments are only descriptions of the preferred embodiments of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A fish feeding method based on a multi-task active learning framework, characterized in that, Including: Step S1: Construct a fish school feeding behavior dataset and a multi-task active learning framework; Among them; Constructing the fish school feeding behavior dataset includes: Step 11: Use the constructed experimental environment to obtain fish school feeding behavior videos; among them, the experimental environment includes: a circulating water aquaculture system, a video acquisition system, and a data transmission system; Step 12: Obtain fish school feeding images based on the fish school feeding behavior videos; among them, the images are screened by extracting one frame every 5s to obtain fish school feeding images; Step 13: Crop the fish feeding image to remove the irrelevant background in the fish feeding image. The resolution of the cropped image is 1024 × 1024. Then, scale the cropped fish feeding image. The final resolution of the scaled image is 512 × 512; Step 14: Use the cropped and scaled fish school feeding images as the fish school feeding behavior dataset; among them, the fish school feeding behavior dataset is divided into a training set, a validation set, and a test set according to the ratio of 7:1:2; The multi-task active learning framework includes: an annotation expert, an unlabeled sample pool , a labeled sample pool , a multi-task learning model, and a multi-task inference network; Step S2: Randomly select image samples in the unlabeled sample pool as initial samples, and have them labeled by annotation experts. After labeling, they will be put into the labeled sample pool, and the multi-task learning model will be initially trained based on the labeled initial samples to obtain an initial model. The multi-task learning model includes: a feature sharing network, a feeding intensity classification network, and a residual bait counting network. The feeding intensity classification network and the residual bait counting network are respectively connected to the feature sharing network; Step S3: Through the multi-task inference network, perform sample inference on the initial model in the unlabeled sample pool, calculate the classification uncertainty score of the feeding intensity and the regression uncertainty score of the residual bait count for each image sample in the unlabeled sample pool respectively, calculate the total uncertainty score by weighted means, and select the top image samples with the highest total uncertainty score as the samples to be labeled, and continue with expert annotation. After annotation, put them into the labeled sample pool, and perform iterative training on the model based on the initial model until the threshold of the evaluation index is reached or the maximum number of iterations is reached, to obtain the final multi-task learning model; Step S4: Input the real-time obtained fish school feeding behavior video into the final multi-task learning model to predict the fish school feeding intensity and the number of remaining baits, and at the same time output the corresponding feeding strategy.

2. The fish feeding method based on the multi-task active learning framework according to claim 1, characterized in that Unlabeled sample pool It includes: all initial fish feeding behavior image samples and the remaining fish feeding behavior image samples after each iteration of the multi-task learning model training; Labeled sample pool It is used to store the fish feeding behavior image sample dataset that has been expert-annotated and the annotated samples that are continuously iteratively updated during the training of the multi-task learning model.

3. The fish feeding method based on the multi-task active learning framework according to claim 2, characterized in that In step S3, the unlabeled sample selection query strategy function is used to determine the samples to be labeled during the iteration process; among them, the query strategy function includes: a classification query strategy and a regression query strategy. The classification query strategy uses the category information entropy to calculate the classification uncertainty of the feeding intensity of each image; the regression query strategy uses an approximate Bayesian inference network to calculate the regression uncertainty of the remaining bait count of each sample image; the total uncertainty score of each image is determined according to the weighted summation method, and the samples to be labeled during the iteration process are determined according to the total uncertainty score.

4. The fish feeding method based on the multi-task active learning framework according to claim 3, wherein, The multi-task inference network includes: an approximate Bayesian network and the multi-task learning model obtained from the current round of training; the approximate Bayesian network is placed after the remaining bait count network in the multi-task learning model obtained from the current round of training; the approximate Bayesian inference network includes a second average pooling layer, a first Dropout layer, a first linear layer, a second Dropout layer, and a second linear layer connected in sequence.

5. A fish feeding device based on a multi-task active learning framework for implementing the fish feeding method based on the multi-task active learning framework described in claim 1, characterized in that, Including: The first processing module is used to construct a fish school feeding behavior dataset and a multi-task active learning framework; Among them; The multi-task active learning framework includes: an annotation expert, an unlabeled sample pool , a labeled sample pool , a multi-task learning model, and a multi-task inference network; The second processing module is used to randomly select image samples as initial samples from the unlabeled sample pool, and these samples are labeled by annotation experts. After labeling, they will be put into the labeled sample pool. Based on the labeled initial samples, the multi-task learning model is initially trained to obtain an initial model. The multi-task learning model includes: a feature sharing network, a feeding intensity classification network, and a residual bait counting network. The feeding intensity classification network and the residual bait counting network are respectively connected to the feature sharing network; The third processing module is used to perform sample inference on the initial model in the unlabeled sample pool through a multi-task inference network, calculate the feeding intensity classification uncertainty score and the residual bait count regression uncertainty score for each image sample in the unlabeled sample pool respectively, calculate the total uncertainty score by a weighted method, and select the top image samples with the highest total uncertainty score as the samples to be labeled, and continue with expert annotation. After annotation, they are put into the labeled sample pool, and the model is iteratively trained on the basis of the initial model until the threshold of the evaluation index is reached or the maximum number of iterations is reached, and the final multi-task learning model is obtained; The fourth processing module is used to input the real-time obtained fish school feeding behavior video into the final multi-task learning model to predict the fish school feeding intensity and the number of remaining baits, and at the same time output the corresponding feeding strategy.

6. The fish feeding device based on the multi-task active learning framework according to claim 5, characterized in that, Unlabeled sample pool Contains: all initial fish feeding behavior image samples and the remaining fish feeding behavior image samples after each iteration training of the multi-task learning model; Labeled sample pool Used to store the fish feeding behavior image sample dataset that has been annotated by experts and the annotated samples that are continuously iteratively updated during the training of the multi-task learning model.

7. The fish feeding device based on the multi-task active learning framework according to claim 6, characterized in that, The third processing device determines the samples to be labeled during the iteration process through the unlabeled sample selection query strategy function; among them, the query strategy function includes: a classification query strategy and a regression query strategy. The classification query strategy uses the category information entropy to calculate the classification uncertainty of the feeding intensity of each image; the regression query strategy uses an approximate Bayesian inference network to calculate the regression uncertainty of the remaining bait count of each sample image; the total uncertainty score of each image is determined according to the weighted summation method, and the samples to be labeled during the iteration process are determined according to the total uncertainty score.

8. The fish feeding device based on the multi-task active learning framework according to claim 7, wherein, The multi-task inference network includes: an approximate Bayesian network and the multi-task learning model obtained from the current round of training; the approximate Bayesian network is placed after the remaining bait count network in the multi-task learning model obtained from the current round of training; the approximate Bayesian inference network includes a second average pooling layer, a first Dropout layer, a first linear layer, a second Dropout layer, and a second linear layer connected in sequence.

Images