Fish group feeding behavior identification method and device and storage medium

Abstract

The invention provides a fish group feeding behavior identification method and device and a storage medium. The fish group feeding behavior identification method comprises the following steps: acquiring to-be-processed fish behavior data; inputting the to-be-processed fish behavior data into a fish ingestion behavior recognition model to obtain an ingestion behavior recognition result of a target fish school; wherein the fish ingestion behavior recognition model is trained by the following steps: acquiring an original video shot at a culture site, and acquiring a fish ingestion behavior data set according to the original video; based on the fish feeding behavior data set, obtaining a feeding behavior recognition result by using a coordinate attention mechanism and a depth separable convolution; and training the fish feeding behavior recognition model based on the feeding behavior recognition result. The recognition model with higher learning ability and adaptive ability is obtained through training, and the accuracy of fish feeding behavior recognition is improved.

Description

Technical Field

[0001] The present invention relates to the field of target recognition technology, and in particular to a method, a device and a storage medium for identifying the feeding behavior of a fish group. Background Art

[0002] Accurate identification of fish feeding behavior is one of the important ways to achieve scientific farming. It can not only provide decision-making information for feeding timing and reduce problems such as feed waste and environmental pollution caused by overfeeding. It can also provide important reference for determining breeding plans and adjusting feeding strategies. However, aquaculture currently still relies mainly on manual identification, which has low accuracy and leads to low feed utilization.

[0003] At present, deep learning can automatically extract image features and learn more image data, and has been widely used in the field of aquatic product feeding recognition. Among them, the Transformer model extracts features by calculating the connection in the input sequence, which is suitable for processing sequence data and provides a new idea for accurately identifying fish feeding behavior.

[0004] However, when filming fish feeding behavior, data of some feeding levels are often lacking, which will lead to an imbalance in the data set samples and ultimately affect the training effect. Moreover, in the complex environment of the recirculating aquaculture system, there are abnormal postures of underwater fish and interference from the feeding environment, which reduces the accuracy of fish feeding behavior recognition. Summary of the invention

[0005] The present invention provides a method, a device and a storage medium for identifying the feeding behavior of a fish group, which are used to solve the technical problem of low accuracy in identifying the feeding behavior of fish in the prior art.

[0006] In a first aspect, the present invention provides a method for identifying feeding behavior of a fish group, comprising: Obtaining fish behavior data to be processed; Inputting the to-be-processed fish behavior data into a fish feeding behavior recognition model to obtain a feeding behavior recognition result of a target fish school; The fish feeding behavior recognition model is trained by the following steps: Obtaining an original video shot at a breeding site, and obtaining a fish feeding behavior dataset based on the original video; Based on the fish feeding behavior dataset, the coordinate attention mechanism and depth-wise separable convolution are used to obtain the feeding behavior recognition results; The fish feeding behavior recognition model is trained based on the feeding behavior recognition result.

[0007] In some embodiments, the method of obtaining a feeding behavior recognition result based on the fish feeding behavior dataset using a coordinate attention mechanism and a depthwise separable convolution includes: Based on the fish feeding behavior dataset, the target feature information is extracted using a coordinate attention mechanism to obtain a first feature map; Inputting the first feature map into a depth-separable gated DSGsted convolution module in the model to obtain a second feature map; the DSGsted convolution module includes a depth-separable convolution network; Inputting the second feature map into a multi-scale module after patch embedding to obtain a multi-scale feature map; A feeding behavior recognition result corresponding to the fish feeding behavior data set is obtained based on the multi-scale feature map.

[0008] In some embodiments, the step of inputting the first feature map into a depth-separable gated DSGsted convolution module in a model to obtain a second feature map comprises: Normalizing the first feature map through a normalization layer in the DSGsted convolution module; The first branch network, the second branch network and the third branch network in the DSGsted convolution module are used to perform dimension transformation and feature extraction on the standardized first feature map respectively to obtain the features output by each branch network; the first branch network includes a linear layer and a depthwise separable convolutional network; the second branch network includes a linear layer and a convolutional network; the third branch network includes a linear layer and an activation layer; The features output by each branch network are fused and linearly transformed to obtain the second feature map.

[0009] In some embodiments, the method of extracting target feature information based on the fish feeding behavior dataset using a coordinate attention mechanism to obtain a first feature map includes: Performing an average pooling operation in the height direction on the input image in the fish feeding behavior dataset to obtain a feature map in the height direction, and performing an average pooling operation in the width direction on the fish feeding behavior dataset to obtain a feature map in the width direction; Connecting and convolving the feature map in the height direction and the feature map in the width direction to obtain a connected feature map, and performing batch normalization and nonlinear activation operations on the connected feature map to obtain an integrated feature map; Inputting the integrated feature map into the first convolution layer and the first activation layer in sequence to obtain a first output feature map, and inputting the integrated feature map into the second convolution layer and the second activation layer in sequence to obtain a second output feature map; The input image, the first output feature map, and the second output feature map are element-wise multiplied to obtain the first feature map.

[0010] In some embodiments, the multi-scale module includes a self-attention module and a residual pooling connection layer; Among them, a relative position encoding layer is embedded in the self-attention module, and the relative position encoding layer is used to determine the relative position encoding corresponding to the key K tensor; the residual pooling connection layer is used to fuse the pooled query Q tensor with the output features of the self-attention module.

[0011] In some embodiments, obtaining an original video shot at a breeding site and obtaining a fish feeding behavior dataset based on the original video includes: The breeding site is filmed with a camera to obtain an original video; the camera is placed in a transparent waterproof box, the waterproof box is arranged at the edge of the fish pond and the camera faces the main area of ​​the breeding pond; Converting the original video into pictures, and adding labels to the converted pictures; the labels are divided according to eating characteristics; The labeled images are subjected to data augmentation and resolution adjustment, and the images are divided into training set, validation set, and test set.

[0012] In some embodiments, the training of the fish feeding behavior recognition model based on the feeding behavior recognition result includes: Based on the feeding behavior recognition results, the fish feeding behavior recognition model is trained using the adaptive moment estimation AdamW optimizer.

[0013] In a second aspect, the present invention provides a device for identifying feeding behavior of fish groups, comprising the following modules.

[0014] An acquisition module, used for acquiring the fish behavior data to be processed; An identification module, used for inputting the to-be-processed fish behavior data into a fish feeding behavior identification model to obtain a feeding behavior identification result of a target fish school; The fish feeding behavior recognition model is trained by the following steps: Obtaining an original video shot at a breeding site, and obtaining a fish feeding behavior dataset based on the original video; Based on the fish feeding behavior dataset, the coordinate attention mechanism and depth-wise separable convolution are used to obtain the feeding behavior recognition results; The fish feeding behavior recognition model is trained based on the feeding behavior recognition result.

[0015] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements any of the above-described methods for identifying feeding behavior of fish groups.

[0016] In a fourth aspect, a non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements any of the above-described methods for identifying feeding behavior of a fish group.

[0017] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method for identifying feeding behavior of fish groups as described in the first aspect above.

[0018] The present invention provides a method, device and storage medium for identifying the feeding behavior of a fish group, which obtains the fish behavior data to be processed; inputs the fish behavior data to be processed into a fish feeding behavior recognition model to obtain the feeding behavior recognition result of the target fish group; wherein the fish feeding behavior recognition model is trained by the following steps: obtaining the original video shot at the breeding site, and obtaining a fish feeding behavior data set based on the original video, based on the fish feeding behavior data set, using a coordinate attention mechanism and a depthwise separable convolution to obtain the feeding behavior recognition result, and training the fish feeding behavior recognition model based on the feeding behavior recognition result, thereby improving the ability of the fish feeding behavior recognition model to capture abnormal posture features and the ability to learn fish features in a complex environment, thereby improving the accuracy of fish feeding behavior recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of the method for identifying the feeding behavior of fish groups provided by the present invention.

[0021] Figure 2 This is one of the structural schematic diagrams of the fish group feeding behavior identification device provided by the present invention.

[0022] Figure 3 It is a structural schematic diagram of the fish feeding behavior recognition model provided by the present invention.

[0023] Figure 4 It is a schematic diagram of the network structure of the coordinate attention module provided by the present invention.

[0024] Figure 5 It is a schematic diagram of the network structure of the DSGated convolution module provided by the present invention.

[0025] Figure 6 Schematic diagram of the network structure of the multi-scale module provided by the present invention.

[0026] Figure 7 This is the second structural schematic diagram of the fish group feeding behavior identification device provided by the present invention.

[0027] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] Figure 1 FIG. 1 is one of the flow charts of the method for identifying the feeding behavior of fish groups provided by the present invention, such as Figure 1 As shown, the present invention provides a method for identifying the feeding behavior of a fish group. The method comprises: Step 101: Obtain fish behavior data to be processed.

[0030] Step 102: input the to-be-processed fish behavior data into a fish feeding behavior recognition model to obtain a feeding behavior recognition result of the target fish school.

[0031] Specifically, the fish behavior data to be processed refers to the relevant image data of the target fish school, and the target fish school can be recirculating aquaculture fish. Fish group feeding behavior recognition refers to identifying or detecting the feeding behavior of fish from relevant images, that is, classifying the behavior of fish into feeding behavior and other behaviors (non-feeding behavior), or identifying the type of fish feeding behavior.

[0032] A camera or other equipment is used to shoot in a fish farm where the target fish school is located to obtain an original video, which is processed to obtain the fish behavior data to be processed. The fish behavior data to be processed is input into a fish feeding behavior recognition model to output the feeding behavior recognition result of the target fish school.

[0033] For example, the computing processor is connected to the camera and the light source respectively. The computing processor can control the camera to collect the behavior video of the target fish school in the farm in real time and extract the video into a video stream. The computing processor can also control the switch of the light source, which is used to fill the light for the camera. The computer sends the video to the computing processor, and the computing processor performs fish feeding behavior recognition based on the trained fish feeding behavior recognition model to obtain the feeding behavior recognition result of the target fish school.

[0034] Among them, the camera includes a camera head, a computing chip and a communication interface in sequence, the controller is connected to the communication interface respectively, the computing chip can control the camera to collect data, and transmit the collected data to the computing processor through the communication interface.

[0035] Optionally, the camera may be a RealSense Depth Camera D455, which is produced and developed by Intel Corporation, has an operating range of 6m, a field of view depth of 86×57, and includes an RGB sensor and a tracking module. The video frame rate is set to 30 frames / s.

[0036] The fish feeding behavior recognition model is trained through the following steps: Obtaining an original video shot at a breeding site, and obtaining a fish feeding behavior dataset based on the original video; Based on the fish feeding behavior dataset, the coordinate attention mechanism and depth-wise separable convolution are used to obtain the feeding behavior recognition results; The fish feeding behavior recognition model is trained based on the feeding behavior recognition result.

[0037] Specifically, firstly, a fish feeding behavior dataset is constructed: an original video shot at a breeding site is obtained, and a fish feeding behavior dataset is obtained based on the original video.

[0038] For example, obtain the original video shot at the breeding site, convert the original video into multiple pictures, annotate the pictures with information to distinguish fish feeding behavior from other behaviors, and then preprocess the pictures to make them image data suitable for inputting the model, thus obtaining a fish feeding behavior dataset.

[0039] Then, the fish feeding behavior dataset is input into the fish feeding behavior recognition model, and the feeding behavior recognition result is obtained by using the coordinate attention mechanism (CA) and the depthwise separable convolution (DSConV). The fish feeding behavior recognition model is trained based on the feeding behavior recognition result to generate a trained fish feeding behavior recognition model (DCA-MVIT).

[0040] The fish feeding status and confidence value can be used as the expected output of the recognition model, and the fish feeding behavior recognition model can be subsequently deployed to edge devices or automatic detection systems.

[0041] The method for identifying the feeding behavior of a group of fish provided in the embodiment of the present application constructs a fish feeding behavior recognition model for the target classification task, targeting abnormal postures in complex underwater environments, environmental interference, and different features in time series data. The method improves the ability of the fish feeding behavior recognition model to capture abnormal posture features and the ability to learn fish features in complex environments, thereby improving the accuracy of fish feeding behavior recognition.

[0042] In some embodiments, obtaining an original video shot at a breeding site and obtaining a fish feeding behavior dataset based on the original video includes: The breeding site is filmed with a camera to obtain an original video; the camera is placed in a transparent waterproof box, the waterproof box is arranged at the edge of the fish pond and the camera faces the main area of ​​the breeding pond; Converting the original video into pictures, and adding labels to the converted pictures; the labels are divided according to eating characteristics; The labeled images are subjected to data augmentation and resolution adjustment, and the images are divided into training set, validation set, and test set.

[0043] Specifically, Figure 2 FIG. 1 is one of the structural schematic diagrams of the device for identifying the feeding behavior of fish groups provided by the present invention. Figure 2 As shown, the recognition device includes a camera, a waterproof box, a light source, and a computer. The camera is placed in a transparent waterproof box, which is set at the edge of the fish pond and faces the main area of ​​the breeding pond, so that the camera can capture the normal swimming of the fish. The computer can send the video captured by the camera to the operation processor, and the operation processor performs fish feeding behavior recognition according to the trained fish feeding behavior recognition model.

[0044] Process the original video: First, perform a simple image conversion on the original video, and annotate the converted images, and divide multiple feeding behavior labels according to the feeding characteristics. Then perform data enhancement such as horizontal flipping, random noise, and random filtering on the input image. At the same time, divide the data set into training set, validation set, and test set according to a certain ratio; the image resolution is adjusted to W × H (W and H are both greater than 0), and finally obtain the fish feeding behavior data set.

[0045] The method for identifying the feeding behavior of a group of fish provided in the embodiment of the present application uses a camera to collect video stream data of fish swimming and feeding under the control of a computing processor. The light source is used to supplement the light for the waterproof camera device. When the underwater light is insufficient, the light source and the light intensity transmitter supplement the light. The underbalanced data set samples are expanded by performing data enhancement such as horizontal flipping, random noise, and random filtering on the acquired raw data, thereby solving the problem of data set imbalance generated in shooting fish feeding behavior and improving the recognition accuracy of the model.

[0046] In some embodiments, the method of obtaining a feeding behavior recognition result based on the fish feeding behavior dataset using a coordinate attention mechanism and a depthwise separable convolution includes: Based on the fish feeding behavior dataset, the target feature information is extracted using a coordinate attention mechanism to obtain a first feature map; Inputting the first feature map into a depth-separable gated DSGsted convolution module in the model to obtain a second feature map; the DSGsted convolution module includes a depth-separable convolution network; Inputting the second feature map into a multi-scale module after patch embedding to obtain a multi-scale feature map; A feeding behavior recognition result corresponding to the fish feeding behavior data set is obtained based on the multi-scale feature map.

[0047] Specifically, in the fish feeding behavior recognition model, the backbone network takes the acquired fish feeding behavior dataset as input, uses the coordinate attention mechanism to extract target feature information, outputs the first feature map, and then inputs the first feature map into the DSGsted convolution module to output the second feature map. The second feature map is then input into the multi-scale module after patch embedding, outputs the multi-scale feature map, and finally obtains the final feeding behavior recognition result based on the multi-scale feature map.

[0048] For example, Figure 3 Schematic diagram of the structure of the fish feeding behavior recognition model provided by the present invention. Figure 3 As shown in the figure, the fish feeding behavior dataset is taken as input, the target feature information required for fish feeding recognition is extracted through the coordinate attention mechanism, and the first feature map with a scale of H × W × C (such as 3×224×224) is output.

[0049] Then, the first feature map is input into the DSGsted convolution module in the model, the model is quickly adjusted and the key domain information of the specific image is learned, and the second feature map with an output scale of H×W×C is obtained. The DSGsted convolution module includes a deep separable convolutional network.

[0050] Next, the second feature map is patch-embedded and input into four multi-scale modules (MultiScale Block), and multi-scale feature maps are output, such as feature maps with output scales of A×B, A / 4×B / 2, A / 16×B / 4, and A / 64×B / 8 (A and B are both greater than 0). Finally, the multi-scale feature map is transposed and reshaped to obtain the final output, thereby realizing the recognition of fish feeding behavior.

[0051] In some embodiments, the method of extracting target feature information based on the fish feeding behavior dataset using a coordinate attention mechanism to obtain a first feature map includes: Performing an average pooling operation in the height direction on the input image in the fish feeding behavior dataset to obtain a feature map in the height direction, and performing an average pooling operation in the width direction on the fish feeding behavior dataset to obtain a feature map in the width direction; Connecting and convolving the feature map in the height direction and the feature map in the width direction to obtain a connected feature map, and performing batch normalization and nonlinear activation operations on the connected feature map to obtain an integrated feature map; Inputting the integrated feature map into the first convolution layer and the first activation layer in sequence to obtain a first output feature map, and inputting the integrated feature map into the second convolution layer and the second activation layer in sequence to obtain a second output feature map; The input image, the first output feature map, and the second output feature map are element-wise multiplied to obtain the first feature map.

[0052] Specifically, the coordinate attention mechanism is used to capture the long-distance dependencies of image features, which can address the limitations of the traditional attention mechanism. Figure 4 is a schematic diagram of the network structure of the coordinate attention module provided by the present invention, such as Figure 4 As shown in the figure, first, the input image in the fish feeding behavior dataset is averaged in the height direction and the width direction to obtain feature maps in two directions: , in, F h ( h ), F w ( w ) and x are the feature map in the height direction, the feature map in the width direction, and the input image, respectively. W and H are the scales of the input image in height and width, respectively.

[0053] Next, the two feature maps are concatenated and convolved to obtain a concatenated feature map, and the concatenated feature map is batch normalized and nonlinearly activated to obtain an integrated feature map. The integrated feature map can be expressed as: Among them, f is the integrated feature map, δ is a nonlinear activation layer, C 1×1 is a one-dimensional convolution operation, Concat () is the connection operation, F h is the feature map in the height direction, F w It is the feature map in the width direction.

[0054] Finally, the integrated feature map is sequentially input into the first convolutional layer and the first activation layer to obtain the first output feature map, and the integrated feature map is sequentially input into the second convolutional layer and the second activation layer to obtain the second output feature map. The input image is element-wise multiplied with the first output feature map and the second output feature map to obtain the reconstructed feature map, i.e., the first feature map. The reconstructed feature map can be expressed as: in, Y is the reconstructed feature map output (i.e., the first feature map), σ is the Sigmoid activation layer, , They are one-dimensional convolutions in the height direction and width direction respectively.

[0055] The method for identifying feeding behavior of fish groups provided in the embodiment of the present application uses a coordinate attention mechanism that selectively focuses on the specific spatial location of the relevant task, which can enhance the model's attention to specific feeding area information. This not only solves the limitations of the traditional attention mechanism, but also enhances the model's ability to capture abnormal posture features.

[0056] In some embodiments, the step of inputting the first feature map into a depth-separable gated DSGsted convolution module in a model to obtain a second feature map comprises: Normalizing the first feature map through a normalization layer in the DSGsted convolution module; The first branch network, the second branch network and the third branch network in the DSGsted convolution module are used to perform dimension transformation and feature extraction on the standardized first feature map respectively to obtain the features output by each branch network; the first branch network includes a linear layer and a depthwise separable convolutional network; the second branch network includes a linear layer and a convolutional network; the third branch network includes a linear layer and an activation layer; The features output by each branch network are fused and linearly transformed to obtain the second feature map.

[0057] Specifically, the depthwise separable gated DSGsted convolution module is used to quickly adjust the model to learn the key domain information of a specific image. In the DSGated convolution module, a multi-step, multi-branch structure is mainly adopted.

[0058] Figure 5 : is a schematic diagram of the network structure of the DSGated convolution module provided by the present invention, such as Figure 5 As shown in the figure, in the DSGsted convolution module, first, the input first feature map is normalized by the normalization layer to accelerate training and improve model stability. Secondly, the normalized feature map is transformed in dimension and extracted by three parallel branch networks, where the first branch network contains a linear layer and a depthwise separable convolutional network, the second branch network contains a linear layer and a convolutional network, and the third branch network contains a linear layer and an activation layer. Finally, the features output by each branch network are fused and linearly transformed to obtain the second feature map.

[0059] DSConV is a convolution operation that aims to reduce the complexity and number of parameters of convolutional neural networks while maintaining model performance. DSConV convolution consists of depthwise convolution and pointwise convolution. Depthwise convolution is used to extract spatial features, and its convolution kernel is a single-channel mode, which requires convolution for each channel of the input. Pointwise convolution is used to extract channel features, and uses a one-dimensional convolution kernel for dimensionality increase.

[0060] In particular, the number of parameters of traditional convolution is , and the number of parameters of DSConV is ,in, is the convolution kernel size, W is the number of convolution kernels for depth-wise convolution, C is the number of convolution kernels for point-by-point convolution. Therefore, DSConV can effectively reduce the number of model parameters compared to traditional convolution.

[0061] In order to solve the problems of environmental interference and different features in time series data, the fish group feeding behavior recognition method provided in the embodiment of the present application adds a new linear layer and DSConV convolution branch in the DSGated convolution module to quickly adjust the convolution operation, which can quickly adjust the model to learn the key domain information of specific images, and improve the model's ability to learn fish features in complex environments. The DSGated convolution can effectively enhance the model's learning of characteristic information in classification tasks, effectively improve the model's ability to learn fish features in complex environments, and can also reduce the number of model parameters and reduce the complexity of the model.

[0062] In some embodiments, the multi-scale module includes a self-attention module and a residual pooling connection layer; Among them, a relative position encoding layer is embedded in the self-attention module, and the relative position encoding layer is used to determine the relative position encoding corresponding to the key K tensor; the residual pooling connection layer is used to fuse the pooled query Q tensor with the output features of the self-attention module.

[0063] Specifically, the multi-scale module is used to capture multi-scale information, gradually reduce the spatial resolution, and enhance the feature expression ability to adapt the model to complex scenes. The multi-scale module includes a self-attention module and a residual pooling connection layer.

[0064] Figure 6 is a schematic diagram of the network structure of the multi-scale module provided by the present invention, such as Figure 6 As shown, a relative position encoding layer is embedded in the self-attention module, which is used to determine the relative position encoding corresponding to the key K tensor to guide K to obtain its position.

[0065] A residual pooling connection is introduced at position Q to fuse the pooled query Q tensor with the output features of the self-attention module, thereby strengthening the transmission of information flow and improving the training efficiency of the pooling module.

[0066] The multi-scale module utilizes linear layers and pooling layers to model global information.

[0067] In some embodiments, the training of the fish feeding behavior recognition model based on the feeding behavior recognition result includes: Based on the feeding behavior recognition results, the fish feeding behavior recognition model is trained using the adaptive moment estimation AdamW optimizer.

[0068] Specifically, the training data set and the validation data set in the fish feeding behavior data set are used as the input of the model training, the initial parameters of the network are set, and the recognition model is trained using the AdamW optimizer to generate a trained fish feeding behavior recognition model.

[0069] Optionally, on a 64-bit Windows 10 operating system platform, a fish feeding behavior recognition model was built based on the Pytorch 1.12.0 and Torchvision0.13.1 deep learning frameworks and using the Python3.8 scripting language, and the model training was completed using an NVIDIA GeForce RTX 3090Ti graphics card. For all comparison models and the model in this paper, the batch size on one GPU is 32 images. The initial learning rate is set to 0.000125, the weight decay is set to 0.05, and the number of training rounds is uniformly set to 200 epochs.

[0070] The fish group feeding behavior recognition method provided in the embodiment of the present application is based on the improved multiscale vision transformer version 2 (MVITv2) algorithm, and a fish feeding behavior recognition model DCA-MVIT is proposed. In general, the use of the fish feeding behavior recognition model for feeding behavior recognition can greatly improve the model's ability to detect different features in time series data while ensuring the accuracy of feeding behavior recognition, and keep the model size basically unchanged.

[0071] Specifically, a coordinate attention mechanism was introduced into the model, which focused the recognition model's attention on the feeding characteristics of fish in characteristic position domains, rather than on background such as feeding bait or splashes, thereby improving the recognition performance of the model in underwater feeding environments.

[0072] Since the coherent video stream data in the same feeding behavior may contain different feeding feature information, such as fast swimming, splashing or no obvious feature information, this may lead to unstable model prediction results. In order to ensure high recognition accuracy while maintaining complete learning of video data stream information, a multi-step, multi-branch structure DSGated convolution module is proposed, which can not only significantly enhance the context feature fusion and model recognition accuracy, but also reduce the number of parameters.

[0073] The method for identifying the feeding behavior of fish groups provided in the embodiment of the present application realizes the accurate identification of the feeding behavior of fish, can be effectively applied in aquaculture environments, and provides a reliable and accurate technical means for the research and monitoring of fish feeding behavior, which is of great significance for the realization of scientific feeding and factory-scale recirculating aquaculture.

[0074] Figure 7 FIG. 2 is a second structural diagram of the device for identifying the feeding behavior of fish groups provided by the present invention. Figure 7 As shown, the present invention provides a device for identifying feeding behavior of a fish group, including an acquisition module 701 and an identification module 702.

[0075] The acquisition module 701 is used to acquire the fish behavior data to be processed; The identification module 702 is used to input the to-be-processed fish behavior data into the fish feeding behavior identification model to obtain the feeding behavior identification result of the target fish school; The fish feeding behavior recognition model is trained by the following steps: Obtaining an original video shot at a breeding site, and obtaining a fish feeding behavior dataset based on the original video; Based on the fish feeding behavior dataset, the coordinate attention mechanism and depth-wise separable convolution are used to obtain the feeding behavior recognition results; The fish feeding behavior recognition model is trained based on the feeding behavior recognition result.

[0076] Specifically, the above-mentioned fish group feeding behavior identification device provided by the present invention can implement all the method steps implemented in the above-mentioned fish group feeding behavior identification method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0077] It should be noted that the division of units / modules in the above-mentioned embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, the functional units in the various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.

[0078] Figure 8 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 8 As shown, the electronic device may include: a processor 801, a communications interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communications interface 802 and the memory 803 communicate with each other via the communication bus 804. The processor 801 may call the logic instructions in the memory 803 to execute the method for identifying the feeding behavior of a fish group, and the method includes: Obtaining fish behavior data to be processed; Inputting the to-be-processed fish behavior data into a fish feeding behavior recognition model to obtain a feeding behavior recognition result of a target fish school; The fish feeding behavior recognition model is trained by the following steps: Obtaining an original video shot at a breeding site, and obtaining a fish feeding behavior dataset based on the original video; Based on the fish feeding behavior dataset, the coordinate attention mechanism and depth-wise separable convolution are used to obtain the feeding behavior recognition results; The fish feeding behavior recognition model is trained based on the feeding behavior recognition result.

[0079] Specifically, the processor 801 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD), and the processor may also adopt a multi-core architecture.

[0080] The logic instructions in the memory 803 can be implemented in the form of software functional units and can be stored in a processor-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0081] In some embodiments, a computer program product is further provided, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the fish group feeding behavior identification method provided by the above-mentioned method embodiments, the method comprising: Obtaining fish behavior data to be processed; Inputting the to-be-processed fish behavior data into a fish feeding behavior recognition model to obtain a feeding behavior recognition result of a target fish school; The fish feeding behavior recognition model is trained by the following steps: Obtaining an original video shot at a breeding site, and obtaining a fish feeding behavior dataset based on the original video; Based on the fish feeding behavior dataset, the coordinate attention mechanism and depth-wise separable convolution are used to obtain the feeding behavior recognition results; The fish feeding behavior recognition model is trained based on the feeding behavior recognition result.

[0082] Specifically, the above-mentioned computer program product provided in the embodiment of the present application can implement all the method steps implemented by the above-mentioned method embodiments, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0083] In some embodiments, a computer-readable storage medium is further provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program is used to enable a computer to execute the fish group feeding behavior identification method provided by the above-mentioned method embodiments, the method comprising: Obtaining fish behavior data to be processed; Inputting the to-be-processed fish behavior data into a fish feeding behavior recognition model to obtain a feeding behavior recognition result of a target fish school; The fish feeding behavior recognition model is trained by the following steps: Obtaining an original video shot at a breeding site, and obtaining a fish feeding behavior dataset based on the original video; Based on the fish feeding behavior dataset, the coordinate attention mechanism and depth-wise separable convolution are used to obtain the feeding behavior recognition results; The fish feeding behavior recognition model is trained based on the feeding behavior recognition result.

[0084] Specifically, the above-mentioned computer-readable storage medium provided by the present invention can implement all the method steps implemented by the above-mentioned method embodiments, and can achieve the same technical effects. The parts and beneficial effects that are the same as the method embodiments in this embodiment will not be described in detail here.

[0085] It should be noted that the computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0086] It should also be noted that the terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are usually of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0087] In the present invention, "determine B based on A" means that the factor A should be considered when determining B. It is not limited to "B can be determined based on A alone", but should also include: "determine B based on A and C", "determine B based on A, C and E", "determine C based on A, and further determine B based on C", etc. In addition, it can also include taking A as a condition for determining B, for example, "when A meets the first condition, use the first method to determine B"; for another example, "when A meets the second condition, determine B", etc.; for another example, "when A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be a condition that takes A as a factor for determining B, for example, "when A meets the first condition, use the first method to determine C, and further determine B based on C", etc.

[0088] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0089] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0091] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0092] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for identifying feeding behavior of fish groups, characterized in that: include: Obtaining fish behavior data to be processed; Inputting the to-be-processed fish behavior data into a fish feeding behavior recognition model to obtain a feeding behavior recognition result of a target fish school; The fish feeding behavior recognition model is trained by the following steps: Obtaining an original video shot at a breeding site, and obtaining a fish feeding behavior dataset based on the original video; Based on the fish feeding behavior dataset, the coordinate attention mechanism and depth-wise separable convolution are used to obtain the feeding behavior recognition results; The fish feeding behavior recognition model is trained based on the feeding behavior recognition result.

2. The method for identifying the feeding behavior of fish groups according to claim 1, characterized in that: The method of obtaining the feeding behavior recognition result based on the fish feeding behavior dataset by using the coordinate attention mechanism and the depth-separable convolution includes: Based on the fish feeding behavior dataset, the target feature information is extracted using a coordinate attention mechanism to obtain a first feature map; Inputting the first feature map into a depth-separable gated DSGsted convolution module in the model to obtain a second feature map; the DSGsted convolution module includes a depth-separable convolution network; Inputting the second feature map into a multi-scale module after patch embedding to obtain a multi-scale feature map; Based on the multi-scale feature map, a feeding behavior recognition result corresponding to the fish feeding behavior data set is obtained.

3. The method for identifying the feeding behavior of fish groups according to claim 2, characterized in that: The step of inputting the first feature map into a depth-separable gated DSGsted convolution module in the model to obtain a second feature map comprises: Normalizing the first feature map through a normalization layer in the DSGsted convolution module; The first branch network, the second branch network and the third branch network in the DSGsted convolution module are used to perform dimension transformation and feature extraction on the standardized first feature map respectively to obtain the features output by each branch network; the first branch network includes a linear layer and a depthwise separable convolutional network; the second branch network includes a linear layer and a convolutional network; the third branch network includes a linear layer and an activation layer; The features output by each branch network are subjected to feature fusion and linear transformation to obtain the second feature map.

4. The method for identifying the feeding behavior of fish groups according to claim 2, characterized in that: Based on the fish feeding behavior dataset, the target feature information is extracted using a coordinate attention mechanism to obtain a first feature map, including: Performing an average pooling operation in the height direction on the input image in the fish feeding behavior dataset to obtain a feature map in the height direction, and performing an average pooling operation in the width direction on the fish feeding behavior dataset to obtain a feature map in the width direction; Connecting and convolving the feature map in the height direction and the feature map in the width direction to obtain a connected feature map, and performing batch normalization and nonlinear activation operations on the connected feature map to obtain an integrated feature map; Inputting the integrated feature map into the first convolution layer and the first activation layer in sequence to obtain a first output feature map, and inputting the integrated feature map into the second convolution layer and the second activation layer in sequence to obtain a second output feature map; The input image, the first output feature map, and the second output feature map are element-wise multiplied to obtain the first feature map.

5. The method for identifying the feeding behavior of fish groups according to claim 2, characterized in that: The multi-scale module includes a self-attention module and a residual pooling connection layer; Among them, a relative position encoding layer is embedded in the self-attention module, and the relative position encoding layer is used to determine the relative position encoding corresponding to the key K tensor; the residual pooling connection layer is used to fuse the pooled query Q tensor with the output features of the self-attention module.

6. The method for identifying the feeding behavior of a fish group according to claim 1, characterized in that: The method of obtaining an original video shot at a breeding site and obtaining a fish feeding behavior dataset based on the original video includes: The breeding site is filmed with a camera to obtain an original video; the camera is placed in a transparent waterproof box, the waterproof box is arranged at the edge of the fish pond and the camera faces the main area of ​​the breeding pond; Converting the original video into pictures, and adding labels to the converted pictures; the labels are divided according to eating characteristics; The labeled images are subjected to data augmentation and resolution adjustment, and the images are divided into training set, validation set, and test set.

7. The method for identifying the feeding behavior of fish groups according to claim 1, characterized in that: The step of training the fish feeding behavior recognition model based on the feeding behavior recognition result comprises: Based on the feeding behavior recognition results, the fish feeding behavior recognition model is trained using the adaptive moment estimation AdamW optimizer.

8. A device for identifying feeding behavior of fish groups, characterized in that: include: An acquisition module, used for acquiring the fish behavior data to be processed; An identification module, used for inputting the to-be-processed fish behavior data into a fish feeding behavior identification model to obtain a feeding behavior identification result of a target fish school; The fish feeding behavior recognition model is trained by the following steps: Obtaining an original video shot at a breeding site, and obtaining a fish feeding behavior dataset based on the original video; Based on the fish feeding behavior dataset, the coordinate attention mechanism and depth-wise separable convolution are used to obtain the feeding behavior recognition results; The fish feeding behavior recognition model is trained based on the feeding behavior recognition result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for identifying the feeding behavior of a fish group as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying the feeding behavior of a fish group as described in any one of claims 1 to 7 is implemented.

Images