A method for intelligent detection and tracking of the behavior of rock snapper and a model building method

Abstract

The present invention provides a method for intelligent detection and tracking of the behavior of a spotted rockfish and a method for building a model. The main part is a model building method, which includes the following steps: obtaining a dataset of spotted rockfish images taken in a field environment and dividing it into a training set and a test set, and preprocessing the data; building a DCN-YOLOv5 model that can focus on biological posture features; training and testing the model; building a multi-category and multi-target behavior detection algorithm model MOMC-Tracking model, and connecting it in series with the trained model DCN-YOLOv5 to form a final model. The model built by the present invention can adaptively change the receptive field according to the characteristics of the fish posture, so as to adapt to the extraction of features when the detected target swims quickly and changes its posture, and can identify the key points of the detected target at the same time. The proposed MOMC algorithm can automatically track the behavior trajectory and calculate the characteristic parameters of the key points of the behavior.

Description

Technical Field

[0001] The present invention belongs to the technical field of aquaculture monitoring, and in particular relates to an intelligent detection and tracking method for the behavior of rock snapper and a model building method. Background Art

[0002] In aquaculture, water pollution and diseases can cause changes in fish behavior. If they are not discovered in time, they will lead to mass death or even complete extinction of fish, causing huge economic losses to the aquaculture industry. Therefore, it is of great practical significance to evaluate the state of fish through their behavior and reduce losses. At present, there are many fish species in aquaculture, among which Oplegnathus punctatus is one of the main species in aquaculture farms and one of the aquaculture products with the highest economic benefits. Therefore, the breeding environment of Oplegnathus punctatus is intelligently monitored, so that the suitability of the breeding environment and the disease conditions of fish can be inferred through the abnormal behavior analysis of Oplegnathus punctatus. For example, in the actual breeding process, excessive fish breeding density, excessive feeding of feed, and accumulation of excrement will produce a large amount of harmful substances such as ammonia nitrogen and nitrite, resulting in a deterioration of the breeding environment. Among them, ammonia nitrogen and nitrite are called invisible killers in aquaculture. Related studies have shown that excessive ammonia nitrogen will have acute and chronic toxic effects on fish, affecting the growth and development of fish and behavioral changes, and in severe cases, it will cause mass death or even complete extinction of fish. However, the detection of environmental parameters such as ammonia nitrogen in aquaculture environments currently requires manual sampling and then sending them to the laboratory for testing. This method is not only time-consuming and laborious, but may also lead to a reduction in fish production or even the spread of infectious diseases due to untimely testing, causing serious economic losses. Especially in offshore aquaculture, due to weather and other reasons, water quality sampling is often difficult or even impossible. It can be seen that if the behavior of fish can be intelligently detected and tracked, the suitability of the fish breeding environment and the disease situation can be inverted. In this way, not only can the abnormal behavior of farmed fish be monitored in a timely manner, but also the problem of untimely detection and difficulty in sampling using artificial chemical methods can be solved.

[0003] As one of the effective ways to achieve intelligent monitoring, real-time tracking of underwater targets is a prerequisite for timely detection of their abnormal behaviors. The premise for achieving real-time tracking is the detection of underwater targets. In the existing research on underwater biological target detection, scholars mainly rely on underwater sonar technology, Brillouin scattering method and gravity gradient method for target detection. However, the above methods can only detect the location of the target, but cannot detect the target category, and the detection efficiency or accuracy is relatively low. With the application of artificial intelligence, especially deep learning in agriculture, ocean and other fields, deep learning technology has also been applied to underwater target recognition research. For example, Zhang et al. proposed a new deep adversarial learning framework AdvFish to address the interference of underwater noise in the fish recognition process. The recognition accuracy of this framework is 81.12%. In order to improve the fish recognition effect in complex water environments, Jiang et al. proposed a transfer learning method with a recognition accuracy of 91.33%. Salman et al. used the motion information of fish in the video to generate fish-related candidate areas through background subtraction and optical flow method, combined the results with the original image, and detected the fish, with a recognition accuracy of 87.44%. From the above research and related studies, it can be seen that the use of deep learning technology for underwater fish recognition has achieved some good results. However, due to the complex underwater environment and the fast and diverse swimming postures of fish, the accuracy of existing target detection models will still be significantly reduced when detecting fish targets. Unfortunately, there is no effective solution in the existing technology.

[0004] On the other hand, the target tracking and behavior analysis of underwater animals is also a hot research field at present, and some results have been achieved. For example, Xu et al. used the dataset established by the research group themselves and proposed a method based on the Faster R-CNN target detection model to study the behavior of ornamental fish under ammonia nitrogen stress. Marcano et al. used the dataset of the Tweed Estuary in Australia to design a target detection and tracking pipeline, which can accurately identify and track 120 of the 169 fish appearing in the video. Li et al. proposed a method for sea cucumber target detection, behavior trajectory detection and analysis based on the Faster R-CNN model. Sadoul et al. used a dataset shot in an aquarium and proposed an algorithm to extract the fish school dispersion index and swimming activity index. This method can detect the shoal and swimming behavior of underwater fish schools. It can be seen from the above research and related literature that underwater animal tracking and behavior analysis can be achieved through target tracking methods. In addition, drawing its motion trajectory based on target tracking is conducive to reflecting the survival status of underwater animals. In fact, compared with the motion trajectory, some behavioral characteristic parameters of underwater animals can reflect the status of fish. For example, the frequency of the fin swing can be used to determine whether the fish is swimming and active in the water, which can be used as an indicator to detect whether the fish behavior is abnormal. However, there is no solution in the prior art that uses key behavioral characteristic parameters for marine animals to reflect the animal's living status, and compared with terrestrial animals, marine animals have fewer key characteristics and are not as obvious as terrestrial animals. Summary of the invention

[0005] In view of the above problems, the present invention proposes a DCN-YOLOv5 model that focuses on biological posture features. The model can change the target receptive field according to the target's posture change, solving the problem of false detection caused by rapid posture changes of fish and missed detection caused by partial occlusion; at the same time, a multi-target multi-category behavior detection and tracking algorithm MOMC-Tracking model is proposed. The algorithm model can perform real-time tracking, automatically draw motion trajectories and calculate the parameter values ​​of key behaviors, solving the problem that existing algorithms cannot simultaneously detect targets and characteristic behavior parameters of targets.

[0006] The first aspect of the present invention provides a method for building an intelligent detection and tracking model for the behavior of a grouper sea bream, comprising the following steps:

[0007] Step 1, obtaining a dataset of images of spotted sea bream taken in a field environment and dividing it into a training set and a test set, preprocessing the image data in the training set and annotating key points of a portion of the data;

[0008] Step 2, building a DCN-YOLOv5 model that can focus on biological posture features based on the YOLOv5 model; the DCN-YOLOv5 model adds a deformable convolutional network structure to the backbone network, so that the receptive field can adaptively adjust the field of view according to the contour and size of the target; at the same time, a number of filters for key point detection are added to the Head layer of the model, and the number of output filters of the convolutional layer is increased, so that the model can complete the target detection and key point detection tasks at the same time. In the loss function part, the loss function calculation of the key point regression is added, and the model is constrained together with the bounding box regression loss, the category loss function and the confidence loss function;

[0009] Step 3: Use the training set and test set to train and test the constructed DCN-YOLOv5 model respectively;

[0010] Step 4, build a multi-category and multi-target behavior detection algorithm model MOMC-Tracking, the MOMC-Tracking model is used to track the target in real time and draw its behavior trajectory, and at the same time, perform statistics and visualization on key behavioral characteristic parameters such as the swing angle of the tail fin and pectoral fin, the tail swinging and pectoral fin swinging frequency, the MOMC-Tracking model includes a SORT module, a trajectory tracking and visualization module, and a key point tracking and visualization module; the MOMC-Tracking model is connected in series with the trained DCN-YOLOv5 model to form the final intelligent detection and tracking model of the grouper sea bream behavior.

[0011] Preferably, in step 1, the image data in the training set are preprocessed by using Labelimg software to annotate the target frame, and the label is named "fish" for training the target detection model; the key points of a portion of the data are annotated by using Labelme software while annotating the target frame, and the key points include the head, left fin, right fin, dorsal fin and caudal fin.

[0012] Preferably, the DCN-YOLOv5 model constructed in step 2 includes a Backbone network layer, a Neck network layer and a Head network layer; when in use, first, the image data set of the spotted sea bream that has been subjected to data enhancement and normalization processing is input into the Backbone network layer of DCN-YOLOv5, the function of the Backbone network layer is to extract the feature map to complete the feature encoding, then, the encoded feature map is input into the Neck network layer, the feature map is decoded, and finally, the extracted features are passed to the Head layer to complete the processing of the target and key point features, and the prediction results are output.

[0013] Preferably, in the step 2, the DCN-YOLOv5 model adds a deformable convolutional network structure to the backbone network, and the deformable convolutional network structure is specifically:

[0014] Deformable convolution mainly adds a 2D offset to each grid sample on the basis of ordinary standard convolution, and adds an additional parameter direction to each element, so as to allow the sampling grid to deform freely and expand the receptive field;

[0015] use Represents the size of the receptive field and the hole, and its definition is as follows:

[0016] The definition of ordinary standard convolution is as follows:

[0017]

[0018] in, is the output feature map, is the weight function of the sampled value, represents the input feature map, Z 0 means output feature map For each position in express Position in

[0019] In a deformable convolution, the offset Used to expand the regular grid ,in , therefore, sampling the offset points and irregular points , the definition of deformable convolution is as follows:

[0020]

[0021] in, Indicates location The offset of; offset in deformable convolution Additional convolutional layers are used to learn from previous feature maps, thereby increasing the adaptability of convolutional neural networks.

[0022] Preferably, the loss function part of the DCN-YOLOv5 model is as follows:

[0023] Including the loss function of bounding box regression, let represents the prediction box area, represents the area of ​​the real box, express and The minimum convex closed box of , the loss function of bounding box regression GIoU Loss is defined as follows:

[0024]

[0025] in, It represents GIoU Loss for bounding box regression, and IoU represents intersection over union;

[0026] Including the loss function of bounding box confidence regression, the loss function BCE Loss of bounding box confidence regression is defined as follows:

[0027]

[0028] in, represents the confidence score in the prediction box, Represents the difference between the predicted box and the corresponding target box value, Represents the BCE Loss on the target confidence;

[0029] Including the loss function of target category regression, the loss function of target category regression and Similar, the difference between the two is that in the calculation Before that, the category confidence score of the prediction box and the category of the target box need to be converted into one-hot form. The loss function BCE Loss of the target category regression is defined as follows:

[0030]

[0031]

[0032] in, Represents the category score of the prediction box, represents the indicator function on the target category, Indicates the number of target categories, represents a category in the target, Indicates the category of the target box, Represents the BCE Loss on target category regression;

[0033] Including the loss function of key point regression, the loss function Smooth L1 Loss of key point regression is defined as follows:

[0034]

[0035] in , represents the true value of the key point, represents the predicted value of the key point, Represents Smooth L1 Loss on key point regression;

[0036] Therefore, the total loss of the DCN-YOLOv5 model is as follows:

[0037]

[0038] in, , and They are the output feature map, cell, and the number of anchors on each cell respectively; is the weight of the corresponding item; Indicated in The output feature map, cell, Whether the sample on the anchor box is a positive sample, if it is a positive sample is 1, otherwise is 0; Weights used to balance the output feature maps at each scale.

[0039] Preferably, the MOMC-Tracking model in step 4 includes a SORT module, a trajectory tracking and visualization module, and a key point tracking and visualization module; the trajectory tracking and visualization module includes a synthetic trajectory unit, a trajectory unit that deletes trajectory units outside the time range, a trajectory unit that deletes trajectory units from the detector view, and a trajectory drawing unit; the key point tracking and visualization module includes a key point and trajectory tracking unit, a swing amplitude and frequency updating unit, and a visualization feature parameter unit; wherein the SORT module and the key point tracking and visualization module receive target detection results and key point detection results respectively.

[0040] Preferably, the working process of the MOMC-Tracking model is as follows: first, the target detection detection frame and key points are obtained by inferring the image through the DCN-YOLOv5 model, then the SORT module and the key point tracking and visualization module receive the target detection results and the key point detection results respectively, and track the target detection results through Kalman filtering and Hungarian algorithm, and according to the results of SORT tracking, the real-time drawing of the fish trajectory is realized in the trajectory tracking and visualization module, wherein the module can delete the trajectory points exceeding the time threshold and the target trajectory leaving the monitoring field of view and visualize the motion trajectory; the key point tracking and visualization module first receives the motion trajectory and the key point detection results, and matches the two, and the module updates the database storing the swing amplitude of the tail fin and the pectoral fin according to these results, and calculates the swing frequency of the tail fin and the pectoral fin and visualizes it; finally, the trajectory, swing amplitude and frequency visualization image are output.

[0041] The second aspect of the present invention provides a method for intelligent detection and tracking of the behavior of a grouper sea bream, comprising the following process:

[0042] S1, obtaining video image data of grouper seabream through a camera;

[0043] S2, using the intelligent detection and tracking model for the behavior of the spotted sea bream as described in the first aspect to process the video image data, first using the DCN-YOLOv5 model to process the spotted sea bream image acquired in real time, obtain the target detection frame and key points, and send them to the MOMC-Tracking model;

[0044] S3, the MOMC-Tracking model performs data processing and outputs trajectory, swing amplitude and frequency visualization images;

[0045] S4, the user analyzes the behavior of the spotted sea bream based on the output visualization images.

[0046] The third aspect of the present invention provides a device for intelligent detection and tracking of the behavior of spotted rockfish, the device comprising at least one processor and at least one memory, the processor and the memory being coupled; the memory stores a computer execution program of the intelligent detection and tracking model of the behavior of spotted rockfish constructed as described in the first aspect; when the processor executes the computer execution program stored in the memory, the processor can perform intelligent detection and tracking of the behavior of spotted rockfish.

[0047] The fourth aspect of the present invention provides a computer-readable storage medium, which stores a computer program or instruction of the intelligent detection and tracking model of the behavior of spotted sea bream constructed as described in the first aspect, and when the program or instruction is executed by a processor, the processor can perform intelligent detection and tracking of the behavior of spotted sea bream.

[0048] Compared with the prior art, the present invention mainly provides a method for intelligent detection and tracking of the behavior of spotted rockfish and a model building method. The present invention uses the YOLOv5 model for the intelligent detection and tracking of the behavior of spotted rockfish for the first time. Under the framework of YOLOv5, the present invention draws on the idea of ​​deformable convolution and adds a deformable convolution network structure to the backbone network, so that the receptive field can adaptively adjust the field of view according to the contour and size of the target, thereby meeting the needs of feature extraction when the spotted rockfish swims, so as to improve the target detection ability of the model; at the same time, in the loss function part, the loss function calculation of key point regression is added, and the model is constrained together with the bounding box regression loss, the category loss function and the confidence loss function to balance the weight of the output feature map of each scale.

[0049] At the same time, a multi-category and multi-target behavior detection algorithm model MOMC-Tracking is proposed. The existing SORT and DeepSORT multi-target tracking algorithms can only track targets. Unlike these two tracking algorithms, the MOMC-Tracking algorithm can not only track the motion trajectory of the spotted sea bream, but also extract its key features (such as the swing frequency and amplitude of the tail fin and pectoral fin) and visualize them. These key features can more accurately reflect the motion characteristics and environmental status of the detected target than the motion trajectory itself.

[0050] The present invention connects two models in series to form an effective intelligent detection and tracking model for the behavior of grouper sea bream. The proposed method and ideas can not only be used for monitoring and behavior tracking of marine targets, but can also be extended to target recognition, behavior tracking and intelligent monitoring of other terrestrial organisms. This research provides theoretical support for the intelligentization of aquaculture and has certain practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of the method for building an intelligent detection and tracking model of the behavior of rock snapper of the present invention;

[0052] Figure 2 Label the graph for the data;

[0053] Figure 3 This is a schematic diagram of the DCN-YOLOv5 model network model structure.

[0054] Figure 4 This is a comparison chart of ordinary convolution sampling points and deformed convolution sampling points.

[0055] Figure 5 This is the structure diagram of the MOMC-Tracking algorithm model.

[0056] Figure 6 This is the model ablation experiment result diagram.

[0057] Figure 7 This is a comparison chart of the model loss curve.

[0058] Figure 8 This is a comparison chart of the test results.

[0059] Fig. 9 This is a visualization of the DCN-YOLOv5 model.

[0060] Fig.10 This is the key point detection result diagram of the DCN-YOLOV5 model.

[0061] Fig.11 These are pictures of spotted sea bream in different water quality environments.

[0062] Fig.12 This is the movement trajectory of the spotted sea bream in different water quality environments.

[0063] Fig.13 This is a diagram of the fin swing amplitude of the spotted sea bream in different water quality environments.

[0064] Fig.14 This is a diagram of the tail swing amplitude of the spotted sea bream in different water quality environments.

[0065] Fig.15 This is a simplified block diagram of the structure of the intelligent detection and tracking device for the behavior of spotted sea bream in Example 2. DETAILED DESCRIPTION

[0066] Embodiment 1:

[0067] This embodiment takes the target detection, tracking and behavior analysis of grouper sea bream in an ammonia nitrogen environment as an example, and further illustrates the method proposed in the present invention through a specific experimental process.

[0068] 1. Sample collection:

[0069] This experiment was conducted at a breeding base of a company in Laizhou, Yantai, and the grouper was provided by the company. The grouper used in the experiment was in good health, with an average weight of about 80+10g. Because the grouper has high requirements for the breeding environment and is sensitive to changes in water quality, the grouper was placed in a cylindrical experimental container with a volume of 150L for 4 days before the experiment began. During the temporary breeding period, the same specification of feed was fed at 08:00 and 17:00 every day.

[0070] In order to simulate the ammonia nitrogen stress experiment of spotted bream in a real breeding environment, four 60×75×45cm fish tanks were used for the experiment. In order to avoid the impact of the surrounding environment on the fish, white wallpaper was pasted around each fish tank. Each fish tank contained 135L of water, and four spotted bream were placed in each fish tank for the ammonia nitrogen stress experiment. A camera was used to record the behavioral changes of spotted bream in different fish tanks. The camera model was a Hikvision high-definition camera (DS-IPC-T12HV3-IA). In order to avoid the influence of the oxygen supply system on the shooting, the fish and the oxygen supply system were separated by partitions and filter cotton in the fish tank to ensure sufficient oxygen in the water.

[0071] A total of three experiments were conducted, and each experiment filmed the spotted rock bream in a normal environment and under three different ammonia nitrogen concentrations for 24 hours. In order to ensure the accuracy and rigor of the experiment, the spotted rock bream stopped feeding 24 hours before the experiment, and the light was on from 08:00 in the day to 20:00 in the evening. In order to study the effects of different ammonia nitrogen concentrations on spotted rock bream, we conducted a preliminary experiment. Through the results of the preliminary experiment, we found that when the ammonia nitrogen concentration was 25 mg / L, all spotted rock bream died within 3 hours, and the tolerable ammonia nitrogen concentration range for spotted rock bream to have abnormal reactions but not die quickly was 5 mg / L~15 mg / L.

[0072] Within the range of ammonia nitrogen concentrations tolerated by the grouper bream, the experiment set up four ammonia nitrogen concentration gradient groups (0mg / L, 5mg / L, 15mg / L and 20mg / L), where 0mg / L represents a normal environment that meets the water quality standards. The ammonia nitrogen standard solution was prepared on-site using ammonium chloride. In order to avoid the large error between the prepared ammonia nitrogen concentration and the actual ammonia nitrogen concentration, which would affect the accuracy of the experimental data. After the experiment began, the water quality of all fish tanks was sampled every 6 hours, and then the ammonia nitrogen content was tested using a spectrophotometer and colorimetry. The test results showed that the error between the actual ammonia nitrogen concentration and the prepared concentration in the water body was ±0.3mg / L. The non-ionic ammonia concentration was converted from the ammonia nitrogen concentration using the conversion formula specified in the "Seawater Quality Standard" (GB3097-1997).

[0073] During the experiment, the dissolved oxygen content, temperature, and pH value in the seawater were tested every 4 hours using a detector (YSI550A). Some of the experimental environmental data obtained are shown in Table 1.

[0074] Table 1 Partial sample data of the experiment

[0075]

[0076] 2. Dataset creation:

[0077] For the four water quality environments in each experiment, videos from the same time period (13:00-14:00) were randomly selected, and the videos were divided into frames into images. 1000 images were taken in different water quality environments, and a total of 4000 images were collected. In order to make the data used more general, 1724 images of spotted sea bream were randomly selected in different water quality environments. 1000 images were randomly selected from the 1724 images, and the target boxes were annotated using Labelimg software, and the labels were named "fish" for training the target detection model. The remaining 724 images were annotated using Labelme software, and key points were annotated while annotating the target boxes to train the key point detection model. Among them, the division ratio of the training set and the test set is 8:2. The schematic diagram of target box annotation and key point annotation is shown in the figure. Figure 2 Key points include the head, left fin, right fin, dorsal fin, and caudal fin.

[0078] 3. DCN-YOLOv5 model construction:

[0079] At present, target detection algorithms can be divided into one-stage detectors and two-stage detectors. One-stage detectors directly extract features in the network to predict the probability and location coordinates of target categories, and the speed is relatively fast. Compared with one-stage detectors, two-stage detectors need to generate regions first, and then use convolutional neural networks to perform target recognition and classification. Generally speaking, the detection speed is much slower.

[0080] Since real-time detection and tracking of targets require high speed, the present invention uses the one-stage detector YOLOv5 model as the basic framework. Based on YOLOv5, a DCN-YOLOv5 model for fish detection is proposed. The DCN-YOLOv5 structural framework is shown in the figure. Figure 3 shown.

[0081] The DCN-YOLOv5 model is mainly composed of the Backbone network layer, the Neck network layer, and the Head network layer. First, the image dataset of the rockfish that has been data enhanced and normalized is input into the Backbone network layer of DCN-YOLOv5. The function of the Backbone network layer is to extract feature maps to complete feature encoding. Then, the encoded feature maps are input into the Neck network layer to decode the feature maps. Finally, the extracted features are passed to the Head layer to complete the processing of the target and key point features, and the prediction results are output. At the same time, the DCN-YOLOv5 model has been improved as follows:

[0082] (1) The classic YOLOv5 model uses an ordinary standard convolution module, which adopts a fixed convolution kernel, and its receptive field and sampling position cannot be changed at will. The object of this invention is the spotted sea bream, which has various postures and perspectives when swimming, resulting in low detection accuracy of YOLOv5. More importantly, YOLOv5 will lose the target when two or more fish overlap. In order to solve this problem, the idea of ​​deformable convolution is borrowed, and a deformable convolution network structure module is added to the backbone network, so that the receptive field can adaptively adjust the field of view according to the contour and deformation of the target, thereby better adapting to the needs of target feature extraction when the posture changes rapidly and improving the target detection ability of the model. The network structure diagram of this part is shown in the figure. Figure 3 (b) as shown.

[0083] In fact, deformable convolution mainly adds a 2D offset to each square sample on the basis of ordinary standard convolution, and adds an additional parameter direction to each element, so as to allow the sampling grid to deform freely and expand the receptive field; the comparison diagram of standard convolution sampling and deformable convolution sampling is shown in the figure Figure 4 shown.

[0084] use Represents the size of the receptive field and the hole, and its definition is as follows:

[0085] The definition of ordinary standard convolution is as follows:

[0086]

[0087] in, is the output feature map, is the weight function of the sampled value, represents the input feature map, Z 0 means output feature map For each position in express Position in

[0088] In a deformable convolution, the offset Used to expand the regular grid ,in , therefore, sampling the offset points and irregular points , the definition of deformable convolution is as follows:

[0089]

[0090] in, Indicates location The offset of; offset in deformable convolution Additional convolutional layers are used to learn from previous feature maps, thereby increasing the adaptability of convolutional neural networks.

[0091] Figure 3 Module (b) is a more focused adaptive network module that can adapt to the receptive field of the deformation model, enhance the feature extraction capability of the model, and improve the target detection accuracy.

[0092] (2) The fins are the main source of power for fish to swim in water. Therefore, the swing frequency of the tail fin and pectoral fin is an important indicator of fish activity. Therefore, in order to obtain the swing frequency of the fins of the spotted sea bream, it is necessary to first detect the key points of the spotted sea bream. Since YOLOv5 only has the target detection function, it cannot complete the task of key point detection. In order to realize the detection of the key points of the spotted sea bream, the present invention proposes the following design ideas:

[0093] During data preprocessing, a key point data enhancement algorithm similar to the Mosaic algorithm was added to enhance data diversity.

[0094] 24 filters for key point detection are added to the Head layer of the model, increasing the number of output filters of the convolutional layer from 18 to 42, so that the model can complete the target detection and key point detection tasks at the same time. Figure 3 (c) as shown.

[0095] In the loss function part, the loss function calculation of key point regression is added, and the model is constrained together with the bounding box regression loss, category loss function and confidence loss function.

[0096] Different from the YOLOv5 model, the loss function of the present invention includes GIoU Loss, BCE Loss and Smooth L1 Loss. Specifically as follows:

[0097] Including the loss function of bounding box regression, let represents the prediction box area, represents the area of ​​the real box, express and The minimum convex closed box of , the loss function of bounding box regression GIoU Loss is defined as follows:

[0098]

[0099] in, It represents GIoU Loss for bounding box regression, and IoU represents intersection over union;

[0100] Including the loss function of bounding box confidence regression, the loss function BCE Loss of bounding box confidence regression is defined as follows:

[0101]

[0102] in, represents the confidence score in the prediction box, Represents the difference between the predicted box and the corresponding target box value, Represents the BCE Loss on the target confidence;

[0103] Including the loss function of target category regression, the loss function of target category regression and Similar, the difference between the two is that in the calculation Before that, the category confidence score of the prediction box and the category of the target box need to be converted into one-hot form. The loss function BCE Loss of the target category regression is defined as follows:

[0104]

[0105]

[0106] in, Represents the category score of the prediction box, represents the indicator function on the target category, Indicates the number of target categories, represents a category in the target, Indicates the category of the target box, Represents the BCE Loss on target category regression;

[0107] Including the loss function of key point regression, the loss function Smooth L1 Loss of key point regression is defined as follows:

[0108]

[0109] in , represents the true value of the key point, represents the predicted value of the key point, Represents Smooth L1 Loss on key point regression;

[0110] Therefore, the total loss of the DCN-YOLOv5 model is as follows:

[0111]

[0112] in, , and They are the output feature map, cell, and the number of anchors on each cell respectively; is the weight of the corresponding item; Indicated in The output feature map, cell, Whether the sample on the anchor box is a positive sample, if it is a positive sample is 1, otherwise is 0; Weights used to balance the output feature maps at each scale.

[0113] 4. Build the algorithm model MOMC-Tracking

[0114] In order to analyze the relationship between the fish's movement trajectory, pectoral fin swing frequency, caudal fin swing frequency and other behavioral characteristic parameters and the ammonia nitrogen content in the water, this paper proposes a multi-category and multi-target behavior detection algorithm MOMC-Tracking. The framework structure of the MOMC-Tracking algorithm is as follows: Figure 5 The MOMC-Tracking algorithm can track the target in real time and draw its behavior trajectory. At the same time, it can count and visualize the key behavioral characteristic parameters such as the swing angle of the tail fin and pectoral fin, the tail swing and the pectoral fin swing frequency, so as to assist in judging the health status of the spotted sea bream.

[0115] The structure of MOMC-Tracking mainly consists of the SORT module, the trajectory tracking and visualization module, and the key point tracking and visualization module. The trajectory tracking and visualization module mainly consists of the modules of synthesizing trajectories, deleting trajectories outside the time range, deleting trajectories from the detector view, and drawing trajectories. The key point tracking and visualization module consists of the modules of tracking key points and trajectories, updating swing amplitude and frequency, and visualizing feature parameters. Among them, the SORT module and the key point tracking and visualization module receive the target detection results and key point detection results respectively.

[0116] First, the video to be detected is input into the DCN-YOLOv5 model, and the target detection frame and key points are obtained by inferring the image through the DCN-YOLOv5 model. Then, the SORT module and the key point tracking and visualization module receive the target detection results and key point detection results respectively, and track the target detection results through Kalman filter and Hungarian algorithm. According to the results of SORT tracking, the real-time drawing of fish trajectories is realized in the trajectory tracking and visualization module, where the module can delete the trajectory points that exceed the time threshold and the target trajectory that leaves the monitoring field of view and visualize the motion trajectory; the key point tracking and visualization module first receives the motion trajectory and key point detection results, and matches the two. According to these results, the module updates the database storing the swing amplitude of the tail fin and pectoral fin, and calculates the swing frequency of the tail fin and pectoral fin and visualizes it. Finally, the trajectory, swing amplitude and frequency visualization images are output.

[0117] 5. Train the DCN-YOLOv5 model

[0118] The hardware and software configuration of the experimental platform used in this experiment is shown in Table 2. The parameter settings of all target detection models in this embodiment are consistent: the number of iterations is set to 400 times, the initial learning rate is set to 0.01, and the momentum and decay coefficient of the learning rate are set to 0.937 and 0.2 respectively. The number of iterations for training the key point detection model is set to 200 times, and the remaining parameters are consistent with the target detection model.

[0119] Table 2 Software and hardware configuration

[0120]

[0121] 6. Model performance index evaluation

[0122] In order to evaluate the recognition effect of the target detection model, this paper uses the accuracy , recall rate , Average Precision and The value is used as the performance indicator of the model, and its definition is as follows:

[0123]

[0124]

[0125]

[0126]

[0127] in, (true positive) indicates the number of positive samples predicted by positive samples. (falsepositive) indicates the number of positive samples predicted by negative samples. (false negative) indicates the number of negative samples predicted by positive samples. Indicates the number of categories.

[0128] In order to verify the effectiveness of the proposed structure, an ablation experiment was conducted. The structure without the deformable convolution module is denoted as DCN-YOLOv5-no, the structure with the deformable convolution module added before the C3 module is denoted as DCN-YOLOv5-up, and the structure with the deformable convolution module added after the C3 module is denoted as DCN-YOLOv5-down. , , and The performance of the model is evaluated by the indicators. The experimental results are as follows Figure 6 shown.

[0129] Depend on Figure 6 It can be seen that compared with the recognition accuracy under the DCN-YOLOv5-up and DCN-YOLOv5-no structures, the DCN-YOLOv5-down structure The results are improved by 2.69% and 24.77% respectively. Therefore, this paper adopts the DCN-YOLOv5-down structure.

[0130] In model training, the loss function is used to measure the degree of inconsistency between the model's predicted value and the true value. This article uses the loss function value to evaluate the effect and stability of model training. Figure 7 The loss function comparison curve of the classic YOLO series algorithm and the DCN-YOLOv5 proposed in this paper is given. As can be seen from the figure, the DCN-YOLOv5 model has the best effect.

[0131] In order to verify the target detection performance of the proposed model, DCN-YOLOv5 is compared with the classic YOLOv3 (Cai et al. 2020b), YOLOv4 (Fu et al. 2022) and YOLOv5 (Teng et al. 2022) models, and the experimental results are shown in Table 3. As can be seen from Table 3, DCN-YOLOv5 has the highest recognition accuracy, recall rate, average precision and F1 index.

[0132] Table 3 Performance comparison of different detection models

[0133]

[0134] Examples of detection results of DCN-YOLOv5 and classic YOLOv5 models are as follows Figure 8 As shown. Figure 8 It can be seen from the figure that when the spotted rock bream is blocked, the DCN-YOLOv5 model can still detect the spotted rock bream, while the YOLOv5 model has a missed detection problem. Figure 8 (a) and (b) are shown in dashed circles. Figure 8 It can also be clearly seen that the DCN-YOLOv5 model has stronger target detection capabilities than the original YOLOv5 model.

[0135] During the model training process, image visualization can help us understand the process of convolutional neural network detection of targets and further understand and analyze data. Due to the different concentrations of ammonia nitrogen stress, the body color and body shape characteristics of the spotted bream are different. In order to facilitate the observation of the changes in the characteristics of the spotted bream under different concentrations of ammonia nitrogen stress. The present invention uses the same convolutional layer to visualize the spotted bream image. The visualization results are shown in Figure 2. Fig. 9 shown.

[0136] from Fig. 9 It can be seen that the visualization images of the spotted rock bream in different water quality environments are indeed different. Compared with the high concentration of ammonia nitrogen environment, the model may pay more attention to texture features in the low concentration of ammonia nitrogen environment. This may be because the body color and body shape characteristics of the spotted rock bream change significantly with the increase of ammonia nitrogen concentration, resulting in different focus points when the model extracts the features of the spotted rock bream in different environments.

[0137] The detection results of the key point detection model are shown in the figure below: Fig.10 As shown. Fig.10 It can be seen that the proposed model can detect the four key points of the head, pectoral fin, dorsal fin and caudal fin of the spotted sea bream. Since the pectoral fins of fish are symmetrical when flapping, in order to improve the detection efficiency, we only detect the key points on one side (left side) of the pectoral fin.

[0138] 7. Mortality and color changes of rockfish under ammonia nitrogen stress

[0139] Through experimental observation, it was found that the body color and behavior of the spotted bream changed significantly after ammonia nitrogen poisoning (Zhang et al. 2018). In order to observe the behavioral changes of the spotted bream under different ammonia nitrogen concentrations, we conducted experiments with different ammonia nitrogen concentrations three times and counted the mortality rate of the spotted bream in ammonia nitrogen poisoning within 24 hours. The statistical results are shown in Table 4.

[0140] Table 4 Mortality of grouper bream poisoned by different concentrations of ammonia nitrogen

[0141]

[0142] As can be seen from Table 4, in the environment of ammonia nitrogen concentration of 0mg / L, 5mg / L, and 15mg / L, there was no death of the spotted bream within 24 hours. In the environment of ammonia nitrogen concentration of 20mg / L, all the spotted bream died within 24 hours. However, although there was no death of the spotted bream under the ammonia nitrogen concentration of 5mg / L and 15mg / L, poisoning reactions also occurred. Fig.11 The images of the spotted sea bream were collected after 3 hours of experiment under four different ammonia nitrogen concentrations. Fig.11 It can be seen that with the increase of ammonia nitrogen concentration, the body color of the spotted sea bream changes significantly, the body loses balance and falls over.

[0143] 8. Motion trajectory drawing

[0144] In order to analyze the behavioral and movement characteristics of rockfish in an ammonia nitrogen environment for a long time, this paper uses the proposed MOMC-Tracking algorithm to track the behavior of rockfish under different ammonia nitrogen concentrations and draw the movement trajectory. Take the video of the 14:00-14:30 time period of the first day of the experiment as an example. Fig.12 The behavioral movement trajectory curves of rock snapper under four different ammonia nitrogen concentrations are given.

[0145] from Fig.12 (a) It can be seen that in a normal water environment, the spotted sea bream in the fish tank swims regularly, the swimming range is maintained within a certain range, and it is basically always in motion and never stands still. Fig.12 It can be clearly seen from Figures 12(b), 12(c) and 12(d) that, compared with the normal water quality environment, the spotted bream swam slowly and often became still in the environment with excessive ammonia nitrogen, and the duration of continuous swimming and the amount of exercise were significantly reduced. Moreover, as the concentration of ammonia nitrogen gradually increased, the stillness time of the spotted bream became longer; when the concentration of ammonia nitrogen increased to 20 mg / L, the spotted bream was still most of the time, although it occasionally ran around, but the duration of this phenomenon was short.

[0146] 9. Statistics and visualization of characteristic behavior parameters

[0147] In order to evaluate the state of the spotted bream through its characteristic behavioral parameters, this experiment used the MOMC-Tracking algorithm to calculate and visualize the swing angles of the caudal fin and pectoral fin of the spotted bream. Fig.12 In the same video segment, the swing angles of the caudal fin and pectoral fin of the spotted snapper are visualized as follows: Fig.13 and Fig.14 As shown. Fig.13 and Fig.14It can be seen that with the increase of ammonia nitrogen concentration, the swinging angle and number of times of the tail fin and pectoral fin of the spotted sea bream decreased significantly.

[0148] Corresponds to Fig.14 The frequency statistics of the caudal fin and pectoral fin of the spotted rock bream are shown in Table 5. The number and frequency of the swings of the caudal fin and pectoral fin in Table 5 reflect the activity level of the spotted rock bream over a period of time and reflect the health status or environmental conditions of the spotted rock bream.

[0149] Table 5 The frequency of caudal and pectoral fin swings

[0150]

[0151] 10. Summary and Analysis

[0152] Through the experiments in this embodiment, it can be known that the method for building a behavior intelligent detection and tracking model of a spotted sea bream proposed in the present invention has the following general process: Figure 1 as shown, and includes the following steps:

[0153] Step 1, obtaining a dataset of images of spotted sea bream taken in a field environment and dividing it into a training set and a test set, preprocessing the image data in the training set and annotating key points of a portion of the data;

[0154] Step 2, building a DCN-YOLOv5 model that can focus on biological posture features based on the YOLOv5 model; the DCN-YOLOv5 model adds a deformable convolutional network structure to the backbone network, so that the receptive field can adaptively adjust the field of view according to the contour and size of the target; at the same time, a number of filters for key point detection are added to the Head layer of the model, and the number of output filters of the convolutional layer is increased, so that the model can complete the target detection and key point detection tasks at the same time. In the loss function part, the loss function calculation of the key point regression is added, and the model is constrained together with the bounding box regression loss, the category loss function and the confidence loss function;

[0155] Step 3: Use the training set and test set to train and test the constructed DCN-YOLOv5 model respectively;

[0156] Step 4, build a multi-category and multi-target behavior detection algorithm model MOMC-Tracking, the MOMC-Tracking model is used to track the target in real time and draw its behavior trajectory, and at the same time, perform statistics and visualization on key behavioral characteristic parameters such as the swing angle of the tail fin and pectoral fin, the tail swinging and pectoral fin swinging frequency, the MOMC-Tracking model includes a SORT module, a trajectory tracking and visualization module, and a key point tracking and visualization module; the MOMC-Tracking model is connected in series with the trained DCN-YOLOv5 model to form the final intelligent detection and tracking model of the grouper sea bream behavior.

[0157] There is no necessary order between the steps. Combined with the constructed model, a method for intelligent detection and tracking of the behavior of spotted sea bream is given:

[0158] Firstly, the video image data of the spotted sea bream is obtained through a camera;

[0159] Then, the video image data is processed using the above-mentioned intelligent detection and tracking model of the behavior of the spotted sea bream, and the real-time acquired spotted sea bream image is processed using the DCN-YOLOv5 model to obtain the target detection detection box and key points, and send them to the MOMC-Tracking model;

[0160] Thirdly, the MOMC-Tracking model performs data processing and outputs trajectory, swing amplitude, and frequency visualization images;

[0161] Finally, the user analyzes the behavior of the spotted sea bream based on the output visualization images.

[0162] Embodiment 2:

[0163] like Fig.15As shown, the present invention also provides a device for intelligent detection and tracking of the behavior of spotted rockfish, the device includes at least one processor and at least one memory, and also includes a communication interface and an internal bus; a computer execution program is stored in the memory; when the processor executes the computer execution program stored in the memory, the processor can execute intelligent detection and tracking of the behavior of spotted rockfish. The internal bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus. The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a U disk, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.

[0164] The device may be provided as a terminal, a server or other forms of devices.

[0165] Fig.15 is a block diagram of a device shown as an example. The device may include one or more of the following components: a processing component, a memory, a power component, a multimedia component, an audio component, an input / output (I / O) interface, a sensor component, and a communication component. The processing component generally controls the overall operation of the electronic device, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component may include one or more processors to execute instructions to complete all or part of the steps of the above method. In addition, the processing component may include one or more modules to facilitate interaction between the processing component and other components. For example, the processing component may include a multimedia module to facilitate interaction between the multimedia component and the processing component.

[0166] The memory is configured to store various types of data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0167] The power supply assembly provides power to various components of the electronic device. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device. The multimedia assembly includes a screen that provides an output interface between the electronic device and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia assembly includes a front camera and / or a rear camera. When the electronic device is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0168] The audio component is configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the electronic device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal. The I / O interface provides an interface between the processing component and the peripheral interface module, and the above-mentioned peripheral interface module can be a keyboard, a click wheel, a button, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

[0169] The sensor assembly includes one or more sensors for providing various aspects of status assessment for the electronic device. For example, the sensor assembly can detect the on / off state of the electronic device, the relative positioning of components, such as the display and keypad of the electronic device, and the sensor assembly can also detect the position change of the electronic device or a component of the electronic device, the presence or absence of user contact with the electronic device, the orientation or acceleration / deceleration of the electronic device and the temperature change of the electronic device. The sensor assembly may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0170] The communication component is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0171] In an exemplary embodiment, the electronic device may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0172] Embodiment 3:

[0173] The present invention also provides a computer-readable storage medium, which stores a computer program or instruction of the intelligent detection and tracking model of the behavior of spotted sea bream constructed as described in Example 1. When the program or instruction is executed by a processor, the processor can perform intelligent detection and tracking of the behavior of spotted sea bream.

[0174] Specifically, a system, device or equipment equipped with a readable storage medium may be provided, on which a software program code for implementing the functions of any of the above-mentioned embodiments is stored, and a computer or processor of the system, device or equipment reads and executes the instructions stored in the readable storage medium. In this case, the program code read from the readable medium itself can implement the functions of any of the above-mentioned embodiments, so the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0175] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk (such as CD-ROM, CD-R, CD-RW, DVD-20 ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tape, etc. The storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0176] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented by a hardware processor or implemented by a combination of hardware and software modules in the processor.

[0177] It should be understood that the storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in a terminal or server.

[0178] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0179] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0180] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0181] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for building a behavior intelligent detection and tracking model for a grouper, characterized in that: The following steps are involved: Step 1, obtaining a dataset of images of spotted sea bream taken in a field environment and dividing it into a training set and a test set, preprocessing the image data in the training set and annotating key points of a portion of the data; Step 2, based on the YOLOv5 model, a DCN-YOLOv5 model capable of focusing on biological posture features is built; the DCN-YOLOv5 model adds a deformable convolutional network structure to the backbone network, which is added after the first C3 module of the backbone network, so that the receptive field can adaptively adjust the field of view according to the contour and size of the target; at the same time, a number of filters for key point detection are added to the Head layer of the model, and the number of output filters of the convolutional layer is increased, so that the model can complete the target detection and key point detection tasks at the same time. In the loss function part, the loss function calculation of the key point regression is added, and the model is constrained together with the bounding box regression loss, the category loss function and the confidence loss function; The deformable convolutional network structure is specifically as follows: Deformable convolution mainly adds a 2D offset to each square sample on the basis of ordinary standard convolution, and adds an additional parameter direction to each element, so as to allow the sampling grid to deform freely and expand the receptive field; use Represents the size of the receptive field and the hole, and its definition is as follows: The definition of ordinary standard convolution is as follows: in, is the output feature map, is the weight function of the sampled value, represents the input feature map, Z 0 means output feature map For each position in , express Position in In a deformable convolution, the offset Used to expand the regular grid ,in , therefore, sampling the offset points and irregular points , the definition of deformable convolution is as follows: in, Indicates location The offset of; offset in deformable convolution Increase the adaptability of convolutional neural networks by learning from previous feature maps through additional convolutional layers; Step 3: Use the training set and test set to train and test the constructed DCN-YOLOv5 model respectively; Step 4, building a multi-category and multi-target behavior detection algorithm model MOMC-Tracking model, the MOMC-Tracking model is used to track the target in real time and draw its behavior trajectory, and at the same time, the key behavior characteristic parameters such as the swing angle of the tail fin and pectoral fin, the tail swing and the pectoral fin swing frequency are counted and visualized, the MOMC-Tracking model includes a SORT module, a trajectory tracking and visualization module, and a key point tracking and visualization module; the MOMC-Tracking model is connected in series with the trained DCN-YOLOv5 model to form the final intelligent detection and tracking model of the behavior of the spotted sea bream; The MOMC-Tracking model includes a SORT module, a trajectory tracking and visualization module, and a key point tracking and visualization module; the trajectory tracking and visualization module includes a synthetic trajectory unit, a trajectory unit that deletes trajectories outside the time range, a trajectory unit that deletes trajectories from the detector view, and a trajectory drawing unit; the key point tracking and visualization module includes a key point and trajectory tracking unit, a swing amplitude and frequency updating unit, and a visualization feature parameter unit; the SORT module and the key point tracking and visualization module receive target detection results and key point detection results respectively; The working process of the MOMC-Tracking model is as follows: first, the target detection frame and key points are obtained by inferring the image through the DCN-YOLOv5 model. Then, the SORT module and the key point tracking and visualization module receive the target detection results and key point detection results respectively, and track the target detection results through Kalman filtering and Hungarian algorithm. According to the results of SORT tracking, the real-time drawing of the fish trajectory is realized in the trajectory tracking and visualization module, where the module can delete the trajectory points that exceed the time threshold and the target trajectory that leaves the monitoring field of view and visualize the motion trajectory; the key point tracking and visualization module first receives the motion trajectory and key point detection results, and matches the two. According to these results, the module updates the database storing the swing amplitude of the tail fin and pectoral fin, calculates the swing frequency of the tail fin and pectoral fin and visualizes it; finally, the trajectory, swing amplitude and frequency visualization image are output.

2. A method for building a behavior intelligent detection and tracking model for a grouper seabream as claimed in claim 1, characterized in that: In the step 1, the image data in the training set are preprocessed by using Labelimg software to mark the target frame, and the label is named fish, which is used to train the target detection model; the key point marking of a part of the data is performed while marking the target frame, and the key points include the head, left fin, right fin, dorsal fin and tail fin.

3. A method for building a behavior intelligent detection and tracking model for a grouper sea bream as claimed in claim 1, characterized in that: The DCN-YOLOv5 model constructed in the step 2 includes a Backbone network layer, a Neck network layer and a Head network layer; when in use, first, the image data set of the spotted sea bream that has been subjected to data enhancement and normalization processing is input into the Backbone network layer of the DCN-YOLOv5, the function of the Backbone network layer is to extract the feature map to complete the feature encoding, then, the encoded feature map is input into the Neck network layer, the feature map is decoded, and finally, the extracted features are passed to the Head layer to complete the processing of the target and key point features, and the prediction results are output.

4. A method for building a behavior intelligent detection and tracking model for a grouper seabream as claimed in claim 1, characterized in that: The loss function part of the DCN-YOLOv5 model is as follows: Including the loss function of bounding box regression, let represents the prediction box area, represents the area of ​​the real box, express and The minimum convex closed box of , the loss function of bounding box regression GIoU Loss is defined as follows: in, It represents GIoU Loss for bounding box regression, and IoU represents intersection over union; Including the loss function of bounding box confidence regression, the loss function BCE Loss of bounding box confidence regression is defined as follows: in, represents the confidence score in the prediction box, Represents the difference between the predicted box and the corresponding target box value, Represents the BCE Loss on the target confidence; Including the loss function of target category regression, the loss function of target category regression and Similar, the difference between the two is that in the calculation Before that, the category confidence score of the prediction box and the category of the target box need to be converted into one-hot form. The loss function BCE Loss of the target category regression is defined as follows: in, Represents the category score of the prediction box, represents the indicator function on the target category, Indicates the number of target categories, represents a category in the target, Indicates the category of the target box, represents BCELoss on target category regression; Including the loss function of key point regression, the loss function Smooth L1 Loss of key point regression is defined as follows: in , represents the true value of the key point, represents the predicted value of the key point, Represents Smooth L1 Loss on key point regression; Therefore, the total loss of the DCN-YOLOv5 model is as follows: in, , and They are the output feature map, cell, and the number of anchors on each cell respectively; is the weight of the corresponding item; Indicated in The output feature map, cell, Whether the sample on the anchor box is a positive sample, if it is a positive sample is 1, otherwise is 0; Weights used to balance the output feature maps at each scale.

5. A method for intelligent detection and tracking of the behavior of a grouper, characterized in that: The process includes: S1, obtaining video image data of grouper seabream through a camera; S2, using the intelligent detection and tracking model for the behavior of the spotted sea bream as described in any one of claims 1 to 4 to process the video image data, first using the DCN-YOLOv5 model to process the spotted sea bream image acquired in real time, obtain the target detection frame and key points, and send them to the MOMC-Tracking model; S3, the MOMC-Tracking model performs data processing and outputs trajectory, swing amplitude and frequency visualization images; S4, the user analyzes the behavior of the spotted sea bream based on the output visualization images.

6. An intelligent detection and tracking device for the behavior of grouper sea bream, characterized in that: The device includes at least one processor and at least one memory, and the processor and the memory are coupled; the memory stores a computer execution program of the intelligent detection and tracking model of the behavior of the spotted rockfish constructed as described in any one of claims 1 to 4; when the processor executes the computer execution program stored in the memory, the processor can perform intelligent detection and tracking of the behavior of the spotted rockfish.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program or instruction of the intelligent detection and tracking model of the behavior of the spotted rockfish constructed as described in any one of claims 1 to 4, and when the program or instruction is executed by the processor, the processor can execute the intelligent detection and tracking of the behavior of the spotted rockfish.

Images