Underwater lateolabrax japonicus individual identification and behavior analysis method

Abstract

The invention discloses an underwater lateolabrax japonicus individual identification and behavior analysis method which is characterized by comprising the following steps: S1, acquiring an underwater lateolabrax japonicus image data set to train a model; s2, segmenting and extracting the lateolabrax japonicus image through a YOLOv8-seg model to obtain mask data of a lateolabrax japonicus main body part; s3, key point detection is carried out through a YOLOv8-pose model, and then key point alignment is carried out through affine transformation; s4, extracting high-dimensional features by using InceptionResNetV2, so as to identify lateolabrax japonicus individuals; s5, calculating a centroid coordinate of the obtained mask data, namely a geometric equilibrium position; and S6, taking the lateolabrax japonicus in the image as a main body, calculating the speed, the acceleration and the path coincidence degree of the main body by using the center-of-mass coordinates obtained in the step S5, and analyzing the related information of the direction change of the main body in combination with the key points obtained in the step S3. The method can provide scientific behavior analysis data for aquaculture culturists, and helps to optimize the culture environment and improve the culture efficiency.

Description

Technical Field

[0001] The invention belongs to the field of computer vision and aquatic technology, and in particular to a method for underwater sea bass individual recognition and behavior analysis. Background Art

[0002] With the rapid development of aquaculture, the demand for monitoring the health status and behavioral analysis of Lateolabrax japonicus is growing. Traditional health assessment methods usually rely on manual observation and sampling, which is not only time-consuming and labor-intensive, but may also cause stress reactions to the fish, thus affecting its health. Therefore, computer vision technology, especially non-invasive detection methods based on deep learning, has gradually become a research hotspot.

[0003] In recent years, deep learning technology, especially target detection and image segmentation algorithms, has been widely used in aquaculture. For example, the YOLO (You Only Look Once) model has become an important tool for target detection in many fields due to its real-time and high-precision characteristics. However, underwater images usually have low contrast, high noise, and color distortion due to light refraction, scattering, and water quality, which poses a huge challenge to the individual identification and behavior analysis of Lateolabrax japonicus.

[0004] In order to solve these problems, scholars began to combine image enhancement technology and key point detection methods to extract the movement characteristics of fish bodies through key point detection to analyze their behavior patterns and health status. In addition, deep learning models such as InceptionResNetV2 have performed well in high-dimensional feature extraction and classification tasks, providing strong support for individual identification of spotted seabass.

[0005] Some studies have shown that the behavior of striped bass is closely related to its health status. For example, diseases such as Vibrio infection and gill rot can affect the swimming pattern of striped bass, causing abnormal behavior. These changes are usually manifested as a decrease in motor ability and a single swimming pattern. Therefore, through these behavioral characteristics, combined with deep learning models, the health status of underwater striped bass can be effectively judged. For example, from May to November 2019, a large-scale death event of Vibrio infection of striped bass (Lateolabraxmaculatus) occurred in Zhuhai, Guangdong, with a mortality rate of up to 30%. The study found that Vibrio infection can cause symptoms such as swimming disorders, loss of appetite, and changes in body color in striped bass, which in turn affect their normal behavior.

[0006] By analyzing the behavioral characteristics of the striped perch and combining it with geometric features (such as the center of mass) to analyze the motion path, the health of the fish can be evaluated more effectively. The present invention combines the YOLOv8 model for target detection and image segmentation, detects and aligns the key points of the fish, extracts the high-dimensional features of the fish, and then further analyzes the motion path. By calculating the speed, acceleration, frequency of direction changes, stagnation, and path overlap of the fish, it can

[0007] By analyzing the behavioral characteristics of fish and combining them with geometric features (such as the center of mass) for motion path analysis, the health of the fish can be evaluated more effectively. The present invention combines the YOLOv8 model for target detection and image segmentation, detects and aligns the key points of the fish, extracts the high-dimensional features of the fish, and then further performs motion path analysis. By calculating the changes in the speed, acceleration and direction of the fish, its health can be judged more accurately. Specifically, a decrease in speed and acceleration may indicate an abnormal or injured condition of the fish, while healthy fish usually have a higher activity frequency and larger speed fluctuations. Summary of the invention

[0008] The purpose of the present invention is to provide a method for underwater striped seabass individual identification and behavior analysis, which is based on a deep learning model and aims to achieve individual identification of striped seabass and detailed analysis of the behavior of individual targets.

[0009] The technical solution of the present invention is a method for underwater japonica individual identification and behavior analysis, which comprises the following steps:

[0010] S1: Obtain underwater Lateolabrax image dataset to train the model;

[0011] S2: Use the YOLOv8-seg model to segment the Lateolabrax image and extract the mask data of the main part of the Lateolabrax;

[0012] S3: Key point detection is performed through the YOLOv8-pose model, and then key point alignment is performed through affine transformation;

[0013] S4: Use InceptionResNetV2 to extract high-dimensional features to identify individual Lateolabrax;

[0014] S5: by calculating the centroid coordinates of the acquired mask data, i.e., the geometric equilibrium position;

[0015] S6: Taking the striped sea bass in the image as the subject, using the center of mass coordinates obtained in step S5, calculate the subject's velocity, acceleration and path overlap, and combine with the key points obtained in step S3 to analyze the relevant information of the subject's direction change.

[0016] A further improvement of the present invention is that in step S2, the YOLOv8-seg model includes a feature extraction module, a feature fusion module, and a detection and segmentation head module; its output includes a detection box, a category probability, and a segmentation mask, wherein the segmentation mask part generates an accurate mask for each target by using the predicted feature map through the segmentation head, and its expression is:

[0017]

[0018] Among them: F m is the output of the segmentation branch in the feature map; F d is the feature related to the target box in the detection branch; σ is the Sigmoid activation function, which is used to map the output to [0,1].

[0019] A further improvement of the present invention is that, in step S3, the YOLOv8-pose model is used to extract the multi-scale features of the input image through the backbone network, thereby mapping the high-dimensional pixel space to the low-dimensional semantic space, which is expressed as:

[0020]

[0021] Where: X is the input image; F is the extracted feature tensor; is the backbone network; θ backbone are the parameters of the backbone network;

[0022] The backbone network of the YOLOv8-pose model maps the original high-dimensional pixel space into a low-dimensional semantic feature space, thereby extracting features containing key visual information;

[0023] The YOLOv8-pose model uses a feature pyramid network to enable the model to capture the key points of small targets and integrate feature maps of different scales. Its expression is as follows:

[0024] F out =FPN(F1,F2,···,θ FPN )

[0025] Among them: F1, F2 are multi-scale feature maps extracted by the backbone network; F out It is the integrated feature map;

[0026] The YOLOv8-pose model uses a top-down and bottom-up fusion approach to enable the model to focus on targets of different scales, and then completes key point prediction through the detection head. Its expression is as follows:

[0027]

[0028] Where: K is the number of key points of each target; is the jth key point coordinate of the predicted i-th target; Make coordinates for real key points;

[0029] The detection head is responsible for predicting key points based on the fused feature map and calculating the loss function This loss function measures the error between the predicted key point coordinates and the true key point coordinates; key point detection is achieved through the above content, and then affine transformation is used to achieve key point alignment.

[0030] A further improvement of the present invention is that, in step S4, InceptionResNetV2 is used as a backbone network, which combines the Inception module and the residual structure; the high-dimensional features provided by InceptionResNetV2 will be mapped to the embedding space and the category space; the embedding vectors output by the network are normalized, and the modulus of each embedding vector is standardized to a unit length; to prevent excessive expansion of features, thereby ensuring the stability and comparability of feature vectors; its expression is as follows:

[0031]

[0032] In order to achieve individual recognition, InceptionResNetV2 uses Triplet Loss and SoftmaxLoss at the same time; by combining the distinguishing ability of feature embedding and the classification ability of clear categories, the overall performance of the model in the task of individual recognition of sea bass is improved;

[0033] The optimization goal of the Triplet Loss function is to make the distance between similar samples closer and the distance between dissimilar samples farther; its expression is as follows:

[0034] L t =max(d(f(x a ),f(x p ))-d(f(x a ),f(x n ))+α,0)

[0035] Where: f(x) is the embedding vector extracted by InceptionResNetV2; d is the Euclidean distance between features; x a ,x p ,x n They represent anchor samples, positive samples, and negative samples respectively; α is the set interval threshold, which is used to ensure that the distance between classes is large enough;

[0036] The optimization objectives are divided into:

[0037] Minimize: Similar samples (x a and x p) between the characteristic distance d(f(x a ),f(x p ));

[0038] Maximize: Different class samples (x a and x n ) between the characteristic distance d(f(x a ),f(x n ));

[0039] The semi-hard strategy is used to select triplet; the distance between the anchor point and the positive sample is smaller than the distance between the anchor point and the negative sample; the distance between the anchor point and the negative sample is not less than the distance between the anchor point and the positive sample plus a set interval threshold; its mathematical description is:

[0040] d(f(x a ),f(x p )) <d(f(x a ),f(x n )) <d(f(x a ),f(x p ))+α

[0041] Softmax converts the original score of the model into a probability distribution, while the cross entropy loss is used to measure the difference between the category probability predicted by the model and the true category, thereby guiding the model to improve the probability of the correct category;

[0042] The expression of Softmax is as follows:

[0043]

[0044] Where: z k is the raw score output (logit) of the model for category k; C is the total number of categories; It applies an exponential function to the score of category k, producing positive values ​​and amplifying higher scores;

[0045] The expression of the cross entropy loss function is:

[0046]

[0047] Where: z y is the original score corresponding to the true category y;

[0048] The loss function composed of Triplet Loss and Softmax Loss in InceptionResNetV2 is:

[0049] L=L t +L s

[0050] The model can be used to achieve individual identification.

[0051] A further improvement of the present invention is that, in step S5, for a binary image mask, the white area in the mask is used to represent the target; the expression of the centroid position is:

[0052]

[0053] Where: M(x i ,y i ) is the pixel value, 1 represents the target and 0 represents the background; (x i ,y i ) is the coordinate of the target pixel; (x c ,y c ) are the coordinates of the center of mass.

[0054] A further improvement of the present invention is that in step S6:

[0055] The centroid coordinate data is smoothed by increasing the frame interval to reduce noise and short-term fluctuations in the data, making the overall change trend more stable and continuous;

[0056] Through individual recognition, tracking and segmentation technology, the speed, acceleration and direction of the subject's movement are obtained using the coordinates of the mask's centroid; the relevant information about the subject's direction change includes the frequency of direction change, movement stagnation and trajectory overlap rate;

[0057] In order to eliminate the influence of perspective changes, the camera used to shoot the striped seabass maintained a fixed perspective, and the movement of the target was limited to the horizontal plane, so that calculations could be performed in the two-dimensional (x, y) plane, ignoring changes in the z-axis; ensuring that the description of the target movement was the velocity and acceleration within the plane, without being affected by changes in perspective in the vertical direction.

[0058] A further improvement of the present invention is that in step S6: during the velocity and acceleration analysis:

[0059] The average speed of the object in time Δt is is the displacement vector divided by the time interval; its expression is:

[0060]

[0061] You can write:

[0062]

[0063] Among them: (x1, y1), (x2, y2): coordinates of the center of mass at the current moment and the next moment; are the unit vectors in the x and y directions respectively; within the time Δt, the acceleration of an object is the change in velocity per unit time, which is calculated by the velocity change and the time difference, and its expression is:

[0064]

[0065] A further improvement of the present invention is that in step S6, the direction calculation expression is:

[0066]

[0067] Where: (x1, y1), (x2, y 2) are the coordinates of the center of mass at the current moment and the next moment;

[0068] A further improvement of the present invention is that in step S6:

[0069] The detection scheme for direction change includes:

[0070] Divide the 2π range into n regions:

[0071]

[0072] Base direction

[0073] If the current direction θ c Satisfy θ k <θ c <θ k+1 , and at the next moment θ c+1 ≥θ k+1 or θ c+1 <θ k , it indicates a change of direction;

[0074] A further improvement of the present invention is that in step S6:

[0075] The calculation process of direction change frequency includes:

[0076] Through the above analysis of direction and direction change, the following expression of direction change frequency is obtained:

[0077] or

[0078]

[0079] Where: H(x): indicator function takes the value 1 when the condition in the brackets is met, otherwise it is 0; T is the number of direction changes;

[0080] The calculation process of motion stagnation includes:

[0081] If the rate It is considered stagnant; the stagnation rate formula is:

[0082]

[0083] in: The velocity at the i-th moment; v L : Threshold of speed stagnation; H(x): indicator function, which is 1 when the condition is met, otherwise 0; S: represents the proportion of stagnation time to the total time;

[0084] The phenomenon of motion stagnation can be roughly divided into three categories:

[0085] High stagnation: low speed points account for > 90%

[0086] Moderate stagnation: low speed points account for 50% to 90%

[0087] Low stagnation: low speed points account for <50%

[0088] The calculation process of stress response includes:

[0089] The speed change formula and acceleration calculation formula obtained by the above analysis are:

[0090]

[0091] Then the expression of stress index is obtained:

[0092] SI=α·Δv+β·a t

[0093] Among them: α and β are weight coefficients in the calculation of stress index;

[0094] The calculation process of trajectory overlap rate includes:

[0095] The expression of trajectory overlap rate is:

[0096]

[0097] in: Two trajectories at time t i δ: the position vector of the overlap; H(x): indicator function, which is 1 if the condition is met, otherwise 0.

[0098] The beneficial effects of the present invention are:

[0099] By combining the YOLOv8seg model, fish target detection and segmentation are achieved. This technology can accurately extract the main part of underwater fish, and even in complex underwater environments, it can effectively reduce the influence of interference factors such as background clutter and overlapping objects, thereby ensuring the accuracy of individual recognition. This meticulous segmentation technology greatly improves the identifiability of fish in different environments and provides a high-quality data foundation for subsequent individual feature extraction and behavior analysis.

[0100] The YOLOv8pose model is used to detect key points of fish, annotate key points of fish bodies (such as eyes, head, body, tail, etc.), and perform affine transformation and alignment of the fish bodies in the image based on the key point information. This method can further optimize the stability and accuracy of individual recognition under the conditions of diverse fish postures and complex lighting conditions, and ensure the reliability of fish posture analysis results.

[0101] The InceptionResNetV2 model is used to extract high-dimensional features from fish images, and the embedding space is optimized with normalization technology to solve the problem of feature over-expansion. By improving the model's ability to distinguish, it can accurately distinguish the characteristics of different individuals of fish, while reducing the risk of overlapping features of similar classes. This feature optimization technology provides strong technical support for individual fish identification.

[0102] Combining the advantages of TripletLoss and SoftmaxLoss, the feature differentiation ability and category recognition accuracy of the individual recognition model are optimized. TripletLoss improves the differentiation ability of the embedding space by shortening the feature distance of samples of the same category and increasing the distance between samples of different categories; SoftmaxLoss further enhances the accurate recognition of fish categories by optimizing the category probability. The combination of the two effectively improves the accuracy and generalization ability of individual recognition.

[0103] By measuring the coordinates of the center of mass of the mask and time, the system can comprehensively analyze the movement path and dynamic characteristics of the fish. Combined with key indicators such as vector velocity (with direction), acceleration (with direction), direction (radian), direction change and direction change frequency, the movement state of the fish can be quantitatively analyzed.

[0104] The movement of healthy fish is usually characterized by steady speed, regular changes in direction, and high track consistency. When fish show signs of stagnation, abnormal fluctuations in the frequency of direction changes, or a decrease in track overlap, it may reflect some underlying abnormal behavior.

[0105] By analyzing these key parameters, the system can provide detailed data support for the movement characteristics of fish, helping farmers to understand the activity status and behavior patterns of fish in real time. This automated analysis method based on behavioral data improves the refinement and scientific nature of aquaculture management, and provides an efficient and low-cost intelligent management tool for the aquaculture industry.

[0106] The present invention combines deep learning technology with aquaculture needs, uses the YOLOv8 model to achieve high-precision detection and individual identification of fish, and further enhances the system's ability to analyze individual characteristics and motion status of fish through key point detection and feature optimization methods. Through real-time behavior monitoring and intelligent early warning, the system effectively supports farmers to detect abnormal situations in a timely manner and reduce potential losses caused by the spread of fish diseases. With the advantages of high precision, low cost and automation, the system has promoted the development of aquaculture towards intelligent management and provided innovative technical solutions and strong support for the industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0107] Figure 1 It is the overall flow chart of the system of the present invention;

[0108] Figure 2 This is a flowchart of the semantic segmentation module based on the YOLOv8-seg model;

[0109] Figure 3 The main mask target image of the Lateolabrax japonicus obtained through the semantic segmentation module;

[0110] Figure 4 This is a modified network architecture diagram based on the InceptionResNetV2 network;

[0111] Figure 5 This is a flow chart for detecting the individual recognition module based on the InceptionResNetV2 network;

[0112] Figure 6 t-SNE 2D graph of the training set data obtained for the individual recognition module after transformation through cluster distribution;

[0113] Figure 7 t-SNE 2D graph of the training and test set data obtained for the individual recognition module after transformation through cluster distribution;

[0114] Figure 8 This is the confusion matrix diagram of the individual recognition module. DETAILED DESCRIPTION

[0115] Example: Figure 1 As shown, the embodiment of the present invention provides a method for underwater sea bass individual identification and behavior analysis based on a deep learning model, which mainly comprises the following steps:

[0116] Step 1: Determine the basic underwater Lateolabrax image dataset to provide basic data support for model training and verification.

[0117] Step 2: Use the YOLOv8-seg model to segment the Lateolabrax image dataset and extract the mask data of the main part of the Lateolabrax.

[0118] Step 3: Detect key points through the YOLOv8-pose model, and then align the key points through affine transformation.

[0119] Step 4: Use InceptionResNetV2 to extract high-dimensional features and use normalization to stabilize the embedding space. Combined with Triplet Loss and Softmax Loss, the model not only optimizes the distinguishing ability of the embedding space, but also improves the accuracy of category prediction, thereby achieving efficient individual identification of Lateolabrax japonicus.

[0120] Step 5: Calculate the center of mass coordinates of the acquired mask data, that is, the geometric equilibrium position, to provide data support for subsequent behavior analysis of the subject.

[0121] Step 6: Using the center of mass coordinates obtained in step 5, the center of mass coordinate data is smoothed by slightly increasing the frame interval, and the speed, acceleration, and path overlap of the subject are calculated. Combined with the key point data extracted in step 3, the relevant information of the subject's direction change is analyzed, and the subject's behavior characteristics are analyzed in detail based on the above data.

[0122] Step 1 specifically includes:

[0123] The "Accurate Red Seareef fish shapes / segmentation" dataset from the MetaVision team on the Kaggle open source dataset website, the "Fish Dataset" dataset from Mark Daniel Lampa user, the "A Large-Scale Dataset for Fish Segmentation and Classification" dataset published by IEEE, and manually acquired and annotated datasets are used to provide basic data support for subsequent model training and verification.

[0124] Step 2 specifically includes:

[0125] like Figure 2 , Figure 3As shown in the figure, the YOLOv8-seg model consists of three parts: Backbone (feature extraction), Neck (feature fusion) and Head (detection and segmentation head). By combining the detection and segmentation heads, adopting efficient feature extraction and fusion strategies, and using targeted loss functions, high-precision and efficient target detection and segmentation are achieved. Its output includes detection boxes, category probabilities and segmentation masks. The segmentation mask part mainly generates accurate masks for each target using the predicted feature map through the segmentation head. Its expression is:

[0126]

[0127] Among them: F m : Output of the segmentation branch in the feature map; F d To detect features related to the target box in the branch;

[0128] σ is the Sigmoid activation function, which is used to map the output to [0,1].

[0129] This step can obtain accurate mask data of the main part, providing support for the subsequent acquisition of centroid data.

[0130] Step three specifically includes:

[0131] Use the YOLOv8-pose model to extract the multi-scale features of the input image through the backbone network. Its function is to map the high-dimensional pixel space to the low-dimensional semantic space. Its expression is as follows:

[0132]

[0133] Where: X is the input image; F is the extracted feature tensor; is the backbone network; θ backbone are the parameters of the backbone network.

[0134] The backbone network maps the original high-dimensional pixel space (pixel value) into a low-dimensional semantic feature space through operations such as convolution, thereby extracting features containing key visual information.

[0135] Using the feature pyramid network, the model can capture the key points of small objects and integrate feature maps of different scales. The expression is as follows:

[0136] F out =FPN(F1,F2,···,θ FPN )

[0137] Among them: F1, F2 are multi-scale feature maps extracted by the backbone network; F out is the integrated feature map.

[0138] Through the fusion of top-down and bottom-up methods, the model can focus on targets of different scales, especially significantly improving the detection performance of key points of small targets.

[0139] The key point prediction is then completed through the detection head, and the formula is as follows:

[0140]

[0141] Where: K is the number of key points of each target; is the jth key point coordinate of the predicted i-th target; Coordinates for real key points.

[0142] The detection head is responsible for predicting key points based on the fused feature map and calculating the loss function This loss function measures the error between the predicted keypoint coordinates and the true keypoint coordinates.

[0143] The above content is used to detect key points. In order to align the fish body images, an affine transformation method is used. Affine transformation is a technique that can map a plane in an image to a new coordinate system through geometric transformations such as translation, rotation, and scaling. By calculating the relationship between key points in the source image and corresponding points in the target image, the affine transformation matrix can be solved and applied to the entire image.

[0144] In order to achieve image alignment, the formula is as follows

[0145]

[0146] in:

[0147] (x1,u1),(x2,u2)(x3,y3)(x4,y4)(x5,y5) are the keypoint coordinates in the 5 source images detected by the YOLOv8-pose model, and (x1′,y1′),(x2′,y2′)(x3′,y3′)(x4′,y4′)(x5′,y5′) are the defined target keypoint coordinates. 11 ,a 12 ,a 21 ,a 22 ,t x ,t y There are 6 parameters that need to be solved. Among them, t x ,t y Control the translation of the image, a 11 ,a 12 ,a 21 ,a 22 Control the rotation, scaling, and shearing of an image.

[0148] For each point (x, y) in the source image, the corresponding new coordinates (x′, y′) after affine transformation are calculated by the following formula

[0149]

[0150] Through the above steps, this embodiment can obtain an affine transformation matrix that aligns the key points of the source image to the target image. This matrix contains transformation parameters such as translation, rotation, and scaling, and can be applied to the entire image to achieve image alignment.

[0151] Step 4 specifically includes:

[0152] like Figure 4 , Figure 5 As shown, the individual recognition module uses InceptionResNetV2 as the backbone network, which combines the Inception module and the residual structure, and can effectively extract high-dimensional features with low computational complexity. In this task, the high-dimensional features provided by InceptionResNetV2 will be mapped to two different spaces: the embedding space and the category space. In order to ensure that the eigenvalues ​​in the embedding space are not too large, this embodiment normalizes the output embedding vector and normalizes the modulus of each embedding vector to unit length. This can prevent excessive expansion of features, thereby ensuring the stability and comparability of feature vectors. Its expression is as follows:

[0153]

[0154] In order to achieve individual recognition, this embodiment uses both Triplet Loss and Softmax Loss loss functions. By combining the distinguishing ability of feature embedding and the classification ability of clear categories, the comprehensive performance of the model in the task of individual recognition of sea bass can be improved.

[0155] The optimization goal of the Triplet Loss function is to make the distance between similar samples closer and the distance between dissimilar samples farther. Its expression is as follows:

[0156] L t =max(d(f(x a ),f(x p ))-d(f(x a ),f(x n ))+α,0)

[0157] Where: f(x) is the embedding vector extracted by InceptionResNetV2. d is the Euclidean distance between features. a ,x p ,x nRepresents anchor samples (Anchor), positive samples (Positive) and negative samples (Negative) respectively. α: The interval threshold set to ensure that the distance between classes is large enough.

[0158] The optimization objectives are divided into:

[0159] Minimize: Similar samples (x a and x p ) between the characteristic distance d(f(x a ),f(x p )).

[0160] Maximize: Different class samples (x a and x n ) between the characteristic distance d(f(x a ),f(x n )).

[0161] The semi-hard strategy is used to select triplets. It can provide effective training signals while avoiding instability caused by excessive difficulty. The distance between the anchor point and the positive sample is smaller than the distance between the anchor point and the negative sample, but not too close. The distance between the anchor point and the negative sample is not less than the distance between the anchor point and the positive sample plus a set interval threshold. Its mathematical description is:

[0162] d(f(x a ),f(x p )) <d(f(x a ),f(x n )) <d(f(x a ),f(x p ))+α

[0163] The combination of Softmax and cross entropy loss functions can help the model assign the correct category probability to each sample. Specifically, Softmax converts the model's original score (logits) into a probability distribution, while cross entropy loss is used to measure the difference between the category probability predicted by the model and the true category, thereby guiding the model to improve the probability of the correct category.

[0164] The expression of Softmax is as follows:

[0165]

[0166] Where: z k is the raw score output (logit) of the model for class k. C is the total number of classes. It applies an exponential function to the score of class k, producing positive values ​​and amplifying higher scores.

[0167] The expression of the cross entropy loss function is:

[0168]

[0169] Where: z y is the raw score (logit) corresponding to the true category y.

[0170] In this task, Triplet Loss focuses on the relative distance between samples. It helps the model learn how to distinguish the differences between different categories and form effective distinctions in the embedding space. Softmax Loss focuses on category prediction. It guides the model to classify by optimizing the accuracy of category prediction. Therefore, the losses of the two can be added together to jointly optimize the model, while improving the classification ability of the model and the effectiveness of the embedding space. The loss function composed of Triplet Loss and Softmax Loss is:

[0171] L=L t +L s

[0172] The model can be used to achieve individual identification.

[0173] Figure 6 The t-SNE 2D graph of the training set data obtained by the individual recognition module used in this embodiment after being transformed through cluster distribution; Figure 7 t-SNE2D graph of the training set and test set data obtained for the individual recognition module after transformation through cluster distribution; Figure 8 This is the confusion matrix diagram of the individual recognition module.

[0174] Figure 6 The following is a two-dimensional image of the high-dimensional embedded vector output by the model on the training set of 12 fish, obtained after dimensionality reduction using the t-SNE method. Figure 6 It can be seen that different samples of the same fish are clustered together, indicating that the model can accurately identify and distinguish the same fish; while samples of different fish are more scattered, indicating that the model can effectively identify different fish. This shows that the model performs well on the training set.

[0175] Figure 7 This is the output of the training set and the test set together. The one without the black border is the training set, and the one with the black border is the test set. The distribution of the training set and the test set is similar, indicating that the model has a good performance on the test set.

[0176] Figure 8is the confusion matrix of the model. In this matrix, each row represents the actual category and each column represents the predicted category. Each cell of the matrix contains the number of samples under the actual category and the predicted category. In summary, the individual recognition module used in this embodiment has a good recognition effect for fish.

[0177] Step 5 specifically includes:

[0178] For a binary image mask, this embodiment uses the white area in the mask to represent the target. The expression of the centroid position is:

[0179]

[0180] Where: M(x i ,y i ) is the pixel value, 1 represents the target and 0 represents the background; (x i ,y i ) is the coordinate of the target pixel; (x c ,y c ) are the coordinates of the center of mass.

[0181] The center of mass coordinates obtained through the above content provide data support for the subsequent behavior analysis of the subject.

[0182] Step 6 specifically includes:

[0183] The center of mass coordinate data is smoothed by slightly increasing the frame interval. This processing method can not only retain the detailed characteristics of the trajectory, but also effectively improve the analyzability of the data, providing a more accurate basis for the subsequent calculation of speed, acceleration and behavioral characteristics analysis.

[0184] Through the system's individual recognition, tracking and segmentation technologies, the mask centroid coordinates are used to obtain data such as the subject's movement speed, acceleration, direction, direction change frequency, movement stagnation and trajectory overlap rate.

[0185] In order to eliminate the influence of the change of the viewing angle, the camera is kept at a fixed viewing angle in this embodiment, and the movement of the target is limited to the horizontal plane. Therefore, this embodiment can perform calculations in the (x, y) plane in a two-dimensional plane, ignoring the change of the z axis. It ensures that the description of the target movement is the velocity and acceleration in the plane, without being disturbed by the change of the viewing angle in the vertical direction.

[0186] Velocity and acceleration analysis:

[0187] The average speed of the object in time Δt is It is the displacement vector divided by the time interval. Its expression is:

[0188]

[0189] You can write:

[0190]

[0191] Among them: (x1, y1), (x2, y2): coordinates of the center of mass at the current moment and the next moment; are unit vectors in the x and y directions respectively.

[0192] In the time Δt, the acceleration of an object is the change in velocity per unit time, which is calculated by the velocity change and time difference. Its expression is:

[0193]

[0194] The calculation of velocity and acceleration can be used to detect the smoothness of the target's motion and drastic changes in motion state, providing a data basis for subsequent behavioral analysis.

[0195] In the process of direction and direction change analysis in this embodiment:

[0196] The direction calculation expression is:

[0197]

[0198] Where: (x1, y1), (x2, y 2) are the coordinates of the center of mass at the current moment and the next moment.

[0199] During orientation change detection:

[0200] Divide 2π into n regions:

[0201]

[0202] Base direction

[0203] If the current direction θ c Satisfy θ k <θ c <θ k+1 , and at the next moment θ c+1 ≥θ k+1 or θ c+1 <θ k , it indicates a change of direction.

[0204] Direction change frequency analysis:

[0205] Through the above analysis of direction and direction change, this embodiment can obtain the following expression of direction change frequency:

[0206] or

[0207]

[0208] Where: H(x) is an indicator function that takes the value 1 when the condition in the brackets is met, otherwise it takes the value 0. T is the number of direction changes.

[0209] By analyzing the frequency of direction changes, we can reveal whether the target is changing its movement trajectory, whether it has frequent turning behaviors, etc., providing support for subsequent target behavior pattern analysis.

[0210] Analysis of motion stagnation:

[0211] If the rate It is considered stagnant, and the stagnation rate formula is:

[0212]

[0213] in: The velocity at the i-th moment; v L : The threshold value of velocity stagnation (usually less than 0.1 cm / s); H(x): indicator function, which is 1 when the condition is met, otherwise it is 0; S: represents the proportion of stagnation time to the total time.

[0214] This embodiment divides the motion stagnation phenomenon into three categories:

[0215] High stagnation: low speed points account for > 90%

[0216] Moderate stagnation: low speed points account for 50% to 90%

[0217] Low stagnation: low speed points account for <50%

[0218] By analyzing the stagnation of movement, the activity state of the target can be reflected.

[0219] Stress response analysis:

[0220] The speed change formula and acceleration calculation formula obtained by the above analysis are:

[0221]

[0222] Then the expression of stress index is obtained:

[0223] SI=α·Δv+β·a t

[0224] Among them: α and β are weight coefficients in the calculation of stress index.

[0225] Stress response analysis helps to identify whether the target has a stress response when encountering external stimuli, and then evaluate its agility and physiological response to environmental changes.

[0226] Trajectory overlap analysis:

[0227] The expression of trajectory overlap rate is:

[0228]

[0229] in: Two trajectories at time t i δ: the position vector of the overlap; H(x): indicator function, which is 1 if the condition is met, otherwise 0.

[0230] By analyzing the trajectory overlap rate, we can compare the similarity of the target's trajectories under the same environmental conditions and evaluate the consistency or repeatability of its behavior.

[0231] The present invention uses deep learning and computer vision technology to realize automatic identification and behavior analysis of individual Lateolabrax japonicus. By analyzing the movement trajectory, speed, acceleration and behavior pattern of fish, it helps researchers and aquaculture industry personnel better understand the growth characteristics, activity patterns and stress responses of fish, so as to promote scientific breeding and ecological protection.

[0232] The present invention uses advanced deep learning and computer vision technologies to achieve efficient and automated identification and behavioral analysis of individual seabass. By combining models such as YOLOv8-seg, YOLOv8-pose, and InceptionResNetV2, the present invention improves the accuracy of behavioral analysis and real-time processing capabilities while ensuring recognition accuracy. This technology has a high degree of automation, significantly reducing manual intervention and labeling work, thereby improving the efficiency of research and data processing, and reducing labor costs and errors. In addition, the invention can analyze the movement characteristics of fish in multiple dimensions, such as speed, acceleration, direction changes, and stagnation, providing a scientific basis for a comprehensive assessment of fish behavior patterns. This method has good scalability and flexibility, can adapt to the behavioral analysis needs of other fish species or animals, and provides strong technical support for aquaculture, ecological protection, and scientific research.

[0233] The technology of the present invention can be widely used in aquaculture, ecological protection, behavioral research and other fields. It provides scientific behavioral analysis data for aquaculturists, helps optimize the breeding environment and improve breeding efficiency; at the same time, it also provides monitoring data support for ecological protection work, which helps to protect endangered aquatic organisms. The above describes in detail the preferred specific embodiments of the present invention. It should be understood that ordinary technicians in this field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in this technical field through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the existing technology should be within the scope of protection determined by the claims.

Claims

1. A method for underwater japonicus individual identification and behavior analysis, characterized in that: The following steps are involved: S1: Obtain underwater Lateolabrax image dataset to train the model; S2: Use the YOLOv8-seg model to segment the Lateolabrax image and extract the mask data of the main part of the Lateolabrax; S3: Key point detection is performed through the YOLOv8-pose model, and then key point alignment is performed through affine transformation; S4: Use InceptionResNetV2 to extract high-dimensional features to identify individual Lateolabrax; S5: by calculating the centroid coordinates of the acquired mask data, i.e., the geometric equilibrium position; S6: Taking the striped sea bass in the image as the subject, using the center of mass coordinates obtained in step S5, calculate the subject's velocity, acceleration and path overlap, and combine with the key points obtained in step S3 to analyze the relevant information of the subject's direction change.

2. The method for underwater japonicus individual identification and behavior analysis according to claim 1, characterized in that: In step S2, the YOLOv8-seg model includes a feature extraction module, a feature fusion module, and a detection and segmentation head module; its output includes a detection box, a category probability, and a segmentation mask, wherein the segmentation mask part generates an accurate mask for each target using the predicted feature map through the segmentation head, and its expression is: Among them: F m is the output of the segmentation branch in the feature map; F d is the feature related to the target box in the detection branch; σ is the Sigmoid activation function, which is used to map the output to [0,1].

3. The method for underwater japonicus individual identification and behavior analysis according to claim 1, characterized in that: In step S3, the YOLOv8-pose model is used to extract the multi-scale features of the input image through the backbone network, thereby mapping the high-dimensional pixel space to the low-dimensional semantic space, which is expressed as: Where: X is the input image; F is the extracted feature tensor; is the backbone network; θ backbone are the parameters of the backbone network; The backbone network of the YOLOv8-pose model maps the original high-dimensional pixel space into a low-dimensional semantic feature space, thereby extracting features containing key visual information; The YOLOv8-pose model uses a feature pyramid network to enable the model to capture the key points of small targets and integrate feature maps of different scales. Its expression is as follows: F out =FPN(F1,F2,···,θ FPN ) Among them: F1, F2 are multi-scale feature maps extracted by the backbone network; F out It is the integrated feature map; The YOLOv8-pose model uses a top-down and bottom-up fusion approach to enable the model to focus on targets of different scales, and then completes key point prediction through the detection head. Its expression is as follows: Where: K is the number of key points of each target; is the jth key point coordinate of the predicted i-th target; Make coordinates for real key points; The detection head is responsible for predicting key points based on the fused feature map and calculating the loss function This loss function measures the error between the predicted key point coordinates and the actual key point coordinates; key point detection is achieved through the above content, and then affine transformation is used to achieve key point alignment.

4. The method for underwater japonicus individual identification and behavior analysis according to claim 1, characterized in that: In step S4, InceptionResNetV2 is used as the backbone network, which combines the Inception module and the residual structure; the high-dimensional features provided by InceptionResNetV2 will be mapped to the embedding space and the category space; the embedding vectors output by the network are normalized, and the modulus of each embedding vector is standardized to unit length; to prevent excessive expansion of features, thereby ensuring the stability and comparability of feature vectors; Its expression is as follows: In order to achieve individual recognition, InceptionResNetV2 uses both Triplet Loss and Softmax Loss functions. By combining the distinguishing ability of feature embedding and the classification ability of clear categories, the overall performance of the model in the task of identifying individual sea bass is improved. The optimization goal of the Triplet Loss function is to make the distance between similar samples closer and the distance between dissimilar samples farther; Its expression is as follows: L t =max(d(f(x a ),f(x p ))-d(f(x a ),f(x n ))+α,0) Where: f(x) is the embedding vector extracted by InceptionResNetV2; d is the Euclidean distance between features; x a ,x p ,x n They represent anchor samples, positive samples, and negative samples respectively; α is the set interval threshold, which is used to ensure that the distance between classes is large enough; The optimization objectives are divided into: Minimize: Similar samples (x a and x p ) between the characteristic distance d(f(x a ),f(x p )); Maximize: Different class samples (x a and x n ) between the characteristic distance d(f(x a ),f(x n )); The semi-hard strategy is used to select the triplets; the distance between the anchor point and the positive sample is smaller than the distance between the anchor point and the negative sample; the distance between the anchor point and the negative sample is not less than the distance between the anchor point and the positive sample plus a set interval threshold; its mathematical description is: d(f(x a ),f(x p ))<d(f(x a ),f(x n ))<d(f(x a ),f(x p ))+α Softmax converts the original score of the model into a probability distribution, while the cross entropy loss is used to measure the difference between the category probability predicted by the model and the true category, thereby guiding the model to improve the probability of the correct category; The expression of Softmax is as follows: Where: z k is the model’s raw score output (logit) for category k; C is the total number of categories; It applies an exponential function to the score of category k, producing positive values ​​and amplifying higher scores; The expression of the cross entropy loss function is: Where: z y is the original score corresponding to the true category y; The loss function composed of Triplet Loss and Softmax Loss in InceptionResNetV2 is: L=L t +L s The model can be used to achieve individual identification.

5. The method for underwater Lateolabrax individual identification and behavior analysis according to claim 1, characterized in that: In step S5, for a binary image mask, the white area in the mask is used to represent the target; the expression of the centroid position is: Where: M(x i ,y i ) is the pixel value, 1 represents the target and 0 represents the background; (x i ,y i ) is the coordinate of the target pixel; (x c ,y c ) are the coordinates of the center of mass.

6. The method for underwater Lateolabrax individual identification and behavior analysis according to claim 1, characterized in that: In step S6: The centroid coordinate data is smoothed by increasing the frame interval to reduce noise and short-term fluctuations in the data, making the overall change trend more stable and continuous; Through individual recognition, tracking and segmentation technology, the speed, acceleration and direction of the subject's movement are obtained using the coordinates of the mask's centroid; the relevant information about the subject's direction change includes the frequency of direction change, movement stagnation and trajectory overlap rate; In order to eliminate the influence of perspective changes, the camera used to shoot the striped seabass maintained a fixed perspective, and the movement of the target was limited to the horizontal plane, so that calculations could be performed in the (x, y) plane in the two-dimensional plane, ignoring changes in the z-axis; ensuring that the description of the target movement was the velocity and acceleration within the plane, without being affected by changes in perspective in the vertical direction.

7. The method for underwater Lateolabrax individual identification and behavior analysis according to claim 6, characterized in that: In step S6: During the velocity and acceleration analysis: The average speed of the object in time Δt is is the displacement vector divided by the time interval; its expression is: You can write: Among them: (x1, y1), (x2, y2): coordinates of the center of mass at the current moment and the next moment; are the unit vectors in the x and y directions respectively; within the time Δt, the acceleration of an object is the change in velocity per unit time, which is calculated by the velocity change and the time difference, and its expression is:

8. The method for underwater Lateolabrax individual identification and behavior analysis according to claim 6, characterized in that: In step S6, the direction change angle calculation expression is: Where: (x1, y1), (x2, y 2) are the coordinates of the center of mass at the current moment and the next moment; θ is the angle of change in direction between the current moment and the next moment.

9. The method for underwater japonica individual identification and behavior analysis according to claim 6, characterized in that: In step S6: The detection scheme for direction change includes: Divide the 2π range into n regions: Base direction If the current direction θ c Satisfy θ k <θ c <θ k+1 , and at the next moment θ c+1 ≥θ k+1 or θ c+1 <θ k , it indicates a change of direction.

10. The method for underwater japonicus individual identification and behavior analysis according to claim 9, characterized in that: In step S6: The calculation process of direction change frequency includes: Through the above analysis of direction and direction change, the following expression of direction change frequency is obtained: or Where: H(x): indicator function takes the value 1 when the condition in the brackets is met, otherwise it is 0; T is the number of direction changes; The calculation process of motion stagnation includes: If the rate <Threshold v L It is considered stagnant; the stagnation rate formula is: in: The velocity at the i-th moment; v L : Threshold of speed stagnation; H(x): indicator function, which is 1 when the condition is met, otherwise 0; S: represents the proportion of stagnation time to the total time; The motion stagnation phenomenon is divided into three categories: High stagnation: low speed points account for > 90% Moderate stagnation: low speed points account for 50% to 90% Low stagnation: low speed points account for <50% The calculation process of stress response includes: The speed change formula and acceleration calculation formula obtained by the above analysis are: Then the expression of stress index is obtained: SI=α·Δv+β·a t Among them: α and β are weight coefficients in the calculation of stress index; The calculation process of trajectory overlap rate includes: The expression of trajectory overlap rate is: in: Two trajectories at time t i δ: the position vector of the overlap; H(x): indicator function, which is 1 if the condition is met, otherwise 0.

Images