Fish ingestion intensity analysis method based on hypergraph convolution

Abstract

The invention relates to a fish ingestion intensity analysis method based on hypergraph convolution, and belongs to the technical field of artificial intelligence aquaculture. The method comprises the following steps: extracting individual characteristics of a fish swarm before, during and after ingestion through target detection and key point detection, extracting the movement speed, position and direction of the fish swarm, and calculating position and distance characteristics between fish individuals; according to the bait position, calculating the distance and the direction characteristic of the nearest bait from the individual fish; taking each individual and bait as hypergraph nodes; building hyperedges according to the position and distance characteristics between the fish individuals and the distance and direction characteristics between the individuals and the bait; and constructing a group analysis hypergraph convolutional neural network model, and performing classification prediction on fish group behaviors. According to the method, the position and behavior characteristics can be extracted by fully utilizing the individual characteristics of the fishes, and the hypergraph convolutional network is constructed by utilizing the bait position relation, so that the fish feeding intensity is analyzed more comprehensively.

Description

Technical Field

[0001] The present invention relates to a fish feeding intensity analysis method based on hypergraph convolution, belonging to the technical field of artificial intelligence aquaculture. Background Art

[0002] As aquaculture expands, feed costs are rising. However, as a core component of production management, feeding is of paramount importance. It not only directly impacts the economic benefits of aquaculture but is also a key factor in ensuring stable growth in fish production. With the rapid advancement of computer vision and deep learning, researchers have developed a variety of intelligent aquaculture feeding strategies designed to optimize the feeding process. These innovative intelligent feeding strategies cleverly combine computer vision and image processing technologies, making it possible to assess fish feeding intensity and, therefore, more accurately understand and meet their feeding needs.

[0003] In the study of feeding intensity, the core link is to refine the interaction characteristics between fish schools, bait and environment. By using advanced target detection algorithms, it is possible to accurately identify and locate the characteristics of fish activity, the amount of bait and its spatial distribution. Furthermore, by extracting the recognizable behavioral characteristics of fish distribution, such as the location, density, and remaining amount of bait, the system deeply explores the dynamic changes in the interaction behavior between fish schools and bait under different feeding conditions. However, in the current field of feeding intensity research, conventional target detection algorithms have limitations. When detecting the characteristics of fish activity, the amount of bait and its spatial distribution, the accuracy and efficiency of conventional algorithms are relatively low, which cannot meet the needs of detailed research. Therefore, this application aims to obtain fish activity images or videos, analyze the behavioral characteristics of fish through the hypergraph convolutional neural network method, and combine it with the distribution of bait to finally realize the analysis of fish group behavior and infer the appetite of fish. Summary of the Invention

[0004] The purpose of the present invention is to provide a fish feeding intensity analysis method based on hypergraph convolution, aiming to solve the technical problems of low precision and low efficiency of conventional target detection algorithms.

[0005] To achieve the above objectives, the present invention provides a method for analyzing fish feeding intensity based on hypergraph convolution. This method uses object detection and keypoint detection to extract behavioral and positional features. A hypergraph is then constructed by defining hypergraph nodes and hyperedge sets. Feature information is then extracted using a hypergraph convolutional neural network, ultimately outputting fish feeding prediction results.

[0006] The specific steps are:

[0007] Step 1: Extract individual features of fish before, during and after feeding through target detection and key point detection;

[0008] Step 2: Extract the movement speed, position, and direction of the fish group based on individual characteristics, and calculate the position and distance characteristics between individual fish;

[0009] Step 3: Calculate the distance and direction characteristics of the fish individual to the nearest bait based on the bait position;

[0010] Step 4: Construct a hypergraph node for each fish individual and bait;

[0011] Step 5: Construct hyperedges based on the position and distance characteristics between individual fish and the distance and direction characteristics between individual fish and bait;

[0012] Step 6: Construct a group analysis hypergraph convolutional neural network model to classify and predict fish group behavior.

[0013] The Step 1 is specifically as follows:

[0014] Step 1.1: Use a camera to record video or image data of fish before, during, and after feeding;

[0015] Step 1.2: Use the target detection algorithm to detect the target in each frame of the image or video, and identify and locate the position of the individual fish;

[0016] Step 1.3: Use the key point detection algorithm to detect the position and bounding box of each individual in the fish group, and then extract the key points.

[0017] The Step 2 is specifically as follows:

[0018] Step 2.1: Use target detection and key point detection algorithms to extract the location features of each fish individual and express them using the center point coordinates;

[0019] Step 2.2: Calculate the movement speed of each fish individual by tracking the position changes of the fish individual between consecutive frames;

[0020] Step 2.3: Based on the position characteristics of each individual fish, calculate the average position and direction of the group. The average position of the group is represented by the average of all individual positions, and the direction of the group is represented by calculating the average direction vector of all individual positions.

[0021] Step 2.4: Calculate the position and distance characteristics between individuals. Analyze the structure and behavior of the group by calculating the position and distance characteristics between each pair of individuals.

[0022] The Step 3 is specifically as follows:

[0023] Step 3.1: Use target detection and key point detection algorithms to obtain the location coordinates of individual fish and bait;

[0024] Step 3.2: Calculate the Euclidean distance between the position of the individual fish and the position of the bait;

[0025] Step 3.3: Based on the position of the individual fish and the position of the bait, use the inverse tangent function to calculate the direction angle of the individual fish relative to the bait.

[0026] The Step 4 is specifically as follows:

[0027] Step 4.1: Set a set of fish nodes, where each node represents the position of a fish in the video;

[0028] Step 4.2: Set the bait node set, where each node represents a bait position in the video;

[0029] Step 4.3: Merge the fish node set and the bait node set into a node set.

[0030] The Step 5 is specifically as follows:

[0031] Step 5.1: Define two hyperedge sets, one representing the relationship between individual fish and the other representing the relationship between fish and bait;

[0032] Step 5.2: Analyze the changing trend of a pair of fish over time, define a weight to represent the movement relationship between the pair of fish, and add the pair of fish and the weight as a hyperedge to the hyperedge set;

[0033] Step 5.3: Analyze the changing trends of fish and bait over time, define a weight to represent the movement relationship between the fish and bait, and add the fish and bait as a hyperedge to the hyperedge set;

[0034] Step 5.4: Merge all the hyperedges of the movement relationships between fish and the hyperedges of the relationship between fish and bait to form a new hyperedge set.

[0035] The Step 6 is specifically as follows:

[0036] Step 6.1: Define the input layer. The input data is a hypergraph, which includes a set of nodes and a set of hyperedges. The node features represent the attributes of each node, and the hyperedges represent the relationships between nodes.

[0037] Step 6.2: Define the hypergraph convolutional layer. Parameter sharing is based on the hypergraph structure. By combining the relationships between nodes, the features of the central node are updated by aggregating the features of neighboring nodes.

[0038] Step 6.3: Use mean aggregation to extract the average value of the features of the neighboring nodes of the central node;

[0039] Step 6.4: Use the average pooling operation to calculate the average value of the obtained neighbor node features again, and use the obtained value as the aggregated feature representation;

[0040] Step 6.5: Based on the obtained feature representation, different types of group behavior are identified as none, strong, medium, and weak to represent the activity level of the fish group. Specifically, the different types of group behavior are none during the calm period without bait input, strong during the period when fish find bait and show fierce competition for food, medium during the period when fish's awareness of preying on bait is weakened, and weak during the period when fish no longer eat.

[0041] The beneficial effect of the present invention is that the present invention can fully utilize the individual characteristics of fish to extract position and behavioral characteristics and utilize the position relationship of bait to construct a hypergraph convolutional network, thereby more comprehensively analyzing the feeding intensity of fish. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the steps of the present invention;

[0043] Figure 2 Hypergraph convolution flow chart;

[0044] Figure 3 Schematic diagram of the hyperedge convolution layer. DETAILED DESCRIPTION

[0045] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0046] Example 1: Figure 1 As shown in Figure 1, a fish feeding intensity analysis method based on hypergraph convolution is shown in Figure 1. The specific steps are as follows:

[0047] Step 1: Extract individual features of fish before, during and after feeding through target detection and key point detection.

[0048] Specifically, a camera or other sensor device is used to record video or image data of fish before, during, and after feeding. The object detection algorithm YOLOv11 is used to perform object detection on each frame of the image or video, identifying and locating the positions of individual fish and bait, and adding bounding boxes to them. The key point detection algorithm RTMPose is then used to detect the positions and bounding boxes of each individual fish and bait in the group, extracting their key points and generating numbers and location information. Finally, the obtained fish and bait location information is used to record the positions in each frame in a file.

[0049] Step 2: Extract the movement speed, position, and direction of the fish group based on individual characteristics, and calculate the position and distance characteristics between individual fish.

[0050] Specifically, the position feature of the i-th fish individual is (x i ,y i ). By tracking the position changes of individual fish between consecutive frames, the movement speed of each individual can be calculated. The position of the i-th fish individual in the t-th frame and the (t-1)-th frame is set to (x i (t),y i (t)) and (x i (t-1),y i (t-1)), then the movement speed of the i-th fish individual in the t-frame is V i (t) can be calculated using the following formula:

[0051]

[0052] Based on the position characteristics of each individual fish, the average position and direction of the group can be calculated. The average position of the group can be expressed as the average of all individual positions, and the direction of the group can be expressed by calculating the average direction vector of all individual positions. Then calculate the position and distance characteristics between individuals: the structure and behavior of the group can be analyzed by calculating the position and distance characteristics between each pair of individuals. Set the position characteristics of two fish individuals i and j as (x i ,y i ) and (x j ,y j ), then the Euclidean distance D between them ij It can be calculated using the following formula:

[0053]

[0054] Step 3: Calculate the distance and direction characteristics of the individual fish from the nearest bait based on the bait position.

[0055] Specifically, the position of the fish individual is set to (x i ,y i ), the position of the bait is (x b ,y b ). Then the Euclidean distance D between them ib It can be calculated using the following formula:

[0056]

[0057] According to the position of individual fish (x i ,y i ) and the bait position (x b ,yb ), the inverse tangent function can be used to calculate the direction angle θ of the fish individual relative to the bait. The specific formula is as follows:

[0058]

[0059] Step 4: Construct a hypergraph node for each fish individual and bait.

[0060] Specifically, we set a fish node set (F), where each node represents the position of a fish in the video. We set a bait node set (B), where each node represents the position of a bait in the video. We merge the fish node set and the bait node set into a node set (V).

[0061] Step 5: Construct hyperedges based on the position and distance characteristics between individual fish and the distance and direction characteristics between individual fish and bait.

[0062] Define two hyperedge sets (E_F) and (E_B), where (E_F) represents the relationship between fish individuals and (E_B) represents the relationship between fish and bait. i and F j Over time, the trend of change, that is, analysis of D ij Change trend, define a weight W ij Represents the movement relationship between the two fish, and adds the pair of fish and weight as a hyperedge to the hyperedge set (E_F), ({F i ,F j},W ij ) is a hyperedge. i and bait B b Analysis of trends over time ib Change trend, define a weight W ib Represents the motion relationship between fish and bait, which is added as a hyperedge to the hyperedge set (E_B), ({F i ,F b},W ib ) is a hyperedge. Combine all the hyperedges of the movement relationship between fish and the hyperedges of the relationship between fish and bait to form a hyperedge set (E).

[0063] Step 6: Construct a group analysis hypergraph convolutional neural network model to classify and predict fish group behavior.

[0064] Specifically, if Figure 2As shown, it is the flowchart of hypergraph convolution. The input layer is defined, and the input data is a hypergraph, which includes a node set and hyperedges. The node features represent the attributes of each node, and the hyperedges represent the relationships between nodes. For each hyperedge (h), the features of the connected nodes are averaged to obtain the feature representation of the hyperedge. In the average aggregation, the feature representation of a node is updated by taking the average of the feature of its neighbor nodes. The formula is as follows:

[0065] h v (l) = mean(h u l , |, u ∈ N (v) ) (5)

[0066] where h v (l) is the feature representation of node v at the (l)-th layer, N (v) is the set of neighbor nodes of node v, mean() represents calculating the average value, u represents the neighbor node of node v, is the feature representation of node u at the (l)-th layer. Map the feature of each hyperedge to the connected nodes to update the feature representation of the nodes. The mapping method uses weighted summation:

[0067] v′ i = v i + weight(h, v i ) · h v (6)

[0068] v′ i represents the node feature after mapping, weight() represents the weighted summation operation, h v represents the hyperedge feature of node v. Then, perform node feature update. For each node v i and the node feature v′ i after mapping are fused. A fully connected neural network is used for feature fusion:

[0069] v″ i = f(W · [v i , v′ i + b) (7)

[0070] where v″ i represents the node feature after fusion of node v i and the node feature v′ i after mapping. W is the weight matrix of the fully connected neural network, b is the bias term, and f is the activation function.

[0071] Define the hypergraph convolution layer. Parameter sharing is based on the hypergraph structure, considering the relationships between nodes, and updating the features of the central node by aggregating the features of neighbor nodes. The core convolution formula is as follows:

[0072]

[0073] Where l is the node feature matrix of the (l)th layer, and each row represents the feature vector of a node. (l+1) represents the adjacency matrix of the (l+1)th layer in the hypergraph convolution, W (l) is the weight matrix of the learnable parameters of the (l)th layer of the hypergraph convolution, where each column represents a convolution kernel. H is the adjacency matrix of the hypergraph, which represents the node connection relationship in the graph. D is the degree matrix. H T represents the transposed matrix of H, Represents a normalization operation that balances the influence of the number of connections between different nodes by taking the inverse matrix of the diagonal square root of the degree matrix. (l) It is a fully connected layer, and the adjacency matrix H of the (l)th layer in the hypergraph convolution is (l) Perform feature transformation, σ is the activation function.

[0074] Further, if Figure 3 The figure shows the structure of a hypergraph and the bidirectional aggregation process of node features and hyperedge features. There are multiple nodes (such as n1, n2, n3, etc.), each with a different color, representing different node features, as well as hyperedges (such as e1, e2, e3, etc.). The left side is the original hypergraph, and the middle part shows the process of node features aggregating to hyperedges. For example, the features of nodes n1, n3, and n7 are aggregated to hyperedge e1 along the dotted arrows, and the features of nodes n3, n4, and n6 are aggregated to hyperedges. The right side shows the hyperedge features after aggregation, and each hyperedge has a corresponding feature representation (indicated by a box). The right side shows the hyperedges and their features. The middle part shows the process of hyperedge features aggregating to nodes. For example, the features of hyperedges e1, e2, and e3 are aggregated to node n6 along the dotted arrows. The left side shows the node features after aggregation, and each node has a new feature representation (indicated by a box). Hyperedge features are aggregated into the central vertex features of the hyperedge set through hyperedge convolution. The Softmax activation function is used to convert the input vector into a probability distribution, which is used for different types of fish group behavior classification output:

[0075]

[0076] where x i is the i-th element in the input vector, softmax is the activation function, Indicates x i is the hyperedge set of the central vertex, and N is the dimension of the input vector. The input data x is obtained by the softmax function i The probability of belonging to the i-th category.

[0077] In the pooling process, the average pooling operation is used to calculate the average value of the neighbor node features as the aggregated feature representation. The formula is as follows:

[0078]

[0079] h pool represents the aggregated feature value obtained by the pooling operation, h u Represents the feature vector of node u. Through the pooling operation, the information of multiple nodes can be integrated into an overall representation to better describe the behavioral characteristics of the entire fish school.

[0080] Based on the obtained feature representation, different types of group behavior are identified as none, strong, medium, and weak to represent the activity level of the fish group. Specifically, the different types of group behavior are none during the calm period without bait input, strong during the period when fish find bait and show fierce competition for food, medium during the period when fish's awareness of preying on bait is weakened, and weak during the period when fish no longer eat.

[0081] The above describes in detail the specific embodiments of the present invention in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by ordinary technicians in this field, various modifications and improvements can be made without departing from the concept of the present invention. These all fall within the scope of protection of this application. Therefore, the scope of protection of the patent for this invention shall be based on the appended claims.

Claims

1. A fish feeding intensity analysis method based on hypergraph convolution, characterized by: Step 1: Extract individual features of fish before, during and after feeding through target detection and key point detection; Step 2: Extract the movement speed, position, and direction of the fish group based on individual characteristics, and calculate the position and distance characteristics between individual fish; Step 3: Calculate the distance and direction characteristics of the fish individual to the nearest bait based on the bait position; Step 4: Construct a hypergraph node for each fish individual and bait; Step 5: Construct hyperedges based on the position and distance characteristics between individual fish and the distance and direction characteristics between individual fish and bait; Step 6: Construct a group analysis hypergraph convolutional neural network model to classify and predict fish group behavior.

2. The fish feeding intensity analysis method based on hypergraph convolution according to claim 1, characterized in that: The Step 1 is specifically as follows: Step 1.1: Use a camera to record video or image data of fish before, during, and after feeding; Step 1.2: Use the target detection algorithm to detect the target in each frame of the image or video, and identify and locate the position of the individual fish; Step 1.3: Use the key point detection algorithm to detect the position and bounding box of each individual in the fish group, and then extract the key points.

3. The fish feeding intensity analysis method based on hypergraph convolution according to claim 1, characterized in that: The Step 2 is specifically as follows: Step 2.1: Use target detection and key point detection algorithms to extract the location features of each fish individual and express them using the center point coordinates; Step 2.2: Calculate the movement speed of each fish individual by tracking the position changes of the fish individual between consecutive frames; Step 2.3: Based on the position characteristics of each individual fish, calculate the average position and direction of the group. The average position of the group is represented by the average of all individual positions, and the direction of the group is represented by calculating the average direction vector of all individual positions. Step 2.4: Calculate the position and distance characteristics between individuals. Analyze the structure and behavior of the group by calculating the position and distance characteristics between each pair of individuals.

4. The fish feeding intensity analysis method based on hypergraph convolution according to claim 1, characterized in that: The Step 3 is specifically as follows: Step 3.1: Use target detection and key point detection algorithms to obtain the location coordinates of individual fish and bait; Step 3.2: Calculate the Euclidean distance between the position of the individual fish and the position of the bait; Step 3.3: Based on the position of the individual fish and the position of the bait, use the inverse tangent function to calculate the direction angle of the individual fish relative to the bait.

5. The fish feeding intensity analysis method based on hypergraph convolution according to claim 1, characterized in that: The Step 4 is specifically as follows: Step 4.1: Set a set of fish nodes, where each node represents the position of a fish in the video; Step 4.2: Set the bait node set, where each node represents a bait position in the video; Step 4.3: Merge the fish node set and the bait node set into a node set.

6. The fish feeding intensity analysis method based on hypergraph convolution according to claim 1, characterized in that: The Step 5 is specifically as follows: Step 5.1: Define two hyperedge sets, one representing the relationship between individual fish and the other representing the relationship between fish and bait; Step 5.2: Analyze the changing trend of a pair of fish over time, define a weight to represent the movement relationship between the pair of fish, and add the pair of fish and the weight as a hyperedge to the hyperedge set; Step 5.3: Analyze the changing trends of fish and bait over time, define a weight to represent the movement relationship between the fish and bait, and add the fish and bait as a hyperedge to the hyperedge set; Step 5.4: Merge all the hyperedges of the movement relationships between fish and the hyperedges of the relationship between fish and bait to form a new hyperedge set.

7. The fish feeding intensity analysis method based on hypergraph convolution according to claim 1, characterized in that: The Step 6 is specifically as follows: Step 6.1: Define the input layer. The input data is a hypergraph, which includes a set of nodes and a set of hyperedges. The node features represent the attributes of each node, and the hyperedges represent the relationships between nodes. Step 6.2: Define the hypergraph convolutional layer. Parameter sharing is based on the hypergraph structure. By combining the relationships between nodes, the features of the central node are updated by aggregating the features of neighboring nodes. Step 6.3: Use mean aggregation to extract the average value of the features of the neighboring nodes of the central node; Step 6.4: Use the average pooling operation to calculate the average value of the obtained neighbor node features again, and use the obtained value as the aggregated feature representation; Step 6.5: Based on the obtained feature representation, different types of group behavior are identified as none, strong, medium, and weak to represent the activity level of the fish group. Specifically, the different types of group behavior are none during the calm period without bait input, strong during the period when fish find bait and show fierce competition for food, medium during the period when fish's awareness of preying on bait is weakened, and weak during the period when fish no longer eat.

Images