Fish pose estimation method and device, system, storage medium
By using the bottom-up fish posture keypoint detection model AHSC-Net, combined with data augmentation and posture structure constraints, the robustness and efficiency issues of fish posture estimation in complex underwater environments are solved, achieving high-precision and fast fish posture detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG OCEAN UNIVERSITY
- Filing Date
- 2026-04-17
- Publication Date
- 2026-07-14
Smart Images
Figure CN122392094A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of machine learning technology, specifically relating to a fish pose estimation method, device, system, and storage medium. Background Technology
[0002] As a cost-effective protein source, fish possess both significant nutritional and economic value globally. To achieve automated aquaculture, it's crucial to monitor and acquire accurate physiological information about fish in real time. Pose estimation technology uses algorithms to automatically detect and locate key points of objects in images or video sequences, then infers their spatial position and orientation. By applying pose estimation algorithms to locate key points and analyze the pose of fish, static parameters such as length and morphology can be directly obtained. Furthermore, based on continuous pose analysis, the system can capture changes in fish posture during movement, enabling effective monitoring of fish behavior and health, and the development of intelligent and precise feeding strategies.
[0003] Traditional methods, relying on manually designed features, suffer from insufficient robustness and struggle to adapt to the complex and ever-changing real-world underwater scenarios and the demands of detailed pose analysis, thus limiting the in-depth and large-scale application of fish behavior research. In recent years, machine vision-based image processing methods have developed rapidly, and pose estimation has become relatively mature in human pose estimation.
[0004] It has been gradually applied to the field of fish posture recognition. Compared with humans, fish posture detection faces more severe challenges: on the one hand, the underwater environment has complex optical interference and low visibility problems; on the other hand, the characteristics of severe occlusion between individual fish and variable scale, coupled with the low visual discrimination of key parts of the fish body, all increase the difficulty of accurate detection.
[0005] In recent years, with the increasing demand for high-precision attitude estimation, improved algorithms based on HRNet have become a research hotspot. Li et al. proposed a non-contact measurement system based on binocular vision, using an improved FPN-HRNet network to extract key points of the fish body and combining it with a 3D curve fitting algorithm to reduce measurement errors caused by fish swimming deformation. Peng et al. developed the HPFPE model for spotted sea bream, by incorporating the CBAM attention mechanism and dilated convolution into the HRNet backbone, expanding the receptive field while maintaining high-resolution features, and effectively solving the problems of underwater blurring and occlusion. Cui Haipeng et al. improved the HRNet network by incorporating pyramid segmentation attention, realizing the detection of key feature points of the fish body. However, they all adopt a "top-down" detection process. Although this process has the advantage of accuracy, in high-density fish school scenarios, it is necessary to detect and analyze each fish one by one, facing bottlenecks such as large computational load and slow inference speed. Meanwhile, significant progress has been made in single-stage algorithms based on YOLO-Pose. Li et al. proposed a keypoint detection method based on the YOLOv8-pose model, which, combined with a line laser system, enables automated measurement of the length and width of fish in tilted underwater postures. Tian et al. proposed a binocular measurement method based on an improved YOLOv8-pose model, which enhances the ability to capture minute keypoints on the fish body while ensuring real-time performance by introducing a multi-scale StemBlock module. Wang et al. constructed a lightweight convolutional neural network, using a rotating detection box to adapt to the changing swimming postures of fish, and combined it with a random forest algorithm to achieve efficient identification and trajectory tracking. However, limited by its ability to regress minute features, the positioning accuracy of YOLO-Pose in complex underwater environments often falls short of the requirements for fine-grained measurements. Summary of the Invention
[0006] To address the problems existing in the prior art, the present invention provides a fish posture estimation method, apparatus, system, and storage medium.
[0007] To achieve the above objectives, the present invention provides the following solution: A fish pose estimation method, characterized by comprising: Step S1: Select different fish densities under different light intensities to collect data and obtain the original dataset; Step S2: Perform data augmentation on the original dataset; Step S3: Based on the data-augmented original dataset, construct a fish pose keypoint detection model; wherein, the fish pose keypoint detection model realizes pose detection in a bottom-up manner, firstly detecting all keypoints of the fish in the image at once, and then obtaining the individual pose through cluster association; Step S4: Input the set of fish images to be detected into the fish pose key point detection model to estimate the fish pose.
[0008] Preferably, in step S2, a rectangular region is randomly selected from the training images of the original dataset and covered with random pixels or the mean value to simulate the actual situation where the target part is occluded; at the same time, a uniform grid occlusion is generated on the image to systematically cover the local area with black squares.
[0009] Preferably, in step S3, the fish pose keypoint detection model first extracts the depth visual features of the input image through a backbone network, and then predicts the precise pose of the fish body step by step through three progressive stages from coarse to fine. In the first stage, the network predicts the keypoints of the fish body based on the 9+1 keypoint heatmap refined by adaptive kernel modulation. These correspond to the 9 keypoints and 1 additional local center keypoint labeled in the dataset. In the second stage, the initial pose of each fish is initially estimated using the predicted local center keypoint and its associated offset vector. In the third stage, the network deeply fuses the keypoint heatmap generated in the first stage with the initial pose obtained in the second stage, and further optimizes and outputs the refined fish body pose through the convolutional message passing module and the local-global context adaptation module.
[0010] The present invention also provides a fish posture estimation device, comprising: The first processing module is used to collect data by selecting different fish densities under different light intensities to obtain the original dataset. The second processing module is used to augment the original dataset; The third processing module is used to construct a fish pose key point detection model based on the data augmented original dataset. The fish pose key point detection model is based on a bottom-up approach to realize pose detection. First, it detects the key points of all fish in the image at once, and then obtains the individual pose through cluster association. The fourth processing module is used to input the set of fish images to be detected into the fish pose key point detection model for fish pose estimation.
[0011] Preferably, the second processing module randomly selects rectangular regions in the training images of the original dataset and covers them with random pixels or average values to simulate the actual situation where the target part is occluded; at the same time, it systematically covers local areas with black squares by generating uniform grid-like occlusion on the image.
[0012] As a preferred approach, the fish pose keypoint detection model first extracts the depth visual features of the input image through a backbone network, and then predicts the precise pose of the fish body through three progressive stages from coarse to fine. In the first stage, the network predicts the keypoints of the fish body based on a heatmap of 9+1 keypoints refined by adaptive kernel modulation. These keypoints correspond to the 9 keypoints and 1 additional local center keypoint labeled in the dataset. In the second stage, the network uses the predicted local center keypoint and its associated offset vector to initially estimate the initial pose of each fish. In the third stage, the network deeply fuses the keypoint heatmap generated in the first stage with the initial pose obtained in the second stage, and further optimizes and outputs the refined fish pose through a convolutional message passing module and a local-global context adaptive module.
[0013] The present invention also provides a fish pose estimation system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program performs a fish pose estimation method when executed by the processor.
[0014] The present invention also provides a storage medium storing a computer program that executes a fish pose estimation method during runtime.
[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes a novel fish pose keypoint detection model, Adaptive-kernel Hybrid-center Structural Constraint Network (AHSC-Net). This model achieves pose detection based on a bottom-up process: first, it detects keypoints of all fish in the image at once, and then obtains individual poses through clustering. This design enables it to have efficient parallel processing capabilities, and in dense fish school scenes, the inference time is largely unaffected by the number of targets. Simultaneously, it exhibits outstanding detection accuracy, laying a solid foundation for the advancement of fish farming technology. Attached Figure Description
[0016] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a flowchart of the fish pose estimation method according to an embodiment of the present invention; Figure 2 Here is a flowchart of the AHSC-Net detection model; Figure 3 This is a diagram of the backbone network structure. Figure 4 This is a schematic diagram showing the offset of the target bounding box center; Figure 5 This is a schematic diagram of embedding feature map coordinate information. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0020] Example 1 like Figure 1 As shown, the present invention provides a fish pose estimation method, comprising: Step S1: Select different fish densities under different light intensities to collect data and obtain the original dataset; Step S2: Perform data augmentation on the original dataset; Step S3: Based on the data augmented original dataset, construct the fish pose keypoint detection model AHSC-Net; the fish pose keypoint detection model realizes pose detection in a bottom-up manner, first detecting all keypoints of the fish in the image at once, and then obtaining the individual pose through cluster association; Step S4: Input the set of fish images to be detected into the fish pose key point detection model to estimate the fish pose.
[0021] In one embodiment of the present invention, in step S1, the original dataset was collected at different fish densities under different light intensities. Using bass as the research object, 3435 images were selected to construct a dataset, which was then divided into training, validation, and test sets in an 8:1:1 ratio. CVAT was used to annotate the dataset, with 9 key points labeled for each fish.
[0022] In one embodiment of the present invention, step S2 employs a comprehensive data augmentation strategy to improve the model's robustness to varying fish postures, scale differences, and common occlusions. Regarding geometric transformations, rotations (±20°), scaling (0.7–1.3 times), and horizontal flips are randomly applied to the training data to simulate the basic deformation of the target in real-world imaging. Furthermore, to simulate keypoint occlusion scenarios, rectangular regions are randomly selected in the training images and covered with random pixels or the mean value to simulate the actual situation of partial target occlusion. Simultaneously, a uniform grid-like occlusion is generated on the image, systematically covering local areas with black squares. This avoids the loss of key features due to excessive occlusion while ensuring that the target remains partially visible, thus providing the model with effective learning signals and enhancing its anti-occlusion performance while constructing a reasonable occlusion scenario.
[0023] In one embodiment of the present invention, in step S3, AHSC-Net extracts the depth visual features of the input image through a backbone network, and then predicts the precise pose of the fish body through three progressive stages from coarse to fine. In the first stage, the network predicts the fish body's key points based on a heatmap of 9+1 keypoints refined by adaptive kernel modulation (corresponding to the 9 keypoints labeled in the dataset and 1 additional local center keypoint). In the second stage, the network initially estimates the pose of each fish using the predicted local center keypoint and its associated offset vector. In the third stage, the network deeply fuses the keypoint heatmap generated in the first stage with the initial pose obtained in the second stage, and further optimizes and outputs the refined fish body pose through a convolutional message passing module and a "local-global" context adaptation module. The third stage's specific process is as follows: First, each coarse keypoint is expanded to generate local sampling meshes (Keypoint Expansion Maps, KEMs). Then, feature sampling is performed on the depth feature map based on these sampling meshes to obtain latent spatial features fused with local context information. On this basis, the topological constraints between keypoints of the fish are learned through a convolutional message passing module, generating corresponding Keypoint Attraction Maps (KAMs) for each keypoint. This is equivalent to a dynamically learned local correction filter. Finally, in the "local-global" context adaptation step, KAMs are used to perform convolution operations on the heatmap extracted centered on the initial keypoints, thereby guiding the keypoints to more precise positions and completing the final pose prediction. AHSC-Net has achieved superior performance in fish pose estimation tasks. The structure of AHSC-Net is as follows: Figure 2 It uses HRNet as its backbone, with the following structure: Figure 3 .like Figure 2 As shown, AHSC-Net extracts depth visual features from the input image through a backbone network. Then, through three progressively coarse-to-fine stages, it uses KAMs to convolve the extracted heatmap centered on the initial keypoints, thereby guiding the keypoints to more precise locations and completing the final pose prediction. Figure 3As shown, the backbone network employs a four-branch architecture with progressively decreasing resolution. The output feature map sizes of each branch correspond to 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image, respectively. Each branch extracts features at the same resolution through consecutively stacked convolutional units, simultaneously achieving multi-dimensional feature representation from fine spatial details of fish keypoints to global pose semantic information. After each convolutional operation, the network achieves bidirectional feature interaction and information fusion between all branches through upsampling and downsampling operations, maintaining a high-resolution feature flow throughout. This effectively alleviates the spatial detail loss problem caused by repeated upsampling and downsampling in traditional serial backbone networks, ultimately completing the aggregation and output of multi-scale features, providing high-quality feature support for subsequent fish keypoint heatmap regression.
[0024] In algorithms that use center points for pose prediction, the center of the target detection bounding box is typically used as the supervision signal. Using the center point as an anchor point for pose prediction provides a core basis for the individual association of keypoints across multiple instances, simplifies the pose decoupling process in dense fish school scenes, and reduces the learning difficulty of pose estimation, making it a mainstream bottom-up approach for multi-target pose estimation. However, this definition method has a significant performance bottleneck when dealing with severely occluded fish targets: for fish whose parts are outside the image or severely occluded, the center of the target bounding box will shift significantly relative to the complete fish body. For example, when the posterior part of the fish is outside the image or severely occluded, the center of the target bounding box will shift significantly towards the head, which is severely mismatched with the statistical regularity learned by the model from a large number of complete samples that "the center point is located in the middle and posterior part of the fish body." Figure 4 As shown: These "difficult samples" typically constitute a small percentage of the dataset, and models tend to treat them as noise during optimization, leading to insufficient localization of occluded targets and ultimately causing missed detections or decreased keypoint regression accuracy. A simple solution is to create center points at more locations on the fish body, allowing the model to effectively adapt to these rarely used center points. To address this, this invention proposes a Stochastic Local Centroid Sampling (SLCS) strategy. This strategy dynamically samples the centroids of local keypoints on the fish body during the training phase as center point supervision signals, simulating the center offset characteristics of occluded scenes. This improves the model's ability to accurately identify fish and corresponding keypoints even when some parts of the fish are clearly visible, enhancing the model's perception and reasoning capabilities in difficult scenarios.
[0025] To establish a more robust regression benchmark, this invention defines the regression center as the geometric centroid of the set of all labeled keypoints, establishing a direct linear mapping between it and the core skeleton of the fish. This invention divides the nine keypoints into two topologically representative subsets: • Anterior anatomical structures group ( (This section includes the mouth, eyes, dorsal fin, and pelvic fins, representing the core structure of the front half of the fish's body.)
[0026] • Posterior anatomical structures group ( ): This includes the four corners of the caudal fin and the anal fin, representing the structure of the posterior half of the fish's body.
[0027] During the model training phase, in order to artificially simulate the distribution shift of the center point supervision signal caused by occlusion, for complete individuals in which all key points are clearly visible, this invention performs random local center sampling with a preset probability: 1. From or A subset is randomly selected from the data.
[0028] 2. Subsequently, the local centroids of the key points within the subset are calculated as the ground truth values of the center point of the current sample.
[0029] In this way, the model receives supervision signals more frequently during training that are similar to the features of real occluded scenes, thereby improving its detection performance in these situations. This approach may result in the same fish having multiple predicted center points; therefore, this invention will select the pose result with the highest confidence as the final output in subsequent processing stages.
[0030] In the pose detection algorithm of this invention, the accuracy of offset regression directly determines the quality of the initial pose. However, traditional offset regression loss functions typically predict the spatial displacement of each keypoint independently, completely ignoring the natural geometric and topological constraints inherent in the fish's anatomical structure. In underwater occlusion or low-contrast scenes, this easily leads to unreasonable poses with structural distortions. Secondly, the inherent translation invariance of standard convolution means the model lacks absolute coordinate references during regression, further amplifying the prediction error of the offset. To systematically solve the above problems, this invention proposes Spatial-Awareness Enhanced Pose Structural Constraint (SAPSC). To endow the network with spatial awareness capabilities, this invention embeds normalized coordinate information into the feature maps extracted from the backbone network. For example... Figure 5 As shown: Specifically, two coordinate channels are generated for the input feature map: horizontal coordinate channel i and vertical coordinate channel j, whose element values are linearly normalized to... Within this range, enhanced feature maps are formed by concatenating them with the original feature maps along the channel dimension. This design overcomes the lack of absolute position awareness in convolution operators, providing a global spatial position reference for offset regression.
[0031] However, relying solely on coordinate-aware local feature extraction still struggles to avoid pose distortion under complex occlusion. Therefore, this invention introduces pose structure constraints. This invention abstracts the fish's pose as a graph structure with keypoints as vertices and adjacent keypoints connected as "skeletons." The skeleton set is defined as follows: Every bone Representative from key points Pointing to key points The geometric vectors are used. By constructing a posture structure constraint loss function, the model is made to follow the inherent physical constraints of the fish more closely when predicting offsets. The posture constraint loss function is divided into bone length ratio constraints and bone angle constraints.
[0032] Skeleton Length Ratio Constraint: This invention chooses length ratio rather than absolute length as the constraint term to eliminate the problem of inconsistent loss intensity at different scales. If an absolute length constraint is used, the loss magnitude generated by large-scale targets will be significantly higher than that of small-scale targets, causing the optimization process to skew towards large fish samples, thereby weakening the model's topological inference ability for small fish. Suppose the model predicts two adjacent keypoints... and Coordinates are and The first one that it constitutes pixel length of a skeletal segment The calculation is as follows: Set as The dataset contains a set of skeleton pairs. For any given skeleton pair, the predicted proportion is... Compared to the true proportions They are represented as follows: Because significant size differences between different bone regions can lead to large ranges in proportional values, directly using L1 loss can easily cause the model to overemphasize the error between bone pairs with large proportional differences. This invention employs a logarithmic transformation, enabling the model to fairly learn skeletal structural features of different magnitudes: Skeletal Angle Constraint: The swimming posture of a fish is strictly limited by the physiological bending angle of its skeleton. To ensure the rationality of posture prediction, this invention introduces an angle constraint based on trigonometric function decomposition. The unit direction vector of two adjacent bones sharing the same joint is defined as... and Let the angle between their physiological curvatures be . By extracting the cosine and sine components of the included angle through the dot product and scalar cross product of two-dimensional vectors, respectively, a scale-free and differentiable model of the included angle is achieved. In the formula, For two-dimensional vector dot product, It is a two-dimensional vector scalar cross product, and the combination of the two can uniquely determine the size of the joint angle and the direction of bending.
[0033] Final angle consistency loss Defined as the sum of the differences between the model's predicted value and the true labeled value in these two components: in, It is a collection of adjacent bone joints. The joint bending angle predicted by the model. The true value calculated for the joint angle.
[0034] Total loss function: Therefore, the total loss function As shown in the formula below. By introducing The gradient can propagate back from the global structural bias to the local offset prediction head, thereby forcing the model to establish a global topological cognition of the fish's structure during the training phase, thus improving the model's attitude generation accuracy in complex underwater environments.
[0035] Among them, and This is a hyperparameter used to balance the weights of the two loss terms.
[0036] Accurate keypoint localization in complex underwater scenes faces a dual challenge. First, imaging blurring and contour degradation caused by water scattering significantly increase the difficulty of extracting stable features. Second, the mismatch of supervision signals caused by drastic changes in target scale leads to the following: fish schools are distributed at varying distances in the scene, resulting in significant differences in their pixel size in the image. The fixed-scale static Gaussian kernel supervision signal used in traditional keypoint heatmap regression is difficult to adapt to changes in keypoint size.
[0037] To address the aforementioned issues, inspired by SWAHR, this invention introduces Adaptive Kernel Modulation Branch (AKMB). Its core logic lies in no longer relying on a preset static Gaussian kernel, but instead autonomously predicting kernel parameters that best match the current visual characteristics. For example... Figure 2 As shown, AKMB runs in parallel with the heatmap prediction head, sharing the deep features of the backbone network, and outputs a scale-modulated map that is exactly the same size as the heatmap. The scale-modulated map represents the spatial location of each keypoint k. Predict a scaling factor This factor represents the model's real-time perception of local spatial scale and localization difficulty. Through this factor, the model gains the ability to dynamically modulate the Gaussian kernel standard deviation. When the boundaries of key points of the target are blurred, AKMB expands the effective coverage of the Gaussian kernel by outputting a larger modulation factor, thereby enhancing the model's tolerance to annotation bias. When the target scale changes drastically, AKMB can adaptively adjust the kernel size according to the actual pixel span of the target, thereby eliminating the mismatch between the fixed-scale supervision signal and targets of different scales.
[0038] To effectively implement the kernel modulation strategy during training, directly recalculating the Gaussian distribution incurs significant computational overhead. Therefore, this invention employs a second-order Taylor expansion to approximate the supervisory signal using a polynomial. The standard Gaussian heatmap is known to be defined as... When a scale factor is introduced, the target dynamic heatmap can be represented as: Therefore, the present invention knows and It is a power-law relationship. Therefore, in order to simplify the calculation and improve the gradient stability, this invention may let... And it is expanded at a standard scale. Given the logarithmic distance factor from a pixel to the center of the Gaussian, the dynamic heatmap can be approximated in the following polynomial form: By transforming complex exponentiation operations into efficient multiplication-addition operations, computational efficiency is significantly optimized while preserving the physical meaning of nuclear modulation, providing robust mathematical support for AKMB's real-time perception of visual diffusion.
[0039] The loss function for adaptive kernel modulation consists of two parts. The first is the scaling regularization loss. This is used to constrain scale prediction and prevent the model from increasing the kernel range indefinitely to reduce the difficulty of fitting. Secondly, there is the loss of dynamic heatmap. Since the kernel modulation process alters the coverage area of the foreground region, it further exacerbates sample imbalance. Therefore, an integrated weight adaptation mechanism is employed to dynamically reconstruct the regression loss by constructing a spatial weight matrix. Among them, the balance weight Drawing inspiration from the design principles of Focal Loss, this approach automatically adjusts the loss contribution based on the difference between the truth heatmap and the predicted response. Therefore, AKMB can automatically reduce the weight of easily learnable background samples, allowing the model to focus on difficult keypoint regions. Through AKMB's kernel modulation mechanism, the model's keypoint detection accuracy in complex underwater environments is systematically improved.
[0040] Example 2 The present invention also provides a fish posture estimation device, comprising: The first processing module is used to collect data by selecting different fish densities under different light intensities to obtain the original dataset. The second processing module is used to augment the original dataset; The third processing module is used to construct a fish pose key point detection model based on the data augmented original dataset. The fish pose key point detection model is based on a bottom-up approach to realize pose detection. First, it detects the key points of all fish in the image at once, and then obtains the individual pose through cluster association. The fourth processing module is used to input the set of fish images to be detected into the fish pose key point detection model for fish pose estimation.
[0041] As one embodiment of the present invention, the second processing module randomly selects rectangular regions in the training images of the original dataset and covers them with random pixels or average values to simulate the actual situation where the target part is occluded; at the same time, it generates uniform grid-like occlusion on the image to systematically cover local areas with black squares.
[0042] As one embodiment of the present invention, the fish pose keypoint detection model first extracts the depth visual features of the input image through a backbone network, and then predicts the precise pose of the fish body step by step through three progressive stages from coarse to fine. In the first stage, the network predicts the keypoints of the fish body based on a 9+1 keypoint heatmap refined by adaptive kernel modulation. The keypoints correspond to the 9 keypoints and 1 additional local center keypoint labeled in the dataset. In the second stage, the initial pose of each fish is initially estimated using the predicted local center keypoint and its associated offset vector. In the third stage, the network deeply fuses the keypoint heatmap generated in the first stage with the initial pose obtained in the second stage, and further optimizes and outputs the refined fish body pose through a convolutional message passing module and a local-global context adaptive module.
[0043] Example 3 The present invention also provides a fish pose estimation system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program performs a fish pose estimation method when executed by the processor.
[0044] Example 4 The present invention also provides a storage medium storing a computer program that executes a fish pose estimation method during runtime.
[0045] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.
Claims
1. A method for estimating fish pose, characterized in that, include: Step S1: Select different fish densities under different light intensities to collect data and obtain the original dataset; Step S2: Perform data augmentation on the original dataset; Step S3: Based on the data-augmented original dataset, construct a fish pose keypoint detection model; wherein, the fish pose keypoint detection model realizes pose detection in a bottom-up manner, firstly detecting all keypoints of the fish in the image at once, and then obtaining the individual pose through cluster association; Step S4: Input the set of fish images to be detected into the fish pose key point detection model to estimate the fish pose.
2. The fish pose estimation method as described in claim 1, characterized in that, In step S2, a rectangular region is randomly selected from the training images of the original dataset and covered with random pixels or average values to simulate the actual situation of the target part being occluded; at the same time, a uniform grid occlusion is generated on the image to systematically cover the local area with black squares.
3. The fish pose estimation method as described in claim 2, characterized in that, In step S3, the fish pose keypoint detection model first extracts the depth visual features of the input image through a backbone network, and then predicts the precise pose of the fish body step by step through three progressive stages from coarse to fine. In the first stage, the network predicts the keypoints of the fish body based on the 9+1 keypoint heatmap refined by adaptive kernel modulation. These keypoints correspond to the 9 keypoints and 1 additional local center keypoint labeled in the dataset. In the second stage, the initial pose of each fish is initially estimated using the predicted local center keypoint and its associated offset vector. In the third stage, the network deeply fuses the keypoint heatmap generated in the first stage with the initial pose obtained in the second stage, and further optimizes and outputs the refined fish body pose through the convolutional message passing module and the local-global context adaptation module.
4. A fish posture estimation device, characterized in that, include: The first processing module is used to collect data by selecting different fish densities under different light intensities to obtain the original dataset. The second processing module is used to augment the original dataset; The third processing module is used to construct a fish pose key point detection model based on the data augmented original dataset. The fish pose key point detection model is based on a bottom-up approach to realize pose detection. First, it detects the key points of all fish in the image at once, and then obtains the individual pose through cluster association. The fourth processing module is used to input the set of fish images to be detected into the fish pose key point detection model for fish pose estimation.
5. The fish posture estimation device as described in claim 4, characterized in that, The second processing module simulates the actual situation where the target part is occluded by randomly selecting rectangular regions in the training images of the original dataset and covering them with random pixels or average values; at the same time, it systematically covers local areas with black squares by generating uniform grid-like occlusion on the image.
6. The fish posture estimation device as described in claim 5, characterized in that, The fish pose keypoint detection model first extracts deep visual features from the input image through a backbone network. Then, through three progressive stages from coarse to fine, it gradually predicts the precise pose of the fish. In the first stage, the network predicts the fish's keypoints based on a heatmap of 9+1 keypoints refined by adaptive kernel modulation. These keypoints correspond to the 9 keypoints labeled in the dataset and 1 additional local center keypoint. In the second stage, the network uses the predicted local center keypoint and its associated offset vector to initially estimate the initial pose of each fish. In the third stage, the network deeply fuses the keypoint heatmap generated in the first stage with the initial pose obtained in the second stage. Through a convolutional message passing module and a local-global context adaptation module, it further optimizes and outputs the refined fish pose.
7. A fish pose estimation system, characterized in that, include: A memory and a processor, wherein the memory stores a computer program executed by the processor, the computer program performing the fish pose estimation method as described in any one of claims 1-3 when executed by the processor.
8. A storage medium, characterized in that, The storage medium stores a computer program that, when executed, performs the fish pose estimation method as described in any one of claims 1-3.