Binocular vision-based non-contact fish body phenotype character detection and measurement method

Abstract

The invention relates to the technical field of machine vision and deep learning, in particular to a binocular vision-based non-contact fish body phenotype character detection and measurement method. Comprising the steps that four feature points on a fish body trunk are recognized through a key point detection network, and then the modeling stage of the fish body trunk is completed with the feature points as control vertexes based on a third-order Bezier curve; a FishIDM-Net model is proposed, the model is based on YOLOv8, and bounding box parameters of target detection and third-order Bezier curve control point parameters of trunk recognition are output by integrating an EAMRF module and a third-order Bezier curve and expanding a regression branch of a decoupling head; designing a composite loss function; acquiring two-dimensional control point parameters of the trunk of the fish body, and measuring the length of the fish body through a three-dimensional curve integration method in combination with three-dimensional space information provided by the binocular camera; and performing performance evaluation on the model. According to the invention, non-contact detection and measurement of the phenotypic characters of the underwater fish body can be realized.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and deep learning technology, and in particular to a non-contact method for detecting and measuring fish phenotypic traits based on binocular vision. Background Technology

[0002] In the field of factory-scale aquaculture, fish body length, as a key phenotypic trait, is an important quantitative basis for genetic selection and provides basic support for assessing fish growth status and formulating stocking strategies.

[0003] Currently, existing technologies for measuring fish body length mainly fall into two categories: The first is traditional manual contact measurement technology, which involves direct human contact with the fish, and if necessary, anesthetizing the fish to remove it from the water before using measuring instruments to complete the length measurement; the second is computer vision-based measurement technology, with early mainstream methods based on semantic segmentation, which rely on pre-acquired fish masks as prior information, such as... Figure 2 As shown in (a), the task of segmenting individual fish is first completed, and then the body length is measured based on the segmentation results; subsequently, measurement methods based on key point detection have been developed, such as... Figure 2 As shown in (b), body length measurement is aided by detecting key points of fish.

[0004] However, the aforementioned existing technologies have significant shortcomings: Traditional manual contact measurement is cumbersome and time-consuming, making it unsuitable for the high-throughput measurement needs of large-scale aquaculture. Measurement results are easily affected by the subjective factors of the operators, making it difficult to guarantee the accuracy and consistency of the data. Furthermore, anesthetizing the fish and removing them from the water can easily trigger stress responses and cause physical damage, which is detrimental to the healthy growth of the fish. For computer vision-based measurement technologies, semantic segmentation requires fish masks as a prerequisite, but fish segmentation in underwater scenarios faces challenges such as varied postures, dynamic swimming, and complex environments, making implementation difficult. Although key point detection-based methods have made progress, the system stability is insufficient in complex underwater environments and scenarios with dynamically swimming fish, making it difficult to meet the requirements of non-contact, high-throughput, and high-precision measurement. Summary of the Invention

[0005] In view of this, the purpose of this invention is to propose a non-contact fish phenotypic detection and measurement method based on binocular vision, in order to solve the problems of traditional manual contact measurement being cumbersome, time-consuming, and lacking in data accuracy and easily damaging the fish; semantic segmentation methods in computer vision relying on fish masks; and the current key point detection methods having poor stability in complex underwater environments and dynamic swimming fish scenarios.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a non-contact method for detecting and measuring fish phenotypic traits based on binocular vision, comprising the following steps:

[0007] S1. In the trunk modeling stage of underwater fish phenotypic traits, four feature points on the fish trunk are first identified through a key point detection network. Then, based on the third-order Bézier curve modeling method, the feature points are used as control vertices to complete the modeling of the fish trunk.

[0008] S2. In the target detection and trunk recognition stage of underwater fish phenotypic traits, the FishIDM-Net model is proposed. The FishIDM-Net model is based on YOLOv8 and integrates the EAMRF module and the third-order Bézier curve. It also extends the regression branch of the decoupling head to output the detection bounding box parameters and the control point parameters of the third-order Bézier curve.

[0009] S3. For the target detection and torso recognition of the FishIDM-Net model, design a composite loss function that includes a target detection loss term, a torso recognition sampling loss term, and an endpoint loss term;

[0010] S4. Based on the FishIDM-Net model, the two-dimensional third-order Bézier curve control point parameters of the fish body can be obtained. Combined with the three-dimensional spatial information provided by the binocular camera, the fish body length can be measured by the three-dimensional curve integration method.

[0011] S5. Evaluate the performance of the FishIDM-Net model.

[0012] Preferably, the four feature points on the fish's body are the head, the base of the tail fin, and two control points in the middle of the body.

[0013] Preferably, the mathematical expression for the third-order Bézier curve is:

[0014]

[0015] in, Points on a third-order Bézier curve are parametric variables. The function, when t changes from 0 to 1, It will depict the entire third-order Bézier curve; The value ranges from 0 to 1, when At that time, corresponding to the starting point of the curve, when At time t corresponds to the endpoint of the curve, and t determines the position of the point on the curve. It is the starting point of the curve, when hour, ; This represents the first control point, which determines the curve's starting point. The direction of departure and the degree of curvature of the curve The tangent direction at that point points to ; This represents the second control point, which determines when the curve reaches its endpoint. The direction and degree of curvature of the curve The tangent direction at that point comes from ; This indicates the endpoint of the curve, when t=1. = Solid dot and Corresponding to the endpoints of the fish's head and the base of the caudal fin; hollow spots and These are two intermediate control points in the middle of the body. By adjusting the spatial coordinates of these two points, the bending shape of the fish's body can be fitted.

[0016] Preferably, the composite loss function is expressed as follows:

[0017]

[0018] in, , and These are the weighting factors for different loss terms. , , ; For target detection loss terms; For torso recognition sampling loss term; Identify endpoint loss terms for the torso.

[0019] Preferably, the The calculation formula is: Where N is the number of sampling points and N=4, For sampling points, For a true third-order Bézier curve, This indicates that the Euclidean distance method is used for calculation. and The shortest distance between; The calculation formula is: ,in, and To predict the two endpoints of a third-order Bézier curve, and The endpoints of a real third-order Bézier curve. This indicates that the Euclidean distance method is used for calculation. and The distance between, This indicates that the Euclidean distance method is used for calculation. and The distance between them.

[0020] Preferably, the specific steps for measuring the fish body length using the three-dimensional curve integral method are as follows: First, let the parameterized curve equation of the fish body in three-dimensional space be... ,in, For parameters, , , These correspond to coordinate functions along the x, y, and z axes in three-dimensional space, respectively; then, the formula for expressing the fish's body length is determined as follows: In the formula , , They are respectively , , For parameters The first derivative.

[0021] The beneficial effects of this invention are as follows: This invention proposes a FishIDM-Net model integrating third-order Bézier curves and an EAMRF module, combined with a composite loss function, which enables non-contact detection and measurement of underwater fish phenotypic traits, improving the accuracy and robustness of fish target detection and trunk recognition. Simultaneously, it designs a non-contact fish phenotypic trait detection and measurement method based on a "cloud-edge-device" collaborative architecture, providing a practical and effective tool for measuring fish phenotypic traits. This solution precisely addresses the challenges of traditional manual contact measurement, which is cumbersome, time-consuming, and prone to data damage; semantic segmentation methods in computer vision that rely on fish masks; and the poor stability of current keypoint detection methods in complex underwater environments and dynamic swimming fish scenarios. It provides an efficient and reliable phenotypic data acquisition technology for factory-scale aquaculture, with broad application prospects. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the process of the present invention;

[0024] Figure 2 This is a schematic diagram comparing the traditional fish body identification method of this invention with the method described in this paper;

[0025] Figure 3 This is a schematic diagram of the non-contact fish phenotypic trait detection and measurement method based on the "cloud-edge-device" collaborative architecture of the present invention.

[0026] Figure 4This is a schematic diagram of the underwater intelligent measurement device integrating a binocular vision system according to the present invention;

[0027] Figure 5 This is a schematic diagram illustrating the third-order Bézier curve of the present invention and its application in fish body trunk modeling;

[0028] Figure 6 This is a schematic diagram of the FishIDM-Net model framework of the present invention;

[0029] Figure 7 This is a schematic diagram of the sampling / endpoint loss of the present invention;

[0030] Figure 8 This is a schematic diagram of the cloud computing platform architecture of the present invention;

[0031] Figure 9 This is a schematic diagram of the FishIDM-Net model torso recognition dataset of the present invention;

[0032] Figure 10 This is a visual comparison diagram of the FishIDM-Net model of the present invention and two two-stage ensemble methods on test sets 1 and 2.

[0033] Figure 11 This is a visual comparison diagram of the ablation experiment of the present invention;

[0034] Figure 12 This is a schematic diagram showing a comparative analysis of the characteristic heatmaps of the EAMRF module of the present invention;

[0035] Figure 13 This is a schematic diagram of the visual analysis of the receptive field of the EAMRF module in this invention;

[0036] Figure 14 This is a schematic diagram illustrating the accuracy analysis of the calibration parameters of the underwater binocular camera of the present invention;

[0037] Figure 15 This is a schematic diagram of the error distribution in fish body length measurement according to the present invention;

[0038] Figure 16 This is a schematic diagram of the grouper length measurement results using the FishIDM-Net model in this invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains.

[0040] like Figure 1As shown, a non-contact method for detecting and measuring fish phenotypic traits based on binocular vision includes the following steps:

[0041] S1. In the trunk modeling stage of underwater fish phenotypic traits, four feature points on the fish trunk are first identified through a key point detection network. Then, based on the third-order Bézier curve modeling method, the feature points are used as control vertices to complete the modeling of the fish trunk.

[0042] Underwater cultured fish exhibit complex posture changes during free swimming, posing significant challenges to accurate modeling of their torso morphology. Bézier curves, with their unique smoothness and curvature, provide an ideal mathematical modeling tool for torso modeling under different postures. Furthermore, Bézier curves have low computational complexity, meeting the requirements for real-time processing and high-throughput measurement. Figure 5 (a) and Figure 2 As shown in (c), in the fish body length measurement task, the keypoint detection network can identify four feature points on the fish's torso: the head, the base of the caudal fin, and two control points in the middle of the torso. These four points correspond precisely to the four control vertices of a third-order Bézier curve, making this curve an ideal modeling tool. The third-order Bézier curve can effectively simulate various posture changes of the fish's torso, especially in high curvature regions. Its mathematical expression is:

[0043] (1)

[0044] in, Points on a third-order Bézier curve are parametric variables. The function, when t changes from 0 to 1, It will depict the entire third-order Bézier curve; The value ranges from 0 to 1, when At that time, corresponding to the starting point of the curve, when At time t corresponds to the endpoint of the curve, and t determines the position of the point on the curve; for example Figure 5 As shown in (b) It is the starting point of the curve, when hour, ; This represents the first control point, which determines the curve's orientation from the starting point. The direction of departure and the degree of curvature of the curve The tangent direction at that point points to ; This represents the second control point, which determines when the curve reaches its endpoint. The direction and degree of curvature of the curve The tangent direction at that point comes from ; This indicates the endpoint of the curve, when t=1. = Solid dot and Corresponding to the endpoints of the fish's head and the base of the caudal fin; hollow spots and These are two intermediate control points in the middle of the body. By adjusting the spatial coordinates of these two points, the bending shape of the fish's body can be fitted.

[0045] S2. In the target detection and trunk recognition stage of underwater fish phenotypic traits, the FishIDM-Net model is proposed. The FishIDM-Net model is based on YOLOv8, and integrates the EAMRF module and the third-order Bézier curve, and extends the regression branch of the decoupled head to output the detection bounding box parameters and the control point parameters of the third-order Bézier curve.

[0046] In factory farming environments, the visual scene of the East Star Grouper is complex, posing two major challenges to target detection and body recognition: first, the high color similarity between individuals makes identification difficult; second, the significant scale difference between the scene captured by underwater measuring equipment and the target makes it difficult to implement full-map key point detection and curve modeling.

[0047] In response to the above problems, such as Figure 6 As shown, this invention proposes the FishIDM-Net (Fish Identification with Dynamic Multi-scale Network) model, which simultaneously achieves fish body recognition and detection as well as curve prediction. Based on YOLOv8, this network achieves comprehensive sharing of parameters and features across multiple tasks through seamless lightweight integration of the EAMRF (Efficient Adaptive Multi-scale Receptive Field) module and third-order Bézier curves. The fish body recognition and detection results of FishIDM-Net are shown below. Figure 2 (d) and Figure 2 As shown in (e), object detection, as a core task, can alleviate the scale difference between the scene and the object, and distinguish the object from the background with low annotation requirements, providing accuracy assurance for subsequent curve detection and modeling. This is in stark contrast to the design of traditional multi-task networks where branches are independent and only share the backbone weights. The pseudocode of the research process proposed in this invention is shown in Algorithm 1, which describes in detail the overall implementation steps of the FishIDM-Net network, including key steps such as input preprocessing, feature extraction, multi-scale receptive field calculation, third-order Bézier curve fitting, and final detection result output.

[0048]

[0049] in, This represents the number of pixels in the vertical direction of the image. This represents the number of pixels in the horizontal direction of the image. The average pixel value of the image. The standard deviation of the image pixels. For target detection loss term and by The regression / classification loss decision, These are cubic Bernstein basis functions. For sampling points, The true third-order Bézier curve of the fish body. To calculate using the Euclidean distance method arrive The shortest distance.

[0050] (1) Multi-task fusion mechanism based on decoupling head output extension

[0051] In underwater target detection, the bounding box localization regression branch contains rich target feature representations. This invention employs a simple and efficient method to fuse target detection and curve modeling tasks by modifying the output of this regression branch. Figure 6 As shown in (a), in FishIDM-Net, the regression branches of multiple decoupled heads are expanded to incorporate Bezier parameter predictions. The new regression output contains two parts: one is the four original regression parameters of the fish target bounding box, corresponding to the center coordinates. The parameters are: 1) width and height; and 2) eight parameters used for backbone modeling, corresponding to the coordinates of four control points of the third-order Bézier curve. The classification branch in the decoupled head retains the original form of YOLOv8, with a single output representing the probability of fish target presence. It's important to note that fish target detection is inherently challenging due to complex background interference and the similarity in fish appearances; the introduction of torso recognition further increases the complexity of the joint learning problem. To achieve accurate torso recognition, curve detection and modeling place higher demands on the encoder's spatial context feature representation capabilities and cross-spatial learning abilities. Therefore, this invention proposes an EAMRF module, which constructs a multi-feature joint perception mechanism by fusing depth and color information, enhancing the model's feature perception and spatial reasoning capabilities in complex underwater environments.

[0052] (2) EAMRF module

[0053] In underwater target detection and phenotypic measurement tasks, accurately identifying fish with different phenotypic traits and postures is a key prerequisite for improving bounding box localization accuracy, curve parameter prediction accuracy, and classification performance. Small-scale features such as fish color and depth information can efficiently capture local details, providing support for the accurate measurement of key phenotypic traits; while large-scale features such as the overall outline and posture of the fish construct a global structural representation. This complementarity and synergy among multi-scale features lays a solid foundation for subsequent high-precision detection, morphological fitting, and phenotypic measurement tasks, thereby improving the overall detection robustness and measurement accuracy of the system.

[0054] While the C2f (Cross-Stage Partial Connection with 2 Convolutional Layers) module in traditional backbone networks enhances target detection performance through cross-stage feature extraction, it lacks an adaptive attention mechanism for key points such as the torso, head, and tail fin base of fish with complex poses, making it difficult to meet the requirements for fine-grained detection. Meanwhile, traditional multi-scale processing methods, such as FPN (Feature Pyramid Network), can capture multi-scale information, but require additional network layers, leading to increased computational cost.

[0055] Inspired by the EMA (Efficient Multi-Scale Attention) module, this invention proposes an innovative cross-space learning general method and a multi-scale parallel sub-network to construct the EAMRF module, such as... Figure 6 As shown in (b), this module effectively avoids the dimensionality reduction problem of traditional convolution operations by reshaping some channel dimensions into batch dimensions. In the keypoint estimation task, it uses multi-path parallel branches combined with the EMA module to capture features at different scales: the large-scale branch achieves rapid localization of key regions, while the medium and small-scale branches gradually and accurately locate the regions. Finally, by weighted fusion of prediction results at different scales, higher-precision localization is achieved. Compared with traditional methods, this multi-branch parallel processing mechanism improves localization accuracy while optimizing computational efficiency.

[0056] The calculation process for EAMRF is as follows:

[0057] (2)

[0058] in, Represents the input feature map, This represents the number of intermediate channels. For space dimensions, The dimension is the output after the first convolution operation, Groups( ) indicates that Dividing by "group", Split means splitting the feature map of a group into two parts, resulting in and , and This represents the two sub-feature maps after splitting.

[0059] (3)

[0060] in, Indicates an iterative index; Indicates the first After secondary processing Initially ; This represents the bottleneck module, used to extract higher-level features; Indicates the first After processing by the secondary bottleneck module .

[0061] (4)

[0062] in, This indicates the first convolution operation, used for dimensionality reduction; This indicates the second convolution operation, used for dimensionality increase; This indicates a residual connection, which will connect the input... The result is added to the result of the two convolutions to avoid gradient vanishing and enhance the feature representation capability.

[0063] (5)

[0064] in, It was through After each Bottleneck module Cat([ ]) indicates channel cascading, which will and splicing along the channel dimension; This is the last convolution operation, used for fusion. and Based on the characteristics, adjust the number of channels and output the final Y.

[0065] S3. For the target detection and torso recognition of the FishIDM-Net model, design a composite loss function that includes a target detection loss term, a torso recognition sampling loss term, and an endpoint loss term.

[0066] As an end-to-end learning framework, the FishIDM-Net model needs to simultaneously perform object detection and torso recognition based on keypoint prediction. Its training performance is highly dependent on an effective loss function design. For the object detection task, there are already well-defined and validated loss functions. The core challenge of the torso recognition task lies in how to accurately quantify the deviation between the predicted curve and the actual curve.

[0067] In torso recognition and length measurement tasks, third-order Bézier curves exhibit dual characteristics. On the one hand, slight deviations at the midpoints of the curve do not compromise the overall integrity, demonstrating a degree of robustness, such as... Figure 7 As shown in (a); on the other hand, small offsets at the endpoints can affect the length measurement results, such as Figure 7 As shown in (b), it exhibits vulnerability. Based on this characteristic, as... Figure 7 As shown in (c), this study designs a composite loss function for the torso recognition task, which includes a fish torso sampling loss term and an endpoint loss term, in order to specifically optimize the curve prediction accuracy.

[0068] (1) Sampling loss

[0069] Sampling loss is used to measure the average deviation of the predicted third-order Bézier curve from the true third-order Bézier curve in terms of overall shape. For example... Figure 7 As shown in (a), considering the robustness of the third-order Bézier curve to small deviations in the intermediate point, the following sampling strategy is adopted: N points are sampled on the predicted third-order Bézier curve based on Gaussian distribution. In this paper, N=4. The shortest distance from each sampling point to the real third-order Bézier curve is calculated. The real third-order Bézier curve is defined by the real control point. The sampling loss is the average of the distances of all sampling points.

[0070] Specifically, a set of points are distributed for sampling. For each sampling point The Euclidean distance method is used to calculate the nearest true third-order Bézier curve. shortest distance The sampling loss is the average distance of all sampling points, as shown in the formula below:

[0071] (6)

[0072] The sampling loss function It can effectively capture the overall shape difference between the predicted curve and the true curve, and is not sensitive to slight shifts in the intermediate control points, which is consistent with the robust characteristics of third-order Bézier curves.

[0073] (2) Endpoint loss

[0074] Endpoint loss is specifically designed to penalize deviations in the predicted third-order Bézier curve at its endpoints. Since endpoint coordinates directly participate in subsequent length calculations, even small errors in endpoint position can lead to significant biases in length estimation. Figure 7 As shown in (b), stronger constraints need to be imposed on the endpoint positions. The endpoint loss is calculated as the sum of the distances between the two endpoints of the predicted curve and the corresponding endpoints of the true curve. The endpoint loss is calculated for the two endpoints of the predicted third-order Bézier curve. and endpoints corresponding to the real third-order Bézier curve and Distance between:

[0075] (7)

[0076] in, This indicates that the Euclidean distance method is used for calculation. and The distance between, This indicates that the Euclidean distance method is used for calculation. and The distance between endpoints. The endpoint loss function directly penalizes endpoint position errors. It effectively mitigates the negative impact of endpoint deviation on curve fitting and final length measurement results, and compensates for the vulnerability of the third-order Bézier curve at the endpoints.

[0077] (3) Composite loss function

[0078] The overall end-to-end learning loss function of FishIDM-Net consists of a torso recognition loss based on a third-order Bézier curve and a general object detection loss from the YOLOv8 detector, as expressed below:

[0079] (8)

[0080] in, , and These are the weighting factors for different loss terms; see Algorithm 1 for specific values.

[0081] S4. Based on the FishIDM-Net model, the control point parameters of the two-dimensional third-order Bézier curve of the fish body can be obtained. Combined with the three-dimensional spatial information provided by the binocular camera, the body length of the fish is measured by the three-dimensional curve integration method.

[0082] The FishIDM-Net described above can be used to easily obtain the two-dimensional curve features of the fish's body. Combined with the three-dimensional spatial information provided by a binocular camera, the length of the fish in the underwater environment can be accurately measured using the three-dimensional curve integration method.

[0083] Specifically, the length of the fish's torso in three-dimensional space can be measured using parametric curve integration. Let the parametric curve equation of the fish's torso in three-dimensional space be:

[0084] (9)

[0085] in, For parameters, , , These correspond to the coordinate functions along the x, y, and z axes in three-dimensional space, respectively. The curve is obtained by converting the control points of the two-dimensional third-order Bézier curve output by FishIDM-Net into depth information obtained from binocular vision. The fish body length is the arc length of this three-dimensional curve from the starting point P(0) to the ending point P(3), and its mathematical expression is shown in formula (10):

[0086] (10)

[0087] In the formula , , They are respectively , , For parameters The first derivative of the curve represents the rate of change of the curve in each dimension. By solving the integral using numerical integration, the actual body length of the fish can be obtained. This measurement method based on three-dimensional curve integration can fully capture the spatial curvature characteristics of the fish's body, providing a high-precision quantitative means for measuring the body length of farmed fish.

[0088] S5. Evaluate the performance of the FishIDM-Net model.

[0089] (1) Research findings

[0090] We propose a non-contact method for detecting and measuring fish phenotypic traits based on a "cloud-edge-device" collaborative architecture. The overall process is as follows: Figure 3 As shown. This system encompasses three layers: perception, algorithms, and data. At the perception layer, i.e., the edge, such as... Figure 3 (a) and Figure 4 As shown, the underwater intelligent measurement device integrating a binocular vision system developed by our research group is used to acquire multimodal images and depth information of the fish in real time; at the algorithm layer, i.e., the edge layer, such as... Figure 3 (h) Figure 3 (i) and Figure 3As shown in (j), to address the challenges of low underwater image quality, complex environments, and variable target poses, a FishIDM-Net model integrating deep learning and 3D vision is proposed. This model performs end-to-end target detection, pose estimation, and keypoint localization, enabling non-contact 3D calculation of fish body length. The data and service layer, i.e., the cloud side, is as follows: Figure 3 As shown in (g), the integrated management, analysis, and visualization of measurement data are achieved through a cloud computing platform, supporting breeding decisions and aquaculture regulation. Among these, Figure 3 (b) indicates an underwater binocular camera. Figure 3 (c) indicates a low-power distribution box. Figure 3 (d) indicates the deployment site of underwater intelligent equipment. Figure 3 (e) represents an image of a fish captured by an underwater detection device. Figure 3 (f) indicates a remote monitoring platform.

[0091] Among them, the cloud computing platform developed a project display interface based on HTML and CSS, such as Figure 8 As shown, this interface uses JSON data storage and HTTP services for local transmission, supporting real-time data visualization. After identifying a target, users can intuitively view real-time video footage and depth images. The system can also statistically analyze the number and average length of target objects in the current frame, and provide statistical charts and parameters of the historical total number of targets, historical average length, and individual target length. Furthermore, we have developed a Flask Application Programming Interface (API) to interface with the system. When administrators access this interface, in aquaculture scenarios, the API and Flask API will use corresponding identifiers and provide ecological information about the fish in the current scene. This system has already been successfully applied in modern fishery industrial parks. Building upon this foundation, we have further constructed a complete intelligent monitoring system for factory farming, including a fish trait measurement system, a factory farming environment and water quality parameter monitoring and abnormal water quality prediction system, a factory farming fish behavior monitoring and abnormal behavior identification system, a factory farming disease monitoring and predictive diagnosis system, a data management and system management module, and a factory farming digital twin system. This forms a complete chain solution from data acquisition and intelligent analysis to decision support, providing technical support for the digital and intelligent transformation of aquaculture.

[0092] To verify the accuracy of the FishIDM-Net model in extracting fish morphological feature point parameters, this study constructed two benchmark datasets for comprehensive evaluation. Dataset 1 was collected from a modern fishery industrial park aquaculture base, using underwater intelligent measurement equipment to capture images in a factory aquaculture environment. It covers typical aquaculture water conditions with natural lighting and natural swimming behavior of fish. This dataset contains 725 RGB images and annotated 2036 grouper samples, which can be directly used to verify the accuracy of third-order Bézier curve fitting and keypoint detection. Dataset 2 is a grouper body trunk keypoint detection dataset covering diverse underwater scenes, including small, medium, and large groupers, reflecting morphological characteristics at different growth stages. A total of 5652 fish were annotated by adding trunk labels to adapt to the task. The two datasets were divided into training / validation sets and test sets, and compared to Dataset 1, Dataset 2 can comprehensively test the model's adaptability and stability in complex environments.

[0093] (2) Benchmark dataset

[0094] Based on the aforementioned datasets, this study conducted body length measurement experiments using the proposed method and compared the model performance with manual measurement results. For the fish torso recognition task, both datasets employed a weak annotation strategy to reduce annotation costs. Detailed statistical information regarding sample size, body length range, and scene type is shown in Table 1. Examples of the two datasets are provided below. Figure 9 As shown.

[0095]

[0096] Where Dataset represents the dataset, and Train / validation represents the training / validation set. Indicates the number of images. The number of fish is represented by "Test", the test set by "Test", the range of fish length by "The range of fish length", the type of scene by "Type of scene", and the resolution by "Resolution".

[0097] (3) Weak annotations used to identify the main branch

[0098] By leveraging the capabilities of third-order Bézier curve modeling, this study eliminates the need for pixel-level annotated mask labels on the grouper's torso; the torso morphology can be described using only a few key points on the curve. This weak annotation method avoids tedious pixel-level labeling, reduces annotation costs, and fully represents the torso curve with a small number of key points, offering the advantages of both cost-effectiveness and information completeness.

[0099] Weak annotation only requires labeling the bounding box and key points of the grouper's body, and the number of these key points can be flexibly adjusted. Considering the diversity of the fish's body posture, such as bending and twisting, four key points are a reasonable choice. This effectively captures the bending features of the body while avoiding excessive labeling burden, achieving a balance between modeling accuracy and labeling efficiency.

[0100] (4) Experimental environment configuration

[0101] To ensure the repeatability and accuracy of the experiments, all key model experiments in this paper were conducted under the same hardware and software configuration. Specific computational configurations are shown in Table 2. The model training process was set to 300 epochs on both datasets 1 and 2, with a fixed batch size of 8. The optimizer used was AdamW, with an initial learning rate of 1e-3 and a weight decay coefficient of 5e-4. The learning rate scheduling strategy employed was cosine annealing, periodically adjusting the learning rate to avoid the model getting trapped in local optima. To ensure the consistency and reproducibility of the experiments, all hyperparameters remained consistent throughout the training process on both datasets.

[0102]

[0103] In terms of data augmentation, random cropping, random horizontal flipping, and mosaic enhancement techniques are used. Random cropping can improve the model's adaptability to targets of different scales, random horizontal flipping increases sample diversity by introducing mirror transformation, and mosaic enhancement expands scene complexity by stitching together multiple images. The three work together to enhance the model's generalization ability.

[0104] (5) Evaluation indicators

[0105] For quantitative evaluation of the grouper target detection task, the official evaluation metrics recommended by the YOLO series were adopted, namely the mean Average Precision (mAP) of the Intersection over Union (IoU) threshold, specifically mAP50 and mAP50-95. Here, mAP50 represents the mAP of the fish detection box when the IoU threshold is 0.5; mAP50-95 represents the mAP of the fish detection box when the IoU threshold is between 0.5 and 0.95. These two metrics comprehensively reflect the model's accuracy in detecting groupers of different scales and poses.

[0106] For the evaluation of the grouper's body curve, since its annotation is based on keypoints on a third-order Bézier curve, the alignment between the predicted and ground truth curves can be measured through point set correspondence. Specifically, 50 equidistant points are uniformly sampled on each curve, and the official COCO metrics are used for evaluation, including mAP (middle accuracy) of keypoint detection and the Percentage of Correct Keypoints (PCK). PCK, by setting a reasonable distance threshold, typically a certain proportion of the diagonal length of the target detection box, statistically measures the proportion of predicted and ground truth keypoints whose distances fall within the threshold range, thus intuitively reflecting the accuracy of keypoint localization.

[0107] (6) Comparative Experiment

[0108] Since curve detection in complex aquaculture scenarios is a relatively new research area, a mature benchmark framework for comparative analysis has not yet been established. Therefore, this study uses the YOLOv8 backbone network CSPDarknet as the internal comparison model for FishIDM-Net, and introduces two integrated two-stage top-down frameworks as external comparison methods: Faster-RCNN+RTMpose and RTMdet+RTMpose. The reasons for choosing these two frameworks are as follows: First, besides the YOLO series, Faster-RCNN and RTMdet are representative deep models in the field of object detection, capable of covering the performance characteristics of different detection paradigms; second, RTMpose, as an advanced pose estimation framework, handles keypoint localization through classification tasks, and can form an effective two-stage collaboration with the aforementioned detectors. The workflow of these two integrated frameworks is as follows: first, the detector generates target bounding boxes, then the grouper within the boxes is uniformly scaled to provide standardized input for subsequent third-order Bézier curve detection. The specific configurations of the above comparison methods are detailed in Table 3. All comparative models were implemented on the OpenMMLab platform and trained on datasets 1 and 2. All methods, including the proposed FishIDM-Net, used the same training / validation and test set partitioning method to ensure the fairness of the comparative experiments and the comparability of the results.

[0109]

[0110] Where Model is the model name, Backbone is the backbone network, Time is the time, Total Loss is the total loss value during training, Params is the number of model parameters, FLOPs is the floating-point operation cost per image, and FPS is the number of image frames that can be processed per second.

[0111] Table 4 presents the quantitative comparison results of FishIDM-Net single-stage and two two-stage ensemble methods on test sets 1 and 2, evaluating performance in grouper detection and body recognition, such as... Figure 10 As shown.

[0112]

[0113] In this study, "Object detection" refers to target detection, and "Fish identification" refers to torso recognition. For parametric modeling of the fish torso, this research compares three mainstream curve modeling methods: B-spline, polyline, and Bézier curve. Considering the practical needs and computational efficiency of target detection, this experiment uniformly uses four control points for parametric modeling of the fish torso.

[0114] B-spline curves offer excellent local control and support for arbitrary order, theoretically providing flexible shape control. However, under the constraint of four control points, their local control advantage is difficult to fully realize, and additional node vector calculations are required. The polyline method is simple, intuitive, and fast, but lacks smoothness and cannot accurately describe the natural curvature of the fish's body with only four control points. Third-order Bézier curves naturally adapt to designs with four control points, possessing endpoint interpolation properties that can precisely locate the fish's head and tail while maintaining the smooth curve of the fish's body, and offering relatively stable calculations.

[0115] To objectively evaluate the performance of different methods, this study conducted comparative experiments from multiple dimensions, including computational complexity, fitting accuracy, training efficiency, and detection performance. The experimental results are shown in Table 5. Under the condition of using four control points uniformly, the computational complexity of the B-spline curve is O(n), with 450 training iterations to converge; the computational complexity of the piecewise linear segment method is O(n), with 280 training iterations to converge; and the computational complexity of the third-order Bézier curve is O(n), with 320 training iterations to converge.

[0116]

[0117] in, This represents the mAP of key points on the fish body when the IoU threshold is 0.5. Comparative experiments on detection accuracy show that the third-order Bézier curve method performs best in all indicators; moreover, in terms of fitting accuracy, the third-order Bézier curve is superior to the B-spline curve and the broken line segment.

[0118] Considering factors such as computational efficiency, modeling accuracy, and parameter settings, the third-order Bézier curve method performs optimally under the constraint of four control points. Its endpoint interpolation characteristics and natural four-point design make it more suitable for parametric modeling tasks of fish torsos. While ensuring high accuracy, this method exhibits moderate computational complexity and good training stability. Therefore, this study adopts the third-order Bézier curve as the primary modeling method for subsequent experiments.

[0119] (7) Ablation test

[0120] To verify the effectiveness of the key improvements and designs in FishIDM-Net, this study conducted a series of ablation experiments for more comprehensive validation. The evaluation results on datasets 1 and 2 are shown in Table 6. The data show that the complete model outperforms all ablation models in the torso recognition task, providing strong evidence for the effectiveness of EAMRF, trunk sampling loss, and endpoint loss.

[0121]

[0122] Here, "Ablations" means ablation, "-w / o-Sampleloss" means removing the Sampleloss module, "-w / o-EAMRF" means removing the EAMRF module, "-w / o-Endpointloss" means removing the Endpointloss module, and "full" means the FishIDM-Net full model.

[0123] As shown in Table 6, Sampleloss and EAMRF not only improve the accuracy of describing the backbone features, but also effectively enhance the overall perception capability of the target region. The introduction of these key enhancement modules enables the model to achieve coordination and co-optimization of target detection and torso recognition tasks within a unified framework, avoiding performance deviations that may be caused by single-task optimization.

[0124] Visual examples of ablation studies, such as Figure 11 As shown, the impact of each module on the curve prediction results is presented intuitively. The following is a detailed analysis of the role of each key module:

[0125] ①Sampleloss ablation study

[0126] This study proposes using SampleLoss as the loss function in FishIDM-Net. Unlike conventional loss functions, SampleLoss adjusts the loss weights based on spatial information in the feature map, paying particular attention to regions that help establish local spatial dependencies. In the grouper detection scenario, the fish morphology is relatively simple, but factors such as turbid water and complex backgrounds pose significant challenges to detection. The application of SampleLoss allows the model to focus more on the spatial structure of the fish and its surrounding environment, effectively suppressing background interference and thus improving the accuracy of target detection and body recognition under complex conditions.

[0127] Experimental results show that on dataset 1, the complete FishIDM-Net model exhibits significant advantages: compared to the model with sample loss removed, it shows improved performance in both object detection and torso recognition tasks. Figure 11 (a) and Figure 11 As can be seen from the visual comparison in (b), the complete model including sample loss generates more comprehensive and better-fitting torso prediction results, indicating its stronger ability to perceive local dependencies. On dataset 2, the advantage of the complete model over the model without sample loss is mainly reflected in the torso recognition task, further verifying the effectiveness of sample loss in enhancing spatial feature representation.

[0128] ②EAMRF Ablation Study

[0129] The EAMRF module, a key component for improving the neck region of the framework, integrates two-dimensional global average pooling and channel-level attention mechanisms. Through this module, the network can transfer attention information between channels, establish cross-spatial dependencies, and thus achieve simultaneous fish target detection and body recognition.

[0130] Figure 12 This presentation showcases the comparative visualization of feature heatmaps for 12 typical fish samples. Each group includes the original input image, the feature activation map after adding the EAMRF module, and the feature activation map after removing the EAMRF module. The comparison clearly shows that after adding EAMRF, the fish body region in the heatmap exhibits a bright, high-activation response, such as... Figure 12 The bright green / yellow areas and low activation intensity in the background region effectively achieved feature focusing and target-background separation. In contrast, after removing the EAMRF module, feature activation became scattered and irregular, the distinction between the target and background decreased significantly, and the activation regions lacked specificity.

[0131] Figure 13 This further demonstrates the impact of the EAMRF module on the network receptive field. Figure 13 (a) and Figure 13(c) The visualization results of the receptive fields after adding the EAMRF module to datasets 1 and 2 are shown respectively. It can be observed that the receptive fields exhibit a regular striped distribution pattern, which is beneficial for capturing the elongated geometric features of the fish body. Figure 13 (b) and Figure 13 (d) shows the receptive field distribution on the two datasets after removing the EAMRF module. The receptive fields are relatively scattered and irregular, lacking a clear structured pattern, and the feature perception ability is significantly weakened.

[0132] Ablation experiments further validated the effectiveness of the EAMRF module in fish target detection and torso recognition tasks, with particularly outstanding performance in torso recognition. On Dataset 1, the FishIDM-Net model improved the mAP50 and mAP50-95 metrics for torso recognition by more than 3.5% and 2.8% respectively compared to the model without EAMRF, while also improving the PCK score by 4%. On Dataset 2, EAMRF also demonstrated significant improvement in torso recognition, proving its crucial role in enhancing the network's feature representation capabilities.

[0133] ③Ablation study of Endpoint loss

[0134] Endpoint loss, as an auxiliary loss term, plays a crucial role in reinforcing the network's constraint on the accuracy of endpoint prediction. As the results of the two-stage method demonstrate, ... Figure 9 (a) Even a small deviation at the endpoints can affect the overall accuracy of the curve.

[0135] Experiments verified this conclusion: when the Endpointloss term is removed, the model faces greater challenges in accurate curve localization, such as... Figure 10 (a) and Figure 10 As shown in (d), the mAP50, mAP50-95, and PCK scores are all lower than those of the full model in the torso recognition task on both datasets. Furthermore, as previously emphasized, there is a trade-off between object detection loss and curve prediction loss within the ensemble framework. The evaluation results in Table 6 for the full model and the model with Endpointloss removed show that the performance improvement brought by the trunk endpoint loss does not come at the expense of object detection accuracy. This finding further confirms the good compatibility of this loss term with existing loss functions and learning strategies, indicating that it can specifically improve endpoint prediction accuracy without interfering with the main task.

[0136] (8) Binocular camera calibration and image correction

[0137] This invention employs Zhang Zhengyou's planar checkerboard camera calibration method to calibrate and correct acquired stereo images. The stereo camera calibration is performed using the Stereo Camera Calibrator module in Matlab, which can directly calculate the complete set of calibration results, including intrinsic parameter matrices, distortion coefficients, rotation and translation matrices, etc. During the calibration image processing, reprojection error data for each pair of images is acquired, and unreliable data is manually removed to improve calibration accuracy. Finally, 30 pairs of images are selected as the calibration results, such as... Figure 14 (a) shows a subset of images. The average error of these 30 image pairs is 0.1 pixels, indicating that the calibration process and algorithm flow have been well implemented. Figure 14 As shown in (b), the calibration results accurately reconstructed the three-dimensional structure of the actual scene, and the accuracy of the position recovery results directly verified the accuracy of the calibration. Figure 14 As shown in (c), the final calibration results show that the average error of all images is 0.15 pixels. The calibration results are shown in Table 7.

[0138] In this study, the distortion of the corrected image was significantly reduced, and the corresponding pixels of the left and right images in the corrected image were on the same horizontal line. The stereoscopic image has achieved basic consistency in correction, which lays the foundation for subsequent disparity calculation and 3D coordinate solving.

[0139]

[0140] During the measurement phase, the intrinsic parameters of the binocular camera are first used to map the coordinates of the grouper trunk curve identified by FishIDM-Net to the three coordinate system generated by the depth map. Then, the length of the grouper trunk is calculated by integrating along the spatial curve. The specific calculation method is as described in equation (10). Figure 16 The paper presents some measurement examples and corresponding depth maps. To evaluate the accuracy of the fish body measurement process, the measurement errors in the experiment are statistically analyzed, and the FishIDM-Net model is used to evaluate the process performance, so as to fully verify the performance of models of different scales in length measurement tasks.

[0141] Measurement error distribution results are as follows Figure 15 As shown, where Figure 15 (a) shows the relative error distribution of FishIDM-Net on dataset 1, fitted with a Gaussian function, and labeled with mean error and standard deviation; Figure 15 (b) shows the absolute and relative error distribution and cumulative distribution of FishIDM-Net on dataset 1; Figure 15 (c) shows the relative error distribution of FishIDM-Net on dataset 2, fitted with a Gaussian function, and the mean error and standard deviation are labeled. Figure 15(d) shows the absolute and relative error distributions and cumulative distributions of FishIDM-Net on dataset 2.

[0142] relative error and absolute relative error The definition is as follows:

[0143] (11)

[0144] (12)

[0145] in, To measure length, This corresponds to the actual length.

[0146] Based on the absolute and relative error distribution, the cumulative probability at different error levels is further calculated and plotted as a curve, such as... Figure 15 (b) and Figure 15 As shown in (d), the results show that the relative errors of FishIDM-Net on both datasets are approximately normally distributed, with the mean error close to zero, indicating no systematic bias. The mean absolute errors are 0.154 and 0.168, respectively, with over 80% of the sample errors below 0.30. These results validate that the proposed method has high measurement accuracy and robustness, enabling highly robust body length measurements.

[0147] The tail of the error distribution shows that a small number of samples have relatively large measurement errors. Analysis indicates that the main reasons include: the depth information acquired by the binocular camera is prone to sudden changes and distortions such as depth gaps. Figure 16 (d) Figure 16 (h) Figure 16 As shown in (i), this affects measurement accuracy; the depth information of the pixel containing the curve is unstable, and the curve may exceed the effective area of ​​the fish body, leading to fluctuations in measurement accuracy; blind spots exist in the fish body at some viewpoints, such as... Figure 16 (a) This can also reduce measurement accuracy. However, as mentioned above, these problems can be effectively solved by averaging multiple measurements to obtain reliable fish length data.

[0148] Figure 16 The paper presents the results of grouper length measurement using the FishIDM-Net model. Each sample includes a front view captured by a stereo camera and its corresponding depth map. For a batch of grouper bodies with a length range of 7-9 cm, the measurement error is less than 0.30. Furthermore, the introduction of EAMRF successfully corrected some failures in trunk prediction, such as... Figure 16 As shown in (g) and 16(h), this further demonstrates the important role of this module in enhancing cross-spatial feature correlation and improving curve prediction accuracy.

[0149] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention is limited to these examples; within the framework of the invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity.

[0150] This invention aims to cover all such substitutions, modifications, and variations that fall within the scope of protection. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A non-contact method for detecting and measuring fish phenotypic traits based on binocular vision, characterized in that, Includes the following steps: S1. In the trunk modeling stage of underwater fish phenotypic traits, four feature points on the fish trunk are first identified through a key point detection network. Then, based on the third-order Bézier curve modeling method, the feature points are used as control vertices to complete the modeling of the fish trunk. S2. In the target detection and trunk recognition stage of underwater fish phenotypic traits, the FishIDM-Net model is proposed. The FishIDM-Net model is based on YOLOv8 and integrates the EAMRF module and the third-order Bézier curve. It also extends the regression branch of the decoupled head to output the detection bounding box parameters and the control point parameters of the third-order Bézier curve. S3. For the target detection and torso recognition of the FishIDM-Net model, design a composite loss function that includes a target detection loss term, a torso recognition sampling loss term, and an endpoint loss term; S4. Based on the FishIDM-Net model, the two-dimensional third-order Bézier curve control point parameters of the fish body can be obtained. Combined with the three-dimensional spatial information provided by the binocular camera, the fish body length is measured by the three-dimensional curve integration method. S5. Evaluate the performance of the FishIDM-Net model.

2. The method according to claim 1, characterized in that, The four characteristic points on the fish's body are the head, the base of the tail fin, and two control points in the middle of the body.

3. The method according to claim 1 or 2, characterized in that, The mathematical expression for the third-order Bézier curve is: in, Points on a third-order Bézier curve are parametric variables. The function, when t changes from 0 to 1, It will depict the entire third-order Bézier curve; The value ranges from 0 to 1, when At that time, corresponding to the starting point of the curve, when At time t corresponds to the endpoint of the curve, and t determines the position of the point on the curve. It is the starting point of the curve, when hour, ; This represents the first control point, which determines the curve's starting point. The direction of departure and the degree of curvature of the curve The tangent direction at that point points to ; This represents the second control point, which determines when the curve reaches its endpoint. The direction and degree of curvature of the curve The tangent direction at that point comes from ; This indicates the endpoint of the curve, when t=1. = Solid dot and Corresponding to the endpoints of the fish's head and the base of the caudal fin; hollow spots and These are two intermediate control points in the middle of the body. By adjusting the spatial coordinates of these two points, the bending shape of the fish's body can be fitted.

4. The method according to claim 1, characterized in that, The expression for the composite loss function is as follows: in, , and These are the weighting factors for different loss terms. , , ; For target detection loss terms; For torso recognition sampling loss term; Endpoint loss terms are identified for the torso.

5. The method according to claim 4, characterized in that, The The calculation formula is: Where N is the number of sampling points and N=4, For sampling points, For a true third-order Bézier curve, This indicates that the Euclidean distance method is used for calculation. and The shortest distance between; The calculation formula is: ,in, and To predict the two endpoints of a third-order Bézier curve, and The endpoints of a real third-order Bézier curve. This indicates that the Euclidean distance method is used for calculation. and The distance between, This indicates that the Euclidean distance method is used for calculation. and The distance between them.

6. The method according to claim 1, characterized in that, The specific steps for measuring fish body length using the three-dimensional curve integral method are as follows: First, let the parameterized curve equation of the fish body in three-dimensional space be... ,in, For parameters, , , These correspond to coordinate functions along the x, y, and z axes in three-dimensional space, respectively; then, the formula for expressing the fish's body length is determined as follows: In the formula , , They are respectively , , For parameters The first derivative.

Images