A multi-fish-species identification and length measurement method and system based on depth information

Abstract

The application provides a multi-fish-species identification and length measurement method based on depth information, relates to the technical field of aquaculture, and comprises the following steps: RGB-D multi-modal data acquisition and physical alignment; feature extraction based on a C2f_Faster lightweight network; pose solution based on a Pose_LSCD exclusive decoding head; three-dimensional space back projection and physical length pre-solution; multi-frame time sequence queue caching and joint filtering output. The lightweight reconstruction feature extraction network provided by the application is compared and optimized with a traditional YOLO network and a standard deep convolution network, the model parameter quantity and the calculation complexity are reduced by about 30%, the edge algorithm bottleneck is broken through, the multi-target concurrent processing efficiency is improved, the low-delay, high-concurrent operation demand of the aquatic processing flow line is met, and the hardware procurement and energy consumption cost are reduced.

Description

Technical Field

[0001] This invention relates to the technical field of aquaculture, and more particularly to a method and system for multi-fish species identification and length measurement based on depth information. Background Technology

[0002] Fish body length is a key indicator for assessing the growth status of aquaculture, processing and sorting, and related scientific research statistics. In land-based aquatic product processing scenarios (such as pallet sorting and conveyor belt operations), achieving rapid and accurate multi-target measurement of live fish without contact is a hot research topic in the industry. Currently, existing technologies in this field mainly fall into two categories:

[0003] Existing technology 1: Traditional measurement method based on two-dimensional image processing and semi-contact components 1. Chinese invention patent CN112017200A (A method for determining fish posture based on visual recognition) proposes to obtain grayscale and binary images, perform ellipse fitting on the fish body contour, and take its major axis as the fish body central axis to determine the posture.

[0004] 2. Chinese invention patent CN112146731A (a component for rapid non-destructive determination of fish body length and weight) proposes to use a flexible wire mesh to laterally and flexibly press and fix the fish placed in a fish container, and then read the fish body length and acquire images through a length measuring ruler.

[0005] Existing limitations: This type of method based on traditional 2D vision and mechanical components has obvious application limitations. 1. Traditional image processing (such as ellipse fitting) is easily affected by water stains on the surface of the fish and complex backgrounds. Furthermore, when a live fish struggles to dehydrate, its body will be severely bent, causing the fitting centerline to fail.

[0006] 2. Even using semi-contact methods such as pressing with flexible wire mesh can still cause stress reactions in live fish; more fatally, pure two-dimensional images lack Z-axis depth information, so when fish of different thicknesses are tilted or flipped, simple two-dimensional pixel calculations will produce huge perspective distortion errors.

[0007] Existing Technology 2: Automatic Length Measurement Method Based on Deep Learning and Computer Vision In recent years, to overcome the shortcomings of traditional methods, measurement methods that incorporate artificial intelligence and depth / binocular cameras have emerged in the industry: 1. Chinese invention patent CN112070736B (A visual measurement method for object volume combining target detection and depth calculation) proposes to use the YOLO model to detect targets and obtain two-dimensional coordinates, and then combine it with a depth camera to obtain the depth information of specified pixels to calculate the size of the object.

[0008] 2. Chinese invention patent CN117029673A (A method for measuring multiple dimensions of fish body surface based on artificial intelligence) proposes to collect fish images containing a "scale bar", label them, and then train a deep learning model to calculate the fish body size.

[0009] 3. Chinese invention patent CN111862048A (Automatic analysis method for fish posture and length based on key point detection and deep convolutional neural network) proposes to estimate fish posture and length by acquiring images through underwater binocular cameras and combining deep convolutional neural network and key point detection ideas.

[0010] Existing shortcomings: Although the aforementioned existing technologies incorporate deep learning, keypoint analysis, or deep computing, they reveal the following three core technical problems that urgently need to be solved when facing real-world aquaculture production lines (high-frequency concurrency, mixed fish species, limited computing power of edge devices, and dynamic jumping of live fish): 1. The network architecture is large and redundant, failing to overcome the bottleneck of edge computing power: Existing technologies (such as CN112070736B and CN111862048A) mostly directly apply standard object detection frameworks or deep convolutional neural networks. The large number of standard convolutions stacked in their feature extraction backbone results in extremely high parameter count and computational complexity (FLOPs). When deployed on edge computing terminals (such as Jetson OrinNano) commonly used in processing plants with severely limited computing power, multi-target concurrency will cause serious inference latency, which cannot meet the millisecond-level real-time requirements of pipelines.

[0011] 2. Over-reliance on reference points and lack of specific spatial decoding mechanisms for non-rigid fish: Live fish exhibit non-rigid distortion in dehydration trays. Existing technologies (such as CN117029673A) mandate the inclusion of a physical scale in the background; however, this two-dimensional scaling immediately fails when the fish has a certain thickness or is leaping. Furthermore, existing keypoint technologies often employ general decoders and global hybrid training, lacking specific optimizations (such as LSCD decoding) for the large-span spatial semantics of slender aquatic products ("head-to-tail"), making them highly susceptible to keypoint feature confusion and location drift when multiple fish species are densely stacked.

[0012] 3. The single-frame exposure mechanism has poor noise resistance and lacks temporal calibration: the mucus on the surface of aquatic organisms can cause specular reflections, leading to flying spots on the depth camera, and live fish exhibit respiratory fluctuations and stress-induced jumping. The aforementioned depth or binocular measurement schemes all rely on real-time predictions from a single frame for 3D back-projection. This mechanism is extremely sensitive to depth noise, and due to the lack of time-series-based multi-frame joint verification and smoothing algorithms, the final output body length and classification data are prone to drastic random jumps and "flickering," resulting in extremely poor robustness for engineering applications.

[0013] In summary, the industry urgently needs a multi-fish species adaptive length measurement method and system that can completely break free from the constraints of physical scale, simultaneously solve the computing power bottleneck of edge devices, possess high-precision head and tail key point positioning capabilities, and eliminate depth noise through temporal filtering. Summary of the Invention

[0014] In view of the technical problems mentioned in the background section, a method and system for multi-fish species identification and length measurement based on depth information is provided.

[0015] The technical means employed in this invention are as follows: A method for multi-fish species identification and length measurement based on depth information includes the following steps: Step 1: RGB-D multimodal data acquisition and physical alignment; Simultaneously acquire color images and depth images, and use the alignment module to map the spatial coordinates of the depth image to the physical coordinate system of the color image to generate a pixel-by-pixel aligned RGB-D fusion frame; Step 2: Feature extraction based on C2f_Faster lightweight network; Input the color image into the reconstructed lightweight backbone network and neck network, optimize the channel reuse and gradient truncation logic of cross-stage local networks through C2f_Faster lightweight network, and complete deep semantic feature extraction while reducing the number of parameters and floating-point operations, and output multi-scale fused feature map. Step 3: Perform pose calculation based on the Pose_LSCD dedicated decoding head; input the multi-scale fused feature map into the Pose_LSCD pose decoding head, and after channel adjustment and feature compression are completed through convolutional layers, it is sent to the target detection branch, category prediction branch and key point prediction branch respectively. In a single forward propagation, the two-dimensional bounding box of the target, the confidence of the fish species category, the two-dimensional pixel coordinates of the fish mouth key point and the fish tail key point are output simultaneously, and the effective detection results are retained by filtering through confidence threshold and non-maximum suppression; Step 4: 3D spatial back projection and physical length pre-calculation; extract the depth values ​​corresponding to the key points of the fish mouth and tail, remove invalid depth values ​​with zero values, missing values, and abrupt changes, and when the depth values ​​of both key points are valid, combine the RGB-D depth camera calibration intrinsic parameters to back project the two-dimensional pixel coordinates to the three-dimensional camera coordinate system to obtain the three-dimensional coordinates of the fish mouth and tail, and calculate the physical straight line length of the fish body based on the three-dimensional space Euclidean distance formula; Step 5: Multi-frame temporal queue buffering and joint filtering output; calculate the Euclidean distance between the center points of the target boxes in adjacent frames to complete the tracking of fish target IDs, determine the same fish target and use the corresponding ID; establish historical cache queues for the fish species classification results and fish length results with the same ID, and after the queues reach the preset frame number threshold, perform mode filtering on the fish species category sequence and median filtering on the length sequence to output stable final results of fish species category and fish physical length.

[0016] Further, step 3 includes the following steps: First, multi-layer convolutional layers are used to adjust the channel dimension and compress redundant features of multi-scale fusion feature maps of different scales; then, the compressed features are input into three parallel branches, where the target detection branch predicts the position parameters of the fish body target box, the category prediction branch outputs the classification confidence of each fish species, and the key point prediction branch outputs the image coordinates of key points of the fish mouth and tail; finally, according to the feature map stride, the predicted offset is restored to the color image pixel coordinate system to complete the decoding of the detection results.

[0017] Furthermore, in step 4, the camera calibration intrinsic parameters include the lateral focal length. Longitudinal focal length x-coordinate of principal point and principal point ordinate ; The formula for calculating the 3D coordinates of the key points of the fish mouth is: ; ; ; The three-dimensional coordinates of the key points of the fish mouth are obtained as follows: ; The formula for calculating the 3D coordinates of the key point of the fish tail is: ; ; ; The three-dimensional coordinates of the key point of the fish tail are obtained as follows: ; The formula for calculating the physical straight-line length of a fish's body is: .

[0018] Furthermore, in step 4, the invalid depth value is determined as follows: if the depth value corresponding to the fish mouth key point or the fish tail key point is zero, the data is missing, or the depth value has an isolated abrupt change compared to the depth values ​​of the surrounding neighboring pixels, then the depth value is determined to be an invalid depth value, the length calculation result of the corresponding fish body target in the current frame is removed, and it is not stored in the length history cache queue.

[0019] Furthermore, in step 5, the fish target ID tracking and determination includes the following steps: The pixel coordinates of the center point of the bounding box are calculated by using the upper and lower boundaries and the left and right boundaries of the target's two-dimensional bounding box. The pixel Euclidean distance between the center points of the candidate targets of the same category in the current frame and the previous frame is then calculated. When the pixel Euclidean distance is less than the preset center point offset threshold, the target is determined to be the same fish and the historical ID is used. When the pixel Euclidean distance is greater than or equal to the preset center point offset threshold, the target is determined to be a brand new fish and a new ID is generated.

[0020] The present invention also includes a multi-fish species identification and length measurement system based on depth information, comprising: The data acquisition and alignment module is used to simultaneously acquire color and depth image streams, complete physical image alignment, and generate RGB-D fused frames. The feature extraction module, equipped with the C2f_Faster lightweight network, is used to extract deep semantic features from images and output multi-scale fused feature maps. The pose calculation module has a built-in Pose_LSCD dedicated decoding head, which is used to output the target bounding box, fish species confidence score, and two-dimensional pixel coordinates of key points of the fish mouth and tail; The 3D back projection and length calculation module is used to filter effective depth values, combine camera intrinsic parameters to complete 3D back projection of 2D coordinates, and calculate the physical length of the fish. The temporal filtering and output module is used to realize fish target tracking, temporal data caching and filtering processing, and output the final identification and length measurement results; The model training module is used to complete the cascaded training of the C2f_Faster lightweight network and the Pose_LSCD decoder head to optimize the model's detection accuracy.

[0021] Furthermore, the timing filtering and output module includes a target tracking unit, a data buffering unit, and a filtering output unit; The target tracking unit is used to calculate the Euclidean distance between the center points of the target boxes in adjacent frames, complete the fish target matching, and maintain the consistency of the same fish ID; The data caching unit is used to establish a category caching queue and a length caching queue for fish with different IDs, and to store valid detection data in chronological order. The filtering output unit is used to perform mode filtering and median filtering on the cache queue data to remove accidental abnormal detection results.

[0022] Furthermore, the three-dimensional back projection and length calculation module incorporates a depth verification unit, a coordinate back projection unit, and a length calculation unit. The depth verification unit is used to identify and remove invalid depth data with zero values, missing values, or abrupt changes. The coordinate back projection unit is used to call the camera calibration intrinsic parameters to convert the two-dimensional pixel coordinates into three-dimensional spatial coordinates in the camera coordinate system. The length calculation unit is used to accurately calculate the physical straight-line length of the fish body based on the three-dimensional Euclidean distance formula.

[0023] Compared with the prior art, the present invention has the following advantages: 1. The lightweight reconstructed feature extraction network provided by this invention, through comparison and optimization with the traditional YOLO network and standard deep convolutional networks, reduces the number of model parameters and computational complexity by about 30%, breaks through the edge computing power bottleneck, improves the efficiency of multi-objective concurrent processing, meets the low latency and high concurrency operation requirements of aquatic product processing lines, and reduces hardware procurement and energy consumption costs.

[0024] 2. The exclusive decoding constraint combined with the cascaded training process provided by this invention, through comparison and optimization with the general key point localization algorithm, realizes the constraint of the output of key points of the head and tail of aquatic organisms, avoids feature conflicts, improves the target localization and three-dimensional length measurement accuracy under distorted and dense working conditions, and solves the key point localization drift problem.

[0025] 3. The multi-frame temporal collaborative filtering mechanism provided by this invention, through comparison and optimization with the traditional single-frame instant processing mechanism, achieves the filtering of most depth noise and misidentified signals, eliminates three-dimensional depth jumps, and greatly improves the robustness of equipment operation in complex industrial environments.

[0026] In summary, the technical solution of this invention addresses the shortcomings of existing technologies, such as limited computing power of edge terminals, low accuracy in locating and measuring non-rigid aquatic targets, high noise levels in depth signals under complex conditions, and poor system stability. These shortcomings are addressed through optimizations using lightweight network reconstruction, cascaded training strategies, and multi-frame temporal filtering mechanisms. Therefore, the technical solution of this invention solves the problems of high deployment barriers for edge hardware, high latency in multi-target processing, aquatic target positioning drift, low measurement accuracy, and poor algorithm robustness in complex environments.

[0027] Based on the above reasons, this invention can be widely applied in fields such as intelligent aquatic product processing, live aquatic product inspection, industrial visual 3D measurement, and edge intelligent recognition. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0030] Figure 2 This is a schematic diagram of the improved target detection and attitude estimation network logic structure of the present invention.

[0031] Figure 3 This is a flowchart of the cascaded training strategy for resolving feature conflicts of non-rigid targets according to the present invention.

[0032] Figure 4 This is a schematic diagram illustrating the principle of the two-dimensional pixel to three-dimensional spatial coordinate back projection mapping based on the camera intrinsic parameter matrix of this invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] like Figures 1-4 As shown, this invention provides a method for multi-fish species identification and length measurement based on depth information, including the following steps: Step 1: RGB-D multimodal data acquisition and physical alignment; Simultaneously acquire color images and depth images, and use the alignment module to map the spatial coordinates of the depth image to the physical coordinate system of the color image to generate a pixel-by-pixel aligned RGB-D fusion frame; Step 2: Feature extraction based on C2f_Faster lightweight network; Input the color image into the reconstructed lightweight backbone network and neck network, optimize the channel reuse and gradient truncation logic of cross-stage local networks through C2f_Faster lightweight network, and complete deep semantic feature extraction while reducing the number of parameters and floating-point operations, and output multi-scale fused feature map. Step 3: Perform pose calculation based on the Pose_LSCD dedicated decoding head; input the multi-scale fused feature map into the Pose_LSCD pose decoding head, and after channel adjustment and feature compression through convolutional layers, send it to the target detection branch, category prediction branch, and keypoint prediction branch respectively. In a single forward propagation, output the target's two-dimensional bounding box, fish species category confidence, fish mouth keypoint, and fish tail keypoint two-dimensional pixel coordinates simultaneously. Valid detection results are retained by filtering through confidence thresholding and non-maximum suppression. This includes the following steps: First, use multiple convolutional layers to adjust the channel dimensions and compress redundant features of the multi-scale fused feature map at different scales; then, input the compressed features into three parallel branches respectively, where the target detection branch predicts the fish body target box position parameters, the category prediction branch outputs the fish species classification confidence, and the keypoint prediction branch outputs the fish mouth and fish tail keypoint image coordinates; finally, according to the feature map stride, restore the predicted offset to the color image pixel coordinate system to complete the decoding of the detection results.

[0036] Step 4: 3D spatial backprojection and physical length pre-calculation; extract the depth values ​​corresponding to the key points of the fish mouth and tail, and remove invalid depth values ​​with zero values, missing values, or abrupt changes (if the depth value corresponding to the key point of the fish mouth or tail is zero, the data is missing, or the depth value has an isolated abrupt change compared to the depth values ​​of the surrounding neighboring pixels, then the depth value is determined to be an invalid depth value, and the length calculation result of the corresponding fish body target in the current frame is removed and not stored in the length history cache queue). When both key point depth values ​​are valid, combine the RGB-D depth camera calibration intrinsic parameters to backproject the two-dimensional pixel coordinates to the three-dimensional camera coordinate system, obtain the three-dimensional coordinates of the fish mouth and tail, and calculate the physical straight line length of the fish body based on the three-dimensional spatial Euclidean distance formula.

[0037] In this application, as a preferred embodiment, in step 4, the camera calibration intrinsic parameters include the lateral focal length. Longitudinal focal length x-coordinate of principal point and principal point ordinate ; The formula for calculating the 3D coordinates of the key points of the fish mouth is: ; ; ; The three-dimensional coordinates of the key points of the fish mouth are obtained as follows: ; The formula for calculating the 3D coordinates of the key point of the fish tail is: ; ; ; The three-dimensional coordinates of the key point of the fish tail are obtained as follows: ; The formula for calculating the physical straight-line length of a fish's body is: .

[0038] Step 5: Multi-frame temporal queue buffering and joint filtering output; calculate the Euclidean distance between the center points of the target boxes in adjacent frames to complete the tracking of fish target IDs, determine the same fish target and use the corresponding ID; establish historical cache queues for the fish species classification results and fish length results with the same ID, and after the queues reach the preset frame number threshold, perform mode filtering on the fish species category sequence and median filtering on the length sequence to output stable final results of fish species category and fish physical length.

[0039] In a preferred embodiment, the fish target ID tracking and determination in this application includes the following steps: The pixel coordinates of the center point of the bounding box are calculated by using the upper and lower boundaries and the left and right boundaries of the target's two-dimensional bounding box. The pixel Euclidean distance between the center points of the candidate targets of the same category in the current frame and the previous frame is then calculated. When the pixel Euclidean distance is less than the preset center point offset threshold, the target is determined to be the same fish and the historical ID is used. When the pixel Euclidean distance is greater than or equal to the preset center point offset threshold, the target is determined to be a brand new fish and a new ID is generated.

[0040] The present invention also includes a multi-fish species identification and length measurement system based on depth information, comprising: The data acquisition and alignment module is used to simultaneously acquire color and depth image streams, complete physical image alignment, and generate RGB-D fused frames. The feature extraction module, equipped with the C2f_Faster lightweight network, is used to extract deep semantic features from images and output multi-scale fused feature maps. The pose calculation module has a built-in Pose_LSCD dedicated decoding head, which is used to output the target bounding box, fish species confidence score, and two-dimensional pixel coordinates of key points of the fish mouth and tail; The 3D back projection and length calculation module is used to filter effective depth values, combine camera intrinsic parameters to complete 3D back projection of 2D coordinates, and calculate the physical length of the fish. The 3D back projection and length calculation module has a built-in depth verification unit, coordinate back projection unit, and length calculation unit. The depth verification unit is used to identify and remove invalid depth data with zero values, missing values, or abrupt changes. The coordinate back projection unit is used to call the camera calibration intrinsic parameters to convert the two-dimensional pixel coordinates into three-dimensional spatial coordinates in the camera coordinate system. The length calculation unit is used to accurately calculate the physical straight-line length of the fish body based on the three-dimensional Euclidean distance formula.

[0041] The time-series filtering and output module is used to realize fish target tracking, time-series data caching and filtering processing, and output the final identification and length measurement results; the time-series filtering and output module includes a target tracking unit, a data caching unit and a filtering output unit; The target tracking unit is used to calculate the Euclidean distance between the center points of the target boxes in adjacent frames, complete the fish target matching, and maintain the consistency of the same fish ID; The data caching unit is used to establish a category caching queue and a length caching queue for fish with different IDs, and to store valid detection data in chronological order. The filtering output unit is used to perform mode filtering and median filtering on the cache queue data to remove accidental abnormal detection results.

[0042] The model training module is used to complete the cascaded training of the C2f_Faster lightweight network and the Pose_LSCD decoder head to optimize the model's detection accuracy.

[0043] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the above embodiments of the present invention, the descriptions of each embodiment have their own emphasis; parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. It should be understood that the disclosed technical content in the several embodiments provided in this application can be implemented in other ways.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for multi-fish species identification and length measurement based on depth information, characterized in that, Includes the following steps: Step 1: Acquisition and physical alignment of RGB-D multimodal data; Simultaneously acquire color and depth images, and use an alignment module to map the spatial coordinates of the depth image to the physical coordinate system of the color image to generate a pixel-by-pixel aligned RGB-D fused frame; Step 2: Feature extraction based on C2f_Faster lightweight network; Input the color image into the reconstructed lightweight backbone network and neck network, optimize the channel reuse and gradient truncation logic of cross-stage local networks through C2f_Faster lightweight network, and complete deep semantic feature extraction while reducing the number of parameters and floating-point operations, and output multi-scale fused feature map. Step 3: Perform pose calculation based on the Pose_LSCD dedicated decoding head; input the multi-scale fused feature map into the Pose_LSCD pose decoding head, and after channel adjustment and feature compression are completed through convolutional layers, it is sent to the target detection branch, category prediction branch and key point prediction branch respectively. In a single forward propagation, the two-dimensional bounding box of the target, the confidence of the fish species category, the two-dimensional pixel coordinates of the fish mouth key point and the fish tail key point are output simultaneously, and the effective detection results are retained by filtering through confidence threshold and non-maximum suppression; Step 4: 3D spatial back projection and physical length pre-calculation; Extract the depth values ​​corresponding to the key points of the fish mouth and tail, remove invalid depth values ​​with zero values, missing values, and abrupt changes. When the depth values ​​of both key points are valid, combine the RGB-D depth camera calibration intrinsic parameters to back-project the two-dimensional pixel coordinates to the three-dimensional camera coordinate system to obtain the three-dimensional coordinates of the fish mouth and tail. Calculate the physical straight line length of the fish body based on the three-dimensional space Euclidean distance formula. Step 5: Multi-frame temporal queue buffering and joint filtering output; The Euclidean distance between the center points of the target boxes in adjacent frames is calculated to complete the tracking of the fish target ID. The same fish target is identified and the corresponding ID is used. Historical cache queues are established for the fish species classification results and fish length results with the same ID. After the queues reach the preset frame number threshold, mode filtering is performed on the fish species category sequence and median filtering is performed on the length sequence to output the final results of stable fish species category and fish physical length.

2. The method for multi-fish species identification and length measurement based on depth information according to claim 1, characterized in that, Step 3 includes the following steps: First, multi-layer convolutional layers are used to adjust the channel dimension and compress redundant features of multi-scale fusion feature maps of different scales; then, the compressed features are input into three parallel branches, where the target detection branch predicts the position parameters of the fish body target box, the category prediction branch outputs the classification confidence of each fish species, and the key point prediction branch outputs the image coordinates of the key points of the fish mouth and tail; finally, according to the feature map stride, the predicted offset is restored to the color image pixel coordinate system to complete the decoding of the detection results.

3. The method for multi-fish species identification and length measurement based on depth information according to claim 1, characterized in that, In step 4, the camera calibration intrinsic parameters include the lateral focal length. Longitudinal focal length x-coordinate of principal point and principal point ordinate ; The formula for calculating the 3D coordinates of the key points of the fish mouth is: ； ； ； The three-dimensional coordinates of the key points of the fish mouth are obtained as follows: ; The formula for calculating the 3D coordinates of the key point of the fish tail is: ； ； ； The three-dimensional coordinates of the key point of the fish tail are obtained as follows: ; The formula for calculating the physical straight-line length of a fish's body is: 。 4. The method for multi-fish species identification and length measurement based on depth information according to claim 1, characterized in that, In step 4, the invalid depth value is determined as follows: if the depth value corresponding to the fish mouth key point or the fish tail key point is zero, the data is missing, or the depth value has an isolated abrupt change compared to the depth values ​​of the surrounding neighboring pixels, then the depth value is determined to be an invalid depth value, the length calculation result of the corresponding fish body target in the current frame is removed, and it is not stored in the length history cache queue.

5. The method for multi-fish species identification and length measurement based on depth information according to claim 1, characterized in that, Step 5, the fish target ID tracking and determination, includes the following steps: The pixel coordinates of the center point of the bounding box are calculated by using the upper and lower boundaries and the left and right boundaries of the target's two-dimensional bounding box. The pixel Euclidean distance between the center points of the candidate targets of the same category in the current frame and the previous frame is then calculated. When the pixel Euclidean distance is less than the preset center point offset threshold, the target is determined to be the same fish and the historical ID is used. When the pixel Euclidean distance is greater than or equal to the preset center point offset threshold, the target is determined to be a brand new fish and a new ID is generated.

6. A multi-fish species identification and length measurement system based on depth information, using the method described in any one of claims 1-5, characterized in that, include: The data acquisition and alignment module is used to simultaneously acquire color and depth image streams, complete physical image alignment, and generate RGB-D fused frames. The feature extraction module, equipped with the C2f_Faster lightweight network, is used to extract deep semantic features from images and output multi-scale fused feature maps. The pose calculation module has a built-in Pose_LSCD dedicated decoding head, which is used to output the target bounding box, fish species confidence score, and two-dimensional pixel coordinates of key points of the fish mouth and tail; The 3D back projection and length calculation module is used to filter effective depth values, combine camera intrinsic parameters to complete 3D back projection of 2D coordinates, and calculate the physical length of the fish. The temporal filtering and output module is used to realize fish target tracking, temporal data caching and filtering processing, and output the final identification and length measurement results; The model training module is used to complete the cascaded training of the C2f_Faster lightweight network and the Pose_LSCD decoder head to optimize the model's detection accuracy.

7. A multi-fish species identification and length measurement system based on depth information according to claim 6, characterized in that, The timing filtering and output module includes a target tracking unit, a data buffering unit, and a filtering output unit; The target tracking unit is used to calculate the Euclidean distance between the center points of the target boxes in adjacent frames, complete the fish target matching, and maintain the consistency of the same fish ID; The data caching unit is used to establish a category caching queue and a length caching queue for fish with different IDs, and to store valid detection data in chronological order. The filtering output unit is used to perform mode filtering and median filtering on the cache queue data to remove accidental abnormal detection results.

8. A multi-fish species identification and length measurement system based on depth information according to claim 6, characterized in that, The three-dimensional back projection and length calculation module has a built-in depth verification unit, coordinate back projection unit and length calculation unit; The depth verification unit is used to identify and remove invalid depth data with zero values, missing values, or abrupt changes. The coordinate back projection unit is used to call the camera calibration intrinsic parameters to convert the two-dimensional pixel coordinates into three-dimensional spatial coordinates in the camera coordinate system. The length calculation unit is used to accurately calculate the physical straight-line length of the fish body based on the three-dimensional Euclidean distance formula.

Images