A deep learning-based intelligent on-ship sorting model for portunid crabs
By using a deep learning-based shipboard intelligent sorting model for swimming crabs, the problems of statistical bias and insufficient detection accuracy in complex backgrounds of traditional manual measurement methods have been solved. This model enables accurate counting and weight calculation of swimming crabs, and allows for the assessment of fishing intensity and resource abundance.
Patent Information
- Application Number
- CN202510712761.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-30
- Publication Date
- 2025-12-09
- Estimated Expiration
- 2045-05-30
AI Technical Summary
Existing technologies for sorting swimming crabs suffer from statistical biases and a lack of refined quality classification in traditional manual measurement methods. Deep learning models also lack sufficient detection accuracy in complex backgrounds, making them difficult to adapt to actual fishing scenarios.
A deep learning-based shipboard intelligent sorting model for swimming crabs is adopted, including a shipboard intelligent detection model, a weight calculation model, and a real-time counting system. By utilizing YOLO-DFAM and ASF-YOLO modules, dynamic scale calibration, multi-frame confidence update, ByteTrack tracking algorithm, and other technologies, the detection accuracy and weight calculation accuracy are improved.
It effectively improves the detection accuracy of swimming crabs in complex sea conditions, and can accurately count swimming crabs of various sizes, calculate their weight, and assess fishing intensity and resource abundance.
Smart Images
Figure CN120635144B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of sorting models, and particularly relates to a shuttle crab shipborne intelligent sorting model based on deep learning. BACKGROUND
[0002] The current practical difficulties of shuttle crab sorting are that the traditional manual measurement method has statistical deviation and cannot make fine division according to the quality of the catch. In recent years, computer vision-based automatic detection systems are gradually replacing manual measurement (Li et al., 2025). For example, in the measurement of the fishing intensity of shrimp and fish and other aquatic products, the electronic monitor has been used to replace the traditional manual monitoring method (Y. Sun et al., 2024), but there is still a lack of intelligent grading system based on quality (such as weight and size) for high-value species such as shuttle crabs. The existing shuttle crab deep learning weight measurement method is mostly trained and tested in a laboratory environment (Chen et al., 2025), and lacks adaptability to actual fishing scenes. In actual application, there are complex backgrounds and dynamic changes on the conveyor belt of the fishing ship, such as sea waves, changes in light, etc., which puts higher requirements on the robustness and accuracy of the model. Therefore, how to effectively apply deep learning models to actual fishing scenes and solve the problem of small target detection in complex backgrounds has become a hot and difficult point of current research.
[0003] The main technical methods for fishing operation detection at home and abroad include traditional image processing methods such as background extraction (Liu et al., 2024), edge detection (Zheng et al., 2024) and morphological processing (Zhao, Qin, Xu, Yu, & Chen, 2025). These methods usually have low precision and are easily disturbed by environmental light and noise, and are prone to miss detection or false detection of small targets (such as shuttle crabs) in complex scenes. Another is the deep learning method represented by convolutional neural network (CNN), which can realize accurate detection and tracking of fishing objects, and has significantly improved precision compared with traditional methods. SUMMARY
[0004] In view of the deficiencies of the above prior art, the purpose of the application is to provide a shuttle crab shipborne intelligent sorting model based on deep learning, which effectively improves the detection accuracy of shuttle crabs in complex sea conditions, not only can accurately count shuttle crabs of various sizes, but also can calculate the weight of shuttle crabs, and evaluate the fishing intensity and resource abundance.
[0005] The application provides a shuttle crab shipborne intelligent sorting model based on deep learning, which comprises a shipborne intelligent detection model, a weight calculation model and a real-time counting system,
[0006] The shipborne intelligent detection model is used for detecting whether it is a Portunus trituberculatus, and if the detection result is Portunus trituberculatus, the full carapace width of the Portunus trituberculatus is detected;
[0007] The body weight calculation model is used for updating the full carapace width of the Portunus trituberculatus, determining the body weight of the Portunus trituberculatus according to the updated full carapace width of the Portunus trituberculatus, and grading the Portunus trituberculatus according to the body weight;
[0008] The real-time counting system is used for counting the Portunus trituberculatus of different body weight grades;
[0009] The shipborne intelligent detection model is a YOLO-DFAM model, the YOLO-DFAM model integrates a FocalModulation module and an ASF-YOLO module, and the ASF-YOLO module integrates attention scale fusion into a YOLO framework;
[0010] The body weight calculation model further includes a dynamic scale calibration module and a multi-frame confidence updating module;
[0011] The real-time counting system improves a ByteTrack tracking algorithm, the improvement includes an exponential weighted moving average filtering module, an angle data processing module, a motion compensation module, and a dynamic confidence threshold adjustment module based on target size and local density, and the motion compensation module adopts a particle filtering and local motion compensation technology based on optical flow.
[0012] Further, in the above-mentioned shipborne intelligent sorting model for Portunus trituberculatus based on deep learning, the working method of the FocalModulation module includes:
[0013] The multi-scale deep convolution captures context features from local details to global semantics, and constructs a pyramid multi-granularity representation;
[0014] The dynamic gating generator is used for realizing adaptive fusion of cross-scale features, and generates selection weights based on a spatial-channel dual attention mechanism in the adaptive fusion process, so that the YOLO-DFAM model can focus on key regions;
[0015] The element-wise affine transformation injects context features from local details to global semantics into original features.
[0016] Further, in the above-mentioned shipborne intelligent sorting model for Portunus trituberculatus based on deep learning, the working method of the ASF-YOLO module includes:
[0017] The scale sequence feature fusion module captures the multi-scale characteristics of Portunus trituberculatus in various scenes on the deck;
[0018] The triple feature encoder module fuses deep and shallow layer features;
[0019] The channel-position attention mechanism dynamically focuses on the blue-gray carapace region and joint key points of the swimming crab.
[0020] Further, in the above-mentioned deep learning-based intelligent on-ship sorting model for swimming crabs, the working method of the dynamic scale calibration module comprises:
[0021] A plurality of sets of calibration point data are set in the vertical direction of the video frame, and a pixel-centimeter conversion relationship is established for the plurality of sets of calibration point data;
[0022] The plurality of sets of calibration point data after the conversion relationship is established are input into a cubic spline interpolator to construct a continuous spatial scale function.
[0023] Further, in the above-mentioned deep learning-based intelligent on-ship sorting model for swimming crabs, the working method of the multi-frame confidence updating module comprises:
[0024] The confidence of the same swimming crab is compared in consecutive frames;
[0025] When the comparison result is that the confidence of the same swimming crab in the current frame is higher than that in the previous frame, the carapace width and body weight of the swimming crab are updated according to the corresponding image of the current frame.
[0026] Further, in the above-mentioned deep learning-based intelligent on-ship sorting model for swimming crabs, the model formula for determining the body weight of the swimming crab according to the updated carapace width of the swimming crab is:
[0027] W=a×(L) b
[0028]
[0029] wherein L represents the updated carapace width of the swimming crab, a=0.0514±0.0008, b=3.0158±0.0123, W obs represents the actual measured body weight value of the swimming crab, W represents the body weight prediction value output by the model, represents the average value of the actual measured body weight of the swimming crab, R 2 represents the determination coefficient, which is the coefficient for evaluating the body weight model of the swimming crab, i=1……n, and represents the number of swimming crabs.
[0030] Further, in the above-mentioned deep learning-based intelligent on-ship sorting model for swimming crabs, the working method of the exponential weighted moving average filtering module comprises:
[0031] In each frame, the coordinates of the four corners of the current detection box, the center coordinates of the detection box, and the smoothed estimation value of the previous frame are weighted and fused according to a set smoothing coefficient, and the center coordinates, width, and height of the target swimming crab are smoothed and updated.
[0032] Further, in the above-mentioned deep learning-based intelligent sorting model for swimming crab on ship, the working method of the processing angle data module comprises:
[0033] The rotation angle adopts a continuous frame smoothing technique, and the angle data is processed by a rolling window or an exponential weighted moving average filtering mode.
[0034] Further, in the above-mentioned deep learning-based intelligent sorting model for swimming crab on ship, the working method of the motion compensation module comprises:
[0035] The particle filter generates a plurality of possible states according to the historical motion trajectory, detects the coordinates of the four corners of the detection frame, assigns a weight to the center coordinates of the detection frame, and predicts the possible position of the target swimming crab in the next frame;
[0036] The local motion compensation calls an image processing function, estimates the real movement of the target swimming crab based on the displacement of the pixels in the target region of the adjacent frames, and corrects the position of the detection frame in real time.
[0037] Further, in the above-mentioned deep learning-based intelligent sorting model for swimming crab on ship, the working method of the dynamic confidence threshold adjustment module based on the target size and local density comprises:
[0038] When extracting the coordinates of the four corners of the detection frame of each target swimming crab and the center coordinates of the detection frame, it is determined whether it is a swimming crab, and the size information and the density information are combined in the process of determining whether it is a swimming crab to dynamically adjust whether to track the corresponding swimming crab.
[0039] The beneficial effects of the present application are as follows: the present application provides a deep learning-based intelligent sorting model for portunid crabs on a ship, which comprises a shipborne intelligent detection model, a body weight calculation model and a real-time counting system. The shipborne intelligent detection model is used to detect whether it is a portunid crab. If the detection result is a portunid crab, the full carapace width of the portunid crab is detected. The body weight calculation model is used to update the full carapace width of the portunid crab, determine the body weight of the portunid crab according to the updated full carapace width of the portunid crab, and grade the portunid crab according to the body weight. The real-time counting system is used to count the portunid crabs of different weight grades. The shipborne intelligent detection model is a YOLO-DFAM model, the FocalModulation module and the ASF-YOLO module are integrated in the YOLO-DFAM model, and the ASF-YOLO module integrates attention scale into the YOLO framework. The body weight calculation model further comprises a dynamic scale calibration module and a multi-frame confidence updating module. The real-time counting system is an improved ByteTrack tracking algorithm, and the improvement of the ByteTrack tracking algorithm comprises an exponential weighted moving average filtering module, an angle data processing module, a motion compensation module and a dynamic confidence threshold adjustment module based on target size and local density. The motion compensation module adopts particle filtering and local motion compensation technology based on optical flow. The present application effectively improves the detection accuracy of portunid crabs in complex sea conditions, can not only accurately count portunid crabs of various sizes, but also calculate the body weight of portunid crabs, and evaluate the fishing intensity and resource abundance. BRIEF DESCRIPTION OF DRAWINGS
[0040] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings:
[0041] Figure 1 A deep learning-based intelligent sorting model for portunid crabs on a ship is provided for the embodiments of the present application.
[0042] Figure 2 A working method of a FocalModulation module is provided for the embodiments of the present application.
[0043] Figure 3 A working method of an ASF-YOLO module is provided for the embodiments of the present application.
[0044] Figure 4 A working method of a dynamic scale calibration module is provided for the embodiments of the present application.
[0045] Figure 5A working method schematic diagram of the multi-frame confidence updating module provided by the embodiment of the present application is provided. DETAILED DESCRIPTION
[0046] In order for those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. It should be understood that these descriptions are only exemplary and are not used to limit the scope of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.
[0047] In addition, in the following description, the description of well-known structures and techniques is omitted to avoid unnecessary confusion of the concepts disclosed in the present application.
[0048] In the description of the present application, the terms "first", "second", "third" are only for descriptive purposes, and cannot be understood or implied as indicating or implying relative importance. The terms "mounting", "connecting", "connecting" should be broadly understood, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.
[0049] The exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Rather, they are merely examples of methods and systems consistent with some aspects of the present application as detailed in the appended claims.
[0050] The present application provides a deep learning-based intelligent sorting model for ship-mounted swimming crabs, which effectively improves the detection accuracy of swimming crabs in complex sea conditions. It not only can accurately count various sizes of swimming crabs, but also can calculate the weight of swimming crabs, evaluate the fishing intensity and resource abundance.
[0051] Model embodiment
[0052] Figure 1 A deep learning-based intelligent sorting model for ship-mounted swimming crabs provided by the embodiment of the present application is shown in the schematic diagram.
[0053] The application provides a deep learning-based intelligent sorting model for portunid crabs on a ship, which comprises a shipborne intelligent detection model 11, a body weight calculation model 12 and a real-time counting system 13,
[0054] The shipborne intelligent detection model 11 is used for detecting whether it is a portunid crab, and if the detection result is a portunid crab, the full carapace width of the portunid crab is detected.
[0055] Specifically, in the embodiment of the application, the shipborne intelligent detection model 11 is used for detecting whether it is a portunid crab, and if the detection result is a portunid crab, the full carapace width of the portunid crab is detected.
[0056] The body weight calculation model 12 is used for updating the full carapace width of the portunid crab, determining the body weight of the portunid crab according to the updated full carapace width of the portunid crab, and grading the portunid crab according to the body weight.
[0057] Specifically, in the embodiment of the application, the full carapace width detected by the shipborne intelligent detection model 11 is not very accurate, the body weight calculation model 12 further detects and updates the full carapace width of the portunid crab, and the accuracy of the detection of the full carapace width is improved, and the principle of the body weight calculation model updating the full carapace width of the portunid crab, determining the body weight of the portunid crab according to the updated full carapace width of the portunid crab, and grading the portunid crab according to the body weight is described in detail below.
[0058] The real-time counting system 13 is used for counting the portunid crabs of different body weight grades.
[0059] Specifically, in the embodiment of the application, the real-time counting system 13 can grade the portunid crabs according to the body weight, which can be divided into three grades of >150g, 100-150g and <100g, and the real-time counting system 13 can count the portunid crabs of different body weight grades, for example, >150g is 230, 100-150g is 355, and <100g is 130, and in some embodiments, the portunid crabs can be divided into four grades or five grades, which does not limit the protection scope of the application.
[0060] The shipborne intelligent detection model 11 is a YOLO-DFAM model, the YOLO-DFAM model integrates a FocalModulation module 101 and an ASF-YOLO module 102, and the ASF-YOLO module 102 integrates attention scale fusion into a YOLO framework.
[0061] The body weight calculation model 12 further comprises a dynamic scale calibration module 103 and a multi-frame confidence updating module 104.
[0062] The real-time counting system 13 is an improved ByteTrack tracking algorithm, and the improved ByteTrack tracking algorithm comprises an exponential weighted moving average filtering module 105, a processing angle data module 106, a motion compensation module 107, and a dynamic confidence threshold adjustment module 108 based on target size and local density.
[0063] Specifically, in the embodiment of the present application, the FocalModulation module can improve the detection ability of the model for small Portunus sanguinolentus and the ability to distinguish dense targets in complex scenes, the ASF-YOLO module effectively captures the multi-scale characteristics of Portunus sanguinolentus in complex scenes on the deck, such as the details of the near-range Portunus sanguinolentus carapace and the outline of the long-range crab body, and enhances the robustness to degraded information such as broken crab legs and blurred shell texture, DFDM is the full name of DualFocus Dynamic Modulation, the dynamic scale calibration module can improve the measurement accuracy of the shell width, the multi-frame confidence updating module reduces the target tracking trajectory breakage rate, the exponential weighted moving average filtering module reduces the detection noise and the sharp change caused by local motion, the processing angle data module solves the problem of sharp fluctuation of the detection box angle when the Portunus sanguinolentus rotates quickly, the motion compensation module effectively alleviates the position deviation caused by fast motion or local occlusion, and the dynamic confidence threshold adjustment module based on target size and local density ensures that each level of Portunus sanguinolentus can be accurately tracked, and the specific implementation method is described in detail below.
[0064] Figure 2 A working method diagram of a FocalModulation module provided in the embodiment of the present application is shown.
[0065] Further, in the above-mentioned deep learning-based Portunus sanguinolentus shipborne intelligent sorting model, the FocalModulation module comprises the following steps: Figure 2 , the working method of the FocalModulation module comprises three steps of S21 to S23:
[0066] S21: Multi-scale deep convolution captures context features from local details to global semantics, and constructs a pyramid multi-granularity representation;
[0067] S22: The dynamic gating generator is used to realize adaptive fusion of cross-scale features, and generates selection weights based on a spatial-channel dual attention mechanism in the adaptive fusion process, so that the YOLO-DFAM model can focus on key areas;
[0068] S23: Element-wise affine transformation injects context features from local details to global semantics into original features.
[0069] Specifically, in this embodiment of the invention, FocalModulation innovates the traditional attention paradigm through a three-stage mechanism of focus contextualization, gated aggregation, and affine transformation. Unlike the fully connected interaction of the self-attention mechanism, the FocalModulation module achieves a balance between computational efficiency and modeling capability, eliminates the computational bottleneck of QKV interaction, and maintains the modeling capability sensitive to the direction of rotating targets while reducing memory consumption.
[0070] It should be understood that spatial attention enhances the positioning accuracy of rotation-sensitive features such as the crab's body edge and claw direction; channel attention maintains feature discrimination in occluded scenarios, such as when the crab's body is obscured by a fishing net, through dynamic filtering of feature channels; and the gating fusion stage uses Softmax normalization instead of Sigmoid, enabling multi-scale feature weights to have competitive adjustment characteristics, which is more suitable for actual fishery detection needs.
[0071] Figure 3 This is a schematic diagram illustrating the working method of an ASF-YOLO module provided in an embodiment of the present invention.
[0072] Furthermore, in the aforementioned deep learning-based intelligent sorting model for swimming crabs on board, combined with... Figure 3 The working method of the ASF-YOLO module includes three steps, S31 to S33:
[0073] S31: Capture the multi-scale characteristics of swimming crabs in various scenarios on the deck using a scale sequence feature fusion module;
[0074] S32: Triple feature encoder module fuses deep and shallow features;
[0075] S33: The channel-position attention mechanism dynamically focuses on the bluish-gray carapace area and key joint points of the swimming crab.
[0076] Specifically, in this embodiment of the invention, the ASF-YOLO module effectively captures the multi-scale characteristics of swimming crabs in complex scenes on the deck by fusing scale sequence features with the SSFF module, such as the details of the carapace of the swimming crab in the foreground and the outline of the crab body in the background. The triple feature encoder TFE module is used to fuse deep and shallow features to enhance the robustness to degradation information such as broken crab legs and blurred carapace texture. The channel-position attention mechanism can dynamically focus on the unique bluish-gray carapace area and key joint points of the swimming crab, and can still accurately locate the target under the interference of shells, seaweed and debris.
[0077] Figure 4 This is a schematic diagram illustrating the working method of the dynamic scale calibration module provided in an embodiment of the present invention.
[0078] Furthermore, in the aforementioned deep learning-based intelligent sorting model for swimming crabs on board, combined with... Figure 4The working method of the dynamic scale calibration module includes two steps S41-S42.
[0079] S41: multiple sets of calibration point data are set in the vertical direction of the video frame, and a pixel-centimeter conversion relationship is established for the multiple sets of calibration point data.
[0080] Specifically, in the shooting of the sorting conveyor belt of the swimming crab, due to the limitation of the shooting angle, the full family width length of the swimming crab close to the camera and far from the camera will change, multiple sets of calibration point data, such as 12 sets of calibration point data, are set in the vertical direction of the video frame, the y-axis coordinate range is 830-1507 pixels, the calibration point data verification adopts a double insurance mechanism, and the invalid span and abnormal points beyond the image boundary are removed x r ≤x l The pixel / cm conversion relationship function is as follows:
[0081]
[0082] The pixel / cm conversion relationship function presents a nonlinear characteristic in the vertical direction, the conversion coefficient in the top area (y>1400px) is 0.098cm / px, and the conversion coefficient in the bottom area (y<500px) is increased to 0.152cm / px, which accurately adapts to the scale change caused by perspective distortion.
[0083] S42: the multiple sets of calibration point data after the conversion relationship is established are input into a cubic spline interpolator to construct a continuous space scale function.
[0084] Specifically, in the embodiment of the present application, the multiple sets of calibration point data after the conversion relationship is established are input into a cubic spline interpolator to construct a continuous space scale function as follows:
[0085]
[0086] The boundary value constraint (S(0)=0.153, S(H)=0.102, H is the image height) ensures the extrapolation stability of the continuous space scale function, and tests show that the dynamic scale calibration module reduces the vertical direction size measurement error from 12.6mm of the fixed scale to 1.8mm, and the error is reduced by 85.7%.
[0087] Figure 5 The working method of the multi-frame confidence updating module provided in the embodiment of the present application is shown in the schematic diagram.
[0088] Further, in the above-mentioned swimming crab shipborne intelligent sorting model based on deep learning, Figure 5 The working method of the multi-frame confidence updating module includes two steps S51-S52.
[0089] S51: comparing the confidence of the same crab in the continuous frames;
[0090] S52: when the comparison result is that the confidence of the same crab in the current frame is higher than that in the previous frame, updating the overall carapace width and the weight of the crab according to the picture corresponding to the current frame.
[0091] Specifically, in actual application, due to the movement of the swimming crab on the conveyor belt, the change in posture, and the instability of the light condition, single-frame detection often has data fluctuation or error due to the shooting angle or instantaneous interference. In order to ensure the accuracy of the final measurement data, a multi-frame confidence updating module is designed to compare the confidence of the same crab in the continuous frames. When the comparison result is that the confidence of the same crab in the current frame is higher than that in the previous frame, the overall carapace width and the weight of the crab are updated according to the picture corresponding to the current frame. The unreliable measurement caused by local occlusion, motion blur, and other interference factors can be automatically filtered out to ensure that the recorded data represents the best detection moment.
[0092] Further, in the above-mentioned swimming crab ship-mounted intelligent sorting model based on deep learning, the model formula for determining the weight of the swimming crab according to the updated overall carapace width of the swimming crab is:
[0093] W=a×(L)+b b
[0094]
[0095] wherein L represents the updated overall carapace width of the swimming crab, a=0.0514±0.0008, b=3.0158±0.0123, W obs represents the actual measured weight value of the swimming crab, W represents the weight prediction value output by the model, represents the average value of the actual measured weight of the swimming crab, R 2 represents the determination coefficient, which is the coefficient for evaluating the weight model of the swimming crab, i=1……n, and represents the number of swimming crabs.
[0096] Specifically, in the embodiment of the present application, the model outputs the weight prediction value of the swimming crab through L, a, and b. The multiple weight prediction values of the swimming crabs, the average value of the actual measured weight of the swimming crabs, and the actual measured weight value of the swimming crabs are brought into the formula to obtain R2. The closer R2 is to 1, the more accurate the weight prediction value of the swimming crab is.
[0097] Here, a = 0.0514 ± 0.0008 (95% confidence interval), b = 3.0158 ± 0.0123 (95% confidence interval), a, b are model parameters determined by iterative optimization, parameter a is a condition factor, which reflects the nutritional reserves and the level of fullness of the individual Portunus trituberculatus, and parameter b is an allometric growth index, and the value close to 3 indicates that the weight growth of Portunus trituberculatus in the study sea area is proportional to the cube of the total carapace width, which meets the theoretical expectation of allometric growth of crustaceans and the like.
[0098] Further, in the above-mentioned deep learning-based intelligent on-ship sorting model for Portunus trituberculatus, the working method of the exponential weighted moving average filtering module comprises:
[0099] In each frame, the coordinates of the four corners of the current detection box, the center coordinates of the detection box and the smoothed estimation value of the last frame are weighted and fused according to a set smoothing coefficient, and the center coordinates, width and height of the target Portunus trituberculatus are smoothed and updated.
[0100] Specifically, in the embodiment of the present application, since Portunus trituberculatus frequently swings the crab legs and claws after entering, the traditional detection is prone to problems such as large and small frame size and position jitter, and an exponential weighted moving average filtering mechanism is designed, in each frame, the coordinates of the four corners of the current detection box, the center coordinates of the detection box and the smoothed estimation value of the last frame are weighted and fused according to a set smoothing coefficient, and the center coordinates, width and height of the target Portunus trituberculatus are smoothed and updated, thereby reducing the detection noise and the dramatic changes caused by local motion.
[0101] Further, in the above-mentioned deep learning-based intelligent on-ship sorting model for Portunus trituberculatus, the working method of the angle data processing module comprises:
[0102] The angle data is processed by using a continuous frame smoothing technique, through a rolling window or an exponential weighted moving average filtering method.
[0103] Specifically, in the embodiment of the present application, in order to solve the problem of dramatic fluctuation of the detection box angle when Portunus trituberculatus rotates rapidly, an angle data processing module is designed, the angle data is processed by using a continuous frame smoothing technique, through a rolling window or an exponential weighted moving average filtering method, so as to avoid frequent jumping of the detection box angle in the range of 0° to 360°, thereby ensuring that the detection box remains stable and continuous.
[0104] Further, in the above-mentioned deep learning-based intelligent on-ship sorting model for Portunus trituberculatus, the working method of the motion compensation module comprises:
[0105] The particle filter generates a plurality of possible states according to the historical motion trajectory, and predicts the possible position of the target Portunus trituberculatus in the next frame by assigning weights to the coordinates of the four corners of the current detection box and the center coordinates of the detection box.
[0106] Specifically, in the embodiment of the present application, the particle filter generates a plurality of possible states using the historical motion trajectory, the possible position of the target in the next frame is predicted by detecting the coordinates of the four corners of the detection frame, the center coordinates of the detection frame are weighted, and the effective completion of the trajectory is realized when the detection is temporarily invalid, so as to ensure that the trajectory is continuous even if there is a short-time missed detection.
[0107] The local motion compensation calls an image processing function, estimates the real movement of the target swimming crab based on the displacement of the pixels of the target region of the adjacent frame, and corrects the position of the detection frame of the current frame in real time.
[0108] Specifically, in the embodiment of the present application, the local motion compensation calls an image processing function, estimates the real movement of the target based on the displacement of the pixels of the target region of the adjacent frame, and corrects the position of the detection frame in real time, effectively alleviating the position deviation caused by fast motion or local occlusion.
[0109] Further, in the above-mentioned deep learning-based intelligent sorting model of swimming crabs on a ship, the working method of the dynamic confidence threshold adjustment module based on target size and local density comprises:
[0110] When the coordinates of the four corners of the detection frame of each target swimming crab are extracted, the center coordinates of the detection frame are determined, whether it is a swimming crab is determined, and whether it is a swimming crab is determined in the process of combining size information and density information to dynamically adjust whether it is a corresponding swimming crab.
[0111] Specifically, in the embodiment of the present application, in order to solve the problem of missed detection caused by low output confidence of small targets or overlapping occluded targets, a dynamic confidence threshold adjustment module based on target size and local density is designed, the dynamic confidence threshold adjustment module based on target size and local density realizes the self-adaptation of the edge target, when the coordinates of the four corners of the detection frame of the target swimming crab are extracted, the center coordinates of the detection frame are determined, whether it is a swimming crab is additionally determined, and the retention standard is dynamically adjusted in the determination process by combining size, density and other information, so as to ensure that each level of swimming crab can be accurately tracked.
[0112] Those skilled in the art can understand that although some embodiments described herein include certain features included in other embodiments rather than other features, the combination of features of different embodiments means that it is within the scope of the present application and forms different embodiments.
[0113] Those skilled in the art can understand that the description of each embodiment is focused on, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.
[0114] Although the embodiments of the present application have been described with reference to the accompanying drawings, various modifications and changes can be suggested to one skilled in the art, and it is intended that the present application encompass such modifications and changes as fall within the scope of the appended claims. The embodiments of the present application described above are combinations of elements and features of the present application. The elements and features can be used alone or in any combination(s). Each of the elements and features can be used alone or in any combination(s). Thus, the foregoing description of the embodiments of the present application is provided for the purpose of illustration only and not for the purpose of limitation, and that the scope of the present application is defined by the claims as set forth below rather than by the foregoing description. The claims are not intended to be limited to the embodiments of the present application described above, but are intended to include any embodiments of the present application falling within the scope of the claims.
[0115] The embodiments of the present application described above are combinations of elements and features of the present application. The elements and features can be used alone or in any combination(s). Each of the elements and features can be used alone or in any combination(s). Thus, the foregoing description of the embodiments of the present application is provided for the purpose of illustration only and not for the purpose of limitation, and that the scope of the present application is defined by the claims as set forth below rather than by the foregoing description. The claims are not intended to be limited to the embodiments of the present application described above, but are intended to include any embodiments of the present application falling within the scope of the claims.
Claims
1. A deep learning-based intelligent on-ship sorting model for portunid crabs, characterized in that, The application relates to a shipborne intelligent detection model, a weight calculation model and a real-time counting system. The shipborne intelligent detection model is used for detecting whether a crab is a Portunus trituberculatus, and if the detection result is a Portunus trituberculatus, the total carapace width of the Portunus trituberculatus is detected. The weight calculation model is used for updating the total carapace width of the Portunus trituberculatus, determining the weight of the Portunus trituberculatus according to the updated total carapace width of the Portunus trituberculatus, and grading the Portunus trituberculatus according to the weight. The real-time counting system is used for counting Portunus trituberculatus of different weight grades. The shipborne intelligent detection model is a YOLO-DFAM model, the YOLO-DFAM model integrates a FocalModulation module and an ASF-YOLO module, and the ASF-YOLO module integrates attention scale fusion into a YOLO framework. The weight calculation model further comprises a dynamic scale calibration module and a multi-frame confidence updating module. The real-time counting system is an improved ByteTrack tracking algorithm, the improved ByteTrack tracking algorithm comprises an exponential weighted moving average filtering module, an angle data processing module, a motion compensation module and a dynamic confidence threshold adjustment module based on target size and local density, and the motion compensation module adopts a particle filtering and local motion compensation technology based on an optical flow. The working method of the FocalModulation module comprises the following steps:
2. The deep learning-based intelligent on-board sorting model for portunid crabs according to claim 1, wherein, A multi-scale deep convolution captures context features from local details to global semantics, and constructs a pyramid multi-granularity representation. A dynamic gating generator is used for realizing adaptive fusion of cross-scale features, and selection weights are generated based on a space-channel dual attention mechanism in the adaptive fusion process, so that the YOLO-DFAM model can focus on key regions. Element-wise affine transformation injects context features from local details to global semantics into original features. The working method of the ASF-YOLO module comprises the following steps:
3. The deep learning-based intelligent on-board sorting model for portunid crabs according to claim 1, wherein, A scale sequence feature fusion module captures multi-scale characteristics of Portunus trituberculatus in various scenes on a deck. A triple feature encoder module fuses deep and shallow layer features. A channel-position attention mechanism dynamically focuses on a Portunus trituberculatus grey carapace region and joint key points. The working method of the dynamic scale calibration module comprises the following steps:
4. The deep learning-based intelligent on-board sorting model for portunid crabs according to claim 1, wherein, Multiple sets of calibration point data are arranged in the vertical direction of a video frame, and a pixel-centimeter conversion relationship is established for the multiple sets of calibration point data. The multiple sets of calibration point data after the conversion relationship is established are input into a cubic spline interpolator to construct a continuous space scale function. The working method of the multi-frame confidence updating module comprises the following steps:
5. The deep learning-based intelligent on-board sorting model for portunid crabs according to claim 1, wherein, The confidence of the same Portunus trituberculatus in consecutive frames is compared. When the comparison result is that the confidence of the same Portunus trituberculatus in the current frame is higher than that in the previous frame, the total carapace width and the weight of the Portunus trituberculatus are updated according to the corresponding image of the current frame. The model formula for determining the weight of the Portunus trituberculatus according to the updated total carapace width of the Portunus trituberculatus is as follows:
6. The deep learning-based intelligent on-board sorting model for portunid crabs according to claim 1, wherein, The working method of the exponential weighted moving average filtering module comprises the following steps: W = a x (L) b where L represents the updated total carapace width of the Portunus sanguinolentus, a = 0.0514 ± 0.0008, b = 3.0158 ± 0.0123, W obs represents the actual measured Portunus sanguinolentus weight value, W represents the weight prediction value output by the model, represents the average value of the actual measured Portunus sanguinolentus weight, R 2 represents the determination coefficient, is the coefficient for evaluating the Portunus sanguinolentus weight model, i = 1 … n, represents the number of Portunus sanguinolentus.
7. The deep learning-based on-vehicle intelligent sorting model for portunid crabs according to claim 1, characterized in that, In each frame, the coordinates of four corners of a current detection box, the center coordinates of the detection box and the smooth estimated value of the previous frame are weighted and fused according to a set smoothing coefficient, and the center coordinates, width and height of the target Portunus trituberculatus are updated. 8. The deep learning-based intelligent on-board sorting model for portunid crabs according to claim 1, wherein, The working method of the processing angle data module comprises: The angle data is processed by using a continuous frame smoothing technique, a rolling window or an exponential weighted moving average filtering mode.
9. The deep learning-based on-vehicle intelligent sorting model for portunid crabs according to claim 1, characterized in that, The working method of the motion compensation module comprises: The particle filter generates a plurality of possible states according to a historical motion track, detects the coordinates of four corners of a detection frame, assigns a weight to the central coordinate of the detection frame, and predicts the possible position of the target swimming crab in the next frame; The local motion compensation calls an image processing function, estimates the real movement of the target swimming crab based on the displacement of the pixels of the target region of the adjacent frame, and corrects the position of the detection frame in the current frame in real time.
10. The deep learning-based on-vehicle intelligent sorting model for portunid crabs according to claim 1, characterized in that, The working method of the dynamic confidence threshold adjustment module based on the target size and local density comprises: When the coordinates of four corners of the detection frame of each target swimming crab are extracted, the central coordinate of the detection frame is determined, and whether the corresponding swimming crab is tracked is dynamically adjusted in the process of determining whether the swimming crab is tracked in combination with the size information and the density information.