Fish Multi-Target Tracking Method for Land-Based Industrial Recirculating Aquaculture Scenarios

Abstract

The present invention relates to the field of computer vision technology, and specifically to a method for multi-object tracking of fish in the scenario of land-based industrialized recirculating aquaculture. First, the present invention obtains the fish movement video and light intensity of a land-based industrialized recirculating aquaculture farm. Secondly, video frames that meet preset conditions are selected from the fish movement video as reference frames. Then, based on the fish movement video and light intensity, video frame features are extracted, and a video frame adjustment coefficient is generated in combination with the reference frames. Then, an adaptive adjustment model for fish movement video frames is constructed according to the video frame adjustment coefficient to adaptively adjust the fish movement video frames to obtain a standard fish movement video. Then, a fish multi-object tracking model is constructed to perform multi-object tracking of fish on the standard fish movement video to obtain fish movement trajectories. Finally, according to the fish movement trajectories and the standard fish movement video, a fish health detection model is constructed to detect the health status of each fish in the standard fish movement video.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and particularly to a method for multi-object tracking of fish in the scenario of land-based industrial recirculating aquaculture. Background Art

[0002] Aquaculture is an important part of the global food supply. With the development of social economy and the growth of population, people's demand for high-protein food is increasing day by day, and the scale and importance of aquaculture are also constantly rising. However, with the increase of aquaculture density and environmental pressure, the aquaculture industry is facing problems such as fish diseases, environmental pollution, and low aquaculture efficiency. In this context, monitoring the behavior of fish becomes particularly crucial. The behavior and movement trajectory of fish can reflect important information such as their health status, feeding situation, and reproductive state. By tracking and analyzing the movement trajectory of fish, the health monitoring of fish groups can be realized, early signs of diseases can be detected, the adaptability of the aquaculture environment can be evaluated, and the feed delivery strategy can be optimized, so as to improve aquaculture efficiency and reduce economic losses. This helps to improve the benefits of aquaculture and ensure its sustainable development.

[0003] Traditional fish behavior monitoring methods usually rely on manual visual inspection or simple video analysis tools, and these methods have limitations such as low efficiency, strong subjectivity, and limited accuracy. In recent years, with the continuous progress of deep learning technology, multi-object tracking algorithms have been increasingly applied in animal analysis research. However, in the scenario of land-based industrial recirculating aquaculture, the dynamic change of light conditions has a significant impact on the detection and tracking performance of multi-object tracking of fish. Therefore, traditional multi-object tracking methods lack the ability to perceive environmental changes, resulting in a decline in target tracking performance in special environments such as low light and strong light.

[0004] Therefore, a method for multi-object tracking of fish in the scenario of land-based industrial recirculating aquaculture is proposed. Summary of the Invention

[0005] The object of the present invention is to provide a fish multi-object tracking method for land-based industrialized recirculating aquaculture scenarios. First, the present invention obtains the fish movement video and light intensity of a land-based industrialized recirculating aquaculture farm; secondly, selects video frames that meet preset conditions from the fish movement video as reference frames; then, based on the fish movement video and light intensity, extracts video frame features, and generates a video frame adjustment coefficient in combination with the reference frames; then, constructs an adaptive adjustment model for fish movement video frames according to the video frame adjustment coefficient, adaptively adjusts the fish movement video frames, and obtains a standard fish movement video; then, constructs a fish multi-object tracking model to perform fish multi-object tracking on the standard fish movement video to obtain fish movement trajectories; finally, according to the fish movement trajectories and the standard fish movement video, constructs a fish health detection model to detect the health status of each fish in the standard fish movement video.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A fish multi-object tracking method for land-based industrialized recirculating aquaculture scenarios, comprising:

[0008] Dividing a land-based industrialized recirculating aquaculture farm into M regions, and recording the fish movement videos of the M regions; the fish movement videos cover different lighting conditions, including: low light, strong light, and normal light; synchronously obtaining the light intensities of the M regions through light sensors;

[0009] Selecting video frames that meet preset conditions from the fish movement video as reference frames;

[0010] Extracting the video frame features of the fish movement video according to the fish movement video and the light intensity; generating a video frame adjustment coefficient through the video frame features and the reference frames; the video frame adjustment coefficient includes: a first adjustment coefficient, a second adjustment coefficient, and a third adjustment coefficient;

[0011] Constructing an adaptive adjustment model for fish movement video frames according to the video frame adjustment coefficient, and adaptively adjusting the fish movement video frames to obtain a standard fish movement video; the formula for the standard fish movement video is:

[0012] ;

[0013] Wherein, is the pixel value of the standard fish movement video frame; is the pixel value of the input fish movement video frame; is the brightness adjustment coefficient; is the color adjustment coefficient; is the local gradient adjustment coefficient; is the brightness gradient at pixel value; is an adjustment parameter for controlling the gradient enhancement intensity;

[0014] Construct a multi-object tracking model for fish to perform multi-object tracking on the standard fish motion video to obtain the fish motion trajectory; according to the fish motion trajectory and the standard fish motion video, construct a fish health detection model to detect the health status of each fish in the standard fish motion video.

[0015] Preferably, before selecting video frames meeting preset conditions from the fish motion video as reference frames, it further includes synchronizing the fish motion video with the light intensity in time and recording the light intensity corresponding to each frame of the fish motion video.

[0016] Preferably, the step of selecting video frames meeting preset conditions from the fish motion video as reference frames is:

[0017] Set the normal light range, and use the sliding window method to screen out the video frames in the fish motion video with the light intensity within the normal light range to obtain normal light video frames;

[0018] Calculate the light intensity change rate corresponding to each normal light video frame, compare the light intensity change rates, and select the normal light video frame with the smallest light intensity change rate as the reference frame.

[0019] Preferably, the video frame features include: average brightness, average saturation, hue value, brightness gradient, and brightness gradient variance; synchronize the light intensity corresponding to each frame of the fish motion video with the video frame features to obtain the video frame features of each frame.

[0020] Preferably, the first adjustment coefficient is the brightness adjustment coefficient, and the calculation formula is:

[0021] ;

[0022] Wherein, is the brightness adjustment coefficient; is the adjustment parameter; is the light intensity of the current frame; is the reference light intensity; is the average brightness of the current frame; is the average brightness of the reference frame; is the adjustment parameter;

[0023] The second adjustment coefficient is the color adjustment coefficient, and the calculation formula is:

[0024] ;

[0025] Wherein, is the color adjustment coefficient; is the average saturation of the reference frame; is the average saturation of the current frame; is the adjustment parameter; is the adjustment parameter; is the hue value of the current frame; is the hue value of the reference frame; is the total number of frames;

[0026] The third adjustment coefficient is the local gradient adjustment coefficient, and the calculation formula is:

[0027] ;

[0028] where is the local gradient adjustment coefficient; is the adjustment parameter; is the average brightness gradient of the current frame; is the average brightness gradient of the reference frame; is the adjustment parameter; is the variance of the brightness gradient of the current frame; is the variance of the brightness gradient of the reference frame.

[0029] Preferably, an adaptive adjustment model for fish motion video frames is constructed according to the video frame adjustment coefficient, and the fish motion video frames are adaptively adjusted to obtain a standard fish motion video; the formula for the standard fish motion video is:

[0030] ;

[0031] where is the pixel value of the standard fish motion video frame; is the pixel value of the input fish motion video frame; is the brightness adjustment coefficient; is the color adjustment coefficient; is the local gradient adjustment coefficient; is the brightness gradient at pixel value; is the adjustment parameter for controlling the gradient enhancement intensity.

[0032] Preferably, the fish multi-object tracking model is constructed by improving the YOLOv8-ByteTrack model, including: a fish motion feature extraction layer, a fish detection layer, and a fish multi-object tracking layer;

[0033] The fish motion feature extraction layer extracts fish features based on the video frames of the fish images retained only in the fish motion video sample through the MSDA model that combines multi-scale dilated convolution and the attention mechanism; the fish features include: the fish body area, main texture, and contour; the fish detection layer inputs the fish features into the YOLOv8 model for fish detection to obtain the fish detection bounding boxes for each frame; the fish multi-object tracking layer associates the trajectories of the detection bounding box markings based on the ByteTrack algorithm and integrates the adaptive Kalman filter to obtain the fish multi-object tracking trajectories.

[0034] Preferably, the fish health detection model includes: a fish motion feature extraction layer, a fish appearance feature extraction layer, a fish feature fusion layer, and a fish health monitoring layer; wherein, the fish motion feature extraction layer extracts fish motion features according to the fish multi-object tracking trajectories detected by the fish multi-object tracking model; the fish motion features include: speed, acceleration, and turning angle; the fish appearance feature extraction layer extracts fish appearance features according to the standard fish motion video; the fish appearance features include: the hue of the fish, the saturation of the fish, and the brightness of the fish; the fish feature fusion layer fuses the fish motion features and the fish appearance features to obtain fish health features; the fish health monitoring layer detects the health status of each fish according to the fish health features; the health status includes: normal, abnormal, and diseased.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] 1. The present invention proposes a fish motion video frame adaptive adjustment model that extracts video frame features based on the fish motion video and the light intensity, and combines with the reference frame to generate adjustment coefficients including the brightness adjustment coefficient, color adjustment coefficient, and local gradient adjustment coefficient to adaptively adjust the video frame, thereby obtaining a standard fish motion video. This step accurately generates various adjustment coefficients by comprehensively extracting multi-dimensional features such as brightness, saturation, hue, brightness gradient, and its variance, effectively eliminating the influence of light changes on the video quality, and achieving the consistency and clarity of brightness, color, and details. The fish motion video frame adaptive adjustment model standardizes each frame of the video according to these adjustment coefficients, significantly improving the overall visual quality and reducing the noise and blur caused by light fluctuations. This not only optimizes the input data quality of the subsequent multi-object tracking algorithm, improves the accuracy and stability of tracking, but also ensures the accurate tracking and health monitoring of fish behavior in a complex light environment, comprehensively improving the automation and intelligent management level of the farm.

[0037] 2. The present invention proposes a multi-object tracking model for fish and performs multi-object tracking on the standard fish motion video obtained by adaptively adjusting the model for fish motion video frames, capable of accurately obtaining the motion trajectories of each fish. By using the improved YOLOv8-ByteTrack model combined with multi-scale feature extraction and adaptive Kalman filtering technology, high-precision and high-efficiency multi-object detection and tracking are achieved, ensuring stable tracking of multiple fish even in complex lighting and dynamic environments. The accurate motion trajectories can not only monitor the behavior patterns of fish in real time, detect abnormalities and potential diseases in a timely manner, but also provide a reliable data basis for subsequent health detection of fish through the fish health detection model.

[0038] 3. The present invention proposes a fish health detection model, which is constructed based on the fish motion trajectories obtained by the fish multi-object tracking model and the standard fish motion video obtained by the adaptive adjustment model of fish motion video frames, and can accurately detect the health status of each fish. By analyzing the motion behaviors and appearance features of fish, this model can identify normal, abnormal, and specific disease states, realizing real-time monitoring and early warning of fish health. This method not only improves the accuracy and timeliness of health detection, reduces the dependence on manual monitoring, but also can quickly respond to potential health problems, prevent the spread of diseases and the occurrence of aquaculture losses. Through the comprehensive analysis of motion trajectories and video data, the health detection model significantly improves the intelligent level of farm management, optimizes the aquaculture environment, ensures the health and production efficiency of fish, and thus comprehensively enhances the operational benefits and sustainable development capabilities of land-based industrialized recirculating aquaculture farms. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flowchart of the fish multi-object tracking method for land-based industrialized recirculating aquaculture scenarios provided by the embodiments of the present invention;

[0040] Figure 2 It is a flowchart of the adaptive adjustment of fish motion video frames provided by the embodiments of the present invention;

[0041] Figure 3 It is a structural diagram of the fish multi-object tracking model provided by the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0043] Aquaculture is an important part of the global food supply. With the development of social economy and the growth of population, the demand for high-protein foods is increasing day by day, and the scale and importance of aquaculture are also constantly rising. However, with the increase in aquaculture density and environmental pressure, the aquaculture industry faces problems such as fish diseases, environmental pollution, and low aquaculture efficiency. The behavior and movement trajectories of fish can reflect important information such as their health status, feeding conditions, and reproductive status. By tracking and analyzing the movement trajectories of fish, health monitoring of fish populations can be achieved, early signs of diseases can be detected, the adaptability of the aquaculture environment can be evaluated, and feed delivery strategies can be optimized, thereby improving aquaculture efficiency and reducing economic losses.

[0044] The present invention proposes a multi-objective fish tracking method for land-based industrial recirculating aquaculture scenarios, realizing multi-objective trajectory tracking and health detection of fish in recirculating aquaculture farms. The specific method flow chart is referred to Figure 1 . To illustrate that the method of the present invention can play the role of multi-objective trajectory tracking and health detection of fish, the effectiveness of the present invention will be described from two embodiments below.

[0045] Please refer to Figures 1 to 3 , the multi-objective fish tracking method of the present invention for land-based industrial recirculating aquaculture scenarios has the following technical solutions:

[0046] Embodiment 1

[0047] In the embodiment of the present application, the method proposed by the present invention is used to elaborate on the process of multi-objective trajectory tracking and health detection of fish. In the embodiment of the present application, the multi-objective trajectory tracking and health detection of fish are for the multi-objective trajectory tracking and health detection of healthy California bass with a length in the range of 10 - 15 cm in Pond A of a certain land-based industrial recirculating aquaculture farm. The area of Pond A is 20 meters × 20 meters. Please refer to Figure 1 , Figure 1The specific process of the method proposed by the present invention includes: dividing the land-based industrialized recirculating aquaculture farm into M regions, and obtaining the fish movement videos and light intensities of the M regions of the land-based industrialized recirculating aquaculture farm under different light conditions; selecting video frames that meet the preset conditions from the fish movement videos as reference frames; extracting the video frame features of the fish movement videos based on the fish movement videos and the light intensities; generating video frame adjustment coefficients according to the video frame features and the reference frames; constructing an adaptive adjustment model for fish movement video frames according to the video frame adjustment coefficients to adaptively adjust the fish movement video frames to obtain a standard fish movement video; constructing a fish multi-target tracking model to perform fish multi-target tracking on the standard fish movement video to obtain fish movement trajectories; and constructing a fish health detection model according to the fish movement trajectories and the standard fish movement video to detect the health status of each fish in the standard fish movement video. The following will elaborate on Figure 1 the multi-target trajectory tracking and health detection process of healthy California bass in this fish pond A according to

[0048] A fish multi-target tracking method for land-based industrialized recirculating aquaculture scenarios includes:

[0049] Dividing the land-based industrialized recirculating aquaculture farm into M regions, and recording the fish movement videos of the M regions; the fish movement videos cover different light conditions, including: low light, strong light, and normal light; synchronously obtaining the light intensities of the M regions through light sensors;

[0050] Specifically, divide fish pond A into 8 regions, and install 1 Hikvision (DS-IPC-B14HV-LT type) camera in each region to record the fish movement videos of the 8 regions in fish pond A at 8:00 am on a sunny day, 12:00 noon on a sunny day, and on a cloudy day, ensuring that the fish movement videos cover three different light conditions of low light, strong light, and normal light;

[0051] Install 1 light sensor in each region, and arrange it near the camera to synchronously obtain the light intensities of the 8 regions.

[0052] Preferably, before selecting the video frames that meet the preset conditions from the fish movement videos as reference frames, it further includes synchronizing the fish movement videos with the light intensities and recording the light intensity corresponding to each frame of the fish movement videos.

[0053] In the embodiment of the present application, the fish motion video is synchronized with the light intensity in time to ensure that the light intensity corresponding to each video frame is accurately recorded. This measure improves the correspondence accuracy between the light information and the video frames, provides accurate light intensity information for the subsequent adaptive adjustment of the video frames, and improves the quality of the fish motion video and the reliability of multi-object tracking.

[0054] Preferably, video frames meeting preset conditions are selected from the fish motion video as reference frames; the steps of selecting video frames meeting preset conditions from the fish motion video are as follows:

[0055] Set the normal light range, and use the sliding window method to screen out the video frames in the fish motion video whose light intensity is within the normal light range to obtain normal light video frames;

[0056] Calculate the light intensity change rate corresponding to each normal light video frame, compare the light intensity change rates, and select the normal light video frame with the smallest light intensity change rate as the reference frame.

[0057] Specifically, the normal light range is set as:

[0058] ;

[0059] wherein, is the low light threshold; is the light intensity; is the high light threshold;

[0060] Use the sliding window method to screen out the video frames in the fish motion video whose light intensity is within the normal light range to obtain normal light video frames; calculate the light intensity change rate corresponding to each normal light video frame, compare the light intensity change rates, and select the normal light video frame with the smallest light intensity change rate as the reference frame.

[0061] In the embodiment of the present application, setting the normal light range and using the sliding window method to screen normal light video frames ensure that the reference frames are selected under stable light conditions, reduce the influence of light fluctuations on the reference frames, and enhance the representativeness and stability of the reference frames. By calculating and selecting the video frame with the smallest light intensity change rate, the consistency of the reference frames in terms of light conditions is further ensured, providing a reliable reference frame for the subsequent adaptive adjustment of the video frames, enhancing the stability and consistency of the adjustment effect, and improving the overall video quality.

[0062] Preferably, based on the fish motion video and the light intensity, video frame features of the fish motion video are extracted; the video frame features include: average brightness, average saturation, hue value, brightness gradient, and variance of brightness gradient.

[0063] Synchronize the light intensity corresponding to each frame of the fish motion video with the video frame features to obtain the video frame features of each frame.

[0064] In the embodiments of the present application, by extracting multi-dimensional video frame features such as the brightness mean value, saturation mean value, hue value, brightness gradient, and brightness gradient variance, the brightness, color, and detail information of the video frame are comprehensively reflected, the richness of feature expression is enhanced, and the ability of subsequent video frame adaptive adjustment and the accuracy of multi-target tracking are improved. Synchronizing the light intensity with the video frame features ensures the accurate correspondence of light information during the feature extraction process, provides reliable light information for the calculation of subsequent adjustment coefficients, makes the adaptive adjustment of video frames more accurate, and effectively optimizes the video quality.

[0065] Preferably, generate video frame adjustment coefficients by combining the video frame features and the reference frame; the video frame adjustment coefficients include: a first adjustment coefficient, a second adjustment coefficient, and a third adjustment coefficient;

[0066] The first adjustment coefficient is the brightness adjustment coefficient, and the calculation formula is:

[0067] ;

[0068] Wherein, is the brightness adjustment coefficient; is the adjustment parameter; is the light intensity of the current frame; is the reference light intensity; is the brightness mean value of the current frame; is the brightness mean value of the reference frame; is the adjustment parameter;

[0069] The second adjustment coefficient is the color adjustment coefficient, and the calculation formula is:

[0070] ;

[0071] Wherein, is the color adjustment coefficient; is the saturation mean value of the reference frame; is the saturation mean value of the current frame; is the adjustment parameter; is the adjustment parameter; is the hue value of the current frame; is the hue value of the reference frame; is the total number of frames;

[0072] The third adjustment coefficient is the local gradient adjustment coefficient, and the calculation formula is:

[0073] ;

[0074] Among them, is the local gradient adjustment coefficient; is the adjustment parameter; is the average brightness gradient of the current frame; is the average brightness gradient of the reference frame; is the adjustment parameter; is the variance of the brightness gradient of the current frame; is the variance of the brightness gradient of the reference frame.

[0075] In the embodiments of the present application, by accurately calculating various adjustment coefficients, including the brightness adjustment coefficient, the color adjustment coefficient, and the local gradient adjustment coefficient, reliable adjustment coefficients are provided for the subsequent adaptive adjustment of fish motion video frames. The brightness adjustment coefficient is adjusted based on the light intensity and the average brightness, which can dynamically adjust the brightness of the video frame, ensure the consistency of brightness under different lighting conditions, and improve the overall visual effect of the video; the color adjustment coefficient is adjusted based on the saturation and hue values, which helps to maintain the consistency and realism of colors, reduce color deviation, improve the visual quality of the video frame image, and ensure the accurate presentation of fish color characteristics; the local gradient adjustment coefficient can enhance or suppress the local details and edge information of the image by adjusting the average brightness gradient and variance, improve the clarity of the image, contribute to subsequent multi-object detection and tracking, and enhance the multi-object tracking and detection ability of the system of the present invention.

[0076] Preferably, referring to Figure 2 the flowchart of the adaptive adjustment of fish motion video frames shown in, an adaptive adjustment model of fish motion video frames is constructed according to the video frame adjustment coefficient, and the fish motion video frames are adaptively adjusted to obtain a standard fish motion video; the formula of the standard fish motion video is:

[0077] ;

[0078] Among them, is the pixel value of the standard fish motion video frame; is the pixel value of the input fish motion video frame; is the brightness adjustment coefficient; is the color adjustment coefficient; is the local gradient adjustment coefficient; is the brightness gradient at pixel value; is the adjustment parameter for controlling the gradient enhancement intensity.

[0079] In the embodiments of the present application, by combining the brightness, color, and local gradient adjustment coefficients, the input video frames are normalized to eliminate the illumination and color differences, and a standard fish motion video with uniform quality is generated. This normalization process ensures that subsequent multi-object tracking of fish can still be carried out efficiently and accurately under different illumination and environmental conditions, significantly improving the consistency and reliability of the tracking effect, and further improving the accuracy of subsequent health monitoring.

[0080] Preferably, referring to Figure 3 the structural diagram of the fish multi-object tracking model shown, a fish multi-object tracking model is constructed to perform fish multi-object tracking on the standard fish motion video to obtain fish motion trajectories; the fish multi-object tracking model is constructed by improving the YOLOv8-ByteTrack model, including: a fish motion feature extraction layer, a fish detection layer, and a fish multi-object tracking layer;

[0081] The fish motion feature extraction layer extracts fish features according to the video frames in the standard fish motion video through the MSDA model that combines multi-scale dilated convolution and attention mechanism; the fish features include: fish body area, main texture, and contour;

[0082] The fish detection layer inputs the fish features into the YOLOv8 model for fish detection to obtain the fish detection bounding boxes for each frame;

[0083] The fish multi-object tracking layer performs trajectory association on the detection bounding box marks based on the ByteTrack algorithm and fuses adaptive Kalman filtering to obtain fish multi-object tracking trajectories.

[0084] Table 1 shows the detection results of fish multi-object tracking trajectories in weak light and strong light environments, including: IDF1, IDP, IDR, recall rate, precision, and multi-object precision; where, the IDF1 is the proportion of correctly identified detections to the ground truth and the average number of calculated detections, which combines precision and recall rate; the IDP is the precision of correctly identified detections; the IDR is the recall rate of correctly identified detections; the multi-object precision is the proportion considering false positives, false negatives, and ID switches comprehensively.

[0085] Table 2 shows the comparison of the detection results of fish multi-object tracking trajectories before and after introducing the fish motion video frame adaptive adjustment model.

[0086] Table 1 Detection Results of Fish Multi-Object Tracking Trajectories

[0087]

[0088] Table 2 Comparison of the Improvement of the Fish Motion Video Frame Adaptive Adjustment Model on Detection Performance

[0089] Index Fish multi-object tracking model Fish motion video frame adaptive adjustment model + Fish multi-object tracking model Detection accuracy (%) 78.51 98.43 Track loss rate (%) 15.2 4.7 False track rate (%) 12.4 3.1 Lighting change adaptability score 6.2 9.5

[0090] In the embodiment of the present application, the MSDA model with multi-scale dilated convolution and attention mechanism effectively extracts multi-scale features of fish, including the fish body area, main texture and contour, enhances the fish feature expression ability, and improves the accuracy of multi-object detection and tracking. Based on YOLOv8, fast and accurate fish detection is realized, the detection bounding boxes of each frame are obtained, precise initial information is provided for multi-object tracking, and the overall detection efficiency is improved. Combining the ByteTrack algorithm and adaptive Kalman filtering realizes efficient association and prediction of fish movement trajectories, improves the stability and accuracy of tracking, and ensures that multiple fish can still be accurately tracked in a complex dynamic environment.

[0091] Preferably, according to the fish movement trajectory and the standard fish movement video, a fish health detection model is constructed to detect the health status of each fish in the standard fish movement video; the fish health detection model includes: a fish movement feature extraction layer, a fish appearance feature extraction layer, a fish feature fusion layer, and a fish health monitoring layer;

[0092] Among them, the fish movement feature extraction layer extracts fish movement features according to the fish multi-object tracking trajectory detected by the fish multi-object tracking model; the fish movement features include: speed, acceleration, and turning angle; the fish appearance feature extraction layer extracts fish appearance features according to the standard fish movement video; the fish appearance features include: the hue of the fish, the saturation of the fish, and the brightness of the fish; the fish health feature fusion layer fuses the fish movement features and the fish appearance features to obtain fish health features; the fish health monitoring layer detects the health status of each fish according to the fish health features; the health status includes: normal, abnormal, and diseased.

[0093] Specifically, the fish health detection model is trained using fish samples with known health status that have been labeled; through the fish movement feature extraction layer, according to the fish multi-object tracking trajectory detected by the fish multi-object tracking model and the fish samples with known health status, fish movement features are extracted;

[0094] The fish movement features include: speed, acceleration, and turning angle; through the fish appearance feature extraction layer, fish appearance features are extracted according to the standard fish movement video;

[0095] The fish appearance features include: the hue of the fish, the saturation of the fish, and the brightness of the fish; through the fish health feature fusion layer, the fish movement features and the fish appearance features are fused to obtain fish health features;

[0096] The fish health detection model is constructed through a decision tree model; the fish health monitoring layer inputs the fish health characteristics into the trained fish health detection model to detect the health status of each fish and outputs the health status of each fish; the health status includes: normal, abnormal, and specific diseases.

[0097] Table 3 shows the health status classification and its characteristic indicators of 5 samples for the fish health status detection; during the fish health status detection process, the fish health characteristic thresholds are verified and adjusted by combining the breeding environment and actual monitoring data.

[0098] Table 3 Fish Health Status Detection Results

[0099] Number Motion feature analysis Appearance feature analysis Health status Remarks 001 Normal speed, stable acceleration, normal turning angle Normal hue, moderate saturation, uniform brightness Normal No abnormality 002 Slow speed, occasional unstable acceleration, small turning angle Dark hue, reduced saturation, uneven brightness Abnormal May have malnutrition or early stage of disease 003 High-speed swimming, frequent acceleration, large turning angle Normal hue, high saturation, moderate brightness Normal Active and healthy 004 Stationary, lack of acceleration, extremely small turning angle Abnormal hue, significantly white, reduced brightness Disease May have fish leukosis 005 Irregular speed, occasional high-speed swimming, unstable turning Red-shifted hue, abnormal saturation, fluctuating brightness Abnormal May be affected by environmental stress

[0100] The embodiment of the present application proposes a fish health detection model, which realizes the accurate detection of the health status of each fish; by extracting speed, acceleration, and turning angle as motion characteristics through the fish motion feature extraction layer, the behavior pattern of fish can be analyzed, abnormal motions can be identified, potential health problems can be discovered in time, and the sensitivity and accuracy of health detection can be improved; by extracting hue, saturation, and brightness as appearance characteristics through the fish appearance feature extraction layer, the appearance changes of fish can be monitored, color abnormalities can be identified, potential health problems that can be distinguished by appearance can be discovered in time, and the comprehensiveness of appearance health detection can be improved; by fusing motion and appearance characteristics through the fish feature fusion layer, a comprehensive fish health assessment is provided, which helps to comprehensively detect the health status of fish; by detecting normal, abnormal, and specific disease states through the fish health monitoring layer, the real-time monitoring and classification of the fish health status are realized, enabling the workers in the fish farm to conduct early monitoring and precise management of abnormal fry, improving the breeding efficiency and the fish health level, and reducing the risk of disease transmission.

[0101] In the embodiments of the present application, by introducing a fish motion video frame adaptive adjustment model, a fish multi-object tracking model, and a fish health detection model, accurate multi-object tracking and health monitoring of fish are achieved. This fish multi-object tracking method divides the land-based industrial recirculating aquaculture farm into multiple regions, records and synchronously obtains the light intensity under different lighting conditions, and realizes stable monitoring in various environments. By selecting a reference frame with stable lighting, extracting multi-dimensional video frame features, and generating an adaptive adjustment coefficient to adaptively adjust the fish motion video, standardizing the video quality, and effectively eliminating the interference of lighting changes on tracking. The improved YOLOv8-ByteTrack model combines multi-scale feature extraction and adaptive Kalman filtering to achieve high-precision multi-object tracking. At the same time, the health detection model that fuses motion and appearance features can monitor the health status of fish in real time, helping to detect abnormalities and diseases in a timely manner. This overall method significantly improves the environmental adaptability, tracking accuracy, and health monitoring ability of the farm, realizes efficient management of automation and intelligence, and improves the production efficiency and fish health level of land-based industrial recirculating aquaculture.

[0102] Embodiment 2

[0103] In Embodiment 1, the method proposed by the present invention successfully achieved multi-object tracking and health detection of fish in the farm. To further verify the effectiveness of the present invention, multi-object tracking and health detection of fish were also carried out on another fish pond B in the embodiments of the present application.

[0104] Divide the land-based industrial recirculating aquaculture farm into M regions, and record the fish motion videos of the M regions; the fish motion videos cover different lighting conditions, including: low light, strong light, and normal light; synchronously obtain the light intensity of the M regions through light sensors.

[0105] Preferably, before selecting a video frame that meets the preset conditions from the fish motion video as a reference frame, it further includes synchronizing the time of the fish motion video with the light intensity and recording the light intensity corresponding to each frame of the fish motion video.

[0106] Preferably, select a video frame that meets the preset conditions from the fish motion video as a reference frame; the steps of selecting a video frame that meets the preset conditions from the fish motion video are:

[0107] Set the normal light range, and use the sliding window method to screen out the video frames in the fish motion video whose light intensity is within the normal light range to obtain normal light video frames;

[0108] Calculate the light intensity change rate corresponding to each of the normal light video frames, compare the light intensity change rates, and select the normal light video frame with the smallest light intensity change rate as the reference frame.

[0109] Preferably, based on the fish movement video and the light intensity, extract the video frame features of the fish movement video; the video frame features include: brightness mean, saturation mean, hue value, brightness gradient, and brightness gradient variance.

[0110] Synchronize the light intensity corresponding to each frame of the fish movement video with the video frame features to obtain the video frame features of each frame.

[0111] Preferably, generate video frame adjustment coefficients by combining the video frame features and the reference frame; the video frame adjustment coefficients include: a first adjustment coefficient, a second adjustment coefficient, and a third adjustment coefficient.

[0112] The first adjustment coefficient is the brightness adjustment coefficient, and the calculation formula is:

[0113] ;

[0114] Where is the brightness adjustment coefficient; is the adjustment parameter; is the light intensity of the current frame; is the reference light intensity; is the brightness mean of the current frame; is the brightness mean of the reference frame; is the adjustment parameter;

[0115] The second adjustment coefficient is the color adjustment coefficient, and the calculation formula is:

[0116] ;

[0117] Where is the color adjustment coefficient; is the saturation mean of the reference frame; is the saturation mean of the current frame; is the adjustment parameter; is the adjustment parameter; is the hue value of the current frame; is the hue value of the reference frame; is the total number of frames;

[0118] The third adjustment coefficient is the local gradient adjustment coefficient, and the calculation formula is:

[0119] ;

[0120] Where is the local gradient adjustment coefficient; is the adjustment parameter; is the average brightness gradient of the current frame; is the average brightness gradient of the reference frame; is the adjustment parameter; is the variance of the brightness gradient of the current frame; is the variance of the brightness gradient of the reference frame.

[0121] Preferably, an adaptive adjustment model for fish motion video frames is constructed according to the video frame adjustment coefficient, and the fish motion video frames are adaptively adjusted to obtain a standard fish motion video; the formula for the standard fish motion video is:

[0122] ;

[0123] where is the pixel value of the standard fish motion video frame; is the pixel value of the input fish motion video frame; is the brightness adjustment coefficient; is the color adjustment coefficient; is the local gradient adjustment coefficient; is the brightness gradient at pixel value; is the adjustment parameter for controlling the gradient enhancement intensity.

[0124] Preferably, a fish multi-object tracking model is constructed to perform fish multi-object tracking on the standard fish motion video to obtain fish motion trajectories; the fish multi-object tracking model is constructed by improving the YOLOv8-ByteTrack model, including: a fish motion feature extraction layer, a fish detection layer, and a fish multi-object tracking layer; the fish motion feature extraction layer uses the MSDA model that combines multi-scale dilated convolution and attention mechanism to extract fish features according to the video frames in the standard fish motion video; the fish features include: fish body area, texture, and contour; the fish detection layer inputs the fish features into the YOLOv8 model for fish detection to obtain the fish detection bounding boxes for each frame; the fish multi-object tracking layer performs trajectory association on the detection bounding box marks based on the ByteTrack algorithm and fuses adaptive Kalman filtering to obtain fish multi-object tracking trajectories.

[0125] Preferably, according to the fish motion trajectories and the standard fish motion video, a fish health detection model is constructed to detect the health status of each fish in the standard fish motion video; the fish health detection model includes: a fish motion feature extraction layer, a fish appearance feature extraction layer, a fish feature fusion layer, and a fish health monitoring layer;

[0126] Among them, the fish motion feature extraction layer extracts fish motion features according to the fish multi-object tracking trajectories detected by the fish multi-object tracking model; the fish motion features include: speed, acceleration, and turning angle; the fish appearance feature extraction layer extracts fish appearance features according to the standard fish motion video; the fish appearance features include: the hue of the fish, the saturation of the fish, and the brightness of the fish; the fish health feature fusion layer fuses the fish motion features and the fish appearance features to obtain fish health features; the fish health monitoring layer detects the health status of each fish according to the fish health features; the health status includes: normal, abnormal, and diseased.

[0127] Table 4 shows the health status classification and their characteristic indicators of 5 fish samples in fishpond B, which are the detection results of fish health status.

[0128] Table 4 Detection Results of Fish Health Status

[0129] Number Motion feature analysis Appearance feature analysis Health status Remarks 001 Normal speed, stable acceleration, moderate turning angle Normal hue, moderate saturation, uniform brightness Normal No abnormality 002 Slow speed, continuous deceleration, decreasing turning angle Yellowish hue, decreased saturation, uneven brightness Abnormal May have respiratory problems 003 Normal speed, moderate acceleration, smooth turning Normal hue, moderate saturation, uniform brightness Normal Active and healthy 004 Suddenly high-speed swimming, frequent sudden stops, large turning angle Abnormal hue, mottled color, unstable brightness Disease May have parasite infection 005 Regular swimming, stable acceleration, moderate turning angle Normal hue, moderate saturation, uniform brightness Normal No abnormality

[0130] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-target fish tracking method for land-based factory-scale recirculating aquaculture scenarios, characterized in that: include: Divide the land-based factory recirculating aquaculture farm into M areas, and record fish movement videos in the M areas; The fish motion video covers different lighting conditions, including: weak light, strong light and normal light; the light intensity of the M areas is synchronously obtained by a light sensor; Selecting a video frame that meets a preset condition from the fish motion video as a reference frame; Based on the fish motion video and the light intensity, the video frame features of the fish motion video are extracted, and the video frame adjustment coefficient is generated in combination with the reference frame; the video frame adjustment coefficient includes: a first adjustment coefficient, a second adjustment coefficient and a third adjustment coefficient; the first adjustment coefficient is a brightness adjustment coefficient, and the calculation formula is: ； in, is the brightness adjustment coefficient; To adjust the parameters; is the light intensity of the current frame; is the reference light intensity; is the mean brightness of the current frame; is the mean brightness of the reference frame; To adjust the parameters; The second adjustment coefficient is a color adjustment coefficient, and the calculation formula is: ； in, is the color adjustment coefficient; is the mean saturation value of the reference frame; is the mean saturation value of the current frame; To adjust the parameters; To adjust the parameters; is the hue value of the current frame; is the hue value of the reference frame; is the total number of frames; The third adjustment coefficient is a local gradient adjustment coefficient, and the calculation formula is: ； in, is the local gradient adjustment coefficient; To adjust the parameters; is the brightness gradient mean of the current frame; is the brightness gradient mean of the reference frame; To adjust the parameters; is the brightness gradient variance of the current frame; is the brightness gradient variance of the reference frame; A fish motion video frame adaptive adjustment model is constructed according to the video frame adjustment coefficient, and the fish motion video frame is adaptively adjusted to obtain a standard fish motion video; the formula of the standard fish motion video is: ； in, is the pixel value of a standard fish motion video frame; is the pixel value of the input fish motion video frame; is the brightness adjustment coefficient; is the color adjustment coefficient; is the local gradient adjustment coefficient; is the brightness gradient at pixel The value of To control the adjustment parameters of the gradient enhancement intensity; Constructing a fish multi-target tracking model to perform fish multi-target tracking on the standard fish motion video to obtain the fish motion trajectory; According to the fish movement trajectory and the standard fish movement video, a fish health detection model is constructed to detect the health status of each fish in the standard fish movement video.

2. The method for tracking multiple fish targets in land-based factory-scale circulating aquaculture scenarios according to claim 1 is characterized in that: Before selecting a video frame that meets preset conditions from the fish motion video as a reference frame, the method also includes time synchronization of the fish motion video and the light intensity, and recording the light intensity corresponding to each frame of the fish motion video.

3. The method for tracking multiple fish targets in land-based factory-scale circulating aquaculture scenarios according to claim 1 is characterized in that: The step of selecting a video frame that meets preset conditions from the fish motion video as a reference frame is: Setting a normal light range, and using a sliding window method to filter out video frames in the fish motion video where the light intensity is within the normal light range, to obtain normal light video frames; The illumination intensity change rate corresponding to each normal light video frame is calculated, the illumination intensity change rates are compared, and the normal light video frame with the smallest illumination intensity change rate is selected as the reference frame.

4. The method for tracking multiple fish targets in land-based factory-scale circulating aquaculture scenarios according to claim 1 is characterized in that: The video frame features include: brightness mean, saturation mean, hue value, brightness gradient and brightness gradient variance; The light intensity corresponding to each frame in the fish motion video is synchronized with the video frame feature to obtain the video frame feature of each frame.

5. The method for tracking multiple fish targets in land-based factory-scale circulating aquaculture scenarios according to claim 1 is characterized in that: The fish multi-target tracking model is constructed by improving the YOLOv8-ByteTrack model, including: a fish motion feature extraction layer, a fish detection layer and a fish multi-target tracking layer; The fish motion feature extraction layer extracts fish features according to the video frames in the standard fish motion video by combining the MSDA model of multi-scale dilated convolution and attention mechanism; the fish features include: fish body area, texture and contour; The fish detection layer inputs the fish features into the YOLOv8 model to perform fish detection, and obtains a fish detection bounding box for each frame; The fish multi-target tracking layer performs trajectory association on the detection boundary box markers based on the ByteTrack algorithm and integrates the adaptive Kalman filter to obtain the fish multi-target tracking trajectory.

6. The method for tracking multiple fish targets in land-based factory-scale circulating aquaculture scenarios according to claim 1 is characterized in that: The fish health detection model includes: a fish movement feature extraction layer, a fish appearance feature extraction layer, a fish feature fusion layer and a fish health monitoring layer; The fish motion feature extraction layer extracts fish motion features according to the fish multi-target tracking trajectory detected by the fish multi-target tracking model; the fish motion features include: speed, acceleration and turning angle; The fish appearance feature extraction layer extracts fish appearance features according to the standard fish motion video; the fish appearance features include: fish hue, fish saturation and fish brightness; The fish feature fusion layer fuses the fish movement feature and the fish appearance feature to obtain the fish health feature; The fish health monitoring layer detects the health status of each fish according to the fish health characteristics; the health status includes: normal, abnormal and disease.

Images