Fish multi-target tracking method for land-based factory recirculating aquaculture scene

Abstract

The invention relates to the technical field of computer vision, in particular to a fish multi-target tracking method oriented to a land-based factory recirculating aquaculture scene. The method comprises the following steps: firstly, acquiring a fish movement video and illumination intensity of a land-based factory circulating water farm; secondly, selecting a video frame meeting a preset condition from the fish movement video as a reference frame; then, based on the fish motion video and the illumination intensity, video frame features are extracted, and a video frame adjustment coefficient is generated in combination with the reference frame; then, constructing a fish movement video frame adaptive adjustment model according to the video frame adjustment coefficient, and performing adaptive adjustment on the fish movement video frame to obtain a standard fish movement video; then, constructing a fish multi-target tracking model to perform fish multi-target tracking on the standard fish movement video to obtain a fish movement track; and finally, according to the fish movement track and the standard fish movement video, constructing a fish health detection model to perform health state detection on each fish in the standard fish movement video.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a fish multi-target tracking method for land-based factory-scale recirculating aquaculture scenarios. Background Art

[0002] Aquaculture is an important part of the global food supply. With the development of social economy and the growth of population, people's demand for high-protein food is increasing, and the scale and importance of aquaculture are also increasing. However, with the increase in farming density and the increase in environmental pressure, the aquaculture industry faces problems such as fish diseases, environmental pollution and low farming efficiency. In this context, monitoring the behavior of fish becomes particularly critical. The behavior and movement trajectory of fish can reflect important information such as their health status, feeding status and reproductive status. By tracking and analyzing the movement trajectory of fish, it is possible to monitor the health of fish schools, detect early signs of disease, evaluate the adaptability of the farming environment, and optimize feed delivery strategies, thereby improving farming efficiency and reducing economic losses. This helps to improve the benefits of aquaculture and ensure its sustainable development.

[0003] Traditional fish behavior monitoring methods usually rely on manual visual inspection or simple video analysis tools, which have limitations such as low efficiency, strong subjectivity, and limited accuracy. In recent years, with the continuous advancement of deep learning technology, multi-target tracking algorithms have been increasingly used in animal analysis research. However, in land-based factory recirculating aquaculture scenarios, the dynamic changes in lighting conditions have a significant impact on the detection and tracking performance of multi-target tracking in fish. Therefore, traditional multi-target tracking methods lack the ability to perceive environmental changes, resulting in a decrease in target tracking performance in special environments such as weak light and strong light.

[0004] Therefore, a fish multi-target tracking method was proposed for land-based factory recirculating aquaculture scenarios. Summary of the invention

[0005] The purpose of the present invention is to provide a fish multi-target tracking method for land-based factory-scale recirculating aquaculture scenarios. The present invention first obtains fish motion videos and light intensity of land-based factory-scale recirculating aquaculture farms; secondly, selects video frames that meet preset conditions as reference frames from the fish motion videos; then, based on the fish motion video and light intensity, extracts video frame features, and generates video frame adjustment coefficients in combination with the reference frames; then, constructs a fish motion video frame adaptive adjustment model according to the video frame adjustment coefficient, and adaptively adjusts the fish motion video frame to obtain a standard fish motion video; then, constructs a fish multi-target tracking model to track fish multi-targets in the standard fish motion video to obtain fish motion trajectories; finally, according to the fish motion trajectories and the standard fish motion videos, constructs a fish health detection model to detect the health status of each fish in the standard fish motion video.

[0006] To achieve the above object, the present invention provides the following technical solutions: The fish multi-target tracking method for land-based factory recirculating aquaculture scenarios includes: The land-based factory-scale recirculating aquaculture farm is divided into M areas, and fish movement videos of the M areas are recorded; the fish movement videos cover different lighting conditions, including: weak light, strong light and normal light; the light intensity of the M areas is synchronously obtained through light sensors; Selecting a video frame that meets a preset condition from the fish motion video as a reference frame; Extracting video frame features of the fish motion video according to the fish motion video and the light intensity; generating video frame adjustment coefficients through the video frame features and the reference frame; the video frame adjustment coefficients include: a first adjustment coefficient, a second adjustment coefficient and a third adjustment coefficient; A fish motion video frame adaptive adjustment model is constructed according to the video frame adjustment coefficient, and the fish motion video frame is adaptively adjusted to obtain a standard fish motion video; the formula of the standard fish motion video is: ; in, is the pixel value of a standard fish motion video frame; is the pixel value of the input fish motion video frame; is the brightness adjustment coefficient; is the color adjustment coefficient; is the local gradient adjustment coefficient; is the brightness gradient at pixel The value of To control the adjustment parameters of the gradient enhancement intensity; A fish multi-target tracking model is constructed to perform fish multi-target tracking on the standard fish motion video to obtain the fish motion trajectory; based on the fish motion trajectory and the standard fish motion video, a fish health detection model is constructed to detect the health status of each fish in the standard fish motion video.

[0007] Preferably, before selecting a video frame with preset conditions from the fish motion video as a reference frame, the method further includes time-synchronizing the fish motion video with the light intensity and recording the light intensity corresponding to each frame of the fish motion video.

[0008] Preferably, the step of selecting a video frame that meets preset conditions from the fish motion video as a reference frame is: Setting a normal light range, and using a sliding window method to filter out video frames in the fish motion video where the light intensity is within the normal light range, to obtain normal light video frames; The illumination intensity change rate corresponding to each normal light video frame is calculated, the illumination intensity change rates are compared, and the normal light video frame with the smallest illumination intensity change rate is selected as a reference frame.

[0009] Preferably, the video frame features include: brightness mean, saturation mean, hue value, brightness gradient and brightness gradient variance; the light intensity corresponding to each frame of the fish motion video is synchronized with the video frame features to obtain the video frame features of each frame.

[0010] Preferably, the first adjustment coefficient is a brightness adjustment coefficient, and the calculation formula is: ; in, is the brightness adjustment coefficient; To adjust the parameters; is the light intensity of the current frame; is the reference light intensity; is the mean brightness of the current frame; is the mean brightness of the reference frame; To adjust the parameters; The second adjustment coefficient is a color adjustment coefficient, and the calculation formula is: ; in, is the color adjustment coefficient; is the mean saturation value of the reference frame; is the mean saturation value of the current frame; To adjust the parameters; To adjust the parameters; is the hue value of the current frame; is the hue value of the reference frame; is the total number of frames; The third adjustment coefficient is a local gradient adjustment coefficient, and the calculation formula is: ; in, is the local gradient adjustment coefficient; To adjust the parameters; is the brightness gradient mean of the current frame; is the brightness gradient mean of the reference frame; To adjust the parameters; is the brightness gradient variance of the current frame; is the brightness gradient variance of the reference frame.

[0011] Preferably, a fish motion video frame adaptive adjustment model is constructed according to the video frame adjustment coefficient, and the fish motion video frame is adaptively adjusted to obtain a standard fish motion video; the formula of the standard fish motion video is: ; in, is the pixel value of a standard fish motion video frame; is the pixel value of the input fish motion video frame; is the brightness adjustment coefficient; is the color adjustment coefficient; is the local gradient adjustment coefficient; is the brightness gradient at pixel The value of It is a parameter for controlling the intensity of gradient enhancement.

[0012] Preferably, the fish multi-target tracking model is constructed by improving the YOLOv8-ByteTrack model, including: a fish motion feature extraction layer, a fish detection layer and a fish multi-target tracking layer; The fish motion feature extraction layer extracts fish features based on the video frames of the fish images retained only in the fish motion video samples by combining the MSDA model of multi-scale dilated convolution and attention mechanism; the fish features include: fish body area, main texture and outline; the fish detection layer inputs the fish features into the YOLOv8 model for fish detection to obtain the fish detection bounding box of each frame; the fish multi-target tracking layer associates the detection bounding box marks with ByteTrack algorithm and integrates adaptive Kalman filtering to obtain the fish multi-target tracking trajectory.

[0013] Preferably, the fish health detection model includes: a fish motion feature extraction layer, a fish appearance feature extraction layer, a fish feature fusion layer and a fish health monitoring layer; wherein the fish motion feature extraction layer extracts the fish motion features according to the fish multi-target tracking trajectory detected by the fish multi-target tracking model; the fish motion features include: speed, acceleration and turning angle; the fish appearance feature extraction layer extracts the fish appearance features according to the standard fish motion video; the fish appearance features include: fish hue, fish saturation and fish brightness; the fish feature fusion layer fuses the fish motion features with the fish appearance features to obtain fish health features; the fish health monitoring layer detects the health status of each fish according to the fish health features; the health status includes: normal, abnormal and disease.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention proposes an adaptive adjustment model for fish motion video frames, which extracts video frame features based on fish motion video and light intensity, and generates adjustment coefficients including brightness adjustment coefficient, color adjustment coefficient and local gradient adjustment coefficient in combination with reference frames to adaptively adjust the video frames, thereby obtaining standard fish motion videos. This step accurately generates a variety of adjustment coefficients by comprehensively extracting multidimensional features such as brightness, saturation, hue, brightness gradient and its variance, effectively eliminating the impact of illumination changes on video quality, and achieving consistency and clarity of brightness, color and details. The adaptive adjustment model for fish motion video frames standardizes each frame of video according to these adjustment coefficients, significantly improving the overall visual quality and reducing noise and blur caused by light fluctuations. This not only optimizes the input data quality of the subsequent multi-target tracking algorithm, improves the accuracy and stability of tracking, but also ensures accurate tracking and health monitoring of fish behavior in complex lighting environments, and comprehensively improves the automation and intelligent management level of farms.

[0015] 2. The present invention proposes a fish multi-target tracking model and performs multi-target tracking on the standard fish motion video obtained by the fish motion video frame adaptive adjustment model, which can accurately obtain the motion trajectory of each fish. By using the improved YOLOv8-ByteTrack model combined with multi-scale feature extraction and adaptive Kalman filtering technology, high-precision and high-efficiency multi-target detection and tracking are achieved, ensuring that multiple fish can still be stably tracked in complex lighting and dynamic environments. Accurate motion trajectories can not only monitor the behavior patterns of fish in real time and detect abnormalities and potential diseases in a timely manner, but also provide a reliable data basis for subsequent health detection of fish through fish health detection models.

[0016] 3. The present invention proposes a fish health detection model, which is based on the fish motion trajectory obtained by the fish multi-target tracking model and the standard fish motion video obtained by the fish motion video frame adaptive adjustment model, and can accurately detect the health status of each fish. By analyzing the movement behavior and appearance characteristics of fish, the model can identify normal, abnormal and specific disease states, and realize real-time monitoring and early warning of fish health. This method not only improves the accuracy and timeliness of health detection, reduces the reliance on human monitoring, but also can quickly respond to potential health problems, prevent the spread of diseases and the occurrence of breeding losses. Combining the analysis of motion trajectory and video data, the health detection model significantly improves the level of intelligent farm management, optimizes the breeding environment, ensures the health and production efficiency of fish, and thus comprehensively improves the operating efficiency and sustainable development capabilities of land-based factory-scale recirculating aquaculture farms. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart of a method for tracking multiple fish targets in a land-based factory-scale recirculating aquaculture scenario provided by an embodiment of the present invention; Figure 2 A flow chart of adaptive adjustment of fish motion video frames provided by an embodiment of the present invention; Figure 3 A structural diagram of a fish multi-target tracking model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Aquaculture is an important part of the global food supply. With the development of social economy and the growth of population, people's demand for high-protein food is increasing, and the scale and importance of aquaculture are also increasing. However, with the increase in breeding density and the increase in environmental pressure, the aquaculture industry faces problems such as fish diseases, environmental pollution and low breeding efficiency. The behavior and movement trajectory of fish can reflect important information such as their health status, feeding status and reproductive status. By tracking and analyzing the movement trajectory of fish, it is possible to monitor the health of fish schools, detect early signs of disease, evaluate the adaptability of the breeding environment, and optimize feed delivery strategies, thereby improving breeding efficiency and reducing economic losses.

[0020] The present invention proposes a fish multi-target tracking method for land-based factory-scale recirculating aquaculture scenarios, which realizes multi-target trajectory tracking and health detection of fish in recirculating aquaculture farms. For the specific method flow chart, refer to Figure 1 In order to illustrate that the method of the present invention can play a role in fish multi-target trajectory tracking and health detection, the effectiveness of the present invention will be described from two embodiments below.

[0021] See also Figures 1 to 3 The present invention provides a multi-target fish tracking method for land-based factory-scale circulating aquaculture scenarios, and the technical solution is as follows: Embodiment 1 In the embodiment of the present application, the method proposed in the present invention is used to describe in detail the fish multi-target trajectory tracking and health detection process. In the embodiment of the present application, the fish multi-target trajectory tracking and health detection are aimed at the multi-target trajectory tracking and health detection of healthy California bass with a length of 10-15 cm in a fish pond A in a land-based factory-scale recirculating aquaculture farm. The area of ​​the fish pond A is 20 meters × 20 meters. Please refer to Figure 1 , Figure 1 The specific process of the method proposed in the present invention includes: dividing a land-based factory-scale recirculating aquaculture farm into M areas, obtaining fish motion videos and light intensities of the M areas of the land-based factory-scale recirculating aquaculture farm under different lighting conditions; selecting video frames that meet preset conditions from the fish motion video as reference frames; extracting video frame features of the fish motion video based on the fish motion video and the light intensity; generating video frame adjustment coefficients according to the video frame features and the reference frame; constructing a fish motion video frame adaptive adjustment model according to the video frame adjustment coefficients, and adaptively adjusting the fish motion video frames to obtain standard fish motion videos; constructing a fish multi-target tracking model to perform fish multi-target tracking on the standard fish motion video to obtain fish motion trajectories; constructing a fish health detection model according to the fish motion trajectory and the standard fish motion video to detect the health status of each fish in the standard fish motion video. The following is based on Figure 1 The content of this article describes in detail the multi-target trajectory tracking and health detection process of healthy California bass in fish pond A: The fish multi-target tracking method for land-based factory recirculating aquaculture scenarios includes: The land-based factory-scale recirculating aquaculture farm is divided into M areas, and fish movement videos of the M areas are recorded; the fish movement videos cover different lighting conditions, including: weak light, strong light and normal light; the light intensity of the M areas is synchronously obtained through light sensors; Specifically, fish pond A was divided into 8 areas, and a Hikvision (DS-IPC-B14HV-LT) camera was installed in each area to shoot fish movement videos in the 8 areas of fish pond A at 8 o'clock in the morning on a sunny day, 12 o'clock in the afternoon on a sunny day, and on a cloudy day, respectively, to ensure that the fish movement videos covered three different lighting conditions: weak light, strong light, and normal light; One light sensor is installed in each area and placed near the camera to synchronously obtain the light intensity of the eight areas.

[0022] Preferably, before selecting a video frame that meets preset conditions from the fish motion video as a reference frame, the method also includes time synchronization of the fish motion video and the light intensity, and recording the light intensity corresponding to each frame of the fish motion video.

[0023] In the embodiment of the present application, the fish motion video is time-synchronized with the light intensity to ensure that the light intensity corresponding to each frame of the video is accurately recorded. This measure improves the correspondence accuracy between the light information and the video frame, provides accurate light intensity information for subsequent video frame adaptive adjustment, and improves the quality of the fish motion video and the reliability of multi-target tracking.

[0024] Preferably, a video frame that meets preset conditions is selected from the fish motion video as a reference frame; the step of selecting a video frame that meets preset conditions as a reference frame from the fish motion video is: Setting a normal light range, and using a sliding window method to filter out video frames in the fish motion video where the light intensity is within the normal light range, to obtain normal light video frames; The illumination intensity change rate corresponding to each normal light video frame is calculated, the illumination intensity change rates are compared, and the normal light video frame with the smallest illumination intensity change rate is selected as the reference frame.

[0025] Specifically, the normal light range is set as: ; in, is the low light threshold; is the light intensity; is the bright light threshold; The sliding window method is used to filter out the video frames in the fish motion video where the light intensity is within the normal light range to obtain normal light video frames; the light intensity change rate corresponding to each normal light video frame is calculated, the light intensity change rates are compared and the normal light video frame with the smallest light intensity change rate is selected as the reference frame.

[0026] In the embodiment of the present application, the normal light range is set and the normal light video frames are screened using the sliding window method to ensure that the reference frame is selected under stable lighting conditions, reduce the impact of lighting fluctuations on the reference frame, and improve the representativeness and stability of the reference frame. By calculating and selecting the video frame with the smallest lighting change rate, the consistency of the reference frame in lighting conditions is further ensured, providing a reliable reference frame for subsequent video frame adaptive adjustment, enhancing the stability and consistency of the adjustment effect, and improving the overall video quality.

[0027] Preferably, based on the fish motion video and the light intensity, video frame features of the fish motion video are extracted; the video frame features include: brightness mean, saturation mean, hue value, brightness gradient and brightness gradient variance; The light intensity corresponding to each frame of the fish motion video is synchronized with the video frame feature to obtain the video frame feature of each frame.

[0028] In the embodiment of the present application, by extracting the multi-dimensional video frame features such as brightness mean, saturation mean, hue value, brightness gradient and brightness gradient variance, the brightness, color and detail information of the video frame are fully reflected, the richness of feature expression is enhanced, and the ability of subsequent video frame adaptive adjustment and the accuracy of multi-target tracking are improved. The light intensity is synchronized with the video frame features to ensure the accurate correspondence of the light information during the feature extraction process, and to provide reliable light information for the calculation of the subsequent adjustment coefficient, so that the adaptive adjustment of the video frame is more accurate and the video quality is effectively optimized.

[0029] Preferably, a video frame adjustment coefficient is generated by combining the video frame feature and the reference frame; the video frame adjustment coefficient includes: a first adjustment coefficient, a second adjustment coefficient and a third adjustment coefficient; The first adjustment coefficient is a brightness adjustment coefficient, and the calculation formula is: ; in, is the brightness adjustment coefficient; To adjust the parameters; is the light intensity of the current frame; is the reference light intensity; is the mean brightness of the current frame; is the mean brightness of the reference frame; To adjust the parameters; The second adjustment coefficient is a color adjustment coefficient, and the calculation formula is: ; in, is the color adjustment coefficient; is the mean saturation value of the reference frame; is the mean saturation value of the current frame; To adjust the parameters; To adjust the parameters; is the hue value of the current frame; is the hue value of the reference frame; is the total number of frames; The third adjustment coefficient is a local gradient adjustment coefficient, and the calculation formula is: ; in, is the local gradient adjustment coefficient; To adjust the parameters; is the brightness gradient mean of the current frame; is the brightness gradient mean of the reference frame; To adjust the parameters; is the brightness gradient variance of the current frame; is the brightness gradient variance of the reference frame.

[0030] In the embodiment of the present application, by accurately calculating a variety of adjustment coefficients, including brightness adjustment coefficients, color adjustment coefficients and local gradient adjustment coefficients, reliable adjustment coefficients are provided for the subsequent adaptive adjustment of fish motion video frames. The brightness adjustment coefficient is based on the adjustment of light intensity and brightness mean, which can dynamically adjust the brightness of the video frame to ensure the consistency of brightness under different lighting conditions and improve the overall visual effect of the video; the color adjustment coefficient is based on the adjustment of saturation and hue values, which helps to maintain color consistency and realism, reduce color deviation, improve the visual quality of the video frame image, and ensure the accurate presentation of fish color features; the local gradient adjustment coefficient can enhance or suppress the local details and edge information of the image by adjusting the brightness gradient mean and variance, improve the clarity of the image, and help the subsequent multi-target detection and tracking, and improve the multi-target tracking and detection capabilities of the system of the present invention.

[0031] Preferably, see Figure 2 The flowchart of the adaptive adjustment of fish motion video frames is shown, and a fish motion video frame adaptive adjustment model is constructed according to the video frame adjustment coefficient, and the fish motion video frames are adaptively adjusted to obtain a standard fish motion video; the formula of the standard fish motion video is: ; in, is the pixel value of a standard fish motion video frame; is the pixel value of the input fish motion video frame; is the brightness adjustment coefficient; is the color adjustment coefficient; is the local gradient adjustment coefficient; is the brightness gradient at pixel The value of It is a parameter for controlling the intensity of gradient enhancement.

[0032] In the embodiment of the present application, the input video frames are standardized by combining brightness, color and local gradient adjustment coefficients to eliminate lighting and color differences and generate standard fish motion videos of uniform quality. This standardization process ensures that subsequent multi-target fish tracking can still be carried out efficiently and accurately under different lighting and environmental conditions, significantly improving the consistency and reliability of the tracking effect, thereby improving the accuracy of subsequent health monitoring.

[0033] Preferably, see Figure 3 The structure diagram of the fish multi-target tracking model is shown, and the fish multi-target tracking model is constructed to track the fish multi-targets in the standard fish motion video to obtain the fish motion trajectory; the fish multi-target tracking model is constructed by improving the YOLOv8-ByteTrack model, including: a fish motion feature extraction layer, a fish detection layer and a fish multi-target tracking layer; The fish motion feature extraction layer extracts fish features according to the video frames in the standard fish motion video by combining the MSDA model of multi-scale dilated convolution and attention mechanism; the fish features include: fish body area, main texture and contour; The fish detection layer inputs the fish features into the YOLOv8 model to perform fish detection, and obtains a fish detection bounding box for each frame; The fish multi-target tracking layer performs trajectory association on the detection boundary box markers based on the ByteTrack algorithm and integrates the adaptive Kalman filter to obtain the fish multi-target tracking trajectory.

[0034] Table 1 gives the detection results of multi-target tracking trajectories of fish in weak light and strong light environments, including: IDF1, IDP, IDR, recall rate, precision and multi-target precision; among them, the IDF1 is the ratio of the number of correctly identified detections to the ground truth and the calculated detection average, combining precision and recall rate; the IDP is the precision of correctly identified detections; the IDR is the recall rate of correctly identified detections; the multi-target precision is the ratio of false positives, missed negatives and ID switching.

[0035] Table 2 gives the comparison of the detection results of fish multi-target tracking trajectories before and after the introduction of the fish motion video frame adaptive adjustment model.

[0036] Table 1 Detection results of fish multi-target tracking trajectories

[0037] Table 2 Comparison of the improvement of detection performance by the fish motion video frame adaptive adjustment model index Fish multi-target tracking model Fish motion video frame adaptive adjustment model + fish multi-target tracking model Detection accuracy (%) 78.51 98.43 Track loss rate (%) 15.2 4.7 False trajectory rate (%) 12.4 3.1 Light change adaptability score 6.2 9.5 In the embodiment of the present application, the multi-scale features of fish, including fish body area, main texture and outline, are effectively extracted through the MSDA model of multi-scale void convolution and attention mechanism, the fish feature expression ability is enhanced, and the accuracy of multi-target detection and tracking is improved. Fast and accurate fish detection is achieved based on YOLOv8, and the detection bounding box of each frame is obtained to provide accurate initial information for multi-target tracking and improve the overall detection efficiency. The ByteTrack algorithm and adaptive Kalman filter are combined to achieve efficient fish motion trajectory association and prediction, improve the stability and accuracy of tracking, and ensure that multiple fish can still be accurately tracked in complex dynamic environments.

[0038] Preferably, according to the fish motion trajectory and the standard fish motion video, a fish health detection model is constructed to detect the health status of each fish in the standard fish motion video; the fish health detection model includes: a fish motion feature extraction layer, a fish appearance feature extraction layer, a fish feature fusion layer and a fish health monitoring layer; Among them, the fish motion feature extraction layer extracts the fish motion features according to the fish multi-target tracking trajectory detected by the fish multi-target tracking model; the fish motion features include: speed, acceleration and turning angle; the fish appearance feature extraction layer extracts the fish appearance features according to the standard fish motion video; the fish appearance features include: fish hue, fish saturation and fish brightness; the fish health feature fusion layer fuses the fish motion features and the fish appearance features to obtain fish health features; the fish health monitoring layer detects the health status of each fish according to the fish health features; the health status includes: normal, abnormal and disease.

[0039] Specifically, the fish health detection model is trained using labeled fish samples of known health status; the fish motion features are extracted through the fish motion feature extraction layer according to the fish multi-target tracking trajectories detected by the fish multi-target tracking model and the fish samples of known health status; The fish motion features include: speed, acceleration and turning angle; the fish appearance features are extracted according to the standard fish motion video through the fish appearance feature extraction layer; The fish appearance features include: fish hue, fish saturation and fish brightness; the fish movement features and the fish appearance features are fused through the fish health feature fusion layer to obtain fish health features; The fish health detection model is constructed through a decision tree model; the fish health monitoring layer inputs the fish health characteristics into the trained fish health detection model to detect the health status of each fish, and outputs the health status of each fish; the health status includes: normal, abnormal and specific diseases.

[0040] Table 3 gives the health status classification of 5 samples and their characteristic indicators and fish health status detection results; during the fish health status detection process, the fish health characteristic thresholds were verified and adjusted by combining the breeding environment and actual monitoring data.

[0041] Table 3 Results of fish health status test serial number Movement feature analysis Appearance feature analysis Health status Remark 001 Normal speed, steady acceleration, normal turning angle Normal hue, moderate saturation, uniform brightness normal No abnormality 002 Slow speed, occasional unstable acceleration, small turning angle Dark hue, reduced saturation, uneven brightness abnormal Possibly malnutrition or early stage of illness 003 High-speed swimming, frequent acceleration, large turning angles Normal hue, high saturation, moderate brightness normal Active Health 004 Stagnant, lack of acceleration, very tight turning angles Abnormal color tone, noticeably white, reduced brightness disease Possible fish leukemia 005 Irregular speed, occasional high-speed swimming, unstable turns Reddish hue, abnormal saturation, and fluctuating brightness abnormal May be affected by environmental stress The embodiment of the present application proposes a fish health detection model, which realizes accurate detection of the health status of each fish; through the fish motion feature extraction layer, speed, acceleration and turning angle are extracted as motion features, the behavior pattern of the fish can be analyzed, abnormal motion can be identified, potential health problems can be discovered in time, and the sensitivity and accuracy of health detection can be improved; through the fish appearance feature extraction layer, hue, saturation and brightness can be extracted as appearance features, the appearance changes of the fish can be monitored, color anomalies can be identified, potential health problems that can be identified by appearance can be discovered in time, and the comprehensiveness of appearance health detection can be improved; through the fish feature fusion layer, motion and appearance features are integrated to provide a comprehensive fish health assessment, which helps to comprehensively detect the health status of fish; through the fish health monitoring layer, normal, abnormal and specific disease states are detected, and real-time monitoring and classification of the health status of fish are realized, so that workers in fish farms can conduct early monitoring and precise management of abnormal fry, improve breeding efficiency and fish health level, and reduce the risk of disease transmission.

[0042] The embodiment of the present application realizes accurate multi-target tracking and health monitoring of fish by introducing an adaptive adjustment model of fish motion video frames, a multi-target tracking model of fish, and a health detection model of fish. The multi-target tracking method of fish realizes stable monitoring in various environments by dividing the land-based factory-scale circulating water aquaculture farm into multiple areas, and recording and synchronously acquiring the light intensity under different lighting conditions. By selecting a reference frame with stable lighting, extracting multi-dimensional video frame features and generating adaptive adjustment coefficients, the fish motion video is adaptively adjusted, the video quality is standardized, and the interference of lighting changes on tracking is effectively eliminated. The improved YOLOv8-ByteTrack model combines multi-scale feature extraction and adaptive Kalman filtering to achieve high-precision multi-target tracking. At the same time, the health detection model that integrates motion and appearance features can monitor the health status of fish in real time, which helps to detect abnormalities and diseases in a timely manner. This overall method significantly improves the environmental adaptability, tracking accuracy and health monitoring capabilities of the farm, realizes efficient management of automation and intelligence, and improves the production efficiency and fish health level of land-based factory-scale circulating water aquaculture.

[0043] Embodiment 2 In Example 1, the method proposed in the present invention successfully achieved multi-target tracking and health detection of fish in a farm. To further verify the effectiveness of the present invention, multi-target tracking and health detection of fish in another fish pond B was also performed in the present application example.

[0044] The land-based factory-scale recirculating aquaculture farm is divided into M areas, and the fish movement videos of the M areas are recorded; the fish movement videos cover different lighting conditions, including: weak light, strong light and normal light; the light intensity of the M areas is synchronously acquired through light sensors.

[0045] Preferably, before selecting a video frame that meets preset conditions from the fish motion video as a reference frame, the method also includes time synchronization of the fish motion video and the light intensity, and recording the light intensity corresponding to each frame of the fish motion video.

[0046] Preferably, a video frame that meets preset conditions is selected from the fish motion video as a reference frame; the step of selecting a video frame that meets preset conditions as a reference frame from the fish motion video is: Setting a normal light range, and using a sliding window method to filter out video frames in the fish motion video where the light intensity is within the normal light range, to obtain normal light video frames; The illumination intensity change rate corresponding to each normal light video frame is calculated, the illumination intensity change rates are compared, and the normal light video frame with the smallest illumination intensity change rate is selected as a reference frame.

[0047] Preferably, based on the fish motion video and the light intensity, video frame features of the fish motion video are extracted; the video frame features include: brightness mean, saturation mean, hue value, brightness gradient and brightness gradient variance; The light intensity corresponding to each frame of the fish motion video is synchronized with the video frame feature to obtain the video frame feature of each frame.

[0048] Preferably, a video frame adjustment coefficient is generated by combining the video frame feature and the reference frame; the video frame adjustment coefficient includes: a first adjustment coefficient, a second adjustment coefficient and a third adjustment coefficient; The first adjustment coefficient is a brightness adjustment coefficient, and the calculation formula is: ; in, is the brightness adjustment coefficient; To adjust the parameters; is the light intensity of the current frame; is the reference light intensity; is the mean brightness of the current frame; is the mean brightness of the reference frame; To adjust the parameters; The second adjustment coefficient is a color adjustment coefficient, and the calculation formula is: ; in, is the color adjustment coefficient; is the mean saturation value of the reference frame; is the mean saturation value of the current frame; To adjust the parameters; To adjust the parameters; is the hue value of the current frame; is the hue value of the reference frame; is the total number of frames; The third adjustment coefficient is a local gradient adjustment coefficient, and the calculation formula is: ; in, is the local gradient adjustment coefficient; To adjust the parameters; is the brightness gradient mean of the current frame; is the brightness gradient mean of the reference frame; To adjust the parameters; is the brightness gradient variance of the current frame; is the brightness gradient variance of the reference frame.

[0049] Preferably, a fish motion video frame adaptive adjustment model is constructed according to the video frame adjustment coefficient, and the fish motion video frame is adaptively adjusted to obtain a standard fish motion video; the formula of the standard fish motion video is: ; in, is the pixel value of a standard fish motion video frame; is the pixel value of the input fish motion video frame; is the brightness adjustment coefficient; is the color adjustment coefficient; is the local gradient adjustment coefficient; is the brightness gradient at pixel The value of It is a parameter for controlling the intensity of gradient enhancement.

[0050] Preferably, a fish multi-target tracking model is constructed to perform fish multi-target tracking on the standard fish motion video to obtain the fish motion trajectory; the fish multi-target tracking model is constructed by improving the YOLOv8-ByteTrack model, including: a fish motion feature extraction layer, a fish detection layer and a fish multi-target tracking layer; the fish motion feature extraction layer extracts fish features according to the video frames in the standard fish motion video by combining the MSDA model of multi-scale hole convolution and attention mechanism; the fish features include: fish body area, texture and contour; the fish detection layer inputs the fish features into the YOLOv8 model for fish detection to obtain a fish detection bounding box for each frame; the fish multi-target tracking layer performs trajectory association on the detection bounding box mark based on the ByteTrack algorithm and integrates the adaptive Kalman filter to obtain the fish multi-target tracking trajectory.

[0051] Preferably, according to the fish motion trajectory and the standard fish motion video, a fish health detection model is constructed to detect the health status of each fish in the standard fish motion video; the fish health detection model includes: a fish motion feature extraction layer, a fish appearance feature extraction layer, a fish feature fusion layer and a fish health monitoring layer; Among them, the fish motion feature extraction layer extracts the fish motion features according to the fish multi-target tracking trajectory detected by the fish multi-target tracking model; the fish motion features include: speed, acceleration and turning angle; the fish appearance feature extraction layer extracts the fish appearance features according to the standard fish motion video; the fish appearance features include: fish hue, fish saturation and fish brightness; the fish health feature fusion layer fuses the fish motion features and the fish appearance features to obtain fish health features; the fish health monitoring layer detects the health status of each fish according to the fish health features; the health status includes: normal, abnormal and disease.

[0052] Table 4 gives the health status classification and characteristic indicators of the five fish samples in fish pond B and the results of fish health status detection.

[0053] Table 4 Results of fish health status test serial number Movement feature analysis Appearance feature analysis Health status Remark 001 Normal speed, stable acceleration, moderate turning angle Normal hue, moderate saturation, uniform brightness normal No abnormality 002 Slow speed, continuous deceleration, and reduced turning angle Yellowish hue, decreased saturation, uneven brightness abnormal Possible respiratory problems 003 Normal speed, moderate acceleration, smooth turning Normal hue, moderate saturation, uniform brightness normal Active Health 004 Sudden high-speed swimming, frequent emergency stops, and large turning angles Abnormal hue, mottled color, unstable brightness disease Possible parasitic infection 005 Regular swimming, stable acceleration, moderate turning angle Normal hue, moderate saturation, uniform brightness normal No abnormality Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-target fish tracking method for land-based factory-scale recirculating aquaculture scenarios, characterized in that: include: Divide the land-based factory recirculating aquaculture farm into M areas, and record fish movement videos in the M areas; The fish motion video covers different lighting conditions, including: weak light, strong light and normal light; the light intensity of the M areas is synchronously obtained by a light sensor; Selecting a video frame that meets a preset condition from the fish motion video as a reference frame; Based on the fish motion video and the light intensity, extract the video frame features of the fish motion video, and generate a video frame adjustment coefficient in combination with the reference frame; the video frame adjustment coefficient includes: a first adjustment coefficient, a second adjustment coefficient and a third adjustment coefficient; According to the video frame adjustment coefficient, a fish motion video frame adaptive adjustment model is constructed, and the fish motion video frame is adaptively adjusted to obtain a standard fish motion video; the formula of the standard fish motion video is: ； in, is the pixel value of a standard fish motion video frame; is the pixel value of the input fish motion video frame; is the brightness adjustment coefficient; is the color adjustment coefficient; is the local gradient adjustment coefficient; is the brightness gradient at pixel The value of To control the adjustment parameters of the gradient enhancement intensity; Constructing a fish multi-target tracking model to perform fish multi-target tracking on the standard fish motion video to obtain the fish motion trajectory; According to the fish movement trajectory and the standard fish movement video, a fish health detection model is constructed to detect the health status of each fish in the standard fish movement video.

2. The method for tracking multiple fish targets in land-based factory-scale circulating aquaculture scenarios according to claim 1 is characterized in that: Before selecting a video frame that meets preset conditions from the fish motion video as a reference frame, the method also includes time synchronization of the fish motion video and the light intensity, and recording the light intensity corresponding to each frame of the fish motion video.

3. The method for tracking multiple fish targets in land-based factory-scale circulating aquaculture scenarios according to claim 1 is characterized in that: The step of selecting a video frame that meets preset conditions from the fish motion video as a reference frame is: Setting a normal light range, and using a sliding window method to filter out video frames in the fish motion video where the light intensity is within the normal light range, to obtain normal light video frames; The illumination intensity change rate corresponding to each normal light video frame is calculated, the illumination intensity change rates are compared, and the normal light video frame with the smallest illumination intensity change rate is selected as the reference frame.

4. The method for tracking multiple fish targets in land-based factory-scale circulating aquaculture scenarios according to claim 1 is characterized in that: The video frame features include: brightness mean, saturation mean, hue value, brightness gradient and brightness gradient variance; The light intensity corresponding to each frame in the fish motion video is synchronized with the video frame feature to obtain the video frame feature of each frame.

5. The method for tracking multiple fish targets in land-based factory-scale circulating aquaculture scenarios according to claim 1 is characterized in that: The first adjustment coefficient is the brightness adjustment coefficient, and the calculation formula is: ； in, is the brightness adjustment coefficient; To adjust the parameters; is the light intensity of the current frame; is the reference light intensity; is the mean brightness of the current frame; is the mean brightness of the reference frame; To adjust the parameters; The second adjustment coefficient is the color adjustment coefficient, and the calculation formula is: ； in, is the color adjustment coefficient; is the mean saturation value of the reference frame; is the mean saturation value of the current frame; To adjust the parameters; To adjust the parameters; is the hue value of the current frame; is the hue value of the reference frame; is the total number of frames; The third adjustment coefficient is the local gradient adjustment coefficient, and the calculation formula is: ； in, is the local gradient adjustment coefficient; To adjust the parameters; is the brightness gradient mean of the current frame; is the brightness gradient mean of the reference frame; To adjust the parameters; is the brightness gradient variance of the current frame; is the brightness gradient variance of the reference frame.

6. The method for tracking multiple fish targets in land-based factory-scale circulating aquaculture scenarios according to claim 1 is characterized in that: A fish motion video frame adaptive adjustment model is constructed according to the video frame adjustment coefficient, and the fish motion video frame is adaptively adjusted to obtain a standard fish motion video; the formula of the standard fish motion video is: ； in, is the pixel value of a standard fish motion video frame; is the pixel value of the input fish motion video frame; is the brightness adjustment coefficient; is the color adjustment coefficient; is the local gradient adjustment coefficient; is the brightness gradient at pixel The value of It is a parameter for controlling the intensity of gradient enhancement.

7. The method for tracking multiple fish targets in land-based factory-scale circulating aquaculture scenarios according to claim 1 is characterized in that: The fish multi-target tracking model is constructed by improving the YOLOv8-ByteTrack model, including: a fish motion feature extraction layer, a fish detection layer and a fish multi-target tracking layer; The fish motion feature extraction layer extracts fish features according to the video frames in the standard fish motion video by combining the MSDA model of multi-scale dilated convolution and attention mechanism; the fish features include: fish body area, texture and contour; The fish detection layer inputs the fish features into the YOLOv8 model to perform fish detection, and obtains a fish detection bounding box for each frame; The fish multi-target tracking layer performs trajectory association on the detection boundary box markers based on the ByteTrack algorithm and integrates the adaptive Kalman filter to obtain the fish multi-target tracking trajectory.

8. The method for tracking multiple fish targets in land-based factory-scale circulating aquaculture scenarios according to claim 1 is characterized in that: The fish health detection model includes: a fish movement feature extraction layer, a fish appearance feature extraction layer, a fish feature fusion layer and a fish health monitoring layer; The fish motion feature extraction layer extracts fish motion features according to the fish multi-target tracking trajectory detected by the fish multi-target tracking model; the fish motion features include: speed, acceleration and turning angle; The fish appearance feature extraction layer extracts fish appearance features according to the standard fish motion video; the fish appearance features include: fish hue, fish saturation and fish brightness; The fish feature fusion layer fuses the fish movement feature and the fish appearance feature to obtain the fish health feature; The fish health monitoring layer detects the health status of each fish according to the fish health characteristics; the health status includes: normal, abnormal and disease.

Images