A computer vision-based continuous monitoring and stress early warning method and system for shellfish behavior

Abstract

The present application belongs to the technical field of aquaculture, and discloses a shellfish behavior continuous monitoring and stress early warning method and system based on computer vision. The method uses a fixed camera to continuously record videos of cultured shellfish, extracts image frames at minute-level intervals through FFmpeg, and uses a YOLOv8n target detection model to automatically classify the shell valve state (open / close) of each individual, and calculate behavior indicators such as opening rate and state switching rate. The present application proposes to use the sign inversion (from positive to negative) of the Pearson correlation coefficient between the opening rate and the state switching rate as the discrimination mark of the stress state at the group level, and realizes early warning 3-5 days before death based on the multi-dimensional behavior trajectory (opening rate decrease + switching rate anomaly + circadian rhythm disorder) of the individual. The method is completely non-contact and does not require physical sensors, and is suitable for aquaculture management and marine environment monitoring.

Description

Technical Field

[0001] This invention belongs to the field of aquaculture technology, and more specifically, relates to a method and system for continuous monitoring and stress early warning of shellfish behavior based on computer vision. Background Technology

[0002] Bivalve mollusks (such as oysters, mussels, and clams) are important economic species in global aquaculture and key indicator organisms of the health of marine and estuarine ecosystems. Bivalves perform basic physiological processes such as filter feeding, respiration, and osmolar regulation through the opening and closing of their vomeres; their gping behavior directly reflects the individual's physiological state and environmental adaptability. Against the backdrop of global climate change, subtropical and tropical estuarine regions face increasingly frequent complex environmental stress events, including extreme heat waves followed by drastic salinity fluctuations. These combined stresses pose a serious threat to the survival and productivity of farmed shellfish. Therefore, developing technologies capable of real-time, continuous, and quantitative monitoring of shellfish behavior and providing early warnings of stress is of significant practical importance for aquaculture management and environmental monitoring.

[0003] Currently, monitoring the behavior of shell valves in mollusks mainly relies on the following technical solutions:

[0004] (1) Manual observation method: Researchers visually record the opening and closing state of the shell valve at regular intervals. This method is labor-intensive, has low temporal resolution (usually on the order of hours or even days), is highly subjective, and cannot achieve continuous monitoring at night or for long periods of time, making it difficult to capture rapid behavioral changes under stress conditions.

[0005] (2) Hall effect sensor method (valvometry): Miniature magnets and Hall effect sensors are attached to the shell lobes respectively, and the degree of opening and closing of the shell lobes is continuously recorded by detecting changes in magnetic field strength. Although this method can provide high-frequency (second-level to ten-second-level) continuous data, it has the following significant drawbacks: it requires physical attachment to the shell lobes of each individual, which may interfere with the animal's natural behavior; the installation of sensors and magnets requires professional operation, which limits the ability to deploy on a large scale; long-term immersion in seawater can easily lead to sensor corrosion and signal drift; the equipment cost is high, which is not conducive to its promotion and application in aquaculture scenarios.

[0006] (3) Infrared photoplethysmography (IR-PPG): This method non-invasively detects heart rate changes by placing an infrared photoelectric sensor near the pericardial region of the shellfish. This method can provide valuable cardiovascular physiological indicators, but it also requires the probe to be physically contacted or fixed to the individual; and it is limited by the number of probes (usually only a few to a dozen), making it difficult to monitor a large number of individuals at the same time; in addition, this method can only obtain heart rate information and cannot directly reflect the shell valve behavior pattern.

[0007] In recent years, deep learning-based computer vision technology has made significant progress in the aquaculture field. Object detection architectures, such as YOLO (You Only Look Once), can perform rapid and accurate automated analysis of video data and have been successfully applied to scenarios such as fish detection and behavior recognition. However, the application of computer vision technology to the automatic classification of bivalve shell lobe states and the continuous quantification of behavior, especially to achieving non-contact monitoring with minute-level temporal resolution in controlled stress experiments, remains a gap in the field.

[0008] Furthermore, existing technologies generally lack a comprehensive stress early warning indicator system based on behavioral dynamics characteristics. Existing research mainly focuses on single indicators such as the degree or frequency of shell flap opening and closing, and has not yet established a method to use changes in the correlation between opening rate and switching rate as a stress identification marker, nor has it developed a mortality prediction model based on multidimensional behavioral trajectories. Summary of the Invention

[0009] In view of the above-mentioned defects in the existing technology, the present invention first provides a method for continuous monitoring of shellfish behavior based on computer vision.

[0010] The second objective of this invention is to provide a computer vision-based method for early warning of shellfish mortality risk.

[0011] The third objective of this invention is to provide a continuous monitoring and stress early warning system for shellfish behavior based on a computer vision system.

[0012] The objective of this invention is achieved through the following technical solution:

[0013] A method for continuous monitoring of shellfish behavior based on computer vision includes the following steps:

[0014] (1) Video capture: Use fixed-installation camera equipment to record 24-hour continuous video of shellfish;

[0015] (2) Use the FFmpeg multimedia framework to extract frames at fixed intervals from the continuous video stream obtained in step (1) to achieve a minute-level time resolution image sequence;

[0016] (3) Automatic identification of shell lobe state: Input the image frames extracted in step (2) into the pre-trained YOLOv8n target detection model to automatically identify the shell lobe state of each mollusk and classify it as open or closed;

[0017] (4) Calculate the opening rate and state switching rate of each shellfish based on the classification results in (3);

[0018] (5) Calculate the Pearson correlation coefficient based on the opening ratio and state switching rate in (4);

[0019] (6) When the Pearson correlation coefficient calculated in (5) changes from a positive value to a negative value, the shellfish is determined to be in a state of stress.

[0020] Preferably, in the above method, the detection accuracy mAP@0.5 of the YOLOv8n target detection model in step (3) is not less than 98%.

[0021] Preferably, in the above method, the opening rate in step (4) includes the minute-by-minute opening rate and the daily average opening rate;

[0022] Among them, the minute-by-minute opening rate formula ① is to determine the shell lobe state of each individual i at time point t based on the YOLO classification results:

[0023] These are not pre-given constants, but rather observations obtained directly from the model's inference results for each frame of image;

[0024] Formula ②, which calculates the average daily opening rate for each individual, is used to determine the average daily opening rate.

[0025] in This represents the number of minutes of valid observations within day d. It is derived from formula ① The derived quantity is then calculated further.

[0026] Preferably, in the above method, the state switching rate detection in step (4) includes the following steps:

[0027] S1, Switching Event Detection

[0028] When the state of the shell lobe changes between two adjacent time points, it is recorded as a switching event:

[0029] Formula ③ where A value of 1 indicates that a state transition occurred at minute t, while a value of 0 indicates that the state remained unchanged.

[0030] S2, Switching Rate Calculation

[0031] The state transition rate is obtained by summing the transition events in hours:

[0032] Formula ④, the daily switching rate is the average of 24 hours. It is derived from formula ③. Further statistical analysis yielded the derived quantities.

[0033] As a preferred technical solution, the above method includes the following steps:

[0034] Step 1: Video Acquisition: A fixed, high-definition camera system is used to continuously record video of the shellfish in the culture containers for 24 hours. The camera system is equipped with an infrared supplementary light to ensure image quality at night. The video is stored in H.265 / HEVC encoding format to reduce storage costs while maintaining image quality.

[0035] Step 2: Video Frame Extraction: The FFmpeg multimedia framework is used to extract frames from the continuous video stream at fixed intervals. The frame rate filter parameter is set to fps=1 / 60, meaning one frame is extracted every 60 seconds, achieving a minute-level temporal resolution image sequence. A sequential reading mode is used to avoid efficiency loss caused by repeated addressing in a HEVC stream with sparse keyframes.

[0036] Step 3: Automatic Identification of Oyster Lobe State: The extracted image frames are input into a pre-trained YOLOv8n (You Only Look Once, Version 8, Nanoscale) target detection model. This model is optimized through iterative training, and the final version achieves a detection accuracy of mAP@0.5=98.4% and mAP@0.5-0.95=98.1% on the test set. The model outputs a classification result (open / closed), confidence score, and bounding box coordinates for each oyster in each frame. The detection results are exported as a structured CSV data file.

[0037] Step 4: Opening Rate Calculation: For each individual, the shell lobe state is determined at each time point based on the model classification results. The opening rate (OR) is defined as a continuous scale from 0 to 1, where 1 indicates the shell lobe is fully open and 0 indicates the shell lobe is fully closed. The daily average opening rate is calculated as a basic indicator for behavioral quantification.

[0038] Step 5: Calculation of the Switching Rate: Define a lobe state switch as an event where the lobe state (open / closed) changes between two adjacent time points. A continuous aperture ratio is converted into discrete open / closed states using a binarization threshold of 0.5. The switching rate (SR) is calculated as the number of state switches per unit time (hour) (switches / hour). This metric reflects the dynamic characteristics of lobe motion and can capture behavioral pattern changes that conventional aperture ratio metrics cannot reflect.

[0039] Step Six: Stress State Identification – Correlation Reversal Detection: Calculate the Pearson correlation coefficient between the opening rate and state switching rate for each group of individuals. Under normal (non-stress) conditions, the opening rate and state switching rate are positively correlated (i.e., individuals with active shell opening also have a higher state switching frequency, reflecting normal filter feeding and respiratory regulation activities). When subjected to environmental stress, this correlation reverses sign, becoming negative (i.e., individuals with a reduced opening rate exhibit a higher state switching frequency, indicating that shell flap control dynamics are disturbed). In this invention, the correlation coefficient for the control group is r = +0.315 (positive correlation), and for the treatment group it is r = -0.231 (negative correlation). The sign reversal of the Pearson correlation coefficient from positive to negative is used as a discriminant for population-level stress state.

[0040] This invention also provides a computer vision-based method for early warning of shellfish mortality risk, which triggers an early warning signal when an individual shellfish exhibits the following combination of characteristics:

[0041] (1) The average daily opening rate has been declining for more than 3 consecutive days; and,

[0042] (2) The shellfish state switching rate increased and deviated from the population mean by 1.5 standard deviations; and / or,

[0043] (3) The difference between the nighttime opening rate and the daytime opening rate is less than the threshold of 0.05.

[0044] Mortality risk assessment is based on individual-level behavioral trajectories. An early warning signal is triggered when an individual exhibits the following combined characteristics: (a) a continuous downward trend in the average daily opening rate over several days; (b) an abnormally high state switching rate, deviating from the group mean; and (c) disrupted diurnal behavioral rhythms, with a reduced difference between nighttime and daytime opening rates. In the experimental verification of this invention, the deceased individuals exhibited the above-mentioned abnormal behavioral patterns 3 to 5 days before death, providing a time window for intervention.

[0045] The present invention also provides the application of the above method in shellfish farming.

[0046] This invention also provides the application of the above-mentioned computer vision-based shellfish mortality risk early warning method in shellfish aquaculture management.

[0047] The present invention also provides a continuous monitoring and stress early warning system for shellfish behavior based on a computer vision system, comprising: a video acquisition module, a video storage module, a computing and processing module, and an early warning output module.

[0048] Preferably, the computation processing module includes:

[0049] The frame extraction module is used to extract image frames from continuous video at set time intervals.

[0050] The shell lobe state recognition module deploys a pre-trained target detection model to automatically identify the open or closed state of the shell lobes of each individual in each frame of the image.

[0051] The behavior index calculation module is used to calculate the opening ratio and state switching rate based on the shell lobe state classification results;

[0052] The early warning output module is used to monitor changes in the correlation between the opening rate and the state switching rate, as well as abnormal individual behavioral trajectories, and output group stress state assessment and individual mortality risk warning signals.

[0053] More preferably, the target detection model in the shell lobe state recognition module is the YOLOv8n model, with no more than 5M parameters and no more than 10 GFLOPs computation.

[0054] In the above-mentioned continuous monitoring and stress early warning system for shellfish behavior based on computer vision system, (1) video acquisition module: including high-definition camera device, infrared fill light and video storage device, used for 24-hour continuous video recording of aquaculture containers; (2) frame extraction module: based on FFmpeg framework to realize video frame extraction at fixed intervals (1 frame per minute); (3) shell lobe state recognition module: deploy pre-trained YOLOv8n target detection model to automatically classify the shell lobe state of the extracted image frames; (4) behavior index calculation module: calculates behavior indicators such as opening rate and state switching rate according to the classification results; (5) early warning output module: by monitoring the changes in the correlation between opening rate and switching rate and the abnormality of individual behavior trajectory, outputs the group stress state assessment and individual mortality risk early warning signal.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] (1) Completely non-contact monitoring: The present invention adopts a pure video + computer vision solution, which does not require the installation of any physical sensors, magnets or electrodes on individual shellfish, completely eliminating the interference of monitoring equipment on the natural behavior of animals. The acquired behavioral data can more realistically reflect the natural response patterns of shellfish under different environmental conditions.

[0057] (2) High temporal resolution continuous monitoring: It realizes automatic identification of shell lobe status and behavior quantification with minute-level temporal resolution. The time accuracy is significantly better than traditional manual observation methods (usually hour-level). It can capture rapid behavioral changes and short-term microclosing events under stress conditions.

[0058] (3) High detection accuracy: After iterative optimization, the YOLOv8n model achieved a detection accuracy of mAP@0.5=98.4%, which ensured the reliability of shell lobe state classification and provided high-quality basic data for subsequent behavioral index calculation.

[0059] (4) Innovative behavioral stress indicators: For the first time, the sign of the Pearson correlation coefficient between the opening rate and the state switching rate was reversed (from positive to negative) as a discriminant of the stress state at the population level. This indicator has clear biological significance and statistical operability.

[0060] (5) Mortality prediction capability: Based on multidimensional behavioral trajectory analysis (declining opening rate + abnormally high switching rate + circadian rhythm disorder), it can issue early warning signals 3 to 5 days before individual death, providing a valuable time window for aquaculture managers to take intervention measures.

[0061] (6) Low cost and scalability: The system only requires conventional camera equipment and general computing platform, without the need for expensive dedicated sensor hardware, making it suitable for large-scale deployment; the YOLOv8n nanoscale model has only 3.01M parameters and low computing cost (8.1 GFLOPs), and can run on edge computing devices, which is conducive to its practical application in aquaculture scenarios. Attached Figure Description

[0062] Figure 1 This is a diagram of the overall system architecture of the present invention; it shows the connection and data flow between the video acquisition module (breeding container, high-definition camera, infrared supplementary light, environmental parameter sensor), video storage module (NVR / local storage, H.265 / HEVC encoding), computing and processing module (GPU workstation, FFmpeg frame extraction engine, YOLOv8n inference engine, Python data analysis) and early warning output module (structured CSV data, group stress state assessment, individual mortality risk early warning signal);

[0063] Figure 2 This is a data processing flowchart; it shows the complete processing chain starting from continuous video recording (H.265 / HEVC, 24 hours), sequentially going through FFmpeg frame extraction (fps=1 / 60, 1 frame per minute), image preprocessing (scaling to 640×640, brightness normalization), YOLOv8n target detection inference (shell lobe state classification: open / closed, mAP@0.5=98.4%), structured data export (CSV), then splitting into two parallel branches: aperture ratio (OR) calculation and state switching rate (SR) calculation, which are then combined to perform Pearson correlation analysis (OR vs SR, calculated by group), and stress state is determined by correlation inversion detection (r changes from positive to negative), and finally outputting a death risk warning signal through individual behavioral trajectory monitoring;

[0064] Figure 3This is an example of the state detection performance of the YOLOv8n model; it shows the inference results of the model on the validation set image frames, with each oyster labeled with its classification category (open / closed), confidence score, and bounding box coordinates;

[0065] Figure 4 The Pearson correlation scatter plot shows the relationship between opening rate and state switching rate. The left plot (or blue) represents the control group, where the opening rate and state switching rate are positively correlated (r=+0.315, p<0.001). The right plot (or red) represents the treatment group, where the opening rate and state switching rate are negatively correlated (r=−0.231, p<0.001), demonstrating the inversion of the correlation sign, which is the core basis for stress state discrimination in this invention.

[0066] Figure 5 This is a flowchart illustrating the logic for determining a mortality warning. It shows the three-level condition determination process after inputting an individual's behavioral time series: Condition 1 - The daily average opening rate decreases for ≥3 consecutive days and is lower than the population mean minus one standard deviation; Condition 2 - The daily average state switching rate deviates from the population mean by more than 1.5 standard deviations; Condition 3 (optional auxiliary condition) - The difference between daytime and nighttime opening rates is less than 0.05, i.e., the daytime and nighttime behavioral rhythm is disordered; If both Condition 1 and Condition 2 are met simultaneously, a mortality risk warning signal is triggered; If either condition is not met, it is determined to be a normal state, and monitoring continues in the next time window. Detailed Implementation

[0067] To better illustrate the purpose, technical solution, and advantages of this invention, the invention will be further described below with reference to specific drawings and embodiments. Unless otherwise specified, the experimental methods used in the embodiments are conventional methods, and the materials and reagents used are commercially available unless otherwise specified.

[0068] Example 1

[0069] I. System Hardware Configuration

[0070] In this embodiment, the monitoring system hardware includes the following components:

[0071] (1) Culture container. Glass aquariums (60 cm × 30 cm × 40 cm) were used, equipped with a circulating filtration system, a heater (300W power, temperature control accuracy ±0.5℃) and an aeration device. The experimental group and the control group each used independent aquariums.

[0072] (2) Video Acquisition Device. A high-definition network camera (1920×1080 pixels resolution, 25fps) is used, fixedly installed about 50 cm above the aquarium, with the lens pointing vertically downwards. It is equipped with an 850nm infrared LED supplementary light array to ensure clear images of oyster shell lobes under 24-hour lighting conditions. The camera is connected to a network video recorder (NVR) via a wired network or directly to a computing device for video storage.

[0073] (3) Environmental parameter monitoring. Equipped with a water temperature sensor (accuracy ±0.1℃) and a salinity meter (accuracy ±0.5‰), environmental parameters are recorded once per minute and synchronized with the shell behavior data.

[0074] (4) Computing equipment. A desktop computer or workstation equipped with an NVIDIA GPU (such as GeForce RTX 3060 or above) will be used for video frame extraction, YOLO model inference, and data analysis. The operating system will be Windows 10 / 11 or Linux (Ubuntu 20.04 or above), with Python 3.8+, PyTorch 1.10+, Ultralytics YOLOv8 framework, and FFmpeg 5.0+ installed.

[0075] II. Video Acquisition and Frame Extraction Process

[0076] (1) Continuous video recording. The camera recorded continuously for 24 hours in H.265 / HEVC encoding format. The video files were automatically segmented and stored by day or by hour. In this embodiment, the experiment lasted for 20 days, generating approximately 480 hours of continuous video data.

[0077] (2) Frame extraction. Use FFmpeg to extract frames from the stored video file at fixed intervals. The specific command format is: ffmpeg -i input.mp4 -vf "fps=1 / 60" -q:v 2 output_%06d.jpg

[0078] Here, `fps=1 / 60` means that one frame is extracted every 60 seconds, achieving minute-level time sampling. Sequential decoding is employed to avoid the reduction in decoding efficiency caused by frequent addressing in HEVC encoded videos with large keyframe intervals.

[0079] (3) Image preprocessing. The extracted image frames are uniformly scaled to the model input size (640×640 pixels) and the necessary brightness normalization is performed to adapt to the changes in day and night lighting.

[0080] III. YOLO Model Training and Deployment

[0081] This invention employs an iterative optimization strategy to train the YOLOv8n object detection model, which specifically consists of the following stages:

[0082] Phase 1 (Feasibility Validation): Researchers manually labeled 31 images, categorizing them as "open" (lobe open) and "closed" (lobe closed). YOLOv8n was used for training, achieving a validation set mAP@0.5 = 75.6%. This phase confirmed the feasibility of the computer vision method in this scenario.

[0083] The second stage (semi-automatic expansion): Using the X-AnyLabeling auxiliary annotation tool, manual corrections were made to the pre-annotated data from the first stage model, expanding the labeled dataset to 555 images. After retraining, mAP@0.5 improved to 95.7%.

[0084] Phase 3 (Final Optimization): The dataset was further expanded to 1396 high-quality annotated images, containing 11,777 individual detection annotations, using a semi-automatic annotation process. The final model achieved a detection accuracy of mAP@0.5=98.4% and mAP@0.5-0.95=98.1%. This version was selected as the standard model for all subsequent analyses.

[0085] Model Deployment: The trained YOLOv8n model (3.01M parameters, 8.1 GFLOPs computation) is deployed on a GPU computing device in PyTorch format. Inference is performed on each extracted image frame, and the output contains the following three types of information:

[0086] - Class: The class of the shell state that the model determines for each detected oyster, either "open" or "closed".

[0087] - Confidence Score: This score reflects the model's confidence that the current detection result belongs to a certain category. The value ranges from 0 to 1, with a higher value indicating greater confidence in the identification result. For example, a confidence score of 0.95 means the model is 95% certain that the classification result is correct. In actual deployment, a confidence threshold is set (0.25 in this example). Detection results below this threshold will be filtered to eliminate false positives.

[0088] - Bounding Box Coordinates: These represent the position and extent of the identified oyster individual in the image, using a four-parameter representation (x, y, w, h). x and y are the horizontal and vertical coordinates of the detection box's center point in the image coordinate system (the origin is located at the top left corner of the image), and w and h are the width and height of the detection box, respectively. Coordinate values ​​can be absolute pixel values ​​or normalized to a ratio (0 to 1) relative to the image's width and height. Bounding box coordinates are used for individual spatial localization, number assignment, and cross-time point individual matching.

[0089] The inference results are automatically exported as a structured CSV file, with the following format: frame number, timestamp, individual number, category, confidence score, and bounding box coordinates.

[0090] IV. Method for Calculating Opening Ratio

[0091] (1) Individual identification and tracking. Based on the bounding box position, individual identification is achieved by spatial location matching, taking advantage of the characteristic that the position of oysters remains basically unchanged in sessile culture. Each oyster is assigned a unique number.

[0092] (2) Opening rate per minute (Formula ①). For each individual i at time point t, the shell lobe state is determined based on the YOLO classification results:

[0093] This formula maps the model classification output to a binary open-end state index, which is the fundamental variable for all subsequent behavioral quantitative analysis in this invention. These are not pre-given constants, but rather observations obtained directly from the model's inference results for each frame of image.

[0094] (3) Average daily opening rate (Formula ②). Calculate the average daily opening rate for each individual:

[0095] in This represents the number of minutes of valid observations within day d (a full day is 1440). This formula aggregates minute-level discrete state values ​​into a continuous daily statistic (values ​​from 0 to 1), facilitating cross-day trend analysis and inter-group comparisons. It is derived from formula ① The derived quantity is then calculated further.

[0096] (4) Diurnal and nighttime opening rates. The average opening rates were calculated separately for daytime (06:00–17:59) and nighttime (18:00–05:59) and used for diurnal behavioral rhythm analysis.

[0097] V. Calculation Method for State Switching Rate

[0098] (1) Switching event detection (Formula ③). Because in Formula ①... Since it is already a binary variable of 0 / 1, it can be directly used for switching detection. A switching event is recorded when the state of the lobe changes between two adjacent time points.

[0099] in A value of 1 indicates that a state transition (from open to closed or from closed to open) occurred at minute t, while a value of 0 indicates that the state remained unchanged. This formula quantifies state change events in a continuous time series into countable discrete events.

[0100] (2) Switching rate calculation (Formula ④). The state switching rate is obtained by summing the switching events over an hourly window:

[0101] The daily switching rate is the average of 24 hours. As an independent behavioral dynamics indicator, the state switching rate reflects the frequency of shell and valve movement and can capture behavioral pattern changes that cannot be reflected by the opening rate alone. It is derived from formula ③. Further statistical analysis yielded the derived quantities.

[0102] VI. Correlation Reversal Detection Algorithm

[0103] (1) Intragroup correlation calculation (Formula ⑤). For each experimental group, collect the daily average opening rate and daily average switching rate data of all individuals ( , ), calculate the within-group Pearson correlation coefficient r:

[0104] The Pearson correlation coefficient is a well-known formula in classical statistics, ranging from -1 to +1. A positive value indicates that the two variables change in the same direction, while a negative value indicates that they change in opposite directions. The innovation of this invention lies in applying this coefficient to the analysis of the relationship between the opening rate and the switching rate, and using its sign reversal (from positive to negative) as a discriminant for the stress state at the population level. The r-value is not a preset parameter, but rather the result obtained through further statistical calculations of the derivatives of formulas ② and ④.

[0105] (2) Reversal judgment. The judgment rule is set as follows: - When the control group r > 0 (positive correlation) and the treatment group r < 0 (negative correlation), the correlation is reversed, indicating that the treatment group is under stress.

[0106] (3) Significance test. Perform a statistical test (t-test or Fisher's Z-transform) on the correlation coefficient to confirm the statistical significance of the correlation.

[0107] VII. Logic for Determining Death Warning

[0108] A multidimensional behavioral time series monitoring system is established for each individual, and a mortality risk warning is triggered when the following conditions are met simultaneously. The threshold parameters in the following conditions are preferred empirical parameters used in this embodiment, and can be adjusted according to the farmed species and environmental conditions in actual applications.

[0109] Condition 1 (Decrease in opening rate, Formula 6): The average daily opening rate of an individual shows a downward trend for 3 consecutive days or more, and the opening rate on the most recent day is lower than the group average. Below one standard deviation (preferred in this embodiment) =1):

[0110] in and These represent the mean and standard deviation of the daily average opening rate of all surviving individuals in the same group on the same day. Both are population-level derivatives obtained through further statistical analysis of the results calculated using Formula ②. The technical effect of this condition is to identify individuals with a persistently low opening rate that significantly deviates from the normal population level.

[0111] Condition 2 (Abnormal Switching Rate, Formula ⑦): The individual's daily average switching rate deviates from the group average by more than [percentage missing]. One standard deviation (preferred in this embodiment) =1.5):

[0112] in and These represent the mean and standard deviation of the daily switching rate for all surviving individuals in the same group on the same day, respectively, both derived from the population-level data obtained through further statistical analysis of the results calculated using formula ④. The technical effect of this condition is to capture abnormalities in shell flap dynamics—stressed individuals often exhibit abnormally high or low switching rates.

[0113] Condition 3 (Circadian rhythm disorder, optional auxiliary condition): The absolute value of the difference between the daytime opening rate and the nighttime opening rate is less than a set threshold. (Preferred in this embodiment) =0.05), indicating that the normal diurnal behavioral differences have disappeared:

[0114] The technical effect of this condition is to detect the loss of diurnal behavioral rhythms. Normal individuals usually have a significant difference in diurnal opening rate, and the disappearance of this difference can serve as an auxiliary signal of the loss of physiological regulatory capacity.

[0115] When conditions one and two are met simultaneously, the system outputs a mortality risk warning signal for the individual and suggests that the farm manager take inspection or intervention measures. Condition three serves as an auxiliary judgment condition, which can improve the specificity of the warning but is not a necessary condition.

[0116] The complete data processing flow for the monitoring described above in this invention is as follows:

[0117] S1. Continuous video recording: 24-hour continuous recording in H.265 / HEVC encoding format;

[0118] S2 and FFmpeg extract frames from stored video files at fixed intervals, meaning one frame is extracted every 60 seconds.

[0119] S3. Image preprocessing: The extracted image frames are uniformly scaled to the model input size (640×640 pixels) and necessary brightness normalization is performed.

[0120] S4, YOLOv8n Inference: The YOLOv8n target detection model is trained using an iterative optimization strategy. The inference results are used to classify the shell lobe state as (open / closed).

[0121] S5. Structured Data Export: The inference results are automatically exported as a structured CSV file, with the following format: frame number, timestamp, individual number, category, confidence score, and bounding box coordinates.

[0122] S6. Based on the structured data, calculate the opening rate (minutely / daily average / day and night) and the state switching rate (binarization + switching monitoring + SR).

[0123] S7. Correlation Analysis: Collect daily average opening rate and daily average switching rate data for all individuals through S6 ( , ), calculate the within-group Pearson correlation coefficient r;

[0124] S8. Stress state determination: The Pearson correlation coefficient is a well-known formula in classical statistics, with a value range of −1 to +1. A positive value indicates that the two variables change in the same direction, and a negative value indicates that they change in opposite directions.

[0125] The above S1-S8 can be used for monitoring the individual behavioral trajectories of shellfish.

[0126] S9. Mortality Risk Warning: There are three conditions: condition 1 is a decrease in opening rate (OR), condition 2 is an abnormal switching rate (SR), and condition 3 is a disruption of circadian rhythm (which can be used as an auxiliary condition). Based on these three conditions, a warning signal and risk management suggestions can be output.

[0127] Example 2: Model Recognition Accuracy Verification

[0128] To verify the accuracy of the shell lobe state recognition system of the present invention, the Hong Kong oyster (Crassostrea hongkongensis) was used as the experimental object. An iterative training strategy was adopted to construct a YOLOv8n detection model, and the accuracy of the model was systematically evaluated.

[0129] (1) Dataset Construction: From 23 consecutive video recordings (single segment length 1 to 20 hours, file size 2 to 21 GB), image frames were extracted using FFmpeg at fps=1 / 1200 (1 frame every 20 minutes) for model training and validation. After three stages of iterative annotation, a dataset containing 1396 high-quality annotated images and 11,777 individual detection annotations was finally constructed. The annotation categories are "open" (lobe open) and "closed" (lobe closed).

[0130] (2) Improved accuracy through iterative training: The model accuracy in the three training stages is shown in the table below:

[0131] Table 1

[0132] mAP@0.5 (mean Average Precision at IoU=0.5) is a commonly used accuracy evaluation index in the field of target detection, representing the average accuracy of each category when the cross-union ratio threshold is 0.5; mAP@0.5-0.95 is a more stringent evaluation index that takes the average of IoU from 0.5 to 0.95 (step size 0.05).

[0133] (3) Confusion matrix analysis: The normalized confusion matrix of the final model on the test set shows that the classification accuracy of the closed category is 98% (i.e., 98% of the samples that are actually closed are correctly identified), and the classification accuracy of the open category is 87%. The overall precision of the model is 92.4%, and the recall is 95.0%.

[0134] (4) Model Comparison and Validation: To further verify the rationality of the model selection, the YOLO26n architecture was used for comparative training. The results showed mAP@0.5=97.6%, parameter count of 2.38M (21% less than YOLOv8n), and computational cost of 5.2 GFLOPs (36% less). The accuracy difference between the two models was small, indicating that the detection method adopted in this invention has robustness and reproducibility across model architectures.

[0135] (5) Accuracy verification conclusion: The detection accuracy of the final model mAP@0.5=98.4% shows that the computer vision shell lobe state recognition system of the present invention can automatically complete the shell lobe state classification with an accuracy close to that of manual annotation, which meets the data quality requirements for subsequent behavioral index calculation and stress early warning analysis.

[0136] Example 3: Validation of the effectiveness of the stress early warning method

[0137] To verify the effectiveness of the stress identification and death warning method of the present invention, the above-mentioned identification system was used to conduct a complete method verification in a 19-day controlled combined environmental stress experiment.

[0138] (1) Experimental design: Adult Hong Kong oysters were randomly divided into a treatment group (n=8, numbered T1-T8) and a control group (n=10, numbered C1-C10, where C7 was excluded due to abnormal behavior, so the actual n=9). The experiment consisted of 5 consecutive phases: adaptation period (day 0, 25℃ / 15‰), gradual warming period (days 1-3, 28→33℃ / 15‰), heat stress period (days 4-10, 33℃ / 15‰), salinity increase period (days 11-13, 33℃ / 20→30‰), and combined stress period (days 14-19, 33℃ / 30‰). The control group maintained baseline conditions (25℃ / 15‰) throughout the experiment. The video acquisition system recorded continuously for 24 hours at a resolution of 1920×1080 and H.265 encoding.

[0139] (2) Survival outcomes: During the experiment, four oysters in the treatment group died: T4 (day 13), T7 (day 16), T8 (day 18), and T1 (day 19), with a cumulative mortality rate of 50% (4 / 8). There were no deaths in the control group. The log-rank test of Kaplan-Meier survival analysis confirmed a significant difference in survival between the two groups (χ²=5.65, df=1, p=0.018).

[0140] (3) Verification of differences in opening rate behavior: The daily average opening rate was analyzed using a linear mixture model (LMM: daily_open ~ group × phase + (1|oyster)). The results showed that the main effect of group was significant (F 1，16.7 =31.68, p=3.18×10 -5 The differences in opening rates between the treatment group and the control group at each stage are shown in Table 2 below.

[0141] Table 2

[0142] The results show that the opening rate quantification index of the present invention can effectively distinguish the behavioral differences between the stress group and the normal group.

[0143] (4) Correlation reversal verification: The Pearson correlation coefficients of the opening rate and switching rate of the two groups were calculated respectively, and the results are shown in Table 3 below.

[0144] Table 3

[0145] In the control group, the opening rate and switching rate were positively correlated, consistent with the expected pattern of active shell-opening individuals with a high switching frequency under normal physiological conditions. In the treatment group, the correlation reversed sign, becoming significantly negative, indicating that the shell flap control dynamics were disturbed under stress. The correlation coefficients of the two groups were of opposite signs (+0.315 vs -0.231), satisfying the reversal judgment condition described in Section 3.6 of this invention, successfully identifying the stress state of the treatment group.

[0146] (5) Validation of mortality warning: The treatment group was divided into a death subgroup (n=4: T1, T4, T7, T8) and a survival subgroup (n=4: T2, T3, T5, T6) according to the survival outcome, and the behavioral trajectories before death were analyzed. The results showed that all four deceased individuals showed a continuous decrease in the average daily opening rate 3 to 5 days before death, accompanied by an abnormal increase in the switching rate (first increase and then decrease pattern); there was no significant baseline difference in the opening rate between the death subgroup and the survival subgroup during the pre-death period (days 0-12) (LMM between-group effect p=0.40), excluding the confounding effect of pre-existing individual differences; the individuals in the survival subgroup also showed abnormal behavior during the compound stress period (e.g., T2 on day 17: switching rate 3.12 switches / h, opening rate only 6.3%), indicating that the surviving individuals also suffered severe physiological stress.

[0147] (6) Conclusion on the effectiveness of the method:

[0148] The above experimental results demonstrate that: (a) the opening rate and switching rate indices of this invention can effectively quantify changes in stress behavior (LMM group main effect p < 0.0001); (b) the correlation sign reversal detection successfully identified the stress state at the group level (control group r = +0.315 vs treatment group r = −0.231); and (c) the death warning based on multidimensional behavioral trajectories issued abnormal behavioral signals 3-5 days in advance in all 4 deceased individuals, verifying the warning potential of this method. Given the small sample size (n = 4), the death warning results in this embodiment should be considered as a verification of the method's feasibility. In practical applications, a larger sample size is needed to further determine the sensitivity and specificity of the warning.

Claims

1. A method for continuous monitoring of shellfish behavior based on computer vision, characterized in that, Includes the following steps: (1) Video capture: Use fixed-installation camera equipment to record 24-hour continuous video of shellfish; (2) Use the FFmpeg multimedia framework to extract frames at fixed intervals from the continuous video stream obtained in step (1) to achieve a minute-level time resolution image sequence; (3) Automatic identification of shell lobe state: Input the image frames extracted in step (2) into the pre-trained YOLOv8n target detection model to automatically identify the shell lobe state of each mollusk and classify it as open or closed; (4) Calculate the opening rate and state switching rate of each mollusk based on the classification results of (3); (5) Calculate the Pearson correlation coefficient based on the opening ratio and state switching rate in (4); (6) When the Pearson correlation coefficient calculated in (5) changes from a positive value to a negative value, the shellfish is determined to be in a state of stress.

2. The method according to claim 1, characterized in that, In step (3), the detection accuracy mAP@0.5 of the YOLOv8n target detection model is not less than 98%.

3. The method according to claim 1, characterized in that, The opening rate mentioned in step (4) includes the minute-by-minute opening rate and the daily average opening rate; Among them, the minute-by-minute opening rate formula ① is to determine the shell lobe state of each individual i at time point t based on the YOLO classification results: These are not pre-given constants, but rather observations obtained directly from the model's inference results for each frame of image; Formula ②, which calculates the average daily opening rate for each individual, is used to determine the average daily opening rate. in This represents the number of minutes of valid observations within day d. It is derived from formula ① The derived quantity is then calculated further.

4. The method according to claim 1, characterized in that, Step (4) of the state switching rate detection includes the following steps: S1, Switching Event Detection When the state of the shell lobe changes between two adjacent time points, it is recorded as a switching event: Formula ③ where A value of 1 indicates that a state transition occurred at minute t, while a value of 0 indicates that the state remained unchanged. S2, Switching Rate Calculation The state transition rate is obtained by summing the transition events in hours: Formula ④, the daily switching rate is the average of 24 hours. It is derived from formula ③. Further statistical analysis yielded the derived quantities.

5. A computer vision-based method for early warning of shellfish mortality risk, characterized in that, A warning signal is triggered when a shellfish exhibits the following combination of characteristics: (1) The average daily opening rate has been declining for more than 3 consecutive days; and, (2) The shellfish state switching rate increased and deviated from the population mean by 1.5 standard deviations; and / or, (3) The difference between the nighttime opening rate and the daytime opening rate is less than the threshold of 0.

05.

6. The application of the method according to any one of claims 1 to 4 in shellfish farming.

7. The application of the computer vision-based shellfish mortality risk early warning method as described in claim 5 in shellfish aquaculture management.

8. A continuous monitoring and stress early warning system for shellfish behavior based on a computer vision system, characterized in that, include: The system includes a video acquisition module, a video storage module, a computing and processing module, and an early warning output module.

9. The continuous monitoring and stress early warning system for shellfish behavior based on a computer vision system according to claim 8, characterized in that, The computation processing module includes: The frame extraction module is used to extract image frames from continuous video at set time intervals. The shell lobe state recognition module deploys a pre-trained target detection model to automatically identify the open or closed state of the shell lobes of each individual in each frame of the image. The behavior index calculation module is used to calculate the opening ratio and state switching rate based on the shell lobe state classification results; The early warning output module is used to monitor changes in the correlation between the opening rate and the state switching rate, as well as abnormal individual behavioral trajectories, and output group stress state assessment and individual mortality risk early warning signals.

10. The continuous monitoring and stress early warning system for shellfish behavior based on a computer vision system according to claim 9, characterized in that, The target detection model in the shell lobe state recognition module is the YOLOv8n model, with no more than 5M parameters and no more than 10 GFLOPs computation.

Images