Method and system for monitoring state of laying ducks based on multi-modal data

Abstract

The application provides a kind of based on multimodal data's egg duck state monitoring method and system, related to poultry breeding technical field, method includes: obtaining duck house video stream data and environmental parameter data;Obtain the bounding box of each egg duck in each frame of duck house video stream data image and track;Based on the video feature of each egg duck in each frame of image;When video feature exceeds set range, send alarm signal;When environmental parameter data exceeds environmental setting threshold range, determine environment anomaly, send environment anomaly signal;Based on video feature and environmental parameter data, obtain egg duck comprehensive state value, and judge egg duck comprehensive state value, when egg duck comprehensive state value is less than comprehensive setting threshold, send comprehensive anomaly signal.The application realizes the multimodal deep fusion of visual behavior characteristics and in-shed environmental parameters, improves the accuracy and foresight of state monitoring.

Description

Technical Field

[0001] This invention relates to the field of poultry farming technology, and in particular to a method and system for monitoring the status of laying ducks based on multimodal data. Background Technology

[0002] With the development of large-scale and intensive poultry farming, monitoring the condition of farmed poultry has become a crucial link in ensuring farming efficiency and food safety. Existing poultry farming monitoring technologies mainly suffer from the following problems and technical bottlenecks:

[0003] Traditional manual observation methods rely heavily on the experience and judgment of farmers, which is highly subjective and inefficient in modern farming, making it difficult to meet the needs of intensive farming. Furthermore, manual inspection methods lack quantifiable assessment indicators, making it difficult to establish a systematic health status evaluation system.

[0004] In recent years, most research has focused on using computer vision technology to analyze the behavior of laying ducks, identifying behaviors such as feeding, drinking, neck stretching, and trampling through improved YOLO algorithms. The image data is intuitive and easy to collect non-contactly. However, existing visual monitoring systems have the following limitations: occlusion problems are particularly serious in high-density cage environments; changes in lighting can affect recognition accuracy; and more importantly, many internal physiological states (such as early diseases and metabolic disorders) and psychological stress are difficult to accurately judge based solely on external behavioral representations. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for monitoring the status of laying ducks based on multimodal data, which realizes multimodal deep fusion of visual behavioral characteristics and indoor environmental parameters, thereby improving the accuracy and foresight of status monitoring.

[0006] A method for monitoring the status of laying ducks based on multimodal data, comprising:

[0007] Acquire video stream data and environmental parameter data for the duck house;

[0008] Obtain the bounding box of each laying duck in each frame of the duck house video stream and track it;

[0009] The video features of each duck are obtained based on the bounding boxes of each duck in each frame of the image; the video features include feeding frequency, drinking frequency, standing ratio, and activity level.

[0010] When the feeding frequency exceeds the set feeding frequency range, an abnormal feeding condition is determined for the laying duck, and an abnormal feeding signal is issued; when the drinking frequency exceeds the set drinking frequency range, an abnormal drinking condition is determined for the laying duck, and an abnormal drinking signal is issued; when the standing ratio is greater than a first standing set value or less than a second standing set value, an abnormal standing condition is determined for the laying duck, and an abnormal standing signal is issued; when the activity level is greater than a first activity set value or less than a second activity set value, an abnormal activity condition is determined for the laying duck, and an abnormal activity signal is issued; the first standing set value is greater than the second standing set value; the first activity set value is greater than the second activity set value;

[0011] The environmental parameter data is judged, and when the environmental parameter data exceeds the environmental set threshold range, an environmental anomaly is determined and an environmental anomaly signal is issued.

[0012] Based on the video features and environmental parameter data, the overall status value of the egg-laying ducks is obtained, and the overall status value of the egg-laying ducks is judged. When the overall status value of the egg-laying ducks is less than the overall set threshold, an overall abnormal signal is issued.

[0013] Optionally, data is acquired using an industrial camera at a set frame rate to obtain video stream data of the duck house; the industrial camera has true wide dynamic range and infrared night vision capabilities.

[0014] Optionally, the method for obtaining the feeding frequency is as follows:

[0015] The relationship between the bounding box of the laying duck and the feed trough in each frame image is determined. If the minimum distance between the bounding box of the laying duck and the bounding box of the feed trough is less than a set distance threshold or there is overlap between the bounding box of the laying duck and the bounding box of the feed trough, it is determined that the laying duck is in a feeding state in the current frame image. The feeding frequency is obtained by calculating the ratio of the number of image frames in the feeding state to the total number of frames of the duck house video stream data.

[0016] Optionally, the method for obtaining the drinking frequency is as follows:

[0017] The relationship between the bounding box of the duck and the waterer in each frame image is determined. If the minimum distance between the bounding box of the duck and the bounding box of the waterer is less than a set distance threshold or there is overlap between the bounding box of the duck and the bounding box of the waterer, it is determined that the duck is in the drinking state in the current frame image. The ratio of the number of image frames in the drinking state to the total number of frames of the duck house video stream data is calculated to obtain the drinking frequency.

[0018] Optionally, the method for obtaining the standing ratio is as follows:

[0019] The aspect ratio of the bounding box of the duck in each frame is determined. When the aspect ratio of the bounding box of the duck is less than a set ratio threshold, it is determined that the duck in the current frame is in a standing state. The ratio of the number of image frames in a standing state to the total number of frames of the duck house video stream data is calculated to obtain the standing ratio.

[0020] Optionally, the method for obtaining the activity level is as follows:

[0021] The average displacement of the bounding box of the duck in two consecutive frames is calculated sequentially, and the sum of the average displacements is taken as the activity quantity.

[0022] Optionally, the environmental parameter data includes temperature data, humidity data, ammonia concentration data, carbon dioxide concentration data, light intensity data, and noise value data;

[0023] The environmental setting threshold ranges include temperature setting threshold range, humidity setting threshold range, ammonia setting threshold range, carbon dioxide setting threshold range, light setting threshold range, and noise setting threshold range;

[0024] When the temperature exceeds the set temperature threshold range, a temperature abnormality is determined and a temperature abnormality signal is issued.

[0025] When the humidity exceeds the set humidity threshold range, an abnormal humidity is determined and a humidity abnormality signal is issued.

[0026] When the ammonia concentration exceeds the set ammonia threshold range, an ammonia concentration abnormality is determined, and an ammonia concentration abnormality signal is issued.

[0027] When the carbon dioxide concentration exceeds the set threshold range, it is determined that the carbon dioxide concentration is abnormal and a carbon dioxide concentration abnormality signal is issued.

[0028] When the light intensity exceeds the light intensity set threshold range, the light intensity is determined to be abnormal, and a light intensity abnormality signal is issued.

[0029] When the noise value exceeds the noise set threshold range, the noise is determined to be abnormal, and a noise abnormality signal is issued.

[0030] The environmental anomaly signals include the temperature anomaly signal, the humidity anomaly signal, the ammonia concentration anomaly signal, the carbon dioxide concentration anomaly signal, the light intensity anomaly signal, and the noise anomaly signal.

[0031] Optionally, the step of obtaining a comprehensive status value of the laying ducks based on the video features and environmental parameter data, and judging the comprehensive status value of the laying ducks, and issuing a comprehensive abnormality signal when the comprehensive status value of the laying ducks is less than a set comprehensive threshold, specifically:

[0032] The video features and environmental parameter data are normalized to obtain normalized feature data and normalized environmental data.

[0033] Based on the normalized feature data and the normalized environmental data, the video features and environmental features are filtered to obtain several filtered features; the environmental features include temperature, humidity, ammonia concentration, carbon dioxide concentration, light intensity, and noise level.

[0034] Based on the normalized data corresponding to several of the screening features, the overall status value of the laying ducks is obtained;

[0035] The overall status value of the egg-laying ducks is judged, and when the overall status value of the egg-laying ducks is less than the overall set threshold, the overall abnormality signal is issued.

[0036] This invention also provides a system for monitoring the status of laying ducks based on multimodal data, comprising:

[0037] The data acquisition module is used to acquire video stream data and environmental parameter data of the duck house;

[0038] The image recognition module is used to obtain and track the bounding boxes of each laying duck in each frame of the duck house video stream data.

[0039] The feature acquisition module is used to obtain the video features of each duck based on the bounding box of each duck in each frame of the image; the video features include feeding frequency, drinking frequency, standing ratio and activity level;

[0040] The feature judgment module is used to determine that the duck's feeding is abnormal and to issue a feeding abnormality signal when the feeding frequency exceeds the set feeding frequency range; to determine that the duck's drinking is abnormal when the drinking frequency exceeds the set drinking frequency range and to issue a drinking abnormality signal; to determine that the duck's standing is abnormal when the standing ratio is greater than a first standing set value or less than a second standing set value and to issue a standing abnormality signal; and to determine that the duck's activity is abnormal when the activity level is greater than a first activity set value or less than a second activity set value and to issue an activity abnormality signal. The first standing set value is greater than the second standing set value; and the first activity set value is greater than the second activity set value.

[0041] The environment judgment module is used to judge the environmental parameter data. When the environmental parameter data exceeds the environmental set threshold range, it determines that the environment is abnormal and issues an environmental abnormality signal.

[0042] The comprehensive judgment module is used to obtain the comprehensive status value of the egg-laying ducks based on the video features and environmental parameter data, and to judge the comprehensive status value of the egg-laying ducks. When the comprehensive status value of the egg-laying ducks is less than the comprehensive set threshold, a comprehensive abnormal signal is issued.

[0043] The effects of this invention are as follows:

[0044] This invention relates to a method for monitoring the status of laying ducks based on multimodal data. The method acquires video features of each laying duck based on video acquisition, including feeding frequency, drinking frequency, standing ratio, and activity level. Each video feature is evaluated to accurately determine the status of each laying duck.

[0045] This invention relates to a method for monitoring the status of laying ducks based on multimodal data. It acquires and judges environmental parameter data. When the environmental parameter data exceeds the set environmental threshold range, it determines that the environment is abnormal, issues an environmental abnormality signal, and further conducts environmental monitoring, thereby avoiding the impact of environmental factors on laying ducks.

[0046] This invention relates to a method for monitoring the status of laying ducks based on multimodal data. It obtains a comprehensive status value of laying ducks based on video features and environmental parameter data, and judges the comprehensive status value of laying ducks. When the comprehensive status value of laying ducks is less than a set comprehensive threshold, a comprehensive abnormal signal is issued. By deeply fusing visual behavioral features with indoor environmental parameters in a multimodal manner, the accuracy and foresight of comprehensive status monitoring are significantly improved. Attached Figure Description

[0047] Figure 1 This is a flowchart of the method for monitoring the status of duck eggs based on multimodal data according to the present invention. Detailed Implementation

[0048] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0049] Figure 1 This is a flowchart of the method for monitoring the status of laying ducks based on multimodal data, as described in this invention. Figure 1 As shown, this invention provides a method for monitoring the status of laying ducks based on multimodal data, which includes:

[0050] S1, acquire video stream data and environmental parameter data of the duck house.

[0051] Preferably, data is acquired using an industrial camera at a set frame rate to obtain video stream data of the duck house; the industrial camera has true wide dynamic range and infrared night vision capabilities.

[0052] First, initial video stream data of the duck house is acquired using an industrial camera. Then, a Gaussian Mixture Model (GMM) is used to perform background subtraction, outlier processing, and data imputation on the initial video stream data to obtain the final duck house video stream. Specifically, a 4K industrial camera with IP67 protection is used to acquire high-definition video stream data from inside the duck house at an adaptive frame rate (1fps-30fps). It also features true wide dynamic range (WDR) and infrared night vision (IR) capabilities to ensure high-quality, interference-free video data can be acquired 24 hours a day, even under complex lighting and environmental conditions in the farm. The entire system is powered and networked via an industrial PoE switch, ensuring simple deployment and long-term operational stability.

[0053] Specifically, the environmental parameter data includes temperature data, humidity data, ammonia concentration data, carbon dioxide concentration data, light intensity data, and noise level data.

[0054] High-precision digital temperature and humidity sensors are used to acquire temperature and humidity data, electrochemical ammonia sensors are used to acquire ammonia concentration data, electrochemical carbon dioxide sensors are used to acquire carbon dioxide concentration data, light intensity sensors are used to acquire light intensity data, and noise sensors are used to acquire noise value data. These sensors are distributed at key points in the duck house and a stable and reliable sensor network is built through RS-485 bus and Modbus-RTU protocol.

[0055] After acquiring the initial environmental parameter data, the dynamic threshold method and the isolated forest algorithm are used to identify and process sensor outliers. Time series prediction methods are used to fill in missing data, and all data are given a unified timestamp to ensure data quality and time series consistency. Edge caching is deployed to temporarily store data when the network is briefly interrupted and retransmit it after the network is restored to ensure that the data is not lost and to ensure data quality and time series consistency, thus obtaining the environmental parameter data.

[0056] It can diagnose the health status of each sensor in real time and provide accurate alarms when a fault is detected, thus guiding maintenance.

[0057] Preferably, the exposure parameters of the industrial camera are automatically adjusted according to the light intensity inside the building, and the infrared mode is automatically activated in low light conditions.

[0058] S2, Obtain the bounding boxes of each laying duck in each frame of the duck house video stream and track them. Preferably, the bounding boxes of each laying duck in each frame of the duck house video stream are obtained based on the YOLOv5 detector, and tracking is performed based on the IoU Tracker.

[0059] S3. Based on the bounding boxes of each duck in each frame of the image, the video features of each duck are obtained; the video features include feeding frequency, drinking frequency, standing ratio and activity level.

[0060] Furthermore, the feeding frequency is obtained using the following method:

[0061] The relationship between the bounding box of the laying duck and the feed trough in each frame image is determined. If the minimum distance between the bounding box of the laying duck and the bounding box of the feed trough is less than the distance set threshold or there is an overlap between the bounding box of the laying duck and the bounding box of the feed trough, it is determined that the laying duck is in the feeding state in the current frame image. The feeding frequency is obtained by calculating the ratio of the number of image frames in the feeding state to the total number of frames of the duck house video stream data.

[0062] The method for obtaining drinking water frequency is as follows:

[0063] The relationship between the bounding box of the duck and the waterer in each frame image is determined. If the minimum distance between the bounding box of the duck and the bounding box of the waterer is less than the set distance threshold or there is an overlap between the bounding box of the duck and the bounding box of the waterer, it is determined that the duck is in the drinking state in the current frame image. The ratio of the number of image frames in the drinking state to the total number of frames of the duck house video stream data is calculated to obtain the drinking frequency.

[0064] The method for obtaining standing proportions is as follows:

[0065] The aspect ratio of the bounding box of the duck in each frame is determined. When the aspect ratio of the bounding box of the duck is less than the set ratio threshold, it is determined that the duck in the current frame is in a standing state. The ratio of the number of image frames in a standing state to the total number of frames in the duck house video stream data is calculated to obtain the standing ratio.

[0066] The methods for obtaining activity points are as follows:

[0067] The average displacement of the bounding box of the duck in two consecutive frames is calculated sequentially, and the sum of these average displacements is taken as the activity value. Specifically, a ROS-based visual localization method is used to calculate the average displacement of the distance the duck moves per unit frame by analyzing the average pixel depth of the corresponding pixel region.

[0068] S4. When the feeding frequency exceeds the set feeding frequency range, an abnormal feeding signal is detected in the laying ducks; when the drinking frequency exceeds the set drinking frequency range, an abnormal drinking signal is detected; when the standing ratio is greater than the first standing set value or less than the second standing set value, an abnormal standing signal is detected; when the activity level is greater than the first activity set value or less than the second activity set value, an abnormal activity signal is detected. The first standing set value is greater than the second standing set value; the first activity set value is greater than the second activity set value. Preferably, the first standing set value is 50%, and the second standing set value is 20%. When the standing ratio is greater than 50%, it indicates that the laying ducks are uncomfortable in the cage facilities or are in a state of fright caused by stress factors such as skin itching, feather damage leading to feather pecking, scratching, sudden driving, strong light, or noise; when the standing ratio is less than 20%, it indicates that there may be infectious diseases, heat stress, or malnutrition in the laying ducks.

[0069] When the activity level is approximately 0 for multiple consecutive frames, and the activity index is significantly lower than the baseline of 30% and this state lasts for more than 2 hours, it indicates that the laying ducks are still for a long time and may be in a state of disease, malnutrition, low temperature or high temperature stress. When there is a sudden large fluctuation and the cumulative displacement in a short period of time far exceeds the daily average level, it indicates that the laying ducks are affected by environmental noise, competition caused by excessive density, or sudden lack of feed or water.

[0070] When the feeding frequency exceeds the set feeding frequency range, it indicates that the laying ducks may be under stress or in a state of illness, or there may be water quality problems or feed problems.

[0071] Preferably, the feeding frequency range, drinking frequency range, first standing setting value, second standing setting value, first activity setting value, and second activity setting value are automatically updated according to the age of the duck flock and seasonal changes.

[0072] S5 judges the environmental parameter data. When the environmental parameter data exceeds the environmental set threshold range, it determines that the environment is abnormal and issues an environmental abnormality signal.

[0073] Specifically, the environmental setting threshold ranges include temperature setting threshold range, humidity setting threshold range, ammonia setting threshold range, carbon dioxide setting threshold range, light setting threshold range, and noise setting threshold range.

[0074] When the temperature exceeds the set temperature threshold range, a temperature anomaly is identified, and a temperature anomaly signal is issued. Preferably, the set temperature threshold range is 15℃-30℃. When the duck house temperature exceeds 30℃, it will have a significant negative impact, and the laying ducks are judged to be under high temperature stress; when the temperature is below 15℃, especially when it is below 10℃ for a long period of time, the laying ducks need to consume a lot of energy to maintain their body temperature, thereby affecting their egg production performance, and the laying ducks are judged to be under low temperature stress.

[0075] When the humidity exceeds the set humidity threshold range, an abnormal humidity condition is determined, and a humidity abnormality signal is issued. Preferably, the set humidity threshold range is 50%-70% or 50%-80%. When the humidity is consistently higher than 70% or 80%, it is considered an abnormally high humidity environment; when the relative humidity is consistently lower than 50%, it is considered an abnormally low humidity environment.

[0076] When the ammonia concentration exceeds the set threshold range, an ammonia concentration abnormality is detected, and an ammonia concentration abnormality signal is issued.

[0077] When the carbon dioxide concentration exceeds the set threshold range, an abnormal carbon dioxide concentration is identified, and an abnormal carbon dioxide concentration signal is issued. Preferably, the set threshold range for ammonia is 0-19.76 mg / kg, and the set threshold range for carbon dioxide is 0-763.63 mg / kg. When the ammonia concentration exceeds the upper limit of 19.76 mg / kg or the carbon dioxide concentration exceeds the upper limit of 763.63 mg / kg, it may be due to untimely manure removal, excessively high stocking density, or insufficient ventilation, and is judged as an abnormal air quality.

[0078] When the light intensity exceeds the set threshold range, an abnormal light intensity is detected, and an abnormal light intensity signal is issued. Furthermore, the set threshold range is 10lx-15lx. When the light intensity exceeds this range, it may be due to lighting equipment malfunction, improper light mode adjustment, uneven lamp arrangement, or obstructions causing localized dark areas.

[0079] When the noise level exceeds the set threshold, an abnormal noise condition is identified, and a noise abnormality signal is issued. Preferably, the set threshold range is 20-85 dB. When the noise level is greater than 85 dB, it may be due to equipment aging, improper maintenance leading to increased noise, or excessive stocking density causing increased load on the ventilation system. When the noise peak briefly exceeds 90 dB and abnormal high-frequency calls occur, it may be due to stress factors causing frequent squawking by the ducks. When the noise level is less than 20 dB, it may be due to equipment malfunction restricting the ducks' activity, a large number of dead ducks causing decreased group vitality, or an extremely unsuitable environment inhibiting the ducks' activity.

[0080] Preferably, in this embodiment, the method further includes performing short-term trend fitting and prediction on environmental parameter data, and obtaining a predicted environmental state based on the prediction results and a set threshold range for the environment. When the predicted environmental state is abnormal, an early trend warning is issued.

[0081] Environmental anomaly signals include abnormal temperature signals, abnormal humidity signals, abnormal ammonia concentration signals, abnormal carbon dioxide concentration signals, abnormal light intensity signals, and abnormal noise signals.

[0082] S6 obtains the overall status value of the egg-laying ducks based on video features and environmental parameter data, and judges the overall status value of the egg-laying ducks. When the overall status value of the egg-laying ducks is less than the overall set threshold, an overall abnormal signal is issued.

[0083] Specifically, the video features and environmental parameter data are normalized to obtain normalized feature data and normalized environmental data. Preferably, the video features and environmental parameter data are subjected to Z-score normalization and Min-Max normalization respectively to eliminate differences in dimensions, achieve a unified representation of multi-source heterogeneous data, and prevent certain features from dominating the model training due to their large dimensions, thereby improving the model convergence speed and prediction accuracy.

[0084] Based on normalized feature data and normalized environmental data, video features and environmental features are filtered to obtain several screening features. Environmental features include temperature, humidity, ammonia concentration, carbon dioxide concentration, light intensity, and noise level. For orderly land acquisition, a filtering method based on Pearson correlation coefficient and maximum information coefficient (MIC) is used, combined with a wrapping method based on XGBoost and recursive feature elimination (RFE) to select the most relevant features from the video and environmental features as screening features.

[0085] The overall status value of the laying ducks is obtained based on the normalized data corresponding to several screening features. Specifically, the overall status value of the laying ducks is obtained using a multilayer perceptron (MLP) or 1D-CNN network structure based on the normalized data corresponding to several screening features.

[0086] The overall status value of the laying ducks is judged, and when the overall status value of the laying ducks is less than the overall set threshold, an overall abnormal signal is issued.

[0087] Specifically, a tiered early warning mechanism is adopted, dividing the overall status value of laying ducks into different levels, as shown in Table 1. 0-60 indicates an abnormal state, 61-75 indicates a sub-healthy state, 76-90 indicates a good state, and 91-100 indicates an optimal state. The overall threshold is set at 60. When the overall status value of the laying ducks falls below the set threshold, an early warning message is automatically sent to the breeding management personnel, prompting them to take appropriate management intervention measures. Simultaneously, all early warning events and their corresponding characteristic data are recorded for continuous optimization of the prediction model.

[0088] Table 1 Health Status Scores and Early Warning Level Classification

[0089]

[0090] This invention also provides a system for monitoring the status of laying ducks based on multimodal data, comprising:

[0091] The data acquisition module is used to acquire video stream data and environmental parameter data of the duck house.

[0092] The image recognition module is used to obtain the bounding boxes of each laying duck in each frame of the duck house video stream and to track them.

[0093] The feature acquisition module is used to obtain the video features of each duck based on the bounding box of each duck in each frame of the image; the video features include feeding frequency, drinking frequency, standing ratio and activity level.

[0094] The feature judgment module is used to determine abnormal feeding in laying ducks and issue an abnormal feeding signal when the feeding frequency exceeds the set feeding frequency range; to determine abnormal drinking in laying ducks and issue an abnormal drinking signal when the drinking frequency exceeds the set drinking frequency range; to determine abnormal standing in laying ducks and issue an abnormal standing signal when the standing ratio is greater than the first standing set value or less than the second standing set value; and to determine abnormal activity in laying ducks and issue an abnormal activity signal when the activity level is greater than the first activity set value or less than the second activity set value. The first standing set value is greater than the second standing set value; and the first activity set value is greater than the second activity set value.

[0095] The environment judgment module is used to judge environmental parameter data. When the environmental parameter data exceeds the set threshold range, it determines that the environment is abnormal and issues an environmental abnormality signal.

[0096] The comprehensive judgment module is used to obtain the comprehensive status value of the egg-laying ducks based on video features and environmental parameter data, and to judge the comprehensive status value of the egg-laying ducks. When the comprehensive status value of the egg-laying ducks is less than the comprehensive set threshold, a comprehensive abnormal signal is issued.

[0097] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for monitoring the status of laying ducks based on multimodal data, characterized in that, It includes: Acquire video stream data and environmental parameter data for the duck house; Obtain the bounding box of each laying duck in each frame of the duck house video stream and track it; The video features of each duck are obtained based on the bounding boxes of each duck in each frame of the image; the video features include feeding frequency, drinking frequency, standing ratio, and activity level. The method for obtaining the feeding frequency is as follows: The relationship between the bounding box of the laying duck and the feed trough in each frame image is determined. If the minimum distance between the bounding box of the laying duck and the bounding box of the feed trough is less than a set distance threshold or there is an overlap between the bounding box of the laying duck and the bounding box of the feed trough, it is determined that the laying duck is in a feeding state in the current frame image. The ratio of the number of image frames in the feeding state to the total number of frames of the duck house video stream data is calculated to obtain the feeding frequency. The method for obtaining the drinking frequency is as follows: The relationship between the bounding box of the duck and the waterer in each frame image is determined. If the minimum distance between the bounding box of the duck and the bounding box of the waterer is less than a set distance threshold or there is an overlap between the bounding box of the duck and the bounding box of the waterer, it is determined that the duck in the current frame image is in a drinking state. The ratio of the number of image frames in a drinking state to the total number of frames of the duck house video stream data is calculated to obtain the drinking frequency. The method for obtaining the standing ratio is as follows: The aspect ratio of the bounding box of the duck in each frame is determined. When the aspect ratio of the bounding box of the duck is less than a set ratio threshold, it is determined that the duck in the current frame is in a standing state. The ratio of the number of image frames in a standing state to the total number of frames of the duck house video stream data is calculated to obtain the standing ratio. When the feeding frequency exceeds the set feeding frequency range, an abnormal feeding condition is determined for the laying duck, and an abnormal feeding signal is issued; when the drinking frequency exceeds the set drinking frequency range, an abnormal drinking condition is determined for the laying duck, and an abnormal drinking signal is issued; when the standing ratio is greater than a first standing set value or less than a second standing set value, an abnormal standing condition is determined for the laying duck, and an abnormal standing signal is issued; when the activity level is greater than a first activity set value or less than a second activity set value, an abnormal activity condition is determined for the laying duck, and an abnormal activity signal is issued; the first standing set value is greater than the second standing set value; the first activity set value is greater than the second activity set value; The environmental parameter data is judged, and when the environmental parameter data exceeds the environmental set threshold range, an environmental anomaly is determined and an environmental anomaly signal is issued. Based on the video features and environmental parameter data, the overall status value of the egg-laying ducks is obtained, and the overall status value of the egg-laying ducks is judged. When the overall status value of the egg-laying ducks is less than the overall set threshold, an overall abnormal signal is issued.

2. The method for monitoring the status of laying ducks based on multimodal data according to claim 1, characterized in that, Data is acquired using an industrial camera at a set frame rate to obtain video stream data of the duck house; the industrial camera has true wide dynamic range and infrared night vision functions.

3. The method for monitoring the status of laying ducks based on multimodal data according to claim 1, characterized in that, The method for obtaining the activity level is as follows: The average displacement of the bounding box of the duck in two consecutive frames is calculated sequentially, and the sum of the average displacements is taken as the activity quantity.

4. The method for monitoring the status of laying ducks based on multimodal data according to claim 1, characterized in that, The environmental parameter data includes temperature data, humidity data, ammonia concentration data, carbon dioxide concentration data, light intensity data, and noise level data. The environmental setting threshold ranges include temperature setting threshold range, humidity setting threshold range, ammonia setting threshold range, carbon dioxide setting threshold range, light setting threshold range, and noise setting threshold range; When the temperature exceeds the set temperature threshold range, a temperature abnormality is determined and a temperature abnormality signal is issued. When the humidity exceeds the set humidity threshold range, an abnormal humidity is determined and a humidity abnormality signal is issued. When the ammonia concentration exceeds the set ammonia threshold range, an ammonia concentration abnormality is determined, and an ammonia concentration abnormality signal is issued. When the carbon dioxide concentration exceeds the set threshold range, it is determined that the carbon dioxide concentration is abnormal and a carbon dioxide concentration abnormality signal is issued. When the light intensity exceeds the light intensity set threshold range, the light intensity is determined to be abnormal, and a light intensity abnormality signal is issued. When the noise value exceeds the noise set threshold range, the noise is determined to be abnormal, and a noise abnormality signal is issued. The environmental anomaly signals include the temperature anomaly signal, the humidity anomaly signal, the ammonia concentration anomaly signal, the carbon dioxide concentration anomaly signal, the light intensity anomaly signal, and the noise anomaly signal.

5. The method for monitoring the status of laying ducks based on multimodal data according to claim 1, characterized in that, Based on the video features and environmental parameter data, a comprehensive status value for the egg-laying ducks is obtained, and the comprehensive status value is judged. When the comprehensive status value is less than a set threshold, a comprehensive abnormality signal is issued. Specifically: The video features and environmental parameter data are normalized to obtain normalized feature data and normalized environmental data. Based on the normalized feature data and the normalized environmental data, the video features and environmental features are filtered to obtain several filtered features; the environmental features include temperature, humidity, ammonia concentration, carbon dioxide concentration, light intensity, and noise level. Based on the normalized data corresponding to several of the screening features, the overall status value of the laying ducks is obtained; The overall status value of the egg-laying ducks is judged, and when the overall status value of the egg-laying ducks is less than the overall set threshold, the overall abnormality signal is issued.

6. A duck egg-laying status monitoring system based on multimodal data, applied to the duck egg-laying status monitoring method based on multimodal data as described in any one of claims 1-5, characterized in that, It includes: The data acquisition module is used to acquire video stream data and environmental parameter data of the duck house; The image recognition module is used to obtain and track the bounding boxes of each laying duck in each frame of the duck house video stream data. The feature acquisition module is used to obtain the video features of each duck based on the bounding box of each duck in each frame of the image; the video features include feeding frequency, drinking frequency, standing ratio and activity level; The feature judgment module is used to determine that the duck's feeding is abnormal and to issue a feeding abnormality signal when the feeding frequency exceeds the set feeding frequency range; to determine that the duck's drinking is abnormal when the drinking frequency exceeds the set drinking frequency range and to issue a drinking abnormality signal; to determine that the duck's standing is abnormal when the standing ratio is greater than a first standing set value or less than a second standing set value and to issue a standing abnormality signal; and to determine that the duck's activity is abnormal when the activity level is greater than a first activity set value or less than a second activity set value and to issue an activity abnormality signal. The first standing set value is greater than the second standing set value; and the first activity set value is greater than the second activity set value. The environment judgment module is used to judge the environmental parameter data. When the environmental parameter data exceeds the environmental set threshold range, it determines that the environment is abnormal and issues an environmental abnormality signal. The comprehensive judgment module is used to obtain the comprehensive status value of the egg-laying ducks based on the video features and environmental parameter data, and to judge the comprehensive status value of the egg-laying ducks. When the comprehensive status value of the egg-laying ducks is less than the comprehensive set threshold, a comprehensive abnormal signal is issued.

Images