A non-contact multi-modal intelligent monitoring and early warning device and method for cattle
By using a non-contact multimodal intelligent monitoring and early warning device for cattle, combined with channel structure and multimodal data fusion technology, the problems of subjectivity and insufficient single-modal assessment in traditional monitoring schemes are solved, and efficient and accurate cattle health monitoring and real-time early warning are achieved.
Patent Information
- Application Number
- CN202610795101.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-03
- Publication Date
- 2026-07-21
AI Technical Summary
Traditional cattle health monitoring relies on manual inspections, which are highly subjective, difficult to monitor continuously around the clock, and unable to capture weak abnormal signals in the subclinical stage. Wearable devices are prone to causing stress and are costly. Existing non-contact monitoring solutions have limited field of view and insufficient accuracy in single-modal data assessment.
The non-contact multimodal intelligent monitoring and early warning device for cattle is adopted, which includes a channel structure, a mobile acquisition mechanism, an acquisition unit, a data processing unit and a remote management platform. Through multimodal data fusion and edge computing, it realizes cattle health assessment and real-time early warning.
It enables contactless, 24/7, and personalized health monitoring, avoids stress responses, improves the ability to identify abnormal signals in the subclinical phase, ensures that no details of abnormal signs are lost, has real-time performance and high assessment accuracy, and supports individual health record management.
Smart Images
Figure CN122423487A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent monitoring technology for livestock farming, and more specifically, to a non-contact, multimodal intelligent monitoring and early warning device and method for cattle. Background Technology
[0002] Traditional cattle health monitoring relies primarily on manual inspections and experience-based judgment, which suffers from issues such as strong subjectivity, inconsistent standards, difficulty in continuous 24 / 7 monitoring, and inability to capture weak abnormal signals during the subclinical phase. While wearable devices (such as smart collars and smart ear tags) can collect some physiological parameters, they are prone to causing stress in cattle, easily damaged in high-temperature, high-humidity, and dusty environments, and large-scale deployment is costly, with limited sensing dimensions, making it difficult to comprehensively assess health status.
[0003] Existing contactless monitoring solutions mostly use fixed image or sound acquisition equipment, which has a limited field of view and makes it difficult to capture individualized, high-precision data on cattle walking continuously in a passage. When abnormal cattle are detected, they cannot automatically switch to a close-range following acquisition mode, resulting in the loss of details of abnormal physical signs. At the same time, existing solutions are mostly based on single-modal data and do not deeply integrate visual, auditory and environmental multi-source information, so the accuracy and real-time performance of health assessment are still insufficient. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention provides a non-contact, multimodal intelligent monitoring and early warning device for cattle, which adopts the following technical solution:
[0005] A non-contact, multimodal intelligent monitoring and early warning device for cattle includes:
[0006] Passage structure: Located on one side of the cattle shed, allowing cattle to enter and exit one at a time, extending along the first direction.
[0007] Mobile acquisition mechanism: includes a mounting frame located above the channel, with a sliding rail and a sliding seat along the channel direction on the mounting frame; the mounting frame also has a servo motor and a lead screw mechanism, with the lead screw arranged along the channel direction and driving the sliding seat to slide; a mounting box is rotatably connected below the sliding seat, and the mounting box contains an acquisition unit assembly, with a transparent window at its opening; a torsion spring is installed at the rotation shaft of the mounting box so that the opening faces downward in the free state; push rods are provided at both ends of the mounting frame, and when the sliding seat moves to the end, the push rods push against the mounting box to deflect it, so that the transparent window faces the other end of the channel.
[0008] The data acquisition unit includes an image acquisition unit, an audio acquisition unit, and an environmental sensing unit, which are used to acquire color image data, depth image data, vocalization audio data of cattle, and environmental parameters of the farm, including temperature, humidity, and ammonia concentration.
[0009] Data processing unit: Located inside or connected to the installation box, and connected to each acquisition unit respectively, including edge computing module and storage module, used to perform edge preprocessing on the acquired multimodal data to generate multimodal monitoring data.
[0010] Multimodal data fusion engine: Built into the data processing unit, it is used to perform spatiotemporal alignment, feature extraction and weighted fusion of multimodal monitoring data to generate fused feature data.
[0011] Cattle health assessment model: Built into the data processing unit, used to generate cattle health scores and risk levels based on fused feature data.
[0012] Early warning module: Connected to the data processing unit, it is used to issue an early warning signal when the health score is lower than the threshold or the risk level reaches the preset level.
[0013] Remote management platform: Connected to the data processing unit via a wireless communication network, it is used to display monitoring data, health scores, risk levels, and early warning signals.
[0014] Control logic: The servo motor drives the sliding seat to move along the channel; when a cow enters the channel, the sliding seat moves to the corresponding end and deflects under the action of the push rod, so that the transparent window faces the cow and collects data for each cow individually; when the data processing unit determines that a cow is abnormal, the servo motor drives the sliding seat to move directly above the cow and follows it to perform close-range synchronous data collection throughout the process.
[0015] Furthermore, the image acquisition unit includes an RGB-D depth camera and an infrared thermal imager, and the audio acquisition unit includes an omnidirectional microphone array.
[0016] Furthermore, the edge computing module includes: a target detection submodule for detecting and tracking cattle from image data and generating spatiotemporal trajectory information; a behavior recognition submodule for recognizing the behavioral state of cattle based on spatiotemporal trajectory information and depth image data; and an audio analysis submodule for extracting voiceprint features from call audio data to identify the stress state and health state of cattle.
[0017] Furthermore, the multimodal data fusion engine includes: a data spatiotemporal alignment module for unifying timestamp alignment and spatial coordinate mapping; a feature extraction module for extracting visual features, behavioral temporal features, acoustic features, and environmental features; and a cross-modal attention fusion module for weighted fusion of features from various modalities using a cross-modal attention mechanism to generate fused feature data.
[0018] Furthermore, the cattle health assessment model adopts a Transformer-based multimodal large language model, including: a health status encoder, used to encode fused feature data into a health status vector; an anomaly detection branch, used to generate anomaly probabilities based on the deviation between the health status vector and a preset benchmark vector; a health scorer, used to map the health status vector into a continuous health score; and a risk classifier, used to classify cattle into at least three risk levels based on the anomaly probability and the health score.
[0019] Furthermore, the early warning module includes: a local audible and visual alarm located above the channel; and a remote push module for pushing early warning messages to a remote management platform via a wireless communication network. The early warning messages include the abnormal cattle's number, location coordinates, abnormality type, and real-time monitoring data segments.
[0020] Furthermore, the device also includes a cattle individual identification module, used to identify cattle based on at least one of their facial features, markings, and body shape features, generate an individual identification result corresponding to the unique identifier of the cattle, and associate and store the multimodal monitoring data with the individual identification result; in the control logic, when an abnormal cattle is identified, the sliding seat moves to the position corresponding to the individual identifier of the abnormal cattle and records the entire tracking data.
[0021] This invention also provides a non-contact, multimodal intelligent monitoring and early warning method for cattle, applied to the aforementioned device, comprising the following steps:
[0022] S1: Guide the cattle one by one into the passage;
[0023] S2: Control the servo motor to drive the sliding seat to the end of the channel, and use the push rod to deflect the mounting box so that the transparent window faces the direction of the cattle's movement, and collect multimodal data for each cow individually, including color images, depth images, vocalization audio and environmental parameters;
[0024] S3: The data processing unit performs edge preprocessing on the collected multimodal data to generate multimodal monitoring data;
[0025] S4: The multimodal data fusion engine performs spatiotemporal alignment, feature extraction, and weighted fusion on the multimodal monitoring data to generate fused feature data;
[0026] S5: The cattle health assessment model generates health scores and risk levels based on fused feature data;
[0027] S6: When the health score is below the threshold or the risk level reaches the preset level, the animal is judged to be abnormal and an early warning signal is issued by the early warning module.
[0028] S7: In response to anomaly detection, the control slide seat moves directly above the abnormal cow and follows its movement to collect data synchronously at close range throughout the entire process. At the same time, the monitoring data, health score, risk level and early warning signal are pushed to the remote management platform for display.
[0029] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:
[0030] 1. Non-contact monitoring, avoiding stress: Data is collected during the natural walking of cattle through the channel structure and mobile collection mechanism, without the need for any wearable devices, thus avoiding physical contact and stress interference with the cattle.
[0031] 2. Multimodal deep fusion for high assessment accuracy: It integrates multi-dimensional data such as color images, depth images, vocal audio, and environmental temperature, humidity, and ammonia concentration. It performs feature-level fusion through cross-modal attention mechanism and combines it with Transformer large language model for health assessment, which significantly improves the ability to identify abnormal signals in the subclinical stage.
[0032] 3. Dual-mode data acquisition, balancing efficiency and accuracy: During normal passage, the sliding seat moves to the end of the channel and deflects to acquire data, enabling rapid single-shot data acquisition for each cow; when an abnormal cow is detected, it automatically switches to top-follow mode for close-range synchronous acquisition throughout the entire process, ensuring that no details of abnormal vital signs are lost.
[0033] 4. Edge computing + cloud platform, strong real-time performance: Data preprocessing and preliminary analysis are completed at the edge, reducing network transmission latency; the remote management platform enables centralized display and early warning push, facilitating timely response by aquaculture personnel.
[0034] 5. Individual Identification and Traceability: Identify cattle by using multiple biological characteristics such as facial features, markings, and body shape, link monitoring data with individuals to form complete health records, and support long-term health trend analysis and precise management. Attached Figure Description
[0035] Figure 1 This is a three-dimensional structural diagram of the present invention;
[0036] Figure 2 This is a schematic diagram of the channel structure of the present invention, excluding the cattle shed portion;
[0037] Figure 3 This is a side view of the channel of the present invention;
[0038] Figure 4 for Figure 3 Enlarged view of point A in the middle;
[0039] Figure 5 This is a schematic diagram of the sliding seat structure of the present invention.
[0040] Legend: 1. Channel; 2. Mounting bracket; 3. Sliding rail; 4. Sliding seat; 5. Servo motor; 6. Lead screw; 7. Lead screw slider; 8. Mounting box; 9. Torsion spring; 10. Push rod. Detailed Implementation
[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0042] Please see Figure 1-5 This invention provides a technical solution for a non-contact, multimodal intelligent monitoring and early warning device for cattle and its monitoring and early warning method:
[0043] This includes a passageway 1 located at the connection between the cattle shed 100 and the grazing area. The passageway 1 is 0.8 meters wide and can only accommodate one adult cow at a time. The passageway 1 is fencing on both sides, with an open top, and extends along the first direction (i.e., the direction in which the cattle are moving), with a total length of 5 meters.
[0044] The mobile data acquisition mechanism is installed directly above channel 1. The mounting frame 2 is an aluminum alloy frame that spans both sides of channel 1 in the width direction and extends along the entire length of channel 1. Two parallel sliding rails 3, which are linear guide rails, are fixed to the top of the mounting frame 2. The sliding seat 4 is a slider assembly that cooperates with the sliding rails 3 and can slide freely along the rails.
[0045] The drive mechanism employs a servo motor 5 in conjunction with a lead screw 6. The lead screw 6 is a ball screw, positioned at the center of the mounting bracket 2 along the length of the channel 1, with both ends fixed to the mounting bracket 2 via bearing seats. The servo motor 5 is fixed to one end of the mounting bracket 2, and its output shaft is connected to the lead screw 6 via a coupling. A lead screw slider 7 is mounted on the lead screw 6, and this nut is fixedly connected to the bottom of the sliding seat 4. The servo motor 5 driver receives control commands and drives the lead screw 6 to rotate forward or reverse, thereby causing the sliding seat 4 to move precisely along the track. The moving speed can be adjusted within the range of 0.5 m / s to 2 m / s.
[0046] The mounting box 8 is connected to the sliding base 4 via a rotating shaft. The mounting box 8 is a rectangular box with dimensions of 30cm × 20cm × 15cm, featuring a waterproof and dustproof design with an IP65 protection rating. One side wall of the mounting box 8 is open, with a high-transmittance optical glass embedded in the opening as a transparent viewing window. The mounting box 8 houses a fixed acquisition unit assembly, including an image acquisition unit, an audio acquisition unit, and an environmental sensing unit. The rotating shaft of the mounting box 8 is located at its top where it connects to the sliding base 4, and a torsion spring 9 is fitted onto the rotating shaft. Under the elastic force of the torsion spring 9, the mounting box 8 maintains a vertically downward orientation when no external force is applied. Push rods 10 are fixed at both ends of the mounting bracket 2 (i.e., above the inlet and outlet ends of channel 1), respectively. The push rods 10 are cylindrical metal rods that extend downwards to positions corresponding to the middle of the side wall of the mounting box 8. When the sliding seat 4 moves to the end, the push rod 10 contacts the side wall of the mounting box 8 and gradually pushes against the mounting box 8, causing it to deflect around the rotation axis against the elastic force of the torsion spring 9. The deflection angle is about 30° to 45°, so that the transparent window faces the entrance or exit of the channel 1.
[0047] The image acquisition unit includes an RGB-D depth camera (Intel RealSense D435) and an infrared thermal imager (FLIR Lepton 3.5), which output color images, depth images, and the surface temperature distribution of the cattle, respectively. The audio acquisition unit is a four-microphone omnidirectional array with a sampling rate of 44.1kHz. The environmental sensing unit includes a temperature and humidity sensor (SHT30) and an ammonia sensor (MQ137).
[0048] The data processing unit uses an NVIDIA Jetson Orin NX edge computing motherboard, which is built into the mounting box 8.
[0049] Edge computing module: The target detection submodule uses YOLOv8n+DeepSORT to detect and track cattle and generate spatiotemporal trajectories; the behavior recognition submodule uses 3D-CNN to recognize behaviors such as walking posture, head-down, and stagnation; the audio analysis submodule uses VGGish to extract voiceprint features and recognize stress / painful cries.
[0050] Multimodal data fusion engine: Spatiotemporally aligns images (30fps), audio (synchronous triggering), and environment (1Hz), extracts visual features (ResNet50), behavioral temporal features, acoustic features (128 dimensions), and environmental features respectively, and outputs 512-dimensional fused features through a cross-modal attention mechanism (8-head attention).
[0051] Cattle health assessment model: The model uses a Transformer multimodal large language model with 200 million parameters, which includes a health status encoder (outputting a 128-dimensional health status vector), an anomaly detection branch (calculating the Mahalanobis distance from the health baseline vector to obtain the anomaly probability), a health scorer (mapped to 0-100 points), and a risk grader (divided into four levels: healthy, sub-healthy, suspected disease, and high risk).
[0052] Cattle individual identification module: It uses facial, markings and body shape features to match a pre-registered feature library, generates a unique identifier and associates it with monitoring data.
[0053] The early warning module includes an audible and visual alarm (red LED + 85dB buzzer) above channel 1 and a 4G remote push module. When the risk level reaches "suspected illness" or "high risk", a local alarm is triggered and a JSON early warning message is pushed to the remote management platform (Web terminal), containing the cattle number, location, anomaly type, and monitoring data snippets from the last 10 seconds.
[0054] Normal passage mode: Cattle enter passage 1 one at a time. The sliding seat 4 moves to the entrance end, and the push rod 10 deflects the mounting box 8, so that the viewing window faces the entrance. The acquisition unit captures 2-3 seconds of multimodal data as the cattle pass by, and the system assesses the health status in real time.
[0055] Abnormal following mode: When the health score is below 60 or the risk level reaches "suspected disease", the servo motor 5 drives the sliding seat 4 to move at 2m / s to directly above the cow (the mounting box 8 returns to vertical and downward). Then, it follows the cow at the same speed, collecting details such as gait, body surface temperature, and breathing sounds at close range until the cow walks out of the passage 1.
[0056] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art can make equivalent substitutions or changes based on the technical solution and inventive concept of the present invention within the scope of the technology disclosed in the present invention, and all such substitutions or changes should be covered within the scope of protection of the present invention.
Claims
1. A non-contact, multimodal intelligent monitoring and early warning device for cattle, characterized in that, include: Passage (1): Located on one side of the cattle shed, allowing cattle to enter and exit one at a time, extending along the first direction; Mobile acquisition mechanism: includes a mounting frame (2) located above the channel (1), with a sliding rail (3) and a sliding seat (4) on the mounting frame (2) along the direction of the channel (1); the mounting frame (2) is also equipped with a servo motor (5) and a screw mechanism, with the screw (6) set along the direction of the channel (1) and driving the sliding seat (4) to slide through the screw slider (7); a mounting box (8) is rotatably connected below the sliding seat (4), with an acquisition unit assembly inside the mounting box (8), and a transparent window at its opening; a torsion spring (9) is provided at the rotation shaft of the mounting box (8) so that the opening faces downward in the free state; push rods (10) are provided at both ends of the mounting frame (2), and when the sliding seat (4) moves to the end, the push rods (10) push against the mounting box (8) to deflect it, so that the transparent window faces the other end of the channel (1); Acquisition unit components: include an image acquisition unit, an audio acquisition unit, and an environmental sensing unit, which are used to acquire color image data, depth image data, vocalization audio data of cattle, and environmental parameters of the farm, including temperature, humidity, and ammonia concentration; Data processing unit: located in or connected to the installation box (8), and connected to each acquisition unit respectively, including edge computing module and storage module, used to perform edge preprocessing on the acquired multimodal data to generate multimodal monitoring data; Multimodal data fusion engine: Built into the data processing unit, it is used to perform spatiotemporal alignment, feature extraction and weighted fusion of multimodal monitoring data to generate fused feature data; Cattle health assessment model: Built into the data processing unit, used to generate cattle health scores and risk levels based on fused feature data; Early warning module: Connected to the data processing unit, it is used to issue an early warning signal when the health score is lower than the threshold or the risk level reaches the preset level; Remote management platform: Connected to the data processing unit via a wireless communication network, used to display monitoring data, health scores, risk levels, and early warning signals; Control logic: The servo motor (5) drives the sliding seat (4) to move along the channel (1); when the cow enters the channel (1), the sliding seat (4) moves to the corresponding end and deflects under the action of the push rod (10), so that the transparent window faces the cow and collects data for each cow individually; when the data processing unit judges that the cow is abnormal, the servo motor (5) drives the sliding seat (4) to move directly above the cow and follow it to perform close-range synchronous data collection throughout the process.
2. The apparatus according to claim 1, characterized in that, The image acquisition unit includes an RGB-D depth camera and an infrared thermal imager, and the audio acquisition unit includes an omnidirectional microphone array.
3. The apparatus according to claim 1, characterized in that, The edge computing module includes: a target detection submodule for detecting and tracking cattle from image data and generating spatiotemporal trajectory information; a behavior recognition submodule for recognizing the behavioral state of cattle based on spatiotemporal trajectory information and depth image data; and an audio analysis submodule for extracting voiceprint features from call audio data to identify the stress state and health state of cattle.
4. The apparatus according to claim 1, characterized in that, The multimodal data fusion engine includes: a data spatiotemporal alignment module for unifying timestamp alignment and spatial coordinate mapping; a feature extraction module for extracting visual features, behavioral temporal features, acoustic features, and environmental features; and a cross-modal attention fusion module for weighted fusion of features from various modalities using a cross-modal attention mechanism to generate fused feature data.
5. The apparatus according to claim 1, characterized in that, The cattle health assessment model adopts a Transformer-based multimodal large language model, including: a health status encoder, used to encode fused feature data into a health status vector; an anomaly detection branch, used to generate anomaly probability based on the deviation between the health status vector and a preset benchmark vector; a health scorer, used to map the health status vector into a continuous health score; and a risk classifier, used to classify cattle into at least three risk levels based on the anomaly probability and the health score.
6. The apparatus according to claim 1, characterized in that, The early warning module includes: a local sound and light alarm, located above the channel (1); and a remote push module, used to push early warning messages to the remote management platform through a wireless communication network. The early warning messages include the number of the abnormal cattle, its location coordinates, the type of abnormality, and real-time monitoring data segments.
7. The apparatus according to claim 1, characterized in that, It also includes a cattle individual identification module, which is used to identify cattle based on at least one of their facial features, markings and body shape features, generate an individual identification result corresponding to the unique identifier of the cattle, and store the multimodal monitoring data in association with the individual identification result; in the control logic, when an abnormal cattle is identified, the sliding seat (4) moves to the position corresponding to the individual identifier of the abnormal cattle and records the full tracking data.
8. A non-contact multimodal intelligent monitoring and early warning method for cattle, applied to the device described in any one of claims 1 to 7, characterized in that, include: S1: Guide the cattle one by one into the passage (1); S2: Control the servo motor (5) to drive the sliding seat (4) to move to the end of the channel (1), and use the push rod (10) to deflect the mounting box (8) so that the transparent window faces the direction of the cattle's movement. Collect multimodal data for each cow individually, including color images, depth images, vocalization audio and environmental parameters. S3: The data processing unit performs edge preprocessing on the collected multimodal data to generate multimodal monitoring data; S4: The multimodal data fusion engine performs spatiotemporal alignment, feature extraction, and weighted fusion on the multimodal monitoring data to generate fused feature data; S5: The cattle health assessment model generates health scores and risk levels based on fused feature data; S6: When the health score is below the threshold or the risk level reaches the preset level, the animal is judged to be abnormal and an early warning signal is issued by the early warning module. S7: In response to the abnormal judgment, control the sliding seat (4) to move directly above the abnormal cow and follow its movement to collect data synchronously at close range throughout the process. At the same time, push the monitoring data, health score, risk level and early warning signal to the remote management platform for display.