Pig feeding amount estimation method and system based on computer vision recognition

Abstract

The present application relates to the technical field of agricultural intelligence and computer vision, and discloses a pig feeding amount estimation method and system based on computer vision recognition, which comprises the following steps: synchronously collecting image sequences and audio signals; extracting live pig head posture features and chewing / swallowing acoustic features; fusing audio-visual features through a cross-modal attention mechanism and discriminating real feeding behaviors; and combining a single swallowing amount model to cumulatively estimate the feeding amount. The system comprises multi-source data collection, visual feature processing, acoustic feature processing, cross-modal fusion and discrimination, and feeding amount quantitative estimation modules. Through audio-visual cross-modal fusion and adaptive weight adjustment, the present application significantly improves the feeding behavior recognition accuracy and system robustness, and realizes individualized accurate feeding amount estimation.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of agricultural intelligence and computer vision, specifically relating to a method and system for estimating pig feed intake based on computer vision recognition. Background Technology

[0002] With the rapid development of smart agriculture and large-scale pig farming, precision feeding management has become a key aspect of improving breeding efficiency and ensuring pig health. Real-time monitoring and analysis of pig feeding data can not only directly reflect the growth and development status of pigs, but also provide a scientific basis for feed formulation optimization and disease early warning, which is of great significance for achieving refined management of the breeding process.

[0003] Non-contact sensing-based pig behavior monitoring technology is a current research hotspot in the field of smart farming. This type of technology typically uses image acquisition devices deployed above the pens to acquire images of pigs' activities in the feeding area. Computer vision algorithms are then used to automatically identify the pigs' identity, location, and posture, thereby enabling automated statistics on individual feeding duration and frequency.

[0004] Traditional computer vision-based techniques for estimating pig feed intake suffer from several drawbacks: First, in high-density farming environments, frequent occlusion between pigs can easily lead to missed detections or target loss in visual recognition algorithms, severely impacting the continuity of feeding behavior monitoring. Second, visual algorithms struggle to accurately distinguish between genuine feeding behavior and visually similar non-feeding distractions such as rooting or scratching, resulting in significant deviations in feed intake estimation. Third, single visual modalities lack stability in farming environments with drastic lighting changes or severe dust interference, and lack a complementary calibration mechanism based on multi-dimensional information. Finally, existing technologies generally ignore the unique acoustic characteristics of the feeding process, failing to effectively verify chewing and swallowing actions from a physiological perspective. These issues collectively contribute to low accuracy in feed intake estimation and insufficient robustness in behavior discrimination. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for estimating pig feed intake based on computer vision recognition, which can effectively solve the problems mentioned in the background art. The primary technical problem to be solved by this invention is: in high-density farming environments, how to overcome the problems of missed detection and misjudgment of individual pigs caused by visual occlusion, and accurately distinguish between the actual feeding actions of pigs and non-feeding interference behaviors such as rooting and scratching, so as to achieve high-precision automatic estimation of individual pig feed intake.

[0006] To achieve the above objectives, this invention proposes a method for estimating pig feed intake based on computer vision recognition, comprising the following steps: S1. Synchronously collect multi-dimensional sensing data of the pig feeding area. The multi-dimensional sensing data includes a continuous monitoring image sequence of the feeding area and the corresponding synchronous audio signal stream. S2. Extract spatial features and segment target instances from continuous monitoring image sequences to identify the head spatial coordinates and real-time motion posture features of individual pigs. S3. Perform time-frequency domain analysis and voiceprint feature extraction on the synchronous audio signal stream to identify physiological acoustic signal features that are highly correlated with eating behavior; S4. Align the extracted motion posture features with the physiological acoustic signal features in the time dimension, and fuse them through a cross-modal attention mechanism to construct an eating behavior discrimination vector; S5. Based on the feeding behavior discrimination vector, the effective feeding time of individual pigs is accumulated, and combined with the preset single swallowing volume assessment model, the estimated value of the cumulative feeding volume of individual pigs is calculated.

[0007] Preferably, step S2 specifically includes the following steps: S21, preprocessing the continuous monitoring image sequence, removing background interference through a background modeling algorithm, and extracting candidate regions containing individual pigs; S22, using a deep residual convolutional neural network as the backbone for feature extraction to obtain multi-scale semantic features of pigs within the candidate regions; S23, performing pixel-level segmentation of the head and torso of individual pigs through an instance segmentation network, and determining the relative position offset of the pig's head relative to the feed trough; S24, combining continuous frame images in the time series to calculate the motion trajectory of the pig's head position and the duration of the head-down posture.

[0008] Preferably, step S3 specifically includes the following steps: S31, performing preset gain compensation on the original synchronous audio signal stream and removing steady-state background noise in the breeding environment through adaptive spectral subtraction; S32, using sliding window technology to perform frame processing on the denoised audio signal and calculating the short-time energy distribution and zero-crossing rate characteristics of each frame of audio signal; S33, extracting the acoustic features of the audio signal in a specific frequency band, focusing on capturing the impact acoustic features generated by the chewing action of pigs and the specific frequency envelope features generated by the swallowing action; S34, performing temporal modeling of the audio features through a deep recurrent neural network and outputting the confidence score of each moment belonging to the feeding acoustic features.

[0009] Preferably, step S4 specifically includes the following steps: S41, constructing a cross-modal temporal alignment matrix to precisely couple the feature frames of the video sequence with the feature segments of the audio stream at the same timestamp; S42, calculating the cross-correlation weights of visual features and audio features based on a cross-modal attention mechanism, and automatically increasing the discrimination weight of audio features when the occlusion ratio in the visual features exceeds a preset occlusion threshold; S43, fusing the weighted visual and audio features to generate a multi-dimensional feature vector to characterize the pig's action attributes at the current moment; S44, using a discriminant classifier to classify the multi-dimensional feature vector and outputting the classification result of whether the current action belongs to real feeding behavior, rooting behavior, or scratching behavior.

[0010] Preferably, the preprocessing process includes histogram equalization of the continuous monitoring images to eliminate the impact of uneven lighting in the farm on image quality, and bilateral filtering algorithm to suppress high-frequency sensor noise while preserving the edge features of pigs, providing high-contrast image input for subsequent instance segmentation.

[0011] Preferably, the feature extraction backbone adopts a deep residual network structure, which solves the gradient vanishing problem in the deep network training process by introducing a skip connection mechanism, and extracts the apparent texture, contour shape and abstract semantic information of pigs at different convolutional layers, so as to ensure that key local features can still be preserved when pigs overlap.

[0012] Preferably, the instance segmentation network generates potential target regions through a region proposal module and uses a mask branch to determine the pixel affiliation of pigs within each target region, thereby accurately separating the independent contour of each pig in a complex compression and occlusion environment, and calculating the normal distance from the center point of the pig's head to the edge of the feed trough.

[0013] Preferably, the method for calculating the duration of the head-down posture is as follows: when the center point of the pig's head enters the preset feed trough trigger area, and the angle between the pig's neck and spine is within the preset feeding posture angle range, it is determined to be in a feeding posture, and the system records the start and end timestamps of the posture.

[0014] Preferably, the adaptive spectral subtraction method estimates the non-stationary noise power spectrum in the environment in real time and subtracts the noise power spectrum component from the power spectrum of noisy speech, thereby suppressing mechanical noise generated by the operation of the fan and the feeding system while preserving the details of transient signals such as chewing sounds.

[0015] Preferably, the acoustic feature extraction process further includes extracting Mel frequency cepstral correlation features, which, by simulating the perceptual characteristics of biological hearing, maps the audio signal from a linear spectrum to a nonlinear scale, thereby significantly enhancing the system's ability to distinguish between high-frequency crackling sounds in chewing sounds and low-frequency peristaltic sounds in swallowing sounds.

[0016] Preferably, the deep recurrent neural network adopts a bidirectional long short-term memory structure, which can simultaneously utilize past and future audio context information to identify the periodic repetitive patterns of chewing actions, thereby effectively eliminating non-feeding audio interference such as sudden impact sounds or pig noises.

[0017] Preferably, the cross-modal attention mechanism involves calculating the dot product similarity between visual feature vectors and audio feature vectors, and dynamically adjusting the contributions of the two modalities in behavior determination through an attention score matrix. When the pixel overlap of the target region in a video frame is detected to be higher than a certain proportion, the system determines that physical occlusion has occurred. At this time, the reliability weight of the visual modality is reduced, while the weight share of the audio modality in behavior classification is increased simultaneously.

[0018] Preferably, the construction of the eating behavior discrimination vector also considers the temporal consistency of the action. That is, only when the head-down posture and the continuous chewing acoustic features are highly overlapping in time and the duration exceeds the preset minimum eating duration threshold, is it determined to be a valid eating event.

[0019] Preferably, the single-swallowing volume assessment model is a regression model constructed based on historical experimental data. Its input parameters include the pig's estimated weight, current growth stage coefficient, signal intensity integral of the feeding audio, and chewing frequency. By integrating the intensity of the chewing sounds, the number of swallows by the pig within that time period is estimated, thereby obtaining the feed intake quality per unit time.

[0020] Preferably, the calculation logic for the cumulative feed intake estimate is as follows: integrate the time interval of all valid feeding events throughout the day, dynamically adjust the instantaneous feeding rate according to the real-time audio feature intensity within each integration unit, and finally obtain the total daily feed intake of an individual pig through an accumulation operation.

[0021] A computer vision-based pig feed intake estimation system, used to implement the aforementioned method, includes: a multi-source data acquisition module, used to simultaneously acquire image sequences and audio waveform data of pigs during the feeding process through visual sensors and an audio acquisition matrix deployed in the feeding area, and to ensure clock synchronization of the two types of data using a preset synchronization signal; a visual feature processing module, whose input is connected to the visual output of the multi-source data acquisition module, and is equipped with a deep learning computing unit, used to perform target detection, tracking, and posture analysis of pigs, and output spatial position descriptors of individual pigs; and an acoustic feature processing module, whose output... The input is connected to the audio output of the multi-source data acquisition module to perform audio denoising, feature conversion and voiceprint recognition, and output acoustic feature vectors that match the feeding action; the cross-modal fusion discrimination module is connected to the visual feature processing module and the acoustic feature processing module respectively, and uses the attention mechanism layer to realize the weighted fusion of audiovisual information, and performs fine classification of behaviors such as feeding, rooting, and scratching based on the fused features; the feed intake quantification estimation module is connected to the output of the cross-modal fusion discrimination module, and is used to filter the effective feeding time period based on the classification results, and call the feed intake quantification model to calculate the final feed intake data.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention breaks through the limitations of traditional computer vision, which relies solely on optical sensing signals, by innovatively introducing a deep cross-modal fusion mechanism of acoustic signal processing and computer vision. First, in high-density farming scenarios where visual occlusion is highly likely, acoustic signals, as a non-contact sensing method unaffected by physical obstructions, effectively fill visual blind spots. This allows the system to maintain continuous monitoring of individual feeding status through voiceprint features even when pigs overlap and visual recognition fails. Second, by meticulously extracting chewing and swallowing sounds, this invention establishes a discrimination logic for verifying feeding behavior at the physiological mechanism level. This accurately eliminates invalid actions such as rooting and scratching, which are visually similar to feeding but do not actually involve food intake, greatly improving the purity and accuracy of feeding behavior recognition. Furthermore, the audiovisual cross-modal fusion mechanism adaptively adjusts perception weights according to environmental changes. When vision is clear, visual localization is the primary mode; when vision is limited or environmental noise is high, the decision-making mode is automatically switched, significantly enhancing environmental anti-interference capabilities. In summary, this invention achieves a leap from extensive identification of feeding behavior to refined measurement, providing robust and reliable individual feeding data support for smart farming.

[0023] 2. The application of a cross-modal attention mechanism enables the system to self-assess the quality of the perceived environment. By dynamically monitoring the occlusion ratio of the visual modality and the signal-to-noise ratio of the audio modality, the system can optimize the feature fusion strategy in real time. When sudden dust occlusion or high concentration of pigs occurs in the farm, the automatic compensation mechanism of audio weights ensures the integrity of the monitoring data, avoids the missed detections and data interruptions common in traditional methods, and ensures the stability of feed intake estimation under all-weather conditions.

[0024] 3. By employing deep residual networks and pixel-level instance segmentation technology, this invention achieves accurate separation and detailed posture description of individual pigs. Compared to traditional bounding box recognition methods, pixel-level segmentation can provide precise interaction relationships between the pig's head and the feed trough edge, thereby eliminating invalid activity interference from non-feeding areas in the spatial dimension.

[0025] 4. By combining in-depth time-frequency domain analysis of acoustic signals with recurrent neural networks, the system can keenly capture the rhythmic characteristics of chewing and swallowing. This physiological rhythm-based discrimination method not only improves the accuracy of eliminating interfering actions such as rooting for feed, but also reflects the pig's appetite status through the speed of chewing, providing a deeper physiological reference indicator for pig health early warning beyond feed intake.

[0026] 5. The multimodal temporal alignment technology adopted in this invention ensures a one-to-one correspondence between the visual head-down action and the acoustic chewing sound on the time axis. This spatiotemporal coupling judgment logic greatly reduces the false alarm rate, enabling the system to effectively distinguish the complex situation of multiple pigs taking turns to eat in the same area, and achieve truly individualized and accurate estimation.

[0027] 6. The system-integrated single-swallowing volume assessment model utilizes a nonlinear mapping between audio signal intensity and chewing frequency. This method considers the changes in feeding rate of pigs at different feeding stages, making the quantitative results of feed intake more consistent with actual biophysical processes, significantly reducing cumulative errors, and improving the scientific validity and reliability of the estimation results.

[0028] 7. The highly integrated design of each module in the system architecture of this invention, combined with adaptive denoising and filtering algorithms, enables the system to operate stably for a long time in harsh environments of breeding farms with high humidity, high dust and high noise, reducing dependence on high-standard hardware environment, and has strong engineering application value and promotion prospects. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the overall technical solution architecture of the present invention; Figure 2 This is a schematic diagram of the core principle framework of audiovisual feature fusion based on cross-modal attention mechanism in this invention; Figure 3 This is a flowchart illustrating the visual feature extraction logic of pig individual head instance segmentation and motion posture analysis in this invention. Figure 4 This is a logical flowchart of the recognition and temporal modeling of physiological acoustic signals related to pig feeding behavior in this invention; Figure 5 This is a flowchart illustrating the logical process of quantitative estimation of food intake based on the feeding behavior discrimination vector and swallowing volume regression model in this invention. Detailed Implementation

[0030] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0031] Example 1, please refer to Figures 1 to 5 This embodiment provides a method for estimating pig feed intake based on computer vision recognition. This method integrates high-dimensional visual information with physiological acoustic signals to achieve accurate quantification of individual pig feeding behavior in complex farming environments. The method in this embodiment is logically divided into four stages: data perception, feature extraction, sound-image fusion, and quantification estimation.

[0032] During the data perception phase, i.e., when executing step S1, the system synchronously acquires multi-dimensional perception data of the pigs' feeding process through multi-source data acquisition devices deployed above the pigs' feeding area. This multi-dimensional perception data includes a continuous sequence of monitoring images of the feeding area and corresponding synchronous audio signal streams. To ensure the accuracy of subsequent cross-modal analysis, the visual sensor uses an industrial-grade camera with high frame rate output characteristics. Its installation height is standardized according to the physical dimensions of the pigpen fence, ensuring that the field of view completely covers the feed trough and its surrounding activity area of ​​at least 1 meter. Audio acquisition is achieved through a high-sensitivity omnidirectional microphone array, with microphones positioned near the side wall of the feed trough to maximize the capture of chewing and swallowing sounds. During acquisition, a high-precision clock source within the system sends synchronous trigger pulses to the visual acquisition unit and the audio acquisition unit, ensuring that the time deviation between each frame of image and the corresponding audio sampling segment is controlled within 5 milliseconds.

[0033] The process proceeds to step S2, which involves spatial feature extraction and target instance segmentation of the continuous monitoring image sequence. The core of this step lies in locating key parts of the pig using pure visual geometric logic. In sub-step S21, the system first preprocesses the acquired raw images. Through histogram equalization, the system automatically adjusts the contrast distribution of the images, restoring clear texture features to areas that were too dark or too bright due to uneven lighting in the farm. Subsequently, a bilateral filtering algorithm is used to denoise the images. This algorithm filters out high-frequency sensor noise while accurately preserving the sharpness of the pig's body edges based on the spatial proximity and color similarity of pixels, preventing edge blurring from interfering with subsequent segmentation. The background modeling algorithm constructs a stable background probability distribution model through statistical learning on more than 100 consecutive frames of empty feed trough images. This allows for the rapid extraction of candidate target regions containing individual pigs when they enter the field of view using background subtraction technology. In sub-step S22, a deep residual convolutional neural network is used as the backbone for feature extraction. This network structure, by introducing multiple skip connection branches, allows low-level geometric features in the image, such as edges, lines, and textures, to be directly transmitted to the deep network, effectively alleviating the information loss problem during deep network training. Through the hierarchical stacking of multiple convolutional kernels, the system extracts the appearance features of pigs at different scales, including skin wrinkles, hair distribution, and semantic information of the overall contour. In sub-step S23, the instance segmentation network plays a role. This network can not only identify the presence of pigs in the image but also achieve pixel-level classification. The system uses mask generation technology to accurately separate pixels belonging to the pig's head from pixels belonging to the torso. Based on this, the system determines the spatial coordinates of the pig's head by calculating the geometric center of the head pixel centroid. Simultaneously, the system compares the physical distance between the head center coordinates and the preset feed trough edge coordinates in real time to obtain the relative position offset of the pig's head relative to the feed trough. In sub-step S24, by associating consecutive video frames on the time axis, the system constructs the motion vector field of the pig's head. When the normal distance of the head is continuously lower than the preset trigger threshold and the height of the head relative to the ground is within the feeding range, the pig is determined to be in a head-down posture, and the start and end timestamps of the posture are recorded to calculate the duration of the head-down posture.

[0034] In step S3, the system synchronously processes the audio signal stream. Sub-step S31 involves gain compensation and noise reduction of the original audio. Adaptive spectral subtraction plays a crucial role here. By monitoring the steady-state noise spectrum of mechanical equipment such as fans and manure scrapers in the environment in real time, it subtracts the corresponding noise component from the power spectrum of the mixed audio, thereby greatly improving the signal-to-noise ratio while preserving faint chewing sounds. Sub-step S32 uses sliding window technology to divide the continuous audio into tiny frames with a length of 20 milliseconds. For each frame of audio, the system calculates its short-time energy distribution, i.e., the sum of the squares of the signal amplitude, to reflect the intensity of the sound; at the same time, it calculates the zero-crossing rate feature, i.e., the frequency at which the signal waveform crosses the zero-level axis, to distinguish high-frequency crackling sounds from low-frequency background sounds. Sub-step S33 focuses on the deep extraction of voiceprint features. The system focuses on capturing the impactful acoustic signatures produced by pigs during feed crushing, which manifest as instantaneous broadband energy bursts in the frequency spectrum. Simultaneously, it captures the specific frequency envelope generated by swallowing movements, reflecting the low-frequency acoustic characteristics produced by the peristalsis of the throat muscles. Sub-step S34 utilizes a deep recurrent neural network to perform long-term temporal modeling of these features. This network employs its internal gating mechanism to learn the temporal recurrence patterns of chewing behavior and outputs a confidence score for each time slice, reflecting that the acoustic signature at that moment belongs to the actual feeding process.

[0035] Step S4 is the core fusion logic of this invention, which couples motion posture with physiological acoustic signals in the time dimension. In sub-step S41, a cross-modal temporal alignment matrix is ​​constructed. Since the video sampling rate and audio sampling rate are different, the system uses an interpolation algorithm to map the feature vectors of both to the same clock step. In sub-step S42, a cross-modal attention mechanism is introduced. When the vision module detects a high degree of pixel overlap in the current pig individual, i.e., severe mutual occlusion occurs, the system automatically reduces the weight of visual features in the classification decision. At this time, the attention weight will tilt towards audio features, mainly relying on chewing sounds to determine the continuity of feeding behavior. In sub-steps S43 and S44, the fused multi-dimensional feature vector is input into the discriminant classifier. This classifier can accurately identify which actions are the pigs actually eating and which actions are merely the pigs rooting and playing around the feed trough or rubbing against the edge of the feed trough to relieve itching, based on the combination rules of audiovisual information.

[0036] Finally, in step S5, the system calculates the total time of all valid feeding events throughout the day based on the discriminant vector. Combined with a preset single swallowing volume assessment model, which comprehensively considers the growth and development stage coefficient of the pig, the intensity integral value of the audio signal, and the chewing frequency, the system calculates the amount of matter intake during each feeding process through nonlinear regression, and finally obtains the estimated cumulative feed intake of the individual pig by summing them up.

[0037] Example 2: This example focuses on describing a pig feed intake estimation system based on computer vision recognition. This system serves as the physical platform for the method described in Example 1, and its hardware architecture and software module coordination mechanism are described.

[0038] The system mainly consists of a multi-source data acquisition module, a visual feature processing module, an acoustic feature processing module, a cross-modal fusion discrimination module, and a feed intake quantification estimation module. These modules are interconnected via a high-speed data bus, ensuring real-time processing of massive amounts of sensor data.

[0039] The multi-source data acquisition module physically comprises an infrared-enhanced high-definition camera and an industrial-grade audio pickup array deployed above the breeding unit. The camera is equipped with a wide dynamic range sensor, enabling it to output clear images of pig activity in both bright morning light and low-light conditions at night. The audio pickup array uses a digital signal output interface to directly convert analog audio signals into high-bit-depth digital pulse-code modulation signals, reducing electromagnetic interference during transmission. The module's integrated synchronization controller uses a precise clock protocol to synchronize the timing of all sensors, ensuring that the starting point for audio and video data acquisition is completely consistent.

[0040] The visual feature processing module utilizes a computing terminal equipped with a high-performance graphics processing unit. Internally, this module runs a deep residual convolutional neural network and a mask region proposal network. Upon receiving the image stream, the visual feature processing module first performs pixel-level image enhancement. In this embodiment, the module performs spatial domain nonlocal mean filtering on each frame of the image. This method utilizes redundant information in the image to significantly reduce noise while greatly improving the clarity of the pig's outline. Subsequently, the module extracts multi-scale feature maps to identify pigs in different receptive fields. For overlapping pigs, the module uses an instance segmentation algorithm to assign a unique identification label to each pig and draws a precise pixel mask of its head. By calculating the overlap between the mask region and the preset feed trough geometry model, the module can output the real-time three-dimensional pose parameters of the pig's head.

[0041] The acoustic feature processing module focuses on spectral analysis of audio signals. This module integrates a dedicated digital signal processor to run adaptive spectral subtraction logic. Under this logic, the module continuously updates the estimation model for non-stationary noise in the environment. When sudden disturbances occur, such as the sound of a feeding cart starting or the squeals of pigs, the module can use time-frequency masking techniques to remove these non-feeding-related audio components. Subsequently, the module calculates Mel-frequency cepstral coefficients to convert the sound signal into a multi-dimensional feature vector that conforms to biological auditory characteristics. These vectors can sensitively capture the physical fragmentation characteristics produced when pigs chew solid pellet feed.

[0042] The cross-modal fusion discrimination module is the decision-making center of the entire system. It receives spatial pose sequences from the visual module and voiceprint confidence sequences from the acoustic module. This module dynamically evaluates the reliability of different modal data through a bidirectional attention mechanism layer. For example, during peak feeding periods, pigs huddle together in front of the feed trough, and visual head positioning is often intermittent due to occlusion. At this time, the cross-modal fusion discrimination module detects a decrease in the confidence of the visual mask, thereby automatically increasing the weight coefficient of acoustic features in the discrimination logic. As long as the pickup array captures continuous and rhythmically stable chewing pulses, the system will still determine that the individual is in a feeding state, thus ensuring the continuity of monitoring data.

[0043] The feed intake quantification module is responsible for converting the identified feeding behaviors into specific weight values. This module incorporates a biophysical regression model trained on a large amount of experimental data. This model not only simply counts the duration but also analyzes the energy intensity of the audio signal. When pigs eat faster and chew more forcefully, the energy integral value of the audio signal increases accordingly, and the module adjusts the instantaneous feed intake rate parameter accordingly. Finally, the module integrates the instantaneous feed intake of all feeding segments throughout the day to obtain the total individual feed intake data accurate to the gram level.

[0044] Example 3: This example further illustrates the application details of the present invention in actual high-density farming scenarios, especially the identification and quantification logic of individual pigs under extremely crowded conditions.

[0045] In large-scale pig farms, it is extremely common for multiple pigs to compete for food simultaneously. In such cases, traditional visual methods often suffer from significant underestimation because they cannot see the heads of occluded individuals. This embodiment addresses this technical challenge by using deep logic to enhance the cross-modal attention mechanism.

[0046] First, during instance segmentation in step S2, the system identifies not only the head but also the back texture and ear tag position of the pig. When the visual feature processing module determines that the head of one pig is completely covered by the torso of another pig for more than 30 consecutive frames, the visual localization logic enters prediction mode. In prediction mode, the system uses the Kalman filter algorithm to estimate the possible lurking position of the pig's head based on its direction and speed of movement before it went missing.

[0047] Simultaneously, the acoustic feature processing module initiates spatial sound source localization logic. The microphone array determines the spatial location of the current chewing sound by calculating the time difference of sound arrival at different microphones. The cross-modal fusion discrimination module compares the sound source azimuth with the visually predicted head position. If the deviation between the sound source azimuth and the predicted position is within a preset error range, and the captured audio features have a clear eating feature envelope, the system determines that the occluded individual is still continuously eating.

[0048] In terms of refined differentiation of feeding behavior, this embodiment deeply deconstructs the rooting behavior and the feeding behavior. At the visual level, rooting is usually accompanied by frequent, violent up-and-down head swings and left-and-right sweeping movements, with a large radius of curvature in its trajectory; while during actual feeding, the head usually remains relatively still or experiences only slight tremors. At the acoustic level, rooting mainly produces the impact sound of plastic or metal feed troughs colliding, which appears as extremely short, high-frequency pulses in the frequency spectrum and lacks periodicity; while the chewing sounds produced by feeding have a clear physiological rhythm, with pulse intervals highly correlated with the chewing frequency of the pigs. By calculating the cross-correlation coefficient between visual motion curvature and audio periodicity scores, the system can reduce the misjudgment rate of the invalid rooting behavior to below 3%.

[0049] Furthermore, this embodiment adjusts the estimation model for different feed types. There are significant differences in the acoustic signature characteristics during chewing between pelleted and powdered feed. The system automatically identifies the physical form of the current feed by extracting the acoustic feature spectrum centroid in step S33. If it is pelleted feed, the system will apply a high-energy-consumption correction coefficient because crushing pellets requires greater chewing force, resulting in higher audio energy intensity. Through this adaptive model switching, the system can maintain a high level of accuracy in estimating feed intake under different feed supply conditions.

[0050] Example 4: This example details the technical aspects of the present invention in terms of environmental disturbance compensation to ensure the long-term stability of the system under harsh aquaculture environments.

[0051] Aquaculture environments typically contain high concentrations of dust and moisture, which can cause dirt to accumulate on the lenses of visual sensors over time, resulting in blurred images. To address this physical limitation, this invention adds an image quality evaluation branch to the preprocessing stage. This branch assesses the sharpness of the current image by calculating the high-frequency gradient components. When the sharpness index falls below a preset alarm threshold, the system automatically activates an acoustic enhancement decision mode. In this mode, visual features serve only as a rough reference for area localization, while the decision-making power for behavior discrimination is completely transferred to the acoustic feature processing module. The acoustic module compensates for the lack of visual information by increasing the overlap rate of the sliding window and performing deeper resampling of the audio stream.

[0052] To address non-stationary noise interference in the environment, such as squeals, fighting sounds, or severe vibrations caused by the operation of the feeding system, this embodiment introduces a multi-sensor cross-calibration mechanism in step S31. The system deploys reference microphones in areas of the pigsty far from the feed troughs to specifically collect background noise. The main processor calculates the coherence function between the reference microphone and the near-field microphone signals to cancel out global noise components unrelated to feeding from the near-field signal.

[0053] In the mathematical logic of quantifying feed intake, this embodiment dynamically calibrates the single-swallowing volume assessment model. Considering the changes in oral anatomy and swallowing ability of pigs at different growth stages (such as nursery, growth, and finishing stages), the system automatically retrieves and matches the corresponding growth coefficient from the model library based on the pig's body size estimated by the visual module. For example, for finishing pigs weighing over 100 kg, the mapping ratio of the acoustic energy integral generated by a single swallow to mass is adjusted upwards according to a preset physiological curve. This dynamic modeling method based on individual biological differences enables the system to consistently provide reliable feed intake data across different growth cycles.

[0054] The system also has an abnormal behavior monitoring function. If the feed intake quantification estimation module detects that a pig's daily feed intake has dropped abnormally sharply within 48 consecutive hours, and the acoustic feature processing module identifies that the individual's chewing frequency has slowed down significantly, the system will automatically trigger a health warning logic and push the abnormal status information of the pig to the breeding management terminal to assist veterinarians in early disease diagnosis.

[0055] Example 5: This example describes the data logic management of the present invention under multi-pig position collaborative monitoring, that is, how to process data from multiple feeding positions simultaneously in one system.

[0056] In a typical pig farming unit, there is usually a long, shared feed trough. This system achieves full coverage monitoring of the entire feed trough area by deploying multiple vision sensors and microphone arrays. The visual feature processing module divides the entire field of view into a grid, defining each feeding position as an independent logical unit.

[0057] When multiple pigs feed simultaneously, the system employs a multi-target tracking algorithm to establish an independent data trajectory for each individual entering the feeding area. The vision module calculates in real time the contact area ratio between each individual's head mask and its corresponding feed trough logic unit. The acoustic feature processing module utilizes blind source separation technology to separate chewing sound components from different physical locations in the mixed audio stream. Because the pigs' heads are extremely close to the feed trough while feeding, the signals captured by the microphone array exhibit significant spatial directivity. By calculating the arrival time difference and intensity difference of each audio signal, the system can accurately attribute specific chewing sound segments to specific pig individuals identified visually.

[0058] The cross-modal fusion discrimination module maintains a multi-dimensional state machine during this process. The state machine records the current action phase of each pig, including the feeding location search phase, the head-down feeding phase, the intermittent head-up phase, and the departure phase. Only when the visual head-down posture and the acoustic belonging chewing sound achieve a synchronous closed loop on the time axis is the time period counted as the effective feeding time.

[0059] The final step in feed intake estimation includes feedback correction for remaining feed. The system uses visual sensors to measure the height of residual feed in the feed trough in real time. After each automatic feeding cycle, the system compares the total feed intake with the sum of the estimated feed intake for all individuals, and automatically fine-tunes the individual estimation model parameters based on changes in residual feed. Through this closed-loop feedback mechanism, the system can eliminate systematic errors caused by individual sensor differences or long-term environmental drift.

[0060] In summary, this invention, through deep fusion of audiovisual cross-modal approaches, not only solves the occlusion problem in high-density aquaculture but also achieves a technological advancement from observing actions to confirming physiological intake by introducing physiological acoustic features. The system demonstrates extremely high robustness and accuracy in complex physical environments and dynamic biological behaviors, providing data asset support for precision aquaculture.

[0061] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for estimating pig feed intake based on computer vision recognition, characterized in that, Includes the following steps: S1. Multi-dimensional sensing data of the pig feeding area is synchronously acquired by a multi-source data acquisition device deployed above the pig feeding area. The multi-dimensional sensing data includes a high frame rate continuous monitoring image sequence and a corresponding synchronous audio signal stream. S2. The continuous monitoring image sequence is preprocessed to eliminate background interference, and spatial feature extraction and target instance segmentation are performed to identify the spatial coordinates of the centroid of the head pixels of the individual pig, the real-time movement trajectory of the pig's head, and the movement posture features represented by the angle between the pig's neck and spine. S3. Perform gain compensation, noise reduction and time-frequency domain depth analysis on the synchronous audio signal stream. By extracting the acoustic envelope features of the audio signal in a specific frequency band, capture the impact voiceprint features generated by the chewing action of pigs and the physiological acoustic signal features generated by the swallowing action, and output the corresponding acoustic feature confidence score. S4. Construct a cross-modal temporal alignment matrix, couple the extracted motion posture features with physiological acoustic signal features under a unified timestamp, and calculate the cross-correlation weights of visual features and audio features through a bidirectional cross-modal attention mechanism. Fuse the weighted visual and audio features to construct a feeding behavior discrimination vector to characterize the pig's action attributes. Use a discriminant classifier to classify the feeding behavior discrimination vector to identify real feeding behavior. S5. Based on the feeding behavior discrimination vector, the effective feeding time of each pig is accumulated, and a preset single swallowing volume assessment model is called. Using the growth stage coefficient of the pig, the signal intensity integral of the feeding audio, and the chewing frequency as input parameters, the estimated cumulative feeding volume of each pig is calculated.

2. The method for estimating pig feed intake based on computer vision recognition according to claim 1, characterized in that, Step S2 specifically includes: statistically learning continuous empty feed trough images through a background modeling algorithm to construct a background probability distribution model, and using background difference technology to extract candidate regions containing individual pigs. A deep residual convolutional neural network is used as the backbone for feature extraction. A skip connection mechanism is used to pass low-level geometric features in the image to the deep network. The apparent texture, contour shape and abstract semantic information of pigs are extracted at different convolutional levels. An instance segmentation network is used to perform pixel-level segmentation of the head and torso of individual pigs to determine the relative position offset of the pig's head relative to the feed trough. Then, a mask generation technique is used to separate the pixels belonging to the pig's head to determine the spatial coordinates. By combining continuous frame images in the time series, the change in normal distance of the pig's head center point relative to the edge of the feed trough is calculated. When the normal distance is continuously lower than a preset trigger threshold and the angle between the pig's neck and spine is within the preset feeding posture angle range, the start and end timestamps of the head-down posture are recorded, thereby calculating the duration of the head-down posture.

3. The method for estimating pig feed intake based on computer vision recognition according to claim 1, characterized in that, The specific steps of step S3 include: using adaptive spectral subtraction to estimate the non-stationary noise power spectrum in the breeding environment in real time, and subtracting the noise power spectrum component from the power spectrum of noisy speech in order to suppress mechanical background noise while preserving the transient chewing sound signal; The denoised audio signal is processed by frame segmentation using the sliding window technique. The short-time energy distribution and zero-crossing rate characteristics of each frame of audio signal are calculated. The short-time energy distribution is used to reflect the signal strength, and the zero-crossing rate characteristics are used to distinguish between high-frequency fragmented sounds and low-frequency background sounds. The instantaneous broadband energy burst characteristics generated when pigs break down feed were captured by time-frequency analysis, as well as the specific frequency envelope characteristics generated by the peristalsis of the throat muscles. The extracted audio features are modeled temporally using a deep recurrent neural network. The internal control mechanism is used to learn the periodic recurrence pattern of chewing behavior and output the confidence score of the acoustic features of eating at each time point.

4. The method for estimating pig feed intake based on computer vision recognition according to claim 1, characterized in that, Step S4 specifically includes: using an interpolation algorithm to map the feature frames of the video sequence and the feature segments of the audio stream to the same clock step, thereby achieving precise coupling between the visual modality and the audio modality in the time dimension; The attention score matrix is ​​generated by calculating the dot product similarity between visual feature vectors and audio feature vectors based on a cross-modal attention mechanism. The pixel overlap of individual pigs in the continuous monitoring image sequence is monitored in real time. When the pixel overlap of an individual pig is detected to be higher than the preset occlusion threshold, it is determined that physical occlusion has occurred. The weight of visual features in behavior determination is automatically reduced, and the weight of audio features in behavior classification is increased simultaneously. The weighted visual features and audio features are concatenated and fused to generate the feeding behavior discrimination vector, and the discrimination classifier is used to output the classification result of whether the current action belongs to real feeding behavior, feeding behavior, or scratching behavior.

5. The method for estimating pig feed intake based on computer vision recognition according to claim 1, characterized in that, Step S5 specifically includes: integrating the time interval of all effective feeding events of pigs throughout the day, and extracting the corresponding audio signal intensity integral value and chewing frequency in each integration unit; The size of pigs is estimated based on their visual characteristics, and the corresponding growth stage coefficients are retrieved and matched in the model library. The growth stage coefficient, audio signal intensity integral, chewing frequency, and estimated weight are input into the single swallowing volume assessment model. The number of swallowings of the pig in this time period is estimated by nonlinear regression algorithm, and then the instantaneous feeding rate is obtained. The instantaneous feeding rates of all feeding segments throughout the day are summed to obtain the total daily feed intake of an individual pig.

6. The method for estimating pig feed intake based on computer vision recognition according to claim 2, characterized in that, The preprocessing process includes: performing histogram equalization on the continuous monitoring image sequence to adjust the contrast distribution of the images in order to eliminate the influence of uneven lighting in the farm on texture features; A bilateral filtering algorithm is used to smooth the image based on the spatial proximity and color similarity of pixels, which removes high-frequency sensor noise while preserving the sharpness of the pig's body edges.

7. The method for estimating pig feed intake based on computer vision recognition according to claim 3, characterized in that, The acoustic feature extraction process also includes: extracting the Mel frequency cepstral correlation features of the audio signal, and enhancing the ability to distinguish between high-frequency crackling sounds in chewing sounds and low-frequency peristaltic sounds in swallowing sounds by mapping the audio signal from a linear spectrum to a nonlinear scale. The deep recurrent neural network employs a bidirectional long short-term memory structure. By utilizing past and future audio context information, it identifies the rhythmic features of chewing actions and eliminates sudden impact sounds or non-eating audio interference.

8. The method for estimating pig feed intake based on computer vision recognition according to claim 4, characterized in that, Step S4 also includes behavior prediction logic for the occluded individual: when it is determined that the centroid of the head pixel of a certain pig individual is occluded by the torso of another pig individual for more than a preset number of consecutive frames, the Kalman filter algorithm is used to predict the latent position of its head based on the direction and speed of the pig individual's movement before it disappears. The spatial sound source localization logic is executed using a microphone array. By calculating the time difference between the arrival of chewing sounds at different microphones, the spatial location of the current chewing sound source is determined. If the spatial source location is compared with the lurking location, and the deviation between the two is within a preset error range, and the captured audio features have a feeding feature envelope, then it is determined that the obscured pig is in a continuous feeding state.

9. A pig feed intake estimation system based on computer vision recognition, used to implement the method described in any one of claims 1 to 8, characterized in that, include: The multi-source data acquisition module is used to synchronously acquire image sequences and audio signal streams during the feeding process of pigs through visual sensors and microphone arrays deployed in the feeding area, and to use a synchronization controller to ensure clock synchronization between visual data and audio data. The visual feature processing module, whose input end is connected to the visual output end of the multi-source data acquisition module, is equipped with a deep learning computing unit for performing pig target detection, pixel-level instance segmentation, motion trajectory tracking and posture analysis, and outputs a spatial pose descriptor for an individual pig. The acoustic feature processing module, whose input is connected to the audio output of the multi-source data acquisition module, is equipped with a digital signal processor for performing adaptive audio denoising, voiceprint feature extraction and confidence scoring, and outputting an acoustic feature vector that matches the eating action. The cross-modal fusion discrimination module is connected to the visual feature processing module and the acoustic feature processing module respectively. It is used to realize the dynamic weighted fusion of audiovisual information by utilizing the cross-modal attention mechanism layer, and to perform behavior classification based on the fused features to identify valid eating events. The food intake quantification estimation module is connected to the output of the cross-modal fusion discrimination module. It is used to filter valid feeding time periods based on the classification results and call the food intake quantification model to calculate the final food intake data using audio energy integral and chewing frequency.

10. The pig feed intake estimation system based on computer vision recognition according to claim 9, characterized in that: The visual sensor includes an infrared-enhanced high-definition camera equipped with a wide dynamic range sensor; the microphone array includes an omnidirectional microphone array for capturing chewing and swallowing sounds. The cross-modal fusion discrimination module has a built-in multi-dimensional state machine, which is used to record the action phases of each pig, including the feeding stage, the head-down feeding stage, and the intermittent head-up stage. Only when the visual head-down posture and the acoustic belonging chewing sound achieve a synchronous closed loop on the time axis is it determined to be the effective feeding time. The feed intake quantification estimation module is equipped with a feedback correction unit. It measures the height of residual feed in the feed trough using the visual sensor, compares the total feed supply and the change in residual feed with the sum of the estimated feed intake of all individuals, and automatically fine-tunes the model parameters of the feed intake quantification model.

Images