Pig raising monitoring management system based on computer vision

Abstract

The invention relates to the technical field of pig raising monitoring management, and discloses a pig raising monitoring management system based on computer vision, which comprises a data acquisition module, a data preprocessing module, a lightweight deep learning module, a multi-modal data fusion module and a termination module. Visual and audio information can be fully utilized through multi-modal data fusion, the defects of single-modal data are made up, the detection precision of various complex behaviors is improved, meanwhile, the problems of shielding and posture diversity can be better solved on the premise of guaranteeing certain precision in cooperation with a lightweight deep learning model, and the detection precision is improved. In addition, the calculation complexity of the lightweight model is low, real-time processing can be achieved under limited calculation resources, behavior information of the pigs can be fed back in time, meanwhile, environment problems such as illumination changes and noise interference can be effectively solved through a multi-modal data collection and data preprocessing method, and the accuracy of pig behavior detection is improved. And the stability and reliability of the system under different environmental conditions are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of pig farming monitoring and management, and more specifically discloses a pig farming monitoring and management system based on computer vision. Background Art

[0002] Pig farming monitoring and management based on computer vision is a method of using computer vision technology to intelligently monitor and manage pig farms, aiming to improve the efficiency, quality and benefits of pig production, while reducing labor costs and management difficulties, and having advantages such as improving production efficiency, precise farming, and early warning.

[0003] However, in the current pig farming monitoring and management system based on computer vision, pigs have various complex postures in the actual breeding environment, and the existing monitoring models cannot accurately identify some special postures, resulting in incorrect behavior classification. When pigs are raised in groups, they are easily blocked by each other, which will hide some body characteristics and affect the accurate judgment of pig behavior by the monitoring algorithm. In addition, computer vision algorithms require a large amount of computing resources to run, and in the actual pig farm monitoring system, they cannot meet the requirements of real-time processing, resulting in delayed behavior monitoring results. At the same time, the interference of light changes and noise also increases the difficulty of monitoring. Summary of the Invention

[0004] The main technical problem to be solved by the present invention is to provide a pig farming monitoring and management system based on computer vision, which can solve the problems of low monitoring accuracy, monitoring delay, and environmental impact on monitoring existing in the current pig farming monitoring and management system based on computer vision.

[0005] To solve the above technical problems, according to one aspect of the present invention, more specifically, a pig farming monitoring and management system based on computer vision includes: a data acquisition module, a data preprocessing module, a lightweight deep learning module, a multi-modal data fusion module, end;

[0006] The data acquisition module monitors the pigsty comprehensively to obtain visual data and simultaneously collects the audio data of pig calls;

[0007] The data preprocessing module preprocesses the obtained visual data and audio data to provide high-quality data for the subsequent lightweight deep learning module and multi-modal data fusion module;

[0008] The lightweight deep learning module uses deep learning algorithms to extract features and train models from the preprocessed data, and constructs an efficient and accurate pig behavior recognition model;

[0009] The multi-modal data fusion module fuses the processed visual features and audio features to form a more comprehensive feature representation, and combines the classification results of vision and audio to improve the accuracy of behavior detection;

[0010] At the end, the fused result is output for generating pig behavior reports and triggering the abnormal behavior warning mechanism, providing decision-making support for pig farming monitoring and management.

[0011] Furthermore, the data acquisition module includes: a visual data acquisition module and an audio data acquisition module;

[0012] The visual data acquisition module uses multiple high-definition cameras to monitor the pigsty from different angles, obtaining image information of pigs from different perspectives, so as to capture pig behavior characteristics more comprehensively, reduce the influence of occlusion problems. At the same time, cameras with a wide dynamic range are adopted to improve the adaptability to light changes;

[0013] The audio data acquisition module collects the vocal audio information of pigs through a microphone. Different behaviors are accompanied by different sound characteristics, and audio data can be used as a supplement to visual data to improve the accuracy of behavior detection.

[0014] Furthermore, the data preprocessing module includes: a visual preprocessing module and an audio preprocessing module;

[0015] The visual preprocessing module performs denoising and enhancement processing on the collected images to improve the image quality. At the same time, adaptive histogram equalization is used for light compensation;

[0016] The audio preprocessing module performs filtering and feature extraction operations on the audio data to extract audio features related to pig behavior.

[0017] Furthermore, the visual preprocessing module includes: a denoising processing module and a light compensation module;

[0018] The denoising processing module uses median filtering or Gaussian filtering methods to remove noise in the images;

[0019] The light compensation module divides the image into multiple small blocks by adaptive histogram equalization, performs histogram equalization processing on each small block separately, and then splices the processed small blocks together by bilinear interpolation to improve the image quality problem caused by uneven lighting.

[0020] Furthermore, the audio preprocessing module includes: a filtering processing module and a feature extraction module;

[0021] The filtering processing module uses low-pass filtering or band-pass filtering to remove high-frequency noise or unwanted frequency components in the audio signal, improving the quality of the audio signal;

[0022] The feature extraction module extracts Mel-frequency cepstral coefficients, pitch, and volume features in the audio, which can effectively describe the characteristics of pig calls and provide a basis for subsequent behavior classification.

[0023] Furthermore, the lightweight deep learning module includes: a behavior classification module and an object detection module;

[0024] The behavior classification module uses a lightweight convolutional neural network to quickly extract features and classify the images of pigs, judge the behavior states of pigs, and at the same time reduce the training time and improve the generalization ability of the model through transfer learning based on the trained model provided by the lightweight convolutional neural network, so as to improve the monitoring of pig behavior;

[0025] The object detection module is based on a single-stage object detection algorithm, and at the same time adopts a lightweight design of depthwise separable convolution and cross-stage partial network to perform real-time positioning and recognition of pigs.

[0026] Furthermore, the multimodal data fusion module includes: a feature fusion module and a decision fusion module;

[0027] The feature fusion module first splices and fuses the processed visual features and audio features in the feature dimension to form a more comprehensive feature representation, and then inputs the spliced and fused features into the attention mechanism fusion network, multiplies the visual and audio features by the corresponding attention weights respectively, and performs addition or splicing operations to obtain the fused features and improve the fusion effect;

[0028] The decision fusion module combines the classification results of vision and audio, and makes a final behavior decision through voting or weighted average methods to improve the accuracy of behavior monitoring.

[0029] The beneficial effects of a pig-raising monitoring and management system based on computer vision according to the present invention are as follows: through multimodal data fusion, visual and audio information can be fully utilized to make up for the deficiencies of single-modal data, improve the detection accuracy of various complex behaviors, and at the same time cooperate with a lightweight deep learning model to better handle occlusion and pose diversity problems on the premise of ensuring a certain accuracy. In addition, the computational complexity of the lightweight model is relatively low, and it can achieve real-time processing under limited computing resources, timely feedback the behavior information of pigs, and at the same time, through multimodal data acquisition and data preprocessing methods, environmental problems such as light changes and noise interference can be effectively addressed to ensure the stability and reliability of the system under different environmental conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The following further describes the present invention in detail with reference to the drawings and specific implementation methods.

[0031] Figure 1 It is a schematic diagram of the system principle. DETAILED DESCRIPTION OF THE INVENTION

[0032] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other.

[0033] According to one aspect of the present invention, as Figure 1 shown, a pig-raising monitoring and management system based on computer vision is provided, including: a data acquisition module that monitors the pigsty comprehensively to obtain visual data and simultaneously collects the audio data of the pigs' calls. This module includes: a visual data acquisition module that uses multiple high-definition cameras to monitor the pigsty from different angles to obtain image information of the pigs from different perspectives, so as to capture the behavioral characteristics of the pigs more comprehensively, reduce the influence of occlusion problems, avoid the loss of some body characteristics caused by occlusion, and thus capture the behavioral characteristics of the pigs more comprehensively. At the same time, a camera with a wide dynamic range is adopted to improve the adaptability to light changes. Given the complex and changeable lighting conditions in the pig farm, with large differences in light intensity at different times and in different areas, the wide dynamic range camera can clearly capture the image details of the bright and dark parts in the scene where strong light and weak light coexist. It uses special image sensors and processing algorithms to perform separate exposure and processing on different brightness regions, expands the dynamic range of the image, and improves the quality of the image under different lighting conditions;

[0034] an audio data acquisition module that collects the audio information of the pigs' calls through a microphone. Different behaviors are accompanied by different sound characteristics (for example, when hungry, it may emit urgent and high-frequency calls; when fighting or being frightened, the calls will be more intense and loud). By analyzing the characteristics such as the frequency, amplitude, and duration of the audio signal, the behavioral state of the pigs can be inferred, and the audio data can be used as a supplement to the visual data to improve the accuracy of behavior detection.

[0035] a data preprocessing module that preprocesses the obtained visual data and audio data to provide high-quality data for the subsequent lightweight deep learning module and multi-modal data fusion module. This module includes: a visual preprocessing module that performs denoising and enhancement processing on the collected images to improve the image quality, and at the same time performs light compensation using adaptive histogram equalization;

[0036] Among them, the visual preprocessing module includes: a denoising processing module that uses median filtering (a non-linear filtering method that replaces the gray value of each pixel point in the image with the median value in its neighborhood, and can better retain the edge information of the image while removing noise) or Gaussian filtering (a linear filtering method that performs convolution operations on the image based on the Gaussian function and smooths the image by weighted averaging of the pixels in the neighborhood) methods to remove the noise in the image and reduce the influence of noise;

[0037] Light compensation module, Adaptive Histogram Equalization (CLAHE) divides the image into multiple small blocks, performs histogram equalization processing on each small block separately, and then stitches the processed small blocks together by bilinear interpolation. This can enhance the overall contrast of the image while preserving the local details of the image, and improve the image quality problem caused by uneven illumination;

[0038] Audio preprocessing module, filters and extracts features from audio data, and extracts audio features related to pig behavior (such as pitch, volume, Mel Frequency Cepstral Coefficients);

[0039] Among them, the audio preprocessing module includes: a filtering processing module, which uses low-pass filtering (a low-pass filter allows signals below a certain cut-off frequency to pass through while attenuating signals above that frequency) or band-pass filtering (only allows signals within a specific frequency range to pass through and suppresses signals of other frequencies) to remove high-frequency noise or unwanted frequency components in the audio signal and improve the quality of the audio signal;

[0040] Feature extraction module, extracts Mel Frequency Cepstral Coefficients (MFCC), pitch, and volume features from the audio. Among them, the Mel Frequency Cepstral Coefficients are a kind of feature parameter widely used in audio processing. It simulates the perception characteristics of the human auditory system for sounds of different frequencies. By converting the audio signal into the Mel frequency domain and then performing discrete cosine transform, a set of coefficients that can reflect the spectral characteristics of the audio signal can be obtained. The pitch reflects the frequency level of the sound, and the volume represents the strength of the sound, which can effectively describe the characteristics of pig calls and provide a basis for subsequent behavior classification.

[0041] Lightweight deep learning module, uses deep learning algorithms to extract features and train models from the preprocessed data, and constructs an efficient and accurate pig behavior recognition model. This module includes: a behavior classification module, which uses a lightweight Convolutional Neural Network (CNN) to quickly extract features and classify the images of pigs, judge the behavior state of pigs, and at the same time rely on the trained model provided by the lightweight Convolutional Neural Network (CNN) to reduce the training time and improve the generalization ability of the model through transfer learning, thereby improving the monitoring of pig behavior;

[0042] The lightweight convolutional neural network (CNN) among them is MobileNetV3, which combines depthwise separable convolution (decomposes traditional convolution into two steps: depthwise convolution and pointwise convolution. In depthwise convolution, each channel of the input feature map is convolved separately to obtain an intermediate feature map with the same number of channels as the input; pointwise convolution then uses a 1x1 convolutional kernel to convolve the intermediate feature map to adjust the number of channels to the required number of output channels. Through this decomposition method, the computational cost of depthwise separable convolution is much smaller than that of traditional convolution, thus greatly reducing the number of model parameters and computational cost) technology to further optimize the performance of the model. In pig behavior classification, MobileNetV3 can quickly extract features and classify the images of pigs, and judge the behavior states of pigs, such as eating, resting, moving, etc., serving as the basis for the pig behavior classification model;

[0043] Transfer learning refers to transferring the knowledge of a model trained on one task (source task) to another related task (target task). In deep learning, pre-trained models are usually trained on large-scale datasets (such as ImageNet). These models learn rich general features, such as edges, textures, shapes, etc. When applied to the target task, the parameters of these pre-trained models can be used as initialization and fine-tuned on the target dataset, thereby accelerating the convergence speed of the model, reducing the amount of data required for training, and improving the generalization ability of the model;

[0044] Through the synergistic effect of the above lightweight convolutional neural network (such as MobileNetV3), transfer learning, and depthwise separable convolution technology, the training time and the cost of data collection are reduced, and the performance of the model in the actual pig farming monitoring scenario is improved through the pre-trained model in transfer learning.

[0045] The object detection module (YOLOv5s), based on a single-stage object detection algorithm, simultaneously adopts the lightweight design of depthwise separable convolution and cross-stage partial network (CSPNet) to perform real-time positioning and recognition of pigs;

[0046] In single-stage detection, YOLOv5s directly predicts the bounding boxes and categories of objects on the input image, without the need to generate candidate regions first like two-stage detection algorithms and then perform classification and regression. Therefore, the detection speed is faster;

[0047] In depthwise separable convolution (which can refer to the explanation in the above behavior classification module), the standard convolution is decomposed into two steps: depthwise convolution and pointwise convolution, greatly reducing the computational cost and the number of parameters of the convolution operation. While ensuring the detection accuracy of the model, the complexity of the model is reduced and the running efficiency is improved;

[0048] In the cross-stage, by splitting and merging the feature maps in the channel dimension, the redundancy of information is reduced, and the feature expression ability and training speed of the model are improved.

[0049] The multi-modal data fusion module fuses the processed visual features and audio features to form a more comprehensive feature representation. At the same time, it combines the classification results of vision and audio to improve the accuracy of behavior detection. This module includes: a feature fusion module that first splices and fuses the processed visual features and audio features in the feature dimension to form a more comprehensive feature representation, and then inputs the spliced and fused features into the attention mechanism fusion network. The visual and audio features are respectively multiplied by the corresponding attention weights, and an addition or splicing operation is performed to obtain the fused features, improving the fusion effect;

[0050] Among them, feature splicing fusion directly splices the preprocessed visual features and audio features in the feature dimension (for example, the visual feature is a vector of length m, and the audio feature is a vector of length n. After splicing, a fused feature vector of length m + n is obtained), forming a more comprehensive feature representation. This method is simple and direct and can retain the original information of both modalities;

[0051] The attention mechanism can help the model automatically focus on the important parts of different modality features, enabling the model to pay more attention to key information. In pig behavior detection, different behaviors may have different dependencies on visual and audio features. The attention mechanism can help the model adaptively adjust the attention to different modality features and improve the fusion effect;

[0052] The decision fusion module combines the classification results of vision and audio and makes the final behavior decision through voting or weighted average methods to improve the accuracy of behavior monitoring;

[0053] The voting therein is based on the majority principle, believing that if the classification results of the two modalities are consistent, then this result is more likely to be correct (for example, there are two behavior categories A and B, the visual classification result is A, and the audio classification result is also A, then the final decision result is A; if the visual classification result is A and the audio classification result is B, then the final result can be determined according to the preset rules (such as majority voting, random selection, etc.)). It is simple and easy to understand and can improve the reliability of decision-making to a certain extent;

[0054] The weighted average method, on the other hand, takes into account the importance of different modalities in classification. By assigning different weights to different modalities, it can more flexibly comprehensively utilize the information of the two modalities (for example, if visual data is more accurate in certain behavior classifications, a higher weight can be assigned to the visual classification result) to improve the fusion effect.

[0055] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples either. Changes, modifications, additions or substitutions made by those of ordinary skill in the art within the scope of the essence of the present invention also fall within the protection scope of the present invention.

Claims

1. A pig - raising monitoring and management system based on computer vision, characterized in that, Including: Data acquisition module, data preprocessing module, lightweight deep learning module, multi-modal data fusion module, end; The data acquisition module monitors the pigsty comprehensively to obtain visual data and simultaneously collects the audio data of the pigs' calls; The data preprocessing module preprocesses the obtained visual data and audio data to provide high-quality data for the subsequent lightweight deep learning module and multi-modal data fusion module; The lightweight deep learning module uses deep learning algorithms to extract features and train models from the preprocessed data, and constructs an efficient and accurate pig behavior recognition model; The multi-modal data fusion module fuses the processed visual features and audio features to form a more comprehensive feature representation, and combines the classification results of vision and audio at the same time to improve the accuracy of behavior detection; End, output the fused results for generating pig behavior reports, triggering abnormal behavior warning mechanisms, and providing decision-making support for pig farming monitoring and management.

2. The pig-raising monitoring and management system based on computer vision according to claim 1, characterized in that: The data acquisition module includes: a visual data acquisition module and an audio data acquisition module; The visual data acquisition module uses multiple high-definition cameras to monitor the pigsty from different angles, obtains image information of pigs from different perspectives, so as to capture pig behavior characteristics more comprehensively, reduce the influence of occlusion problems. At the same time, a wide dynamic range camera is adopted to improve the adaptability to light changes; The audio data acquisition module collects the audio information of the pigs' calls through microphones. Different behaviors are accompanied by different sound characteristics, and the audio data can be used as a supplement to visual data to improve the accuracy of behavior detection.

3. The pig farming monitoring and management system based on computer vision according to claim 1, wherein: The data preprocessing module includes: a visual preprocessing module and an audio preprocessing module; The visual preprocessing module denoises and enhances the collected images to improve the image quality, and at the same time uses adaptive histogram equalization for light compensation; The audio preprocessing module filters and extracts features from the audio data to extract audio features related to pig behavior.

4. The pig-raising monitoring and management system based on computer vision according to claim 3, characterized in that: The visual preprocessing module includes: a denoising processing module and a light compensation module; The denoising processing module uses median filtering or Gaussian filtering methods to remove noise in the images; The light compensation module divides the image into multiple small blocks by adaptive histogram equalization, performs histogram equalization processing on each small block respectively, and then stitches the processed small blocks together by bilinear interpolation to improve the image quality problem caused by uneven illumination.

5. The pig-raising monitoring and management system based on computer vision according to claim 3, wherein: The audio preprocessing module includes: a filtering processing module and a feature extraction module; The filtering processing module uses low-pass filtering or band-pass filtering to remove high-frequency noise or unwanted frequency components in the audio signal to improve the quality of the audio signal; The feature extraction module extracts Mel-frequency cepstral coefficients, pitch, and volume features in the audio, which can effectively describe the characteristics of pig calls and provide a basis for subsequent behavior classification.

6. The pig-raising monitoring and management system based on computer vision according to claim 1, wherein: The lightweight deep learning module includes: a behavior classification module and an object detection module; The behavior classification module uses a lightweight convolutional neural network to quickly extract features and classify images of pigs, determine the behavioral states of pigs, and at the same time reduce the training time and improve the generalization ability of the model through transfer learning based on the trained model provided by the lightweight convolutional neural network, thereby improving the monitoring of pig behavior; The object detection module is based on a single-stage object detection algorithm and adopts a lightweight design of depthwise separable convolution and cross-stage partial network to perform real-time positioning and recognition of pigs.

7. The pig farming monitoring and management system based on computer vision according to claim 1, characterized in that: The multi-modal data fusion module includes: a feature fusion module and a decision fusion module; The feature fusion module first splices and fuses the processed visual features and audio features in the feature dimension to form a more comprehensive feature representation, and then inputs the spliced and fused features into the attention mechanism fusion network, multiplies the visual and audio features by the corresponding attention weights respectively, and performs addition or splicing operations to obtain the fused features and improve the fusion effect; The decision fusion module combines the classification results of vision and audio and makes the final behavior decision through voting or weighted average methods to improve the accuracy of behavior monitoring.

Images