A pig health and behavior monitoring and automatic response method, system, electronic device, storage medium and product program based on acoustic intelligence

Abstract

This invention discloses a method, system, electronic device, storage medium, and product program for monitoring and automatically responding to pig health and behavior based on acoustic intelligence, belonging to the fields of smart farming and acoustic intelligence technology. It aims to solve the problem of insufficient recognition accuracy in complex farming environments. Real-time edge preprocessing is performed through distributed intelligent audio acquisition nodes. A multi-scale time-frequency fusion algorithm is used to extract multi-dimensional sound biomarkers containing wavelet packet energy and enhanced Mel-frequency cepstral spectra. Based on a Transformer network, the acoustic semantics of pigs are analyzed, and a multi-source fusion sound source-individual binding algorithm is used to achieve accurate association of vocalizing individuals. A semantically driven control strategy is adopted, and control commands are generated through priority arbitration and feedback optimization, forming a perception-cognition-interaction closed loop. This invention enables non-contact, accurate monitoring and automated management of pig health and behavior, improving recognition accuracy and farming management efficiency.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of smart farming and agricultural information technology, and discloses a method, system, electronic device, storage medium, and product program for monitoring and automatically responding to pig health and behavior based on acoustic intelligence. Background Technology

[0002] In large-scale, intensive farming, early detection and rapid intervention are crucial for disease control. Traditional methods mainly rely on manual inspections and experience-based judgment, but manual inspections are infrequent, have limited coverage, and are difficult to identify in the early stages of disease.

[0003] Modern animal husbandry not only focuses on productivity but also increasingly emphasizes animal welfare. Pigs' vocalizations are important signals expressing needs and emotions (such as pain, hunger, stress, and socialization). Currently, the interpretation of these vocal signals relies entirely on the personal experience of farmers, lacking objective and quantifiable standards. This leads to untimely and inaccurate responses to the welfare status and individual needs of pigs.

[0004] Limitations of traditional sound monitoring technology: Simple sound pressure / frequency analysis: Some existing technologies only monitor the level of ambient noise or simple call frequencies, and cannot distinguish the specific type of call and the complex biological meaning behind it. They have a high false alarm rate and low practicality.

[0005] Relying on manual listening: Although experienced farmers can identify some abnormal sounds, this method is not scalable, not continuous, highly subjective, and costly, and cannot be applied to modern large-scale pig farms.

[0006] The shortcomings of existing automated health monitoring technologies: Video surveillance primarily focuses on behavior and posture, but is not sensitive to sound signals such as coughing and breathing, and is greatly affected by light and obstruction, resulting in high computational complexity.

[0007] Wearable sensors (such as thermometers and accelerometers) require contact with animals, which may cause stress, and there are problems such as device damage, loss, battery life and data transmission, making them unsuitable for long-term, large-scale group monitoring.

[0008] Existing sound analysis technologies are mostly limited to the basic stage of anomaly detection (such as judging "there is an abnormal sound"), and cannot achieve semantic analysis (e.g., whether it is a cough, wheezing or fighting), thus failing to provide specific and actionable guidance for management decisions.

[0009] The technological gap in sound biomarker extraction and analysis: Marker extraction is difficult: Pig calls have a low signal-to-noise ratio in complex pigsty environments (fan noise, calls from other pigs), and there are differences between individuals, breeds, and ages. The core challenge is how to stably and accurately extract "sound biomarkers" that are strongly associated with specific health states or behaviors. Summary of the Invention

[0010] This invention provides a method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence, in order to solve the problem of insufficient recognition accuracy in complex farming environments.

[0011] An acoustic intelligence-based method for monitoring and automatically responding to pig health and behavior includes the following steps: S1. Multiple intelligent audio acquisition nodes distributed in the breeding environment continuously collect environmental sounds. The edge computing unit built into the node performs real-time preprocessing on the raw audio stream, filters out environmental noise and extracts the effective vocal segments of the pigs, and packages and uploads the processed audio data with the corresponding time and location tags. S2. The preprocessed audio data is transmitted to an edge server or cloud analysis platform. The sound segments are analyzed by a multi-scale time-frequency fusion feature extraction algorithm to calculate a set of multi-dimensional digital sound biomarkers. The biomarkers include time-frequency features, acoustic features, and high-order features automatically learned by a deep learning network. S3. Input the extracted sound biomarker vector into the pig acoustic semantic parsing model based on deep learning. The model infers and outputs one or more semantic labels corresponding to the pig's health status or behavioral needs. S4. Using a sound source-individual binding algorithm, the parsed semantic tags are accurately associated with the individual identity of the pig making the sound, generating a complete event record containing individual identifiers, semantic tags, timestamps, and location information; S5. Integrate event records with historical growth data of pigs and environmental data, and adopt a semantic-driven equipment control strategy generation algorithm. Through semantic-action mapping, priority arbitration, adaptive delay control and feedback closed-loop optimization, automatically generate corresponding control commands, trigger visual warnings, generate management reports or drive automated execution equipment to respond, forming a perception-cognition-interaction closed loop.

[0012] Further, S2 includes S2.1, multi-resolution wavelet packet decomposition: decomposing the input audio signal into... Layered wavelet packet trees are used to obtain the wavelet packet coefficients for each frequency band. Calculate the first Layer Energy characteristics of each frequency band : ; In the formula, For the first Layer The total number of wavelet packet coefficients in each frequency band; For the first Layer The first frequency band Wavelet packet coefficients; Calculate the normalized energy coefficient for each frequency band. : ; In the formula, For the first Layer Normalized energy coefficients for each frequency band, For the first Layer The original energy characteristics of each frequency band For the first Layer The original energy characteristics of each frequency band; S2.2 Calculate the standard Mel-frequency cepstral coefficients And extract the first difference of the standard Mel-Cepstral coefficients. and second-order difference : ; ; In the formula, For the first Vermeer cepstral coefficients at the th First-order difference of frames, This represents the frame offset within the differential window. For the first Vermeer cepstral coefficients at the th The static value of the frame. The size of the difference window. These are the difference normalization coefficients. For the first Vermeer cepstral coefficients at the th The second difference of the frame, For the first Vermeer cepstral coefficients at the th First-order difference of a frame; For each frame of audio signal, the fundamental frequency is estimated, harmonic components and noise components are separated, and the harmonic-to-noise ratio is calculated. : ; In the formula, For harmonic component energy, The energy of the noise component; S2.3 Input the audio time-frequency map into a lightweight residual convolutional network, and output a 128-dimensional deep acoustic embedding vector. ; S2.4. Concatenate all features to form the final voice biomarker vector: ; In the formula, The wavelet packet energy feature vector, For standard Mel-Cepstral eigenvectors, It is a first-order difference Mel-Cepstral eigenvector. It is the second-order difference Mel-Cepstral eigenvector. This is a characteristic of the harmonic-to-noise ratio. This is a deep acoustic embedding vector.

[0013] Furthermore, the pig acoustic semantic parsing model in S3 is a Transformer-based multi-label classification network, and its working process includes: S3.1, Incorporating traditional acoustic characteristics With deep acoustic embedding features Perform weighted fusion: ; In the formula, For fusion weights; The fused features are input into a Transformer encoder consisting of L encoder layers to obtain contextual features. ; S3.3. Decoding is performed using a three-level tag structure, and the total loss function is calculated. : ; In the formula, , , The cross-entropy loss is calculated for first-level, second-level, and third-level labels, respectively. The loss is due to hierarchical consistency constraints. To constrain weights; The loss of hierarchical consistency constraints for: ; In the formula, This represents the total number of first-level tags. Predict the probability for the first-level label. As a first-level tag All corresponding second-level subclasses; Will go to the same area The historical analysis results of each hour are input into an LSTM network to obtain time-series features. ; context features With time series characteristics After concatenation, the input is given to a fully connected layer, which outputs the probability of each semantic label: ; In the formula, For the first A semantic tag, It is the Sigmoid activation function. and These are the parameters for the fully connected layer.

[0014] Furthermore, the sound source-individual binding algorithm described in S4 specifically includes: S4.1 Utilizing the time difference of arrival of sound waves received by a microphone array Calculate the three-dimensional coordinates of the sound source: ; In the formula, For the sound wave to reach the first The and the first The time difference between the microphones For the first The coordinates of each microphone, The speed of sound in air; S4.2 Project the sound source location onto the video screen and filter out the sound sources closest to the sound source. The individual pigs are considered as a candidate set. ; S4.3 Calculate the voiceprint feature vector of the current call. With candidate individual voiceprint template Cosine similarity: ; In the formula, This is the function for calculating cosine similarity. Comprehensive sound source localization reliability Video tracking confidence Similarity to cosine voiceprint Calculate the final binding confidence level : ; In the formula, , , The weighting coefficients for each confidence level are given and satisfy the following conditions: The individual with the highest confidence level and greater than the threshold was selected as the individual who spoke.

[0015] Furthermore, the semantically driven device control policy generation algorithm in S5 specifically includes: S5.1 Establish a mapping table between semantic tags and device actions. Each tag corresponds to a set of device control instructions. When multiple semantic tags exist in the same area, calculate the effective priority of each tag: ; In the formula, For the preset category weights, The confidence level of the label; S5.2 Record the last execution time of each action. and system real-time timestamp Set minimum execution interval ; When satisfied Only after a certain time is it allowed to execute again. After the action is executed, the changes in the acoustic state of the area in the subsequent time are monitored, and the acoustic state improvement rate is calculated. : ; In the formula, and These represent the frequency of abnormal noises before and after the action is executed. When R < 30%, the response level is automatically upgraded.

[0016] Furthermore, the semantic tags include: health tags, behavior tags, and emotion tags.

[0017] To address the technical problem that existing acoustic monitoring systems for pig farming generally employ a centralized cloud computing architecture, which fails to form a complete automated control loop, this invention also provides, in a second aspect, an acoustic intelligence-based pig health and behavior monitoring and automatic response system, comprising: Perception layer: Multiple distributed intelligent audio acquisition nodes, each node containing a microphone array, a pre-processor, an embedded AI chip, and a wireless communication module; Edge computing layer: a field-level edge server that is wirelessly connected to the perception layer to receive pre-processed audio data and run sound biomarker extraction algorithms and lightweight semantic parsing models; Cloud platform layer: communicates bidirectionally with the edge computing layer, used for storing historical data, training and optimizing deep learning models, and running the global decision engine; Application interaction layer: This includes user terminals and automated execution devices. User terminals are used to receive early warning information and view management reports, while automated execution devices are used to receive control commands and execute corresponding actions.

[0018] To address the technical problems of existing acoustic monitoring solutions for pig farming, which rely on centralized cloud computing leading to high response latency, and the high cost of large-scale deployment and poor scenario adaptability in completing the entire closed-loop process of audio acquisition, analysis, and equipment control locally in the pig farm, the present invention also provides an electronic device, comprising a processor and a memory communicatively connected to the processor; the memory stores instructions executed by the processor, which, when executed by the processor, causes the processor to perform an acoustic intelligence-based method for monitoring and automatically responding to pig health and behavior.

[0019] To address the technical problems of existing acoustic monitoring algorithms for pig health lacking standardized and portable storage media and incurring high promotion costs, in a fourth aspect, this invention also provides a storage medium storing a computer program, which is executed by a processor to provide a method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence.

[0020] To address the technical challenges of existing pig monitoring programs being fragmented, unable to be directly integrated into various smart farming management systems, and requiring significant implementation and adaptation work with high application barriers, the fifth aspect of this invention also provides a program product. This program product enables a processor to execute a method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence.

[0021] Compared with the prior art, the present invention has the following beneficial effects: This method can continuously capture and identify these subclinical sound signals, achieving early warnings at the hourly to dayly level. This enables administrators to accurately isolate and treat individual pigs before the spread of an epidemic, greatly reducing the morbidity and mortality rates of the herd and significantly improving the overall health and survival rate of the pig population. AI-based semantic analysis objectively and quantitatively correlates sound features with specific health states, providing consistent judgment standards, reducing human error, and making health monitoring data digitized and standardized, providing a direct basis for precision medication. By analyzing sound semantics, the system can objectively and continuously assess the animal welfare level of individuals and groups, providing direct data support for improving the feeding environment and optimizing management processes, making it a powerful tool for achieving standardized animal welfare certification.

[0022] This invention solves the problem of "hearing completely" through a distributed acoustic sensing network, addresses the industry challenge of "understanding" through AI-driven biomarkers and semantic analysis, and ultimately solves the problem of "hearing effectively" by integrating with the production system in a closed loop. Compared to existing technologies, it achieves a qualitative leap in prenatal early warning quality, animal welfare, management decision-making accuracy, and production efficiency. It also offers significant advantages in saving on medicines and feed raw materials, simplifying management procedures, and eradicating antibiotic-induced environmental pollution. It is a key innovation driving the transformation of animal husbandry towards intelligent, refined, and sustainable development. Attached Figure Description

[0023] Figure 1 This is a flowchart of the technology of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention are described clearly and completely below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] Example 1 An acoustic intelligence-based method for monitoring and automatically responding to pig health and behavior, such as... Figure 1 As shown, it includes the following steps: S1. Multiple intelligent audio acquisition nodes distributed in the breeding environment continuously collect environmental sounds. The edge computing unit built into the node performs real-time preprocessing on the raw audio stream, filters out environmental noise and extracts the effective vocal segments of the pigs, and packages and uploads the processed audio data with the corresponding time and location tags. S2. The preprocessed audio data is transmitted to an edge server or cloud analysis platform. The sound segments are analyzed by a multi-scale time-frequency fusion feature extraction algorithm to calculate a set of multi-dimensional digital sound biomarkers. The biomarkers include time-frequency features, acoustic features, and high-order features automatically learned by a deep learning network. Multi-resolution wavelet packet decomposition: decomposes the input audio signal into... Layered wavelet packet trees are used to obtain the wavelet packet coefficients for each frequency band. Calculate the first Layer Energy characteristics of each frequency band : ; In the formula, For the first Layer The total number of wavelet packet coefficients in each frequency band For the first Layer The first frequency band Wavelet packet coefficients; Calculate the normalized energy coefficient for each frequency band. : ; In the formula, For the first Layer Normalized energy coefficients for each frequency band, For the first Layer The original energy characteristics of each frequency band For the first Layer The original energy characteristics of each frequency band; S2.2 Calculate the standard Mel-frequency cepstral coefficients And extract the first-order difference. and second-order difference : ; ; In the formula, For the first Vermeer cepstral coefficients at the th First-order difference of frames, This represents the frame offset within the differential window. For the first Vermeer cepstral coefficients at the th The static value of the frame. The size of the difference window. These are the difference normalization coefficients, used to eliminate amplitude deviations caused by window length. For the first Vermeer cepstral coefficients at the th The second difference of the frame, For the first Vermeer cepstral coefficients at the th First-order difference of a frame; For each frame of audio signal, the fundamental frequency is estimated, harmonic components and noise components are separated, and the harmonic-to-noise ratio is calculated. : ; In the formula, For harmonic component energy, The energy of the noise component; S2.3. Input the audio time-frequency map into the convolutional network. The convolutional network has 3 sets of convolutional blocks: the kernel size is 3×3, the number of convolutional channels is 32, 64 and 128 respectively, and the maximum pooling stride is 2; the number of neurons in the two fully connected layers is 256 and 128 respectively, and the final output is a 128-dimensional deep acoustic embedding vector. ; S2.4 Feature Fusion: Concatenate all features to form the final voice biomarker vector: ; In the formula, The wavelet packet energy feature vector, For standard Mel-Cepstral eigenvectors, It is a first-order difference Mel-Cepstral eigenvector. It is the second-order difference Mel-Cepstral eigenvector. This is a characteristic of the harmonic-to-noise ratio. This is a deep acoustic embedding vector.

[0026] S3. Input the extracted sound biomarker vector into the pig acoustic semantic parsing model based on deep learning. The model infers and outputs one or more semantic labels corresponding to the pig's health status or behavioral needs. The pig acoustic semantic parsing model in S3 is a Transformer-based multi-label classification network. Its workflow includes: S3.1, Incorporating traditional acoustic characteristics With deep acoustic embedding features Perform weighted fusion: ; In the formula, The fusion weights are obtained based on the intensity of on-site noise. The fused features are input into a Transformer encoder consisting of L encoder layers to obtain contextual features. ; S3.3. Decoding is performed using a three-level tag structure, and the total loss function is calculated. : ; In the formula, , , The cross-entropy loss is calculated for first-level, second-level, and third-level labels, respectively. The loss is due to hierarchical consistency constraints. To constrain weights and This is used to adjust the proportion of classification loss and label hierarchy consistency damage; The , , The model outputs the predicted probabilities of level 1, level 2, and level 3 labels, respectively. These probabilities are then compared with the corresponding manually labeled real labels at each level using multi-label cross-entropy calculations. The results are obtained by solving the cross-entropy loss for the level 1 label layer by layer. Cross-entropy loss of second-level labels Cross-entropy loss of third-level labels ; The loss of hierarchical consistency constraints for: ; In the formula, This represents the total number of first-level tags. Predict the probability for the first-level label. As a first-level tag All corresponding second-level subclasses; Will go to the same area The historical analysis results of each hour are input into an LSTM network to obtain time-series features. ; context features With time series characteristics After concatenation, the input is given to a fully connected layer, which outputs the probability of each semantic label: ; In the formula, For the first A semantic tag, It is the Sigmoid activation function. and These are the parameters for the fully connected layer; The semantic tags include at least: health tags, behavior tags, and emotion tags.

[0027] S4. Using a sound source-individual binding algorithm, the parsed semantic tags are accurately associated with the individual identity of the pig making the sound, generating a complete event record containing individual identifiers, semantic tags, timestamps, and location information; S4.1 Utilizing the time difference of arrival of sound waves received by a microphone array Calculate the three-dimensional coordinates of the sound source: ; In the formula, For the sound wave to reach the first The and the first The time difference between the microphones For the first The coordinates of each microphone, The speed of sound in air; S4.2 Project the sound source location onto the video screen and filter out the sound sources closest to the sound source. The individual pigs are considered as a candidate set. ; S4.3 Calculate the voiceprint feature vector of the current call. With candidate individual voiceprint template Cosine similarity: ; In the formula, This is the function for calculating cosine similarity. Comprehensive sound source localization reliability Video tracking confidence Similarity to cosine voiceprint Calculate the final binding confidence level : ; In the formula, , , The weighting coefficients for each confidence level are given and satisfy the following conditions: And it was obtained based on the results of multi-source matching experiments; Individuals with the highest confidence level and a confidence level greater than a threshold were selected as the speaking individuals. This threshold was obtained from an individual mismatch test.

[0028] S5. Integrate event records with historical growth data of pigs and environmental data, and adopt a semantic-driven equipment control strategy generation algorithm. Through semantic-action mapping, priority arbitration, adaptive delay control and feedback closed-loop optimization, automatically generate corresponding control commands, trigger visual warnings, generate management reports or drive automated execution equipment to respond, forming a perception-cognition-interaction closed loop.

[0029] The semantic-driven device control policy generation algorithm specifically includes: S5.1 Establish a mapping table between semantic tags and device actions. Each tag corresponds to a set of device control instructions. When multiple semantic tags exist in the same area, calculate the effective priority of each tag: ; In the formula, For the preset category weights, The confidence level of the label; S5.2 Record the last execution time of each action. and system real-time timestamp Set minimum execution interval , To prevent frequent equipment operation and provide a cooling interval, the last execution time... System real-time timestamp and minimum execution interval It is set based on the mechanical wear characteristics of the equipment; When satisfied Only after a certain time is it allowed to execute again. After the action is executed, the changes in the acoustic state of the area in the subsequent time are monitored, and the acoustic state improvement rate is calculated. : ; In the formula, and These represent the frequency of abnormal noises before and after the action is executed. When R < 30%, the response level is automatically upgraded.

[0030] Example 2 Embodiments of the present invention also provide a pig health and behavior monitoring and automatic response system based on acoustic intelligence, comprising: Perception layer: Multiple distributed intelligent audio acquisition nodes, each node containing a microphone array, a pre-processor, an embedded AI chip, and a wireless communication module; Edge computing layer: a field-level edge server that is wirelessly connected to the perception layer to receive pre-processed audio data and run sound biomarker extraction algorithms and lightweight semantic parsing models; Cloud platform layer: communicates bidirectionally with the edge computing layer, used for storing historical data, training and optimizing deep learning models, and running the global decision engine; Application interaction layer: This includes user terminals and automated execution devices. User terminals are used to receive early warning information and view management reports, while automated execution devices are used to receive control commands and execute corresponding actions.

[0031] Example 3 The device in the embodiments of this invention can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a GPU box, mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. The embodiments of this application do not specifically limit the scope of the invention.

[0032] Example 4 Embodiments of the present invention also provide a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of a method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence as disclosed in the embodiments of this application.

[0033] Example 5 Embodiments of the present invention also provide a computer program product that, when run on an electronic device, enables a processor to execute the steps of a method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence as disclosed in the embodiments of this application.

[0034] The following description, in conjunction with the accompanying drawings and embodiments, provides further details.

[0035] Example 6 This embodiment was carried out in the pigsty of a large-scale pig farm. Four intelligent audio acquisition nodes were evenly deployed in the pigsty at a height of 2.5m to continuously collect environmental sounds. The node's built-in edge computing unit performs real-time preprocessing on the raw audio stream, including: The steady-state environmental noise from fans, feed line motors, etc., is filtered out by spectral subtraction, and the noise estimation adopts the silent segment sliding update method. In this embodiment, the audio sampling rate is 16kHz, the single frame length is 256 sampling points, and the frame shift is 128 sampling points; the noise estimation sliding window length for the silent segment is 100 frames, and the sliding update step size is 5 frames. A short-time energy and zero-crossing rate joint decision algorithm was used to extract valid vocal segments from pigs. The energy threshold was set to 0.05 and the zero-crossing rate threshold was set to 50 times / frame. The processed valid vocal segments were packaged with the corresponding timestamps and location tags and uploaded to the edge server. After receiving the audio data, the edge server ran a multi-scale time-frequency fusion feature extraction algorithm to calculate the sound biomarker vector. The sampling, frame processing, and noise window parameters are adapted to the main frequency of pig vocalization, which is a general configuration for acoustic acquisition. The energy threshold and zero-crossing rate threshold are calibrated by sound samples from the pig farm and are used to distinguish between pig noise and equipment noise. The preprocessed single-frame time-domain audio signal is used as the input for wavelet packet transform, and the wavelet packet's first... The total number of layer frequency bands is , To accommodate the inherent mathematical characteristics of wavelet packet hierarchical decomposition, the Daubechies 8 wavelet basis is used for 8-level wavelet packet decomposition. This approach balances low-frequency pig vocalization details with high-frequency noise separation, adapts to the spectral distribution of pig vocalizations, and yields three-dimensional wavelet packet coefficients across 256 frequency bands. Calculate the first Layer Energy characteristics of each frequency band : ; In the formula, For the first Layer The total number of wavelet packet coefficients in each frequency band For the first Layer The first frequency band Wavelet packet coefficients; Further calculations were performed on the normalized energy coefficients for each frequency band. Eliminate the effects of differences caused by absolute volume: ; In the formula, For the first Layer Normalized energy coefficients for each frequency band, For the first Layer The original energy characteristics of each frequency band For the first Layer The original energy characteristics of each frequency band; Standard Mel cepstral coefficients were calculated using a 64-Melb filter bank. The Mel filter bank's frequency range is set to 50Hz–8kHz, which fully covers the entire spectrum of pig growth, retaining 12 effective static features, and further extracting the first-order and second-order difference dynamic features of the Mel cepstral coefficients: ; ; In the formula, For the first Vermeer cepstral coefficients at the th First-order difference of frames, This represents the frame offset within the differential window. For the first Vermeer cepstral coefficients at the th The static value of the frame. The size of the difference window. For the first Vermeer cepstral coefficients at the th The second difference of the frame, For the first Vermeer cepstral coefficients at the th First-order difference of frames, These are the difference normalization coefficients, used to eliminate amplitude deviations caused by window length; Simultaneously, the autocorrelation method is used for fundamental frequency estimation, harmonic and noise components are separated, and the harmonic-to-noise ratio is calculated for each frame. ; In the formula, For harmonic component energy, The energy of the noise component; The audio clips were converted into Mel-time spectrograms and input into a convolutional network. The size, channels, and neuron parameters of the convolutional network were optimized using multiple sets of live pig acoustic samples. The convolutional network contains three convolutional blocks: a 3×3 kernel size, 32, 64, and 128 convolutional channels respectively, and a maximum pooling stride of 2. The two fully connected layers have 256 and 128 neurons respectively, and the final output is a 128-dimensional deep acoustic embedding vector. ; All features are concatenated according to their dimensions to form a sound biomarker vector: ; In the formula, The energy characteristics of the 256-dimensional wavelet packet are... It is a 12-dimensional static Mel cepstral feature. It is a 12-dimensional first-order difference feature. It is a 12-dimensional second-order difference feature. It is a 1D harmonic-to-noise ratio feature. It is a 128-dimensional deep acoustic embedding feature; Input the sound biomarker vector into a Transformer-based multi-label classification network, and infer semantic labels; 293-dimensional traditional acoustic features With 128-dimensional deep acoustic embedding features Perform weighted fusion: ; In the formula, To integrate weights and This is obtained based on the intensity of the on-site noise. The fused features are input into a Transformer encoder consisting of four encoder layers. Each layer contains an 8-head self-attention mechanism and a feedforward neural network, outputting 512-dimensional contextual features. .

[0036] Using a three-level labeling structure, the total loss function is calculated: ; In the formula, , , The cross-entropy loss is calculated for first-level, second-level, and third-level labels, respectively. The loss is due to hierarchical consistency constraints. To constrain weights and This is used to adjust the proportion of classification loss and label hierarchy consistency damage; The , , The model outputs the predicted probabilities of level 1, level 2, and level 3 labels, respectively. These probabilities are then compared with the corresponding manually labeled real labels at each level using multi-label cross-entropy calculations. The results are obtained by solving the cross-entropy loss for the level 1 label layer by layer. Cross-entropy loss of second-level labels Cross-entropy loss of third-level labels ; The loss of hierarchical consistency constraints for: ; In the formula, This represents the total number of first-level tags. Predict the probability for the first-level label. As a first-level tag All corresponding second-level subclasses; Let the time-series history window be The window duration is derived from the statistical analysis of the duration of disease-causing behavior in pigs, used to capture long-term abnormal acoustic patterns; past data from the same region... The historical data analysis results for each hour are input into an LSTM network with a hidden layer dimension of 128, and the output is a 128-dimensional temporal feature. ; The contextual features and temporal features are concatenated and then fed into a fully connected layer. After passing through the Sigmoid activation function, the probability of each semantic label is output: ; In the formula, For the first A semantic tag, It is the Sigmoid activation function. and These are the parameters for the fully connected layer.

[0037] A multi-source fusion algorithm combining 3D localization, video tracking, and voiceprint matching is used to bind semantic tags to the individual speaker; Using the time difference of arrival of sound waves received by a microphone array The Chan algorithm is used to solve for the three-dimensional coordinates of the sound source. : ; In the formula, For the sound wave to reach the first The and the first The time difference between the microphones For the first The coordinates of each microphone, The speed of sound in air, at normal temperature and pressure. The value is 340m / s, and the average positioning error in this embodiment is less than 0.5m; The sound source coordinates are projected onto the synchronously acquired video feed, and the number of candidate pigs within the video field of view is set according to the pig house stocking density and the camera's shooting range. Filter the closest to the sound source The first pig as a candidate individual set ; Extract the 128-dimensional i-vector voiceprint features of the current call. With candidate individual voiceprint template Calculate cosine similarity: ; In the formula, This is the function for calculating cosine similarity. Comprehensive sound source localization reliability Video tracking confidence Similarity to cosine voiceprint Calculate the final binding confidence level : ; In the formula, , , The weighting coefficients for each confidence level are given and satisfy the following conditions: Furthermore, based on the results of multi-source matching experiments, individuals with the highest confidence level and a threshold greater than 0.7 were selected as the vocal individuals. This threshold was obtained from individual mismatch experiments.

[0038] Establish a mapping table between semantic tags and device actions. When multiple semantic tags are triggered simultaneously in the same area, calculate the effective priority of each tag. ; In the formula, Assigning values ​​to pre-defined categories based on the severity of disease, stress, and routine behavior. The confidence level of the label; Record the last execution time of each action. and system real-time timestamp Set minimum execution interval , Cooling interval for equipment to prevent frequent operation; the last execution time System real-time timestamp and minimum execution interval It is set based on the mechanical wear characteristics of the equipment; When satisfied Only after a certain time is it allowed to execute again. After the action is executed, the changes in the acoustic state of the area in the subsequent time are monitored, and the acoustic state improvement rate is calculated. : ; In the formula, and These represent the frequency of abnormal noises before and after the action is executed. When R < 30%, the response level is automatically upgraded.

[0039] The multi-scale time-frequency fusion feature extraction method proposed in this invention can accurately characterize the multi-dimensional acoustic features of pig calls, effectively improving the recognition accuracy of abnormal calls in complex breeding environments. The sound source-individual binding algorithm realizes the accurate association of individuals making sounds in group breeding scenarios, solving the industry problem that traditional acoustic monitoring cannot locate a single individual. The semantic-driven automatic response mechanism can automatically trigger corresponding control actions based on the parsing results, forming a stable and reliable perception-cognition-interaction closed loop, fully meeting the intelligent monitoring and management needs of modern pig farming. Moreover, the SAP algorithm has been deployed in multiple pig farms, with an equipment response accuracy rate of 98.2%, effectively avoiding misoperation and resource waste.

[0040] This invention overcomes the technical limitations of insufficient accuracy in single feature recognition, inability to trace individuals in group monitoring, and disconnect between monitoring and control. Through a layered collaborative architecture between the edge and the cloud, it balances local real-time response with global model iterative optimization. The non-contact acoustic monitoring method does not cause stress to pigs and can achieve uninterrupted continuous monitoring. It can effectively reduce labor costs in breeding, provide early warning of health risks, and improve the level of refined breeding, thus having significant economic benefits and animal welfare value.

[0041] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence, characterized in that, Includes the following steps: S1. Multiple intelligent audio acquisition nodes distributed in the breeding environment continuously collect environmental sounds. The edge computing unit built into the node performs real-time preprocessing on the raw audio stream, filters out environmental noise and extracts the effective vocal segments of the pigs, and packages and uploads the processed audio data with the corresponding time and location tags. S2. The preprocessed audio data is transmitted to an edge server or cloud analysis platform. The sound segments are analyzed by a multi-scale time-frequency fusion feature extraction algorithm to calculate a set of multi-dimensional digital sound biomarkers. The biomarkers include wavelet packet energy feature vectors, standard Mel-Cepstral feature vectors, first-order difference Mel-Cepstral feature vectors, second-order difference Mel-Cepstral feature vectors, harmonic-to-noise ratio features, and deep acoustic embedding vectors. S3. Input the extracted sound biomarker vector into the pig acoustic semantic parsing model based on deep learning. The model infers and outputs one or more semantic labels corresponding to the pig's health status or behavioral needs. The acoustic semantic parsing model for pigs is a Transformer-based multi-label classification network that uses a three-level label structure for decoding. context features With time series characteristics The concatenated data is input into a fully connected layer, and the output is the probability of each semantic label. ; S4. Using a sound source-individual binding algorithm, the parsed semantic tags are accurately associated with the individual identity of the pig making the sound, generating a complete event record containing individual identifiers, semantic tags, timestamps, and location information; The sound source-individual binding algorithm is based on the comprehensive sound source localization reliability. Video tracking confidence Similarity to cosine voiceprint Calculate the final binding confidence level : ； In the formula, , , The weighting coefficients for each confidence level are given and satisfy the following conditions: ; Select the individual with the highest confidence level that is greater than the threshold as the individual who speaks; S5. Integrate event records with historical growth data of pigs and environmental data, and adopt a semantic-driven equipment control strategy generation algorithm. Through semantic-action mapping, priority arbitration, adaptive delay control and feedback closed-loop optimization, automatically generate corresponding control commands, trigger visual warnings, generate management reports or drive automated execution equipment to respond, forming a perception-cognition-interaction closed loop.

2. The method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence according to claim 1, characterized in that, S2 includes S2.1, multi-resolution wavelet packet decomposition: decomposing the input audio signal into... Layered wavelet packet trees are used to obtain the wavelet packet coefficients for each frequency band. Calculate the first Layer Energy characteristics of each frequency band : ； In the formula, For the first Layer The total number of wavelet packet coefficients in each frequency band For the first Layer The first frequency band Wavelet packet coefficients; Calculate the normalized energy coefficient for each frequency band. : ； In the formula, For the first Layer Normalized energy coefficients for each frequency band, For the first Layer The original energy characteristics of each frequency band For the first Layer The original energy characteristics of each frequency band; S2.2 Calculate the standard Mel-frequency cepstral coefficients And extract the first difference of the standard Mel-Cepstral coefficients. and second-order difference : ； ； In the formula, For the first Vermeer cepstral coefficients at the th First-order difference of frames, This represents the frame offset within the differential window. For the first Vermeer cepstral coefficients at the th The static value of the frame. The size of the difference window. These are the difference normalization coefficients. For the first Vermeer cepstral coefficients at the th The second difference of the frame, For the first Vermeer cepstral coefficients at the th First-order difference of a frame; For each frame of audio signal, the fundamental frequency is estimated, harmonic components and noise components are separated, and the harmonic-to-noise ratio is calculated. : ； In the formula, For harmonic component energy, The energy of the noise component; S2.3 Input the audio time-frequency map into a lightweight residual convolutional network, and output a 128-dimensional deep acoustic embedding vector. ; S2.4 Feature Fusion: Concatenate all features to form the final voice biomarker vector: ； In the formula, The wavelet packet energy feature vector, For standard Mel-Cepstral eigenvectors, It is a first-order difference Mel-Cepstral eigenvector. It is the second-order difference Mel-Cepstral eigenvector. This is a characteristic of the harmonic-to-noise ratio. This is a deep acoustic embedding vector.

3. The method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence according to claim 1, characterized in that, The pig acoustic semantic parsing model in S3 is a Transformer-based multi-label classification network, and its working process includes: S3.1, Incorporating traditional acoustic characteristics With deep acoustic embedding features Perform weighted fusion: ； In the formula, For fusion weights; S3.

2. Input the fused features into a Transformer encoder consisting of L encoder layers to obtain contextual features. ; S3.

3. Decoding is performed using a three-level tag structure, and the total loss function is calculated. : ； In the formula, , , The cross-entropy loss is calculated for first-level, second-level, and third-level labels, respectively. The loss is due to hierarchical consistency constraints. To constrain weights; The loss of hierarchical consistency constraints for: ； In the formula, This represents the total number of first-level tags. Predict the probability for the first-level label. As a first-level tag All corresponding second-level subclasses; Will go to the same area The historical analysis results of each hour are input into an LSTM network to obtain time-series features. ; context features With time series characteristics After concatenation, the input is given to a fully connected layer, which outputs the probability of each semantic label: ； In the formula, For the first A semantic tag, It is the Sigmoid activation function. and These are the parameters for the fully connected layer.

4. The method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence according to claim 1, characterized in that, The sound source-individual binding algorithm described in S4 specifically includes: S4.1 Utilizing the time difference of arrival of sound waves received by a microphone array Calculate the three-dimensional coordinates of the sound source: ； In the formula, For the sound wave to reach the first The and the first The time difference between the microphones For the first The coordinates of each microphone, The speed of sound in air; S4.2 Project the sound source location onto the video screen and filter out the sound sources closest to the sound source. The individual pigs are considered as a candidate set. ; S4.3 Calculate the voiceprint feature vector of the current call. With candidate individual voiceprint template Cosine similarity: ； In the formula, This is the function for calculating cosine similarity.

5. The method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence according to claim 1, characterized in that, The semantically driven device control strategy generation algorithm in S5 specifically includes: S5.1 Establish a mapping table between semantic tags and device actions. Each tag corresponds to a set of device control instructions. When multiple semantic tags exist in the same area, calculate the effective priority of each tag: ； In the formula, For the preset category weights, The confidence level of the label; S5.2 Record the last execution time of each action. and system real-time timestamp Set minimum execution interval ; When satisfied Only after a certain time is it allowed to execute again. After the action is executed, the changes in the acoustic state of the area in the subsequent time are monitored, and the acoustic state improvement rate is calculated. : ； In the formula, and These represent the frequency of abnormal noises before and after the action is executed. When R < 30%, the response level is automatically upgraded.

6. The method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence according to claim 1, characterized in that, The semantic tags include: health tags, behavior tags, and emotion tags.

7. A pig health and behavior monitoring and automatic response system based on acoustic intelligence, characterized in that, A method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence as described in any one of claims 1-6, comprising: Perception layer: Multiple distributed intelligent audio acquisition nodes, each node containing a microphone array, a pre-processor, an embedded AI chip, and a wireless communication module; Edge computing layer: a field-level edge server that is wirelessly connected to the perception layer to receive pre-processed audio data and run sound biomarker extraction algorithms and lightweight semantic parsing models; Cloud platform layer: communicates bidirectionally with the edge computing layer, used for storing historical data, training and optimizing deep learning models, and running the global decision engine; Application interaction layer: This includes user terminals and automated execution devices. User terminals are used to receive early warning information and view management reports, while automated execution devices are used to receive control commands and execute corresponding actions.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory communicatively connected to the processor; the memory stores instructions that are executed by the processor to cause the processor to perform an acoustic intelligence-based method for monitoring and automatically responding to pig health and behavior as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores a computer program, which is executed by a processor as described in any one of claims 1 to 6, a method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence.

10. A program product, characterized in that, The program product is used to enable the processor to execute a method for monitoring and automatically responding to pig health and behavior based on acoustic intelligence as described in any one of claims 1 to 6.

Images