Construction method of abnormal pig sound monitoring system

Abstract

The invention discloses a construction method of an abnormal pig sound monitoring system, which comprises the following steps: S1, setting an audio segmentation node, an audio feature extraction node and a deep learning analysis calculation node, and registering the function and state information of each node to a cluster manager when the node is started; s2, an audio segmentation node receives original audio data, preprocesses the original audio data, accurately segments the audio into independent sound segments by using a double-threshold endpoint detection technology, and minimizes invalid data input; s3, an audio feature extraction node extracts key spectrogram data from the processed audio clips, and necessary input is provided for deep learning analysis; and S4, the deep learning analysis calculation node receives data from the audio feature extraction node, carries out deep analysis on the data by using the audio feature matrix, and focuses on the detection of abnormal sounds. According to the method, deep audio analysis is carried out on audio feature extraction nodes and processing nodes, and the method is specially used for audio classification tasks, so that abnormal sounds can be effectively identified and classified.

Description

Technical Field

[0001] The present invention relates to the technical field of pig call monitoring, and in particular to a method for constructing an abnormal pig call monitoring system. Background Art

[0002] With the development of artificial intelligence, more and more smart devices have been introduced into all walks of life. In the traditional animal husbandry industry, the biggest difficulty lies in care. The staff needs to pay close attention to possible abnormal phenomena such as viruses, biting and fighting in pigs and other animals. It takes a lot of manpower to supervise, and there may be situations where supervision is not timely. Therefore, the breeding industry has introduced artificial intelligence and proposed the "pig call recognition" technology, which can preliminarily judge the abnormality of pigs through pig calls, and issue disease warnings at the first time and report to the veterinarian or breeder of the pig farm.

[0003] At present, the pig sound recognition method mainly uses a combination of complex audio features to analyze and process pig calls. The computational cost is high and distributed computing technology of computer networks is also needed to improve the operating efficiency of the system. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a method for constructing an abnormal pig call monitoring system, which can effectively improve the recognition accuracy of the entire abnormal pig call and the system operation efficiency.

[0005] To this end, the technical solution of the present invention is: a method for constructing an abnormal pig call monitoring system, comprising the following steps:

[0006] S1. Set up audio segmentation nodes, audio feature extraction nodes, and deep learning analysis computing nodes, and register the function and status information of each node to the cluster manager when it is started;

[0007] S2, the audio segmentation node receives the original audio data, performs preprocessing, and uses dual-threshold endpoint detection technology to accurately segment the audio into separate call segments, minimizing invalid data input;

[0008] S3, the audio feature extraction node extracts key spectrogram data from the processed audio clip, and after all audio features are calculated, fuses them to form an audio feature matrix; the audio feature extraction node packages each audio feature matrix into a deep learning analysis computing task, and sends it to the message queue of the deep learning analysis computing node for extracting data, waiting for the cluster manager to calculate the load of the deep learning analysis computing node and allocate computing tasks;

[0009] S4, the deep learning analysis computing node receives data from the audio feature extraction node, uses the audio feature matrix to conduct in-depth analysis of the data, and focuses on the detection of abnormal calls; the overall process of deep learning analysis is as follows:

[0010] S4.1. The entire feature matrix is ​​divided into multiple data blocks of N*N elements to ensure that the local frequency features can be preserved. At the same time, the time and frequency dimensions of these data blocks also overlap to smooth the boundaries of the data blocks and ensure audio continuity.

[0011] S4.2, convert each data block into a 1-dimensional vector through a patch embedding layer for learning and capturing deeper features;

[0012] Patch embedding layer vi,j = Conv(Bi,j), the convolution layer Conv extracts key features from the data block by applying a set of learned filters and compresses them into a one-dimensional vector; the size, step size and padding of the convolution kernel can be adjusted as needed to control the dimension and feature coverage of the output vector;

[0013] At the same time, the position of the data block relative to the original feature matrix is ​​encoded through the two-dimensional position encoding layer and superimposed on the one-dimensional vector to form an embedded vector; the two-dimensional position vector is calculated as follows:

[0014]

[0015]

[0016]

[0017]

[0018] 2i and 2i+1 traverse to the dimension d of time encoding t Half of 2j and 2j+1 are traversed to the dimension d of the frequency encoding f Half of t and d f It is usually set to d / 2 to evenly distribute the embedding vector dimensions to time and frequency; t and f represent the specific index positions of time and frequency respectively;

[0019] 10000^2i / d t and 10000^2j / d f is a scaling factor used to adjust the periods of sine and cosine waves of different frequencies so that the position encoding covers a larger range of position indexes;

[0020] S4.3, using multiple superimposed attention mechanism encoding layers to process the embedding vector. The attention mechanism encoding layer uses a multi-head attention mechanism to enable the correlation between data blocks of each dimension in the embedding vector to be calculated and processed;

[0021] The embedded vector is processed using the multi-head attention mechanism MultiHead(Q,K,V), where Q, K, and V are query, key, and value matrices respectively:

[0022] A=MultiHead(v' i,j ,v' i,j ,v' i,j )

[0023]

[0024] S4.4. The vector processed by the transformer encoder is output to a fully connected layer, which maps the features to different categories, normal or abnormal calls; the activation function sigmoid is used to generate the predicted probability of each category.

[0025] On the basis of the above scheme and as a preferred scheme of the above scheme: in step S2, the audio processing steps are as follows:

[0026] S2.1, Frame Windowing Smoothing: Set the frame length and frame shift to divide the audio into multiple short-time frames. The overlapping parts between adjacent short-time frames are used to reduce the frame boundary effect. The influence of non-periodic boundaries is reduced by applying the Hanning window function on the short-time frames.

[0027] S2.2, Windowed logarithmic short-time energy calculation: Short-time energy is used to distinguish between sound and silence. It is calculated by summing the squares of audio samples in a fixed window.

[0028]

[0029] Where: y[n+iM] represents the sample value of the audio signal, n is the sample index in the current frame, iM represents the starting point of the i-th frame, and M is the frame shift or step size, that is, the sample number offset between two adjacent frames;

[0030] w[n] is the window function value applied to each sample to reduce spectral leakage at frame boundaries. The window function tapers to zero at the beginning and end of each frame to smooth the data.

[0031] ∈ is a small positive number, added to the energy summation to avoid division by zero errors in logarithmic operations;

[0032] S2.3, effective audio extraction: use the calculated moving average and moving standard deviation, combined with a pre-set sensitivity coefficient k to calculate the threshold T[m] of each point:

[0033] T[m]=μ[m]+k·σ[m]

[0034]

[0035]

[0036] Where: T[m] is the threshold at time point m, which is used to determine whether the point is an outlier. If the value of a data point exceeds this threshold, it may be marked as an outlier.

[0037] μ[m] is the average of the M data points closest to time point m. This average reflects the central trend of the M points closest to time point m.

[0038] σ[m] is the standard deviation of the M data points closest to time point m, and k is a coefficient used to adjust the threshold to control the sensitivity of anomaly detection;

[0039] Based on the calculated T[m], calculate the start and end points of the valid audio segment in the input audio segment. The start point is the m1 value when start[m1] is 1, and the end point is the m2 value when end[m2] is 1, where m1 and m2 are used to represent the index in the frame sequence;

[0040]

[0041]

[0042] S2.4, the audio segmentation node extracts and packages valid audio segments into computing tasks, and sends them to the message queue of the feature extraction node for extracting data, waiting for the cluster manager to calculate the load of the feature extraction node and allocate computing tasks.

[0043] On the basis of the above scheme and as a preferred scheme of the above scheme: in the step S2.4, the information in each computing task includes: an array of valid audio sequences, a unique code of the camera or original audio data source, the acquisition time of each valid audio sequence, and the required amount of computing resources.

[0044] On the basis of the above scheme and as a preferred scheme of the above scheme: in step S3, the whole feature extraction process includes the following steps:

[0045] S3.1, Audio feature extraction node management: When an audio feature extraction node runs successfully, it sends a network request to the cluster manager for node registration. The cluster manager assigns a unique computing node number. The cluster manager uses a heartbeat mechanism to determine whether a node is available.

[0046] S3.2, cluster manager allocates computing tasks: the task set in the audio feature extraction message queue is represented by T = {t1, t2, ... tn}, and the amount of computing resources required for each task ti is marked as ri,

[0047] At the same time, for the audio feature extraction computing nodes managed by the cluster manager, N = {n1, n2, ..., nm} is used to represent the maximum resource capacity of each node nj is cj, and the current load is L j , by minimizing the standard deviation of the load of each node to ensure load balancing, for each new task ti, calculate the new load L after it is assigned to each node nj j ′=L j +ri;

[0048] Evaluate each L j The absolute value of the difference between the node and the average load L is used to select the node with the smallest difference for allocation and update the node's current load L j :

[0049]

[0050] S3.3, audio feature calculation node calculation process: in the process of audio feature extraction, multiple audio features of valid audio segments are extracted respectively. After the extraction is completed, all audio features are connected to form a feature matrix for subsequent deep learning analysis;

[0051] S3.4. The audio feature extraction node packages each audio feature matrix into a deep learning analysis computing task, and sends it to the message queue of the deep learning analysis computing node for extracting data, waiting for the cluster manager to calculate the load of the deep learning analysis computing node and allocate computing tasks; the information in each computing task includes: the audio feature matrix array, the unique code of the audio feature matrix source, the collection time of the valid audio of the audio feature matrix source, and the required computing resources.

[0052] On the basis of the above scheme and as a preferred scheme of the above scheme: in the step S3.1, the heartbeat is a signal that the node sends periodically to the cluster manager, and the heartbeat signal contains the current status information of the node, namely CPU load, memory usage, and network status; the heartbeat signal can be sent periodically, and when the cluster manager does not receive the heartbeat of a node within a predetermined time window, the node will be judged as unavailable or faulty.

[0053] On the basis of the above scheme and as a preferred scheme of the above scheme: the heartbeat mechanism includes a heartbeat generator, a heartbeat receiver and a timeout detector;

[0054] The heartbeat generator runs on each node and is responsible for periodically generating and sending heartbeat signals;

[0055] The heartbeat receiver and the timeout detector are both located in the cluster manager. The heartbeat receiver is responsible for receiving the heartbeat signals sent by all nodes. The heartbeat receiver can record the timestamp of each received heartbeat to perform timeout detection.

[0056] The timeout detector is responsible for monitoring the time interval for receiving heartbeats. If the heartbeat of a node is not received for more than a predetermined threshold, the timeout detector will trigger an alarm or other response measures, including rescheduling the tasks on the node to other healthy nodes.

[0057] On the basis of the above scheme and as a preferred scheme of the above scheme: in the step S3.3, the audio features to be extracted include: logarithmic Mel spectrum features, Mel-frequency cepstral coefficients, spectral chromaticity features, sound network diagram, spectral contrast, spectral Rolloff value, spectral bandwidth; after all audio features are calculated, they are fused to form an audio feature matrix. Since the audio features are aligned in the time frame, the audio features are the same in the time dimension, but different in the information dimension. The feature matrices can be directly spliced ​​in the information dimension, and all audio columns are arranged side by side to form a feature matrix for deep learning analysis.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] 1. The system uses message queues to achieve efficient data transmission and synchronization, ensuring data reliability and transmission efficiency between nodes; message queues support asynchronous data processing and response, improving the overall response speed and processing capabilities of the system; the cluster's fault tolerance and recovery mechanism ensures stable system operation, and through automatic fault detection and task redistribution functions, even if some nodes fail, tasks can be quickly redistributed to other healthy nodes.

[0060] 2. Use the audio segmentation node to efficiently process the original audio data through adaptive threshold endpoint detection technology, greatly reduce invalid data input, and improve the processing efficiency of the overall system.

[0061] 3. By extracting multi-dimensional audio features from the processed audio, the advanced effects of multiple spectrum feature extraction methods are effectively integrated to form more refined features.

[0062] 4. Use advanced deep learning analysis models to perform in-depth audio analysis on audio feature extraction nodes and processing nodes, specifically targeting audio classification tasks, to effectively identify and classify abnormal calls.

[0063] 5. Capture the local and deep features of the audio by fine-grained segmentation and patch embedding of the multi-dimensional audio feature matrix, and optimize the feature expression using the multi-head self-attention mechanism of the transformer encoder. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a flow chart of the calculation pipeline of the present invention;

[0065] Figure 2 is a flow chart of the method of the present invention;

[0066] Figure 3 The audio segmentation calculation node flow chart of the present invention;

[0067] Figure 4 The flowchart of the audio feature extraction calculation node of the present invention;

[0068] Figure 5 It is a deep learning analysis calculation node flow chart of the present invention;

[0069] Figure 6 It is a flow chart between different types of computing task nodes of the present invention. DETAILED DESCRIPTION

[0070] See the attached drawings. The method for constructing the abnormal pig call monitoring system described in this embodiment includes the following steps:

[0071] S1. Set up audio segmentation nodes, audio feature extraction nodes, and deep learning analysis computing nodes, and register the function and status information of each node to the cluster manager when it starts, so as to perform effective task scheduling and resource allocation.

[0072] S2, the audio segmentation node receives the original audio data, performs preprocessing, and uses dual-threshold endpoint detection technology to accurately segment the audio into separate call segments, minimizing invalid data input.

[0073] S2.1, frame windowing and smoothing: Set the frame length (15ms) and frame shift (7ms) to divide the audio into multiple short-time frames. The overlapping parts between adjacent short-time frames are used to reduce the frame boundary effect. The influence of non-periodic boundaries is reduced by applying the Hanning window function on the short-time frames.

[0074] S2.2. Windowed logarithmic short-time energy calculation: Short-time energy is a method to describe the energy intensity of an audio clip. It can be used to distinguish between sound and silence. The short-time energy can be calculated by summing the squares of audio samples within a fixed window. By using the windowed logarithmic short-time energy measure, it is more in line with human auditory characteristics.

[0075]

[0076] Where: E[i] is the short-time energy of the i-th frame, y[n+iM] represents the sample value of the audio signal, n is the sample index in the current frame, iM represents the starting point of the i-th frame, and M is the frame shift or step size, that is, the sample number offset between two adjacent frames;

[0077] w[n] is the window function value applied to each sample to reduce spectral leakage at frame boundaries. The window function tapers to zero at the beginning and end of each frame to smooth the data.

[0078] ∈ is a small positive number that is added to the energy sum to avoid division by zero errors when performing logarithmic operations; this value ensures that logarithms can be safely calculated even in very quiet frames. The value of ∈ needs to be selected according to the specific task, and is generally selected to be several orders of magnitude smaller than the sum in the log. For example, the sum in the log is generally greater than 100 in all data, so the value of ∈ is a positive number less than 0.01.

[0079] S2.3, effective audio extraction based on adaptive threshold method: the threshold T[m] of each point is calculated by using the calculated moving average and moving standard deviation, combined with a pre-set sensitivity coefficient k. This threshold will change adaptively according to the actual statistical characteristics of the audio data, so that abnormal or significant events can be identified more flexibly.

[0080] T[m]=μ[m]+k·σ[m]

[0081]

[0082]

[0083] Where: T[m] is the threshold at time point m, which is used to determine whether the point is an outlier. If the value of a data point exceeds this threshold, it may be marked as an outlier.

[0084] μ[m] is the average of the M data points closest to time point m. This average reflects the central trend of the M points closest to time point m.

[0085] σ[m] is the standard deviation of the M data points closest to time point m,

[0086] k is a coefficient used to adjust the threshold to control the sensitivity of anomaly detection. The value of k is generally a decimal between [0, 1], which is determined according to the detection sensitivity requirements. If a more sensitive intercept is required, it should be reduced.

[0087] Based on the calculated T[m], the start and end points of the valid audio segment in the input audio segment are calculated. The start point is the m1 value when start[m1] is 1, and the end point is the m2 value when end[m2] is 1, where m1 and m2 are used to represent the index in the frame sequence.

[0088]

[0089]

[0090] S2.4, the audio segmentation node extracts and packages the valid audio segments into computing tasks, and sends them to the message queue of the feature extraction node for extracting data, waiting for the cluster manager to calculate the load of the feature extraction node and assign computing tasks. The information in each computing task includes: an array of valid audio sequences, a unique code of the camera or the source of the original audio data (used to determine the data source information of the collected data), the collection time of each valid audio sequence, and the required computing resources (defined by the sum of the frames of the valid audio segments).

[0091] S3, the audio feature extraction node extracts key spectrogram data from the processed audio clips to provide the necessary input for deep learning analysis. The entire feature extraction process includes the following steps:

[0092] S3.1. Audio feature extraction node management: When an audio feature extraction node runs successfully, it sends a network request to the cluster manager for node registration. The cluster manager assigns a unique computing node number. The cluster manager uses a heartbeat mechanism to determine whether a node is available.

[0093] Heartbeat is a signal that a node sends to the manager periodically, usually containing information about the node's current status, such as CPU load, memory usage, network status, etc. Heartbeat signals have the following key features: Periodic transmission: The node sends heartbeat signals at a predetermined time interval, such as once every few seconds. Timeout detection: If the manager does not receive a heartbeat from a node within a predetermined time window, it will determine that the node is unavailable or faulty. Status information: The heartbeat is not only a signal, but also carries key performance and health indicators of the node.

[0094] The heartbeat mechanism consists of the following:

[0095] ① Heartbeat generator, a lightweight process or thread running on each node, responsible for periodically generating and sending heartbeat signals. This generator can be a simple script or a more complex application, depending on the complexity of the parameters that need to be monitored.

[0096] ②Heartbeat receiver, a component in the cluster manager, is responsible for receiving heartbeat signals sent by all nodes. The receiver usually records the timestamp of each received heartbeat for timeout detection.

[0097] ③Timeout detector, another key component in the cluster manager, is responsible for monitoring the time interval for receiving heartbeats. If the heartbeat of a node is not received for more than a predetermined threshold, the timeout detector will trigger an alarm or other response measures, such as rescheduling the tasks on the node to other healthy nodes.

[0098] S3.2, cluster manager allocates computing tasks: the task set in the audio feature extraction message queue is represented by T = {t1, t2, ... tn}, and the amount of computing resources required for each task ti is marked as ri,

[0099] At the same time, for the audio feature extraction computing nodes managed by the cluster manager, N = {n1, n2, ..., nm} is used to represent the maximum resource capacity of each node nj is cj, and the current load is L j , by minimizing the standard deviation of the load of each node to ensure load balancing, for each new task ti, calculate the new load L after it is assigned to each node nj j ′=L j +ri;

[0100] Evaluate each L j With average load The absolute value of the difference is used to select the node with the smallest difference for allocation and update the current load L of the node j :

[0101]

[0102] S3.3, audio feature calculation node calculation process: in the process of audio feature extraction, multiple audio features of valid audio segments are extracted respectively. After the extraction is completed, all audio features are connected to form a feature matrix for subsequent deep learning analysis;

[0103] The audio features that need to be extracted include:

[0104] Log Mel Spectrogram Features are based on the Mel spectrum and simulate the nonlinear auditory perception of the human ear by applying a Mel filter bank and taking the logarithm of the energy.

[0105] Mel-frequency cepstral coefficients (MFCC Coefficients) further perform discrete cosine transform (DCT) on the logarithmic Mel spectrum to extract the main information.

[0106] Spectral Chroma Features capture the intensity of pitch and reflect the energy distribution of different pitches in the audio.

[0107] Tonnetz are used to analyze harmonic relationships by mapping notes to a network of perfect fourths, fifths, and minor thirds.

[0108] Spectral Contrast measures the contrast between spectral peaks and valleys in different frequency bands, reflecting the dynamic range and texture of the audio.

[0109] The spectral rolloff value represents the frequency point at which the total energy of the audio is a certain percentage, and is used to identify the nature of the sound source.

[0110] Spectral bandwidth measures the width of the spectrum energy concentration and is closely related to the timbre of the sound.

[0111] After all audio features are calculated, they are fused to form an audio feature matrix. Since each audio feature is aligned in the time frame, each audio feature is the same in the time dimension, but different in the information dimension. Each feature matrix can be spliced ​​in the information dimension, and all audio columns can be arranged side by side to form a feature matrix for deep learning analysis.

[0112] S3.4. The audio feature extraction node packages each audio feature matrix into a deep learning analysis computing task, and sends it to the message queue of the deep learning analysis computing node for extracting data, waiting for the cluster manager to calculate the load of the deep learning analysis computing node and allocate computing tasks; the information in each computing task includes: an audio feature matrix array, a unique code of the source of the audio feature matrix, the acquisition time of the valid audio of the source of the audio feature matrix, and the required amount of computing resources (defined by the length of the time dimension of the audio feature matrix).

[0113] S4, the deep learning analysis computing node receives data from the audio feature extraction node, uses the audio feature matrix to conduct in-depth analysis of the data, and focuses on the detection of abnormal calls; the overall process of deep learning analysis is as follows:

[0114] S4.1. The entire feature matrix is ​​divided into multiple data blocks of N*N elements to ensure that the local frequency features can be preserved. At the same time, the time and frequency dimensions of these data blocks also overlap to smooth the boundaries of the data blocks and ensure audio continuity.

[0115] S4.2, convert each data block into a 1-dimensional vector through a patch embedding layer for learning and capturing deeper features;

[0116] Patch embedding layer vi,j = Conv(Bi,j), the convolution layer Conv extracts key features from the data block by applying a set of learned filters and compresses them into a one-dimensional vector; the size, step size and padding of the convolution kernel can be adjusted as needed to control the dimension and feature coverage of the output vector;

[0117] At the same time, the position of the data block relative to the original feature matrix is ​​encoded through the two-dimensional position encoding layer and superimposed on the one-dimensional vector to form an embedded vector; the two-dimensional position vector is calculated as follows:

[0118]

[0119]

[0120]

[0121]

[0122] d t is the dimension size of the time encoding, d f is the dimension size of frequency encoding, 2i and 2i+1 traverse to the dimension d of time encoding t Half of 2j and 2j+1 are traversed to the dimension d of the frequency encoding f Half of t and d f It is usually set to d / 2, where d is the dimension, to evenly distribute the embedding vector dimension to time and frequency; t and f represent the specific index positions of time and frequency respectively;

[0123] 10000^2i / d t and 10000^2j / d f is a scaling factor used to adjust the periods of sine and cosine waves of different frequencies so that the position encoding covers a larger range of position indices.

[0124] S4.3, using multiple superimposed attention mechanism encoding layers to process the embedding vector. The attention mechanism encoding layer uses a multi-head attention mechanism to enable the correlation between data blocks of each dimension in the embedding vector to be calculated and processed;

[0125] The embedded vector is processed using the multi-head attention mechanism MultiHead(Q,K,V), where Q, K, and V are query, key, and value matrices respectively:

[0126] A=MultiHead(v' i,j ,v' i,j ,v' i,j )

[0127]

[0128] S4.4, the vector processed by the transformer encoder is output to a fully connected layer, which maps the features to different categories, such as normal or abnormal calls; the activation function sigmoid is used to generate the predicted probability of each category. The sigmoid function is also called the Logistic function, which is used for the output of hidden layer neurons and has a value range of (0,1). It can map a real number to the interval of (0,1) and can be used for binary classification.

[0129] This embodiment has the following advantages:

[0130] ① Efficient audio processing flow: The audio segmentation node is used to efficiently process raw audio data through adaptive threshold endpoint detection technology, which greatly reduces invalid data input and improves the processing efficiency of the overall system.

[0131] ② Multi-dimensional audio feature matrix extraction: By extracting multi-dimensional audio features from the processed audio, the advanced effects of multiple spectrum feature extraction methods are effectively integrated to form more refined features.

[0132] ③Precise classification driven by deep learning: Advanced deep learning analysis models are used to perform in-depth audio analysis on audio feature extraction nodes and processing nodes, specifically targeting audio classification tasks to effectively identify and classify abnormal calls.

[0133] ④ Innovative data processing technology: Through fine-grained segmentation and patch embedding of the multi-dimensional audio feature matrix, the local and deep features of the audio are captured, and the multi-head self-attention mechanism of the transformer encoder is used to optimize the feature expression.

[0134] ⑤ Efficient and reliable data transmission mechanism: The system achieves efficient and reliable data transmission and synchronization through memory message queues (data transmission within a single machine) or message queue programs, supports the processing of large amounts of data and improves the overall response speed.

[0135] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for constructing an abnormal pig call monitoring system, characterized in that: The following steps are involved: S1. Set up audio segmentation nodes, audio feature extraction nodes, and deep learning analysis computing nodes, and register the function and status information of each node to the cluster manager when it is started; S2, the audio segmentation node receives the original audio data, performs preprocessing, and uses dual-threshold endpoint detection technology to accurately segment the audio into separate call segments, minimizing invalid data input; S3, the audio feature extraction node extracts key spectrogram data from the processed audio clips, and after all audio features are calculated, fuses them to form an audio feature matrix; The audio feature extraction node packages each audio feature matrix into a deep learning analysis computing task, and sends it to the message queue of the deep learning analysis computing node for extracting data, waiting for the cluster manager to calculate the load of the deep learning analysis computing node and allocate computing tasks; S4, the deep learning analysis computing node receives data from the audio feature extraction node, uses the audio feature matrix to conduct in-depth analysis of the data, and focuses on the detection of abnormal calls; the overall process of deep learning analysis is as follows: S4.

1. The entire feature matrix is ​​divided into multiple data blocks of N*N elements to ensure that the local frequency features can be preserved. At the same time, the time and frequency dimensions of these data blocks also overlap to smooth the boundaries of the data blocks and ensure audio continuity. S4.2, convert each data block into a 1-dimensional vector through a patch embedding layer for learning and capturing deeper features; Patch embedding layer vi,j = Conv(Bi,j), the convolution layer Conv extracts key features from the data block by applying a set of learned filters and compresses them into a one-dimensional vector; the size, step size and padding of the convolution kernel can be adjusted as needed to control the dimension and feature coverage of the output vector; At the same time, the position of the data block relative to the original feature matrix is ​​encoded through the two-dimensional position encoding layer and superimposed on the one-dimensional vector to form an embedded vector; the two-dimensional position vector is calculated as follows: 2i and 2i+1 traverse to the dimension d of time encoding t Half of 2j and 2j+1 are traversed to the dimension d of the frequency encoding f Half of t and d f It is usually set to d / 2 to evenly distribute the embedding vector dimensions to time and frequency; t and f represent the specific index positions of time and frequency respectively; 10000^2i / d t and 10000^2j / d f is a scaling factor used to adjust the periods of sine and cosine waves of different frequencies so that the position encoding covers a larger range of position indexes; S4.3, using multiple superimposed attention mechanism encoding layers to process the embedding vector. The attention mechanism encoding layer uses a multi-head attention mechanism to enable the correlation between data blocks of each dimension in the embedding vector to be calculated and processed; The embedded vector is processed using the multi-head attention mechanism MultiHead(Q,K,V), where Q, K, and V are query, key, and value matrices respectively: A=MultiHead(v′ i,j 'in' i,j 'in' i,j ) S4.

4. The vector processed by the transformer encoder is output to a fully connected layer, which maps the features to different categories, normal or abnormal calls; the activation function sigmoid is used to generate the predicted probability of each category.

2. The method for constructing an abnormal pig call monitoring system according to claim 1, characterized in that: In step S2, the audio processing steps are as follows: S2.1, Frame Windowing Smoothing: Set the frame length and frame shift to divide the audio into multiple short-time frames. The overlapping parts between adjacent short-time frames are used to reduce the frame boundary effect. The Hanning window function is applied to the short-time frames to reduce the impact of non-periodic boundaries. S2.2, Windowed logarithmic short-time energy calculation: Short-time energy is used to distinguish between sound and silence. It is calculated by summing the squares of audio samples in a fixed window. Where: y[n+iM] represents the sample value of the audio signal, n is the sample index in the current frame, iM represents the starting point of the i-th frame, and M is the frame shift or step size, that is, the sample number offset between two adjacent frames; w[n] is the window function value applied to each sample to reduce spectral leakage at frame boundaries. The window function tapers to zero at the beginning and end of each frame to smooth the data. ∈ is a small positive number, added to the energy summation to avoid division by zero errors in logarithmic operations; S2.3, effective audio extraction: use the calculated moving average and moving standard deviation, combined with a pre-set sensitivity coefficient k to calculate the threshold T[m] of each point: T[m]=μ[m]+k·σ[m] Where: T[m] is the threshold at time point m, which is used to determine whether the point is an outlier. If the value of a data point exceeds this threshold, it may be marked as an outlier. μ[m] is the average of the M data points closest to time point m. This average reflects the central trend of the M points closest to time point m. σ[m] is the standard deviation of the M data points closest to time point m, and k is a coefficient used to adjust the threshold to control the sensitivity of anomaly detection; Based on the calculated T[m], calculate the start and end points of the valid audio segment in the input audio segment. The start point is the m1 value when start[m1] is 1, and the end point is the m2 value when end[m2] is 1, where m1 and m2 are used to represent the index in the frame sequence; S2.4, the audio segmentation node extracts and packages valid audio segments into computing tasks, and sends them to the message queue of the feature extraction node for extracting data, waiting for the cluster manager to calculate the load of the feature extraction node and allocate computing tasks.

3. The method for constructing an abnormal pig call monitoring system as claimed in claim 2, characterized in that: In step S2.4, the information in each computing task includes: an array of valid audio sequences, a unique code of the camera or the original audio data source, the acquisition time of each valid audio sequence, and the required computing resources.

4. The method for constructing an abnormal pig call monitoring system according to claim 1, characterized in that: In step S3, the entire feature extraction process includes the following steps: S3.1, Audio feature extraction node management: When an audio feature extraction node runs successfully, it sends a network request to the cluster manager for node registration. The cluster manager assigns a unique computing node number. The cluster manager uses a heartbeat mechanism to determine whether a node is available. S3.2, cluster manager allocates computing tasks: the task set in the audio feature extraction message queue is represented by T = {t1, t2, ... tn}, and the amount of computing resources required for each task ti is marked as ri, At the same time, for the audio feature extraction computing nodes managed by the cluster manager, N = {n1, n2, ..., nm} is used to represent the maximum resource capacity of each node nj is cj, and the current load is L j , by minimizing the standard deviation of the load of each node to ensure load balancing, for each new task ti, calculate the new load L after it is assigned to each node nj j ′=L j +ri; Evaluate each L j With average load The absolute value of the difference is used to select the node with the smallest difference for allocation and update the current load L of the node. j : S3.3, audio feature calculation node calculation process: in the process of audio feature extraction, multiple audio features of valid audio segments are extracted respectively. After the extraction is completed, all audio features are connected to form a feature matrix for subsequent deep learning analysis; S3.4, the audio feature extraction node packages each audio feature matrix into a deep learning analysis computing task, and sends it to the message queue of the deep learning analysis computing node for extracting data, waiting for the cluster manager to calculate the load of the deep learning analysis computing node and allocate computing tasks; The information in each computing task includes: an audio feature matrix array, a unique code of a source of the audio feature matrix, a collection time of valid audio of the source of the audio feature matrix, and a required amount of computing resources.

5. The method for constructing an abnormal pig call monitoring system as claimed in claim 4, characterized in that: In step S3.1, the heartbeat is a signal that the node sends periodically to the cluster manager. The heartbeat signal contains the current status information of the node, namely, CPU load, memory usage, and network status. The heartbeat signal can be sent periodically. When the cluster manager does not receive the heartbeat of a node within a predetermined time window, the node will be determined to be unavailable or faulty.

6. The method for constructing an abnormal pig call monitoring system according to claim 5, characterized in that: The heartbeat mechanism includes a heartbeat generator, a heartbeat receiver and a timeout detector; The heartbeat generator runs on each node and is responsible for periodically generating and sending heartbeat signals; The heartbeat receiver and the timeout detector are both located in the cluster manager. The heartbeat receiver is responsible for receiving the heartbeat signals sent by all nodes. The heartbeat receiver can record the timestamp of each received heartbeat to perform timeout detection. The timeout detector is responsible for monitoring the time interval for receiving heartbeats. If the heartbeat of a node is not received for more than a predetermined threshold, the timeout detector will trigger an alarm or other response measures, including rescheduling the tasks on the node to other healthy nodes.

7. The method for constructing an abnormal pig call monitoring system according to claim 4, characterized in that: In the step S3.3, the audio features that need to be extracted include: logarithmic Mel spectrum features, Mel frequency cepstral coefficients, spectrum chromaticity features, sound network diagram, spectral contrast, spectral Rolloff value, and spectral bandwidth; after all audio features are calculated, they are fused to form an audio feature matrix. Since each audio feature is aligned in the time frame, each audio feature is the same in the time dimension, but different in the information dimension. The feature matrices can be directly spliced ​​in the information dimension, and all audio columns are arranged side by side to form a feature matrix for deep learning analysis.

Images