An abnormal voiceprint recognition monitoring system based on water pump operation characteristic spectrum

Abstract

The application discloses an abnormal voiceprint recognition monitoring system based on a water pump operation characteristic spectrum, and belongs to the technical field of water pump state monitoring and voiceprint recognition. The system comprises an audio acquisition module, a dynamic frame division module, a spectrum extraction module, a state segmentation module, a characteristic spectrum aggregation module and an abnormality recognition module; the audio acquisition module acquires original audio signals when the water pump is running; the characteristic spectrum aggregation module performs spectrum line aggregation on sparse spectrum lines in each running segment to obtain a running characteristic spectrum; and the abnormality recognition module performs frequency point-by-frequency point difference on the running characteristic spectra of the current running segment and the previous running segment, extracts an extreme point frequency point set in a differential spectrum sequence as an abnormal voiceprint recognition basis. The scheme reduces the interference caused by the water pump state switching process on the abnormal voiceprint recognition through dynamic frame division and spectrum difference extreme value recognition between running segments.

Description

Technical Field

[0001] This invention relates to the field of water pump status monitoring and voiceprint recognition technology, specifically an abnormal voiceprint recognition and monitoring system based on the characteristic spectrum of water pump operation. Background Technology

[0002] Water pump anomaly soundprint recognition and monitoring involves collecting audio signals from water pumps during operation, extracting spectral features from these signals, and identifying pump anomalies based on changes in these spectral features. Related technologies often employ fixed frame lengths and fixed inter-frame overlap rates for water pump audio, or use fixed-parameter short-time Fourier transforms to obtain the spectrum of each frame. Then, the spectra of adjacent frames or adjacent time periods are differentially analyzed or compared using templates, and the magnitude of the spectral differences is used to determine if abnormal soundprints have appeared. However, water pumps undergo startup, shutdown, speed regulation, and load changes during actual operation, resulting in audio signals with significant non-stationary characteristics; the main frequency components change with the operating state. Fixed framing parameters cannot simultaneously accommodate both rapidly changing and slowly changing soundprint signals. When signal changes are drastic, adjacent frame spectra may jump or alias, while when the signal is stable, inter-frame feature redundancy is easily generated, affecting the timeliness and resolution of spectral feature extraction. Current technologies fail to differentiate between spectral changes caused by operational state switching and those caused by faults when identifying anomalies. When a pump's operational state switches naturally, major frequency components in the spectrum shift, disappear, or emerge. These changes are directly factored into the spectral differences between adjacent frames or time periods, triggering anomaly alarms. Conversely, during stable operation, minor frequency component changes caused by localized abnormal sound patterns are easily masked by fluctuations in the overall spectral baseline, leading to missed anomalies. To address these shortcomings, it is necessary to introduce an inter-frame overlap rate that varies with the signal's dynamic characteristics during the framing stage of the pump's audio, allowing the sliding window to adaptively adjust its time resolution when the zero-crossing rate changes. Furthermore, before anomaly identification, the frequency offset of sparse spectral lines in adjacent frames should be used to determine the operational state switching frame boundaries, dividing the pump audio into different operational segments. Then, feature spectral differences and extreme frequency point extraction should be used between adjacent operational segments to identify spectral differences caused by localized abnormal sound patterns, suppressing the impact of overall spectral changes caused by operational state switching on the identification results. Summary of the Invention

[0003] To address the problems of fixed framing parameters in existing water pump acoustic signature recognition and monitoring systems being unable to adapt to non-stationary operating conditions, and the failure to distinguish between spectral changes during operating state transitions and differences in abnormal acoustic signature spectra, leading to false alarms and missed alarms, this paper proposes an abnormal acoustic signature recognition and monitoring system based on the characteristic spectrum of water pump operation. This system uses a sliding window with a dynamically adjusted inter-frame overlap rate based on the zero-crossing rate to segment the original audio signal of the water pump. It marks the state transition frame boundaries and divides the operating segments based on the frequency offset of sparse spectral lines in adjacent frames. The system aggregates the sparse spectral lines within each operating segment to obtain the characteristic spectrum of operation. It then performs frequency-by-frequency difference analysis on the characteristic spectrum of the current operating segment and the previous operating segment, extracting the set of extreme frequency points from the difference spectrum sequence as the basis for abnormal acoustic signature recognition. This reduces the interference of natural operating state transitions on the abnormal recognition results and enhances the ability to identify local abnormal acoustic signature frequency components.

[0004] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides an abnormal voiceprint recognition and monitoring system based on the characteristic spectrum of water pump operation, including an audio acquisition module, a dynamic framing module, a spectrum extraction module, a state segmentation module, a feature spectrum aggregation module, and an abnormal recognition module. The audio acquisition module acquires the raw audio signal during water pump operation; the dynamic framing module uses a sliding window with dynamically adjustable inter-frame overlap rate to segment the raw audio signal into multiple frames; the spectrum extraction module performs frequency band energy distribution calculation on each frame of audio signal, extracts the frequency points corresponding to the frequency band energy peaks, and obtains the sparse spectrum lines of each frame of audio signal; the state segmentation module determines the moment when the water pump's operating state changes based on the frequency offset of the sparse spectrum lines between adjacent frames, marks the frame boundaries where the frequency offset in adjacent frames exceeds a preset threshold as state switching frame boundaries, and divides the multiple frames of audio signal into several operating segments based on the state switching frame boundaries; the feature spectrum aggregation module performs spectrum line aggregation on the sparse spectrum lines in each operating segment to obtain the operating feature spectrum of each operating segment; the anomaly identification module performs frequency point-by-frequency difference between the operating feature spectrum of the current operating segment and the operating feature spectrum of the previous operating segment to obtain a difference spectrum sequence, extracts the extreme point frequency point set in the difference spectrum sequence, and if the extreme point frequency point set is not empty, the extreme point frequency point set is used as the basis for abnormal voiceprint identification. This system performs segment-level differential analysis by dividing the operation into sparse spectral lines and operational segments. It can highlight the spectral differences caused by anomalies in the context of changes in the pump's operating status, thereby improving the anti-interference capability of abnormal voiceprint recognition.

[0005] As a technical solution of this invention, the dynamic framing module acquires the zero-crossing rate of the original audio signal, adjusts the inter-frame overlap rate of the sliding window according to the value of the zero-crossing rate, decreases the inter-frame overlap rate when the zero-crossing rate increases, and increases the inter-frame overlap rate when the zero-crossing rate decreases; the original audio signal is framing using the adjusted inter-frame overlap rate to obtain multiple frames of audio signal; the start and end times of each frame of audio signal are recorded as the time-domain localization basis for spectrum analysis. By dynamically adjusting the inter-frame overlap rate, redundancy can be reduced when the signal changes drastically, and more time-domain details can be preserved when the signal is flat, balancing computational efficiency and time-domain localization accuracy of spectrum analysis.

[0006] As a technical solution of this invention, the spectrum extraction module performs a Fourier transform on each frame of audio signal to obtain the frequency domain amplitude spectrum of that frame. It then scans the frequency domain amplitude spectrum in ascending frequency order, sequentially calculating the amplitude difference between each frequency point and its preceding frequency point. Frequency points where the amplitude difference changes from positive to negative are marked as local peak frequencies. The frequency values ​​of these local peak frequencies are used as the frequency coordinates of the sparse spectrum lines, and the amplitude values ​​corresponding to these local peak frequencies are used as the amplitude coordinates of the sparse spectrum lines, forming a set of sparse spectrum lines for that frame. This method can compress the complete amplitude spectrum into a sparse representation composed of the main energy peaks, reducing the amount of data processed while retaining key audiogram peak information.

[0007] As a technical solution of this invention, the state segmentation module performs frequency point matching on the sparse spectrum line sets of two adjacent frames. Each sparse spectrum line in the previous frame is paired with the sparse spectrum line with the smallest frequency difference in the next frame, and the frequency difference between the paired frequency points is calculated as the frequency offset. The sum of the absolute values ​​of the frequency offsets of all paired frequency points is used as the overall offset value of the adjacent frames, or the ratio of the sum of the absolute values ​​of the frequency offsets of all paired frequency points to the total number of pairs is used as the overall offset value. When the overall offset value is greater than a preset threshold, the frame boundary between two adjacent frames is marked as the state switching frame boundary. This determination, using the frequency point offset of sparse spectrum lines to reflect changes in the spectral structure, can effectively capture the moment of water pump operation state switching.

[0008] As a technical solution of this invention, the state segmentation module uses the state switching frame boundary as the segmentation point, merging consecutive frames between two adjacent state switching frame boundaries in a multi-frame audio signal into a running segment; aligning the sparse spectral lines of all frames within the running segment according to frequency coordinates, and taking the maximum amplitude value at the same frequency coordinate as the aggregated amplitude value at that frequency point; using all aggregated amplitude values ​​and their corresponding frequency coordinates to construct the aggregated spectral lines of the running segment, as an initial estimate of the running feature spectrum. This separates the acoustic features on both sides of the state change boundary, avoiding the mixing of features from different running states.

[0009] As a technical solution of this invention, the feature spectrum aggregation module merges the frequency coordinates of sparse spectral lines of all frames within a running segment into a frequency set, and sorts the frequency values ​​in the frequency set from smallest to largest. It then traverses the sorted frequency set, merging adjacent frequency values ​​with a frequency interval less than a set threshold into the same frequency band cluster. The weighted average of all frequency values ​​within the frequency band cluster is used as the representative frequency of that frequency band cluster, and the average of the amplitude values ​​of all sparse spectral lines within the frequency band cluster is used as the representative amplitude of that frequency band cluster. The representative frequencies and representative amplitudes of all frequency band clusters constitute the running feature spectrum of the running segment. This aggregation method can merge multiple spectral lines caused by slight frequency drift into the same frequency band, reducing the impact of frequency jitter on the running feature spectrum and making the segment features more stable.

[0010] As a technical solution of this invention, the anomaly identification module merges representative frequencies in the operating characteristic spectra of the current running segment and the previous running segment to obtain a full frequency reference set. For each frequency value in the full frequency reference set, if the difference between the representative frequency and the target frequency value in the operating characteristic spectrum of the current or previous running segment is less than a preset frequency tolerance threshold, then the representative amplitude corresponding to that representative frequency is taken as the effective amplitude of the corresponding segment at that frequency value; otherwise, zero is taken as the amplitude value of the corresponding segment. The current amplitude value at each frequency value is subtracted from the previous amplitude value to obtain the difference value of that frequency value. All difference values ​​are arranged in frequency order to form a difference spectrum sequence. By unifying the frequency reference and tolerance comparison, frequency-point alignment and differential analysis of the operating characteristic spectra of two segments are achieved, which can amplify the spectral differences between adjacent segments.

[0011] As a technical solution of this invention, the anomaly identification module scans the differential spectrum sequence in ascending frequency order, sequentially calculating the difference between the differential value of each frequency point and the differential value of the previous frequency point to obtain the first-order differential change. Frequency points where the first-order differential change changes from positive to negative are marked as local maxima of the differential spectrum, and frequency points where the first-order differential change changes from negative to positive are marked as local minima of the differential spectrum. The frequency values ​​of the local maxima and local minima are merged to obtain a set of extreme point frequencies. If the set of extreme point frequencies is not empty, it indicates a significant spectral difference between adjacent segments, and this set of extreme point frequencies is used as the basis for anomaly voiceprint identification. This method, by extracting the extreme point frequencies of the differential spectrum, can focus on local spectral changes caused by anomalies, reducing the impact of overall amplitude fluctuations on the identification results.

[0012] The technical effects and advantages provided by the present invention in the above technical solution are as follows: The dynamically adjustable sliding window for inter-frame overlap rate adjusts the overlap rate based on the zero-crossing rate of the original audio signal. When the zero-crossing rate increases, the overlap rate decreases; when the zero-crossing rate decreases, the overlap rate increases. During periods of high zero-crossing rate and rapid signal change, a smaller overlap rate allows adjacent frames to contain more difference information, reducing inter-frame spectral redundancy and preserving transient acoustic variations. During periods of low zero-crossing rate and relatively flat signal, a larger overlap rate maintains high spectral line continuity during framing, facilitating stable extraction of sparse spectral lines. Thus, the framing process adaptively changes with the non-stationarity of the pump's operating state, avoiding feature ambiguity caused by a fixed overlap rate during rapid changes or insufficient temporal resolution during flat periods. This allows subsequent spectrum extraction and state switching location to more accurately reflect the actual operating characteristics of the pump. The frequency offset of sparse spectral lines in adjacent frames is used to determine the moment when the pump's operating state changes. Frame boundaries with frequency offsets exceeding a preset threshold are marked as state switching frame boundaries, and multi-frame audio signals are divided into several operating segments. Spectral line aggregation is performed on the sparse spectral lines within each operating segment to obtain the operating characteristic spectrum of that segment. The operating characteristic spectrum of the current operating segment is differentially analyzed point by point on the merged full-frequency reference set with the operating characteristic spectrum of the previous operating segment. Missing frequencies are filled with zero values ​​to obtain a differential spectrum sequence. The first-order differential change of the differential spectrum sequence is scanned in ascending frequency order, and the frequency points that change from positive to negative and from negative to positive are marked to extract the set of extreme point frequencies. Anomaly detection is performed only on the extreme point frequencies in the differential spectrum sequence, without directly utilizing the overall differential energy or average spectrum difference. In this way, the overall spectrum shift and energy change caused by pump start-up and shutdown, speed adjustment, load changes, etc., are separated as different state baselines after the operating segment is divided, and will not form a continuous large-scale difference in the differential of adjacent segments. The discrete frequency component changes caused by local abnormal sound patterns are manifested as local maxima or minima in the differential spectrum, which are easy to extract. This reduces false alarms caused by natural switching of operating states and enhances the response capability to abnormal sound pattern components with small amplitude and concentrated frequency domain location. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0014] Figure 1 This is a flowchart of an abnormal acoustic signature recognition and monitoring method based on the characteristic spectrum of water pump operation; Figure 2 This is a flowchart of the dynamic framing method for audio signals; Figure 3It is a curve showing the change of the zero-crossing rate of the original audio signal and the inter-frame overlap rate after adjustment over time; Figure 4 It is a schematic diagram of the frequency domain amplitude spectrum and sparse spectral lines of an audio signal frame; Figure 5 It is the curve of the overall frequency offset value of adjacent frames for the water pump operation status switching frame boundary detection. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] See Figure 1 This invention provides an abnormal voiceprint recognition and monitoring system based on the characteristic spectrum of water pump operation, including an audio acquisition module, a dynamic framing module, a spectrum extraction module, a status segmentation module, a feature spectrum aggregation module, and an abnormal recognition module.

[0017] The audio acquisition module acquires the raw audio signal during water pump operation. The dynamic framing module uses a sliding window with dynamically adjustable inter-frame overlap rate to segment the raw audio signal into multiple frames. The spectrum extraction module performs frequency band energy distribution calculation on each frame of audio signal, extracting the frequency points corresponding to the frequency band energy peaks to obtain the sparse spectrum lines of each frame. The state segmentation module determines the moment when the water pump's operating state changes based on the frequency offset of the sparse spectrum lines between adjacent frames, marking the frame boundaries where the frequency offset exceeds a preset threshold as state switching frame boundaries, and dividing the multi-frame audio signal into several operating segments based on these state switching frame boundaries. The feature spectrum aggregation module performs spectrum line aggregation on the sparse spectrum lines within each operating segment to obtain the operating feature spectrum of each operating segment. The anomaly detection module performs frequency point-by-frequency difference between the operating feature spectrum of the current operating segment and the operating feature spectrum of the previous operating segment to obtain a difference spectrum sequence, extracting the extreme point frequency point set from the difference spectrum sequence. If the extreme point frequency point set is not empty, it is used as the basis for anomaly voiceprint identification.

[0018] See Figure 2The dynamic framing module acquires the original audio signal. The original audio signal is input from the audio acquisition module, with a sampling rate set to 48 kHz. The dynamic framing module reads the amplitude values ​​of adjacent sampling points from the original audio signal and compares the signs of the amplitude values ​​of the preceding and following sampling points. A sign change is recorded when the amplitude value of the preceding sampling point is greater than zero and the amplitude value of the following sampling point is less than zero, or vice versa. The zero-crossing rate of the original audio signal is obtained by dividing the number of sign changes by the total number of adjacent sampling points. The total number of adjacent sampling points is the total number of sampling points in the original audio signal minus one.

[0019] In practice, the zero-crossing rate of the original audio signal is used to calculate the adjusted inter-frame overlap rate, which is calculated using the following formula: in, This indicates the adjusted inter-frame overlap rate. This represents the lower limit of the inter-frame overlap rate. This indicates the upper limit of the inter-frame overlap rate. This represents the lower limit of the zero-crossing rate. This represents the upper limit of the zero-crossing rate. Indicates the zero-crossing rate of the original audio signal. Lower limit of inter-frame overlap rate. Set to 0.25, the upper limit of inter-frame overlap rate. Set to 0.75. Lower limit for zero-crossing rate. Set to 0.01, upper limit of zero crossing rate. Set to 0.3. The lower limit of the zero-crossing rate is determined based on the zero-crossing rate range of the power frequency component and steady-state noise in the original audio signal during stable pump operation. The upper limit of the zero-crossing rate is determined based on the zero-crossing rate range of the transient impact component and high-frequency component in the original audio signal after the pump's operating state is switched. When the zero-crossing rate of the original audio signal... Less than the lower limit of the zero crossing rate At that time, the zero-crossing rate of the original audio signal Adjusted to the lower limit of the zero-crossing rate When the zero-crossing rate of the original audio signal Greater than the upper limit of zero crossing rate At that time, the zero-crossing rate of the original audio signal Adjusted to the upper limit of the zero-crossing rate Calculated using the above formula, as the zero-crossing rate increases, the adjusted inter-frame overlap rate... Decrease; when the zero-crossing rate decreases, the adjusted inter-frame overlap rate Increase.

[0020] The dynamic framing module uses an adjusted inter-frame overlap rate to perform sliding frame segmentation on the original audio signal. The sliding window length is set to 40 milliseconds. At a sampling rate of 48 kHz, the sliding window length corresponds to 1920 sampling points. The dynamic framing module then uses the adjusted inter-frame overlap rate... Calculate the frame shift length, which is equal to the sliding window length multiplied by 1 minus the adjusted inter-frame overlap rate. The difference is calculated as follows: The initial starting point of the sliding window is set to the first sample point of the original audio signal, and the initial ending point is the starting point plus the sliding window length minus 1. The sliding window extracts sample points from the original audio signal between the starting and ending points, forming one frame of audio signal. Subsequently, the starting point of the sliding window is increased by the number of sample points corresponding to the frame shift length, and the ending point is increased accordingly, continuing to extract the next frame of audio signal. When the ending point of the sliding window exceeds the total number of sample points of the original audio signal, the sliding frame division stops.

[0021] In practice, the start and end times of each audio frame are recorded. The start time is calculated by subtracting 1 from the starting sampling point number of the sliding window and dividing by the sampling rate. The end time is calculated by subtracting 1 from the ending sampling point number of the sliding window and dividing by the sampling rate. The sampling rate is set to 48000 Hz, and the units for the start and end times are seconds. The start and end times of each audio frame serve as the time-domain localization basis for subsequent spectrum analysis.

[0022] See Figure 3 The figure shows the relationship between the zero-crossing rate of the original audio signal acquired by the dynamic framing module in the above embodiment and the corresponding adjusted inter-frame overlap rate over time. The horizontal axis represents the sampling time in seconds, ranging from 0 to 10 seconds. The left vertical axis represents the zero-crossing rate, represented by a solid blue line, ranging from 0 to 0.3. The right vertical axis represents the adjusted inter-frame overlap rate, represented by a dashed red line, ranging from approximately 0.3 to 0.75.

[0023] As can be seen from the figure, during the period from 0 seconds to about 5 seconds, the zero-crossing rate of the original audio signal remains at a low level, about 0.02, and the corresponding adjusted inter-frame overlap rate is near the upper limit, about 0.75, which meets the inter-frame overlap rate calculation rule shown in formula (1) of the dynamic framing module: the inter-frame overlap rate is higher when the zero-crossing rate is low. Subsequently, the zero-crossing rate increases rapidly from 5 seconds, reaching a maximum of about 0.28, and the adjusted inter-frame overlap rate decreases significantly, dropping to a minimum of about 0.3, which is consistent with the design logic of the formula that the inter-frame overlap rate decreases when the zero-crossing rate increases. The peak range of the zero-crossing rate is between about 6 seconds and 8 seconds, during which the inter-frame overlap rate remains at a low level.

[0024] After 8 seconds, the zero-crossing rate rapidly dropped back to a lower level, and the adjusted inter-frame overlap rate correspondingly rose to around 0.75. This trend verifies that the dynamic framing module dynamically adjusts the inter-frame overlap rate based on the zero-crossing rate of the original audio signal, ensuring that the sliding window overlap rate is adaptively adjusted according to the frequency characteristics of the audio signal, effectively balancing time resolution and frequency resolution.

[0025] In practice, the spectrum extraction module performs a Fourier transform on each frame of the audio signal. One frame of the audio signal is output by the dynamic framing module, with a total of 1920 sampling points and a sampling rate set to 48000 Hz. The Fourier transform uses the discrete Fourier transform form, and the expression for the discrete Fourier transform of one frame of the audio signal is: in, This indicates that a frame of audio signal is in the 1st... Complex spectral coefficients at each frequency point For frequency point index, Take 0 to integers, This represents the total number of sampling points in a frame of audio signal. The value is 1920. For time-domain sampling point index, Take 0 to integers, Represents the first [frame] of an audio signal. The amplitude value of each sampling point is the base of the natural logarithm. The imaginary unit, In Pi is the mathematical constant of a circle.

[0026] The frequency domain amplitude spectrum is obtained by taking the modulus of the complex spectral coefficients of a frame of audio signal. The first... The amplitude value corresponding to each frequency point is Frequency index The corresponding actual frequency is the frequency point index. Multiply by the sampling rate and divide by the total number of sampling points The sampling rate is 48000 Hz, therefore the frequency index is... The corresponding frequency interval is 25 Hz.

[0027] Scan the frequency domain amplitude spectrum in ascending order of frequency. Start scanning from frequency index 1 to frequency index 2. For the first Calculate the frequency point. The amplitude value corresponding to the frequency point and the first The difference between the amplitude values ​​corresponding to each frequency point is used as the first frequency point. The amplitude difference at each frequency point. After calculating the amplitude difference for each frequency point, determine the position where the amplitude difference changes from positive to negative. The amplitude difference of each frequency point is greater than zero and the i-th When the amplitude difference of the nth frequency point is less than zero, the nth frequency point will be... The 0th frequency point is marked as a local peak frequency point. Individual frequency points are not included in the local peak frequency point marking.

[0028] Using the frequency values ​​of local peak frequencies as the frequency coordinates of sparse spectral lines, and the amplitude values ​​of the frequency domain amplitude spectrum of the local peak frequencies as the amplitude coordinates of the sparse spectral lines, a set of sparse spectral lines for a frame of audio signal is formed. The frequency coordinate is calculated by multiplying the frequency index of the local peak frequency point by the sampling rate and dividing by the total number of sampling points, i.e., multiplying the frequency index of the local peak frequency point by 25 Hz. The amplitude coordinate is the amplitude value corresponding to the local peak frequency point in the frequency domain amplitude spectrum. All the sparse spectral lines formed by the local peak frequency points together constitute the sparse spectral line set of a frame of audio signal.

[0029] See Figure 4 In the graph, the horizontal axis represents frequency in Hertz, ranging from 0 Hertz to approximately 13,000 Hertz; the vertical axis represents amplitude, ranging from 0 to approximately 1.5. The blue solid line in the graph represents the frequency domain amplitude spectrum of an audio signal frame, and the red dots represent sparse spectral lines identified by the spectrum extraction module, i.e., local peak frequency points in the frequency domain amplitude spectrum.

[0030] As shown in the figure, the frequency domain amplitude spectrum exhibits a significant amplitude peak in the low-frequency range, especially near 0 Hz, where the highest amplitude peak is approximately 1.45. The amplitude then decreases rapidly, dropping to between approximately 0.4 and 0.3 around 500 Hz. Within the frequency range of 500 Hz to 2000 Hz, the frequency domain amplitude spectrum shows multiple local peaks, accurately marked by sparse red spectral lines. Above 2000 Hz, the overall amplitude of the frequency domain amplitude spectrum tends to stabilize and remain low, below 0.1. The scattered sparse red spectral lines indicate the presence of a few local peaks in this frequency band.

[0031] In practical implementation, the state segmentation module obtains the sparse spectrum line set of two adjacent frames. The two adjacent frames are the previous frame and the next frame, respectively. The sparse spectrum line set of the previous frame is obtained by processing the audio signal of the previous frame through the spectrum extraction module, and the sparse spectrum line set of the next frame is obtained by processing the audio signal of the next frame through the spectrum extraction module.

[0032] The state segmentation module performs frequency point matching on the sparse spectrum line set of the previous frame and the sparse spectrum line set of the next frame. For each sparse spectrum line in the previous frame's sparse spectrum line set, its frequency coordinates are obtained. In the sparse spectrum line set of the next frame, the sparse spectrum line with the smallest absolute value of the difference between its frequency coordinates and those of the sparse spectrum line in the previous frame is searched. Specifically, the search method is as follows: traverse each sparse spectrum line in the sparse spectrum line set of the next frame, calculate the absolute value of the difference between the frequency coordinates of the sparse spectrum line in the next frame and those of the sparse spectrum line in the previous frame, and record the sparse spectrum line in the next frame with the smallest absolute value of the difference as the pairing target. The sparse spectrum line in the previous frame and the sparse spectrum line in the next frame, which is the pairing target, are combined to form a pairing frequency point group.

[0033] The frequency offset of a paired frequency point group is defined as the difference between the frequency coordinates of the sparse spectrum lines in the previous frame and the frequency coordinates of the sparse spectrum lines in the next frame. The frequency offset can be positive or negative. A positive value indicates that the frequency coordinates of the sparse spectrum lines in the next frame are higher than those in the previous frame, while a negative value indicates that the frequency coordinates of the sparse spectrum lines in the next frame are lower than those in the previous frame.

[0034] Sum the absolute values ​​of the frequency offsets of all paired frequency point groups, and count the total number of paired frequency point groups. Divide the sum of the absolute values ​​of the frequency offsets by the total number of paired frequency point groups to obtain the overall offset value between adjacent frames. The overall offset value between adjacent frames is calculated as follows: in, This represents the overall offset value between adjacent frames. This indicates the total number of paired frequency point groups. For the sequence number index of the paired frequency point group, Take 1 to integers, Indicates the first Frequency offset of a pair of frequency points Indicates the first The absolute value of the frequency offset of each paired frequency group. When the total number of paired frequency groups... When it is 0, the overall offset value between adjacent frames Set it directly to 0.

[0035] In practice, the preset threshold is set to 10 Hz. The basis for setting the preset threshold to 10 Hz is as follows: when the pump's operating state changes, the frequency position of the power frequency harmonic components in adjacent frames shifts, and the average frequency offset of paired sparse spectral lines exceeds 10 Hz; when the pump's operating state does not change, the average frequency offset of sparse spectral lines in adjacent frames does not exceed 10 Hz. When the overall offset value of adjacent frames is greater than 10 Hz, the frame boundary between two adjacent frames is marked as a state switching frame boundary. When the overall offset value of adjacent frames is less than or equal to 10 Hz, the frame boundary between two adjacent frames is not marked as a state switching frame boundary.

[0036] The state segmentation module uses state transition frame boundaries as dividing points, merging consecutive frames between two adjacent state transition frame boundaries in a multi-frame audio signal into a single running segment. The multi-frame audio signal is arranged chronologically, with each frame having a start and end time. The state segmentation module scans all frame boundaries chronologically, using state transition frame boundaries as dividing positions. If multiple state transition frame boundaries exist, the first running segment begins with the first frame of the multi-frame audio signal and ends with the frame preceding the first state transition frame boundary; the second running segment begins with the frame following the first state transition frame boundary and ends with the frame preceding the second state transition frame boundary; subsequent running segments follow the same pattern; the last running segment begins with the frame following the last state transition frame boundary and ends with the last frame of the multi-frame audio signal. If no state transition frame boundaries exist, all frames in the multi-frame audio signal are merged into a single running segment.

[0037] In practical implementation, the state segmentation module aligns the sparse spectral lines of all frames within a runtime segment according to frequency coordinates. First, it collects all frequency coordinates appearing in the sparse spectral lines of all frames within the runtime segment, forming a frequency coordinate set. For each frequency coordinate in the frequency coordinate set, it searches for the amplitude value at that frequency coordinate in the sparse spectral line set of each frame within the runtime segment. If a frequency coordinate does not exist in the sparse spectral line set of a certain frame, the amplitude value at that frequency coordinate for that frame is recorded as 0. For each frequency coordinate in the frequency coordinate set, it compares the maximum amplitude values ​​of all frames within the runtime segment at that frequency coordinate, and takes the maximum value as the aggregated amplitude value for that frequency coordinate.

[0038] The aggregated spectrum line of the running segment is constructed using all frequency coordinates and their corresponding aggregated amplitude values. The aggregated spectrum line serves as the initial estimate of the running characteristic spectrum. Each spectrum line in the aggregated spectrum line consists of a frequency coordinate and an aggregated amplitude value. The frequency coordinate is derived from the set of frequency coordinates, and the aggregated amplitude value is the maximum amplitude value at that frequency coordinate.

[0039] See Figure 5 In the graph, the horizontal axis represents the frame boundary number, and the vertical axis represents the overall offset value between adjacent frames, in Hertz. The blue solid line curve shows the trend of the overall offset value between adjacent frames, and the red dashed line represents the preset threshold of 10 Hertz, used to determine whether the water pump's operating status has changed. The green triangles in the graph mark the state switching frame boundaries.

[0040] As shown in the figure, the overall offset value of most frame boundaries remains at a low level, far below the preset threshold of 10 Hz, indicating that the water pump is operating stably without significant frequency shift. Two significant peaks appear at approximately frames 350 and 750, respectively, where the overall offset value rises sharply, far exceeding the threshold, with the highest peak approaching 25 Hz. The corresponding green triangle markers clearly indicate that these two peaks are identified as state transition frame boundaries.

[0041] According to the working principle of the state segmentation module in the above embodiment, when the overall offset value of adjacent frames exceeds a preset threshold of 10 Hz, it is considered that the water pump's operating state has changed, and the frame boundary is marked as the state switching frame boundary. The two peak positions in the figure represent this type of determination, indicating that the water pump's operating state has switched at the boundary between these two frames.

[0042] The overall offset value in the figure fluctuates little in the non-state switching range, and the value is stable and below the threshold. This reflects that the frequency offset of the sparse spectrum lines is small during the stable operation of the water pump, and the frequency matching offset does not exceed the threshold range.

[0043] In practice, the feature spectrum aggregation module acquires the sparse spectrum lines of all frames within the running segment. The running segment is output by the state segmentation module and contains several frames of audio signals. The sparse spectrum lines of each frame of audio signal are output by the spectrum extraction module, and each sparse spectrum line contains frequency and amplitude coordinates.

[0044] The feature spectrum aggregation module merges the frequency coordinates of sparse spectral lines from all frames within the running segment into a single frequency set. This frequency set is a deduplicated collection of all frequency coordinates that have appeared in the sparse spectral lines of all frames within the running segment; each frequency coordinate appears only once. The frequency values ​​in the frequency set are then sorted in ascending order to obtain the sorted frequency set.

[0045] In some embodiments, the feature spectrum aggregation module traverses the sorted frequency set. Starting from the first frequency value in the sorted frequency set, it sequentially compares the frequency intervals of two adjacent frequency values. The frequency interval is the difference between the previous and subsequent frequency values ​​in the sorted frequency set. The threshold is set to 5 Hz. The basis for setting the threshold to 5 Hz is: the frequency coordinates of sparse spectral lines come from the local peak frequency points of the frequency domain amplitude spectrum, and the frequency interval between adjacent frequency point indices in the frequency domain amplitude spectrum is 25 Hz; the frequency interval between adjacent harmonic components in the pump operation feature spectrum is greater than 10 Hz, while the frequency deviation of the same harmonic component due to framing processing and frequency point truncation does not exceed 5 Hz, therefore the threshold is set to 5 Hz.

[0046] When the frequency interval between two adjacent frequency values ​​is less than 5 Hz, the two adjacent frequency values ​​are merged into the same frequency band cluster. The merging process is as follows: Maintain the current frequency band cluster, which already contains several frequency values. Take a frequency value from the sorted frequency set and determine whether the frequency interval between this frequency value and the last frequency value in the current frequency band cluster is less than 5 Hz. If the frequency interval is less than 5 Hz, add the frequency value to the current frequency band cluster. If the frequency interval is greater than or equal to 5 Hz, end the current frequency band cluster and use this frequency value as the first frequency value of the new frequency band cluster. Repeat the above process until all frequency values ​​in the sorted frequency set have been traversed.

[0047] In practice, for each frequency band cluster, the weighted average of all frequency values ​​within the cluster is calculated as the representative frequency of that cluster. The weight of a frequency value within a frequency band cluster is the average of the amplitude values ​​of all sparse spectral lines corresponding to that frequency value. Specifically, for a frequency value within a frequency band cluster, the sparse spectral line corresponding to that frequency value is found among the sparse spectral lines of all frames within the running segment. The amplitude values ​​of all sparse spectral lines corresponding to that frequency value are obtained, and the arithmetic mean of these amplitude values ​​is calculated as the weight of that frequency value. The formula for calculating the representative frequency of a frequency band cluster is as follows: in, Indicates the first The representative frequency of each frequency band cluster This is the index of the frequency band cluster. Take 1 to integers, This indicates the total number of frequency band clusters. Indicates the first The number of frequency values ​​within a frequency band cluster For the first The index of the frequency value within a frequency band cluster Take 1 to integers, Indicates the first Within the frequency band cluster, the first The average of the amplitude values ​​of all sparse spectral lines corresponding to each frequency value. Indicates the first Within the frequency band cluster, the first Frequency coordinates of each frequency value.

[0048] The average amplitude of all sparse spectral lines within a frequency band cluster is calculated as the representative amplitude of that frequency band cluster. Specifically, for each frequency value within the frequency band cluster, the amplitude values ​​of all sparse spectral lines corresponding to that frequency value in all frames within the running segment are obtained. All sparse spectral line amplitude values ​​corresponding to all frequency values ​​are collected into an amplitude set, and the arithmetic mean of all amplitude values ​​in the amplitude set is calculated to obtain the representative amplitude of that frequency band cluster.

[0049] The operational characteristic spectrum of the operational segment is constructed using the representative frequencies and representative amplitudes of all frequency band clusters. Each spectral line in the operational characteristic spectrum consists of the representative frequency and representative amplitude of a frequency band cluster, with the representative frequency serving as the frequency coordinate and the representative amplitude serving as the amplitude coordinate.

[0050] In practice, the anomaly detection module acquires the operational characteristic spectrum of the current running segment and the operational characteristic spectrum of the previous running segment. The operational characteristic spectrum of the current running segment is output by the characteristic spectrum aggregation module, as is the operational characteristic spectrum of the previous running segment. The operational characteristic spectrum of the current running segment contains multiple spectral lines, each consisting of a representative frequency and a representative amplitude. The operational characteristic spectrum of the previous running segment also contains multiple spectral lines, each consisting of a representative frequency and a representative amplitude.

[0051] The anomaly detection module merges all representative frequencies from the current running segment's runtime characteristic spectrum with all representative frequencies from the previous running segment's runtime characteristic spectrum to obtain a full frequency reference set. Each frequency value in the full frequency reference set appears only once. The frequency values ​​in the full frequency reference set are then sorted in ascending order.

[0052] In some embodiments, the anomaly detection module sets a frequency tolerance threshold of 2.5 Hz. The basis for setting the frequency tolerance threshold to 2.5 Hz is that the representative frequency of the operating characteristic spectrum is obtained by the weighted average of frequency values ​​within a frequency band cluster; when merging frequency band clusters, the interval between adjacent frequency values ​​is less than 5 Hz; and the deviation between the representative frequency and the true frequency value within the frequency band cluster does not exceed half of the frequency band cluster merging interval, i.e., not exceeding 2.5 Hz. Each frequency value in the full frequency reference set is traversed as a target frequency value.

[0053] For a target frequency value in the full-frequency reference set, a representative frequency is searched in the operational characteristic spectrum of the current operational segment. If there is a representative frequency in the operational characteristic spectrum of the current operational segment whose absolute value of the difference between the representative frequency and the target frequency value is less than 2.5 Hz, then the representative frequency is determined to be the effective amplitude at the target frequency value, and the representative amplitude corresponding to the representative frequency is taken as the current amplitude value. If there are multiple representative frequencies in the operational characteristic spectrum of the current operational segment whose absolute value of the difference between the representative frequency and the target frequency value is less than 2.5 Hz, the representative amplitude corresponding to the representative frequency with the smallest absolute value of the difference between the representative frequency and the target frequency value is taken as the current amplitude value. If there is no representative frequency in the operational characteristic spectrum of the current operational segment whose absolute value of the difference between the representative frequency and the target frequency value is less than 2.5 Hz, then zero is taken as the current amplitude value.

[0054] For the same target frequency value, a representative frequency is searched in the operating characteristic spectrum of the previous operating segment. If there is a representative frequency in the operating characteristic spectrum of the previous operating segment whose absolute value of the difference between the representative frequency and the target frequency value is less than 2.5 Hz, then the representative frequency is determined to be the effective amplitude at the target frequency value, and the representative amplitude corresponding to the representative frequency is taken as the previous amplitude value. If there are multiple representative frequencies in the operating characteristic spectrum of the previous operating segment whose absolute value of the difference between the representative frequency and the target frequency value is less than 2.5 Hz, the representative amplitude corresponding to the representative frequency with the smallest absolute value of the difference between the representative frequency and the target frequency value is taken as the previous amplitude value. If there is no representative frequency in the operating characteristic spectrum of the previous operating segment whose absolute value of the difference between the representative frequency and the target frequency value is less than 2.5 Hz, then zero is taken as the previous amplitude value.

[0055] Subtract the previous amplitude value from the current amplitude value at the same target frequency to obtain the difference value for that target frequency. Perform this process for each target frequency value in the full frequency reference set to obtain the difference value for each target frequency value. Arrange all difference values ​​in ascending order of frequency values ​​in the full frequency reference set to form a differential spectrum sequence. Each element in the differential spectrum sequence corresponds to a frequency value and a difference value.

[0056] In practical implementation, the anomaly detection module scans the differential spectrum sequence in ascending frequency order, sequentially calculating the difference between the difference value of each frequency point and the difference value of the previous frequency point, thus obtaining the first-order differential change. The number of frequency points in the differential spectrum sequence is denoted as... The first-order difference change at the first frequency point is set to 0. For the... The formula for calculating the first-order difference change at each frequency point is as follows: in, Represents the first digit in the differential spectral sequence. The first-order difference change at each frequency point Represents the first digit in the differential spectral sequence. The difference value at each frequency point Represents the first digit in the differential spectral sequence. The difference value at each frequency point This is the index of the frequency point in the differential spectrum sequence. Take 2 to integers, This represents the total number of frequency points in the differential spectrum sequence.

[0057] From the second frequency point to the... After the first frequency point is reached, the anomaly detection module sequentially judges the sign change of the first-order differential change at each frequency point. If the first frequency point is reached... The first-order difference change at the nth frequency point is greater than 0 and the i-th If the first-order difference change at the nth frequency point is less than 0, then the nth frequency point will be... The nth frequency point is marked as a local maximum frequency point of the differential spectrum. If the nth frequency point is a local maximum frequency point of the differential spectrum, then the nth frequency point is a local maximum frequency point of the differential spectrum. The first-order difference change at the nth frequency point is less than 0 and the i-th If the first-order difference change at the nth frequency point is greater than 0, then the nth frequency point will be... The first frequency point is marked as a local minimum frequency point in the differential spectrum. These frequency points are not included in the marking of local maxima and local minima.

[0058] The frequency values ​​of the local maxima and local minima of the differential spectrum are combined to obtain the extreme point frequency set. Each frequency value in the extreme point frequency set comes from the frequency value of the corresponding frequency point in the differential spectrum sequence. If the extreme point frequency set is not empty, it is used as the basis for abnormal voiceprint identification.

[0059] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An abnormal voiceprint recognition and monitoring system based on the characteristic spectrum of water pump operation, characterized in that, include: The audio acquisition module collects the raw audio signals during the operation of the water pump; The dynamic framing module uses a sliding window with dynamically adjustable inter-frame overlap rate to perform frame segmentation on the original audio signal, resulting in multi-frame audio signals. The spectrum extraction module performs frequency band energy distribution calculations on each frame of audio signal, extracts the frequency points corresponding to the frequency band energy peaks, and obtains the sparse spectrum lines of each frame of audio signal; The state segmentation module determines the moment when the pump's operating state changes based on the frequency offset of sparse spectral lines between adjacent frames. It marks the frame boundaries in adjacent frames where the frequency offset exceeds a preset threshold as state switching frame boundaries and divides the multi-frame audio signal into several operating segments based on the state switching frame boundaries. The feature spectrum aggregation module performs spectrum line aggregation on the sparse spectrum lines within each running segment to obtain the running feature spectrum of each running segment; The anomaly detection module performs frequency-point differential analysis between the current running segment's running feature spectrum and the previous running segment's running feature spectrum to obtain a differential spectrum sequence. It then extracts the set of extreme point frequencies from the differential spectrum sequence. If the set of extreme point frequencies is not empty, it uses the set of extreme point frequencies as the basis for anomaly voiceprint identification.

2. The abnormal acoustic signature recognition and monitoring system based on the characteristic spectrum of water pump operation according to claim 1, characterized in that, The dynamically adjustable inter-frame overlap rate sliding window performs frame segmentation processing on the original audio signal, specifically including: Obtain the zero-crossing rate of the original audio signal, and adjust the inter-frame overlap rate of the sliding window according to the value of the zero-crossing rate. When the zero-crossing rate increases, decrease the inter-frame overlap rate; when the zero-crossing rate decreases, increase the inter-frame overlap rate. The original audio signal is divided into multiple frames by sliding frames using the adjusted inter-frame overlap rate. Record the start and end times of each frame of audio signal as the basis for time-domain localization in subsequent spectrum analysis.

3. The abnormal acoustic signature recognition and monitoring system based on the characteristic spectrum of water pump operation according to claim 1, characterized in that, The step of performing frequency band energy distribution calculation on each frame of audio signal, extracting the frequency points corresponding to the frequency band energy peaks, and obtaining the sparse spectrum lines of each frame of audio signal specifically includes: Perform a Fourier transform on each frame of audio signal to obtain the frequency domain amplitude spectrum of that frame; Scan the frequency domain amplitude spectrum in ascending order of frequency, calculate the amplitude difference between each frequency point and the previous frequency point in turn, and mark the frequency point where the amplitude difference changes from positive to negative as the local peak frequency point. The sparse spectrum line set of the frame is formed by using the frequency value of the local peak frequency point as the frequency coordinate of the sparse spectrum line and the amplitude value corresponding to the local peak frequency point as the amplitude coordinate of the sparse spectrum line.

4. The abnormal acoustic signature recognition and monitoring system based on the characteristic spectrum of water pump operation according to claim 1, characterized in that, The step of determining the moment when the pump's operating state changes based on the frequency offset of sparse spectral lines between adjacent frames specifically includes: Frequency point matching is performed on the sparse spectrum line sets of two adjacent frames. Each sparse spectrum line in the previous frame is paired with the sparse spectrum line with the smallest frequency difference in the next frame, and the frequency difference of the paired frequency points is calculated as the frequency offset. The sum of the absolute values ​​of the frequency offsets of all paired frequency points is used as the overall offset value between adjacent frames; When the overall offset value is greater than the preset threshold, the frame boundary between two adjacent frames is marked as the state switching frame boundary.

5. The abnormal acoustic signature recognition and monitoring system based on the characteristic spectrum of water pump operation according to claim 1, characterized in that, The method of dividing multi-frame audio signals into several running segments based on state switching frame boundaries specifically includes: Using the state switching frame boundary as the dividing point, consecutive frames between two adjacent state switching frame boundaries in a multi-frame audio signal are merged into a single running segment. Align the sparse spectral lines of all frames within the running segment according to the frequency coordinates, and take the maximum amplitude value at the same frequency coordinate as the aggregate amplitude value of that frequency point. The aggregated spectrum line of the running segment is constructed using all aggregated amplitude values ​​and their corresponding frequency coordinates, serving as the initial estimate of the running characteristic spectrum.

6. The abnormal acoustic signature recognition and monitoring system based on the characteristic spectrum of water pump operation according to claim 1, characterized in that, The step of performing spectral line aggregation on the sparse spectral lines within each running segment to obtain the running characteristic spectrum of each running segment specifically includes: The frequency coordinates of sparse spectral lines of all frames within the running segment are merged into a frequency set, and the frequency values ​​in the frequency set are sorted from smallest to largest. Traverse the sorted frequency set, merge adjacent frequency values ​​with a frequency interval less than a set threshold into the same frequency band cluster, use the weighted average of all frequency values ​​in the frequency band cluster as the representative frequency of the frequency band cluster, and use the average of the amplitude values ​​of all sparse spectral lines in the frequency band cluster as the representative amplitude of the frequency band cluster. The operating characteristic spectrum of this operating segment is constructed using the representative frequencies and representative amplitudes of all frequency band clusters.

7. The abnormal acoustic signature recognition and monitoring system based on the characteristic spectrum of water pump operation according to claim 1, characterized in that, The step of performing frequency-point-by-frequency difference between the operating characteristic spectrum of the current operating segment and the operating characteristic spectrum of the previous operating segment to obtain a difference spectrum sequence specifically includes: The representative frequencies in the operating characteristic spectrum of the current running segment and the previous running segment are merged to obtain a full frequency reference set; For each frequency value in the full frequency reference set, if a representative amplitude of that frequency value exists in the operating characteristic spectrum of the current running segment, then that representative amplitude is taken as the current amplitude value; otherwise, zero is taken as the current amplitude value. For each frequency value in the full frequency reference set, if a representative amplitude of that frequency value exists in the operating characteristic spectrum of the previous running segment, then that representative amplitude is taken as the previous amplitude value; otherwise, zero is taken as the previous amplitude value. Subtract the previous amplitude value from the current amplitude value at each frequency value to obtain the difference value of that frequency value. All difference values ​​are arranged in frequency order to form a differential spectrum sequence.

8. The abnormal acoustic signature recognition and monitoring system based on the characteristic spectrum of water pump operation according to claim 1, characterized in that, The extraction of the extreme point frequency set from the differential spectrum sequence specifically includes: Scan the differential spectrum sequence in ascending order of frequency, and calculate the difference between the differential value of each frequency point and the differential value of the previous frequency point in turn to obtain the first-order differential change. The frequency points where the first-order difference change changes from positive to negative are the local maxima of the difference spectrum, and the frequency points where the first-order difference change changes from negative to positive are the local minima of the difference spectrum. By merging the frequency values ​​of local maxima and local minima, we obtain the set of extreme point frequencies.

9. The abnormal acoustic signature recognition and monitoring system based on the characteristic spectrum of water pump operation according to claim 4, characterized in that, The overall offset value is the ratio of the sum of the absolute values ​​of the frequency offsets of all paired frequency points to the total number of pairs. The average offset is used as the basis for determining whether the pump's operating status has changed.

10. The abnormal acoustic signature recognition and monitoring system based on the characteristic spectrum of water pump operation according to claim 7, characterized in that, For each frequency value in the full frequency reference set, if the difference between the representative frequency and the target frequency value in the operating characteristic spectrum of the current or previous operating segment is less than the preset frequency tolerance threshold, then the representative frequency is determined to be the effective amplitude of the target frequency value.

Images