Method, system and device for automatically extracting time-frequency characteristics of marine mammal whistle signals

Abstract

The invention relates to a method, a system and a device for automatically extracting time-frequency characteristics of marine mammal whistle signals, and belongs to the technical field of signal processing, and the method comprises the steps of noise reduction preprocessing, signal time-frequency spectrogram calculation, manual point selection and positioning, and candidate contour set acquisition. Performing contour trend judgment and straight line fitting; extrapolating and correcting the starting point and the ending point; the invention further provides a system and a device for operating the method. The method has better anti-interference capability, and the accuracy of contour extraction is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of signal processing, and particularly relates to an automatic extraction method, system and device for the time-frequency characteristics of marine mammal whistle signals. Background Art

[0002] Whistle signals are important social signals of marine mammals and an important way for them to communicate and express emotions among individuals or groups. The typical characteristics of such signals are narrowband, long-duration and frequency modulation: the duration is usually between dozens of milliseconds and several seconds, the frequency range varies from dozens of hertz to hundreds of kilohertz, and multiple harmonic components are often accompanied. After Fourier transform, the whistle signal can be decomposed into the fundamental frequency and harmonics. The fundamental frequency refers to the lowest frequency component obtained by decomposition, while the harmonics refer to other components, which are integers of the fundamental frequency. In the time-frequency spectrum, the signal usually appears as one or more contour curves, where the curve with the lowest frequency corresponds to the fundamental frequency component, and the rest are harmonic components.

[0003] The feature extraction of whistle signals is the key to using underwater acoustics to monitor marine mammals. It can not only improve the accuracy of population identification, but also be the only basis for individual identification through voiceprints. Existing extraction methods usually target the fundamental frequency, ignoring harmonic information; at the same time, they have high requirements for the signal-to-noise ratio of the signal, and their anti-interference ability and robustness are poor. These limitations seriously restrict the development and application of underwater acoustic monitoring technology for marine mammals. Summary of the Invention

[0004] The present method innovatively adopts a technical solution of trend fitting. There is often noise in the marine environment, which not only leads to a low signal-to-noise ratio, but also interferes with the extraction of the target signal. Conventional energy maximum values will cause deviations, while trend fitting can have better effects. Specifically, it includes: First, on the time-frequency diagram, manually select the start and end points of the contour line (appropriately increase the selected points according to the prompts during the fitting process); secondly, locate the manually selected points; then, automatically obtain the contour curve according to the contour trend and the fitted straight line; finally, determine the start and end points of the contour according to the contour curve and save the contour parameters.

[0005] The present invention is generally implemented through the following steps: An automatic extraction method for the time-frequency characteristics of marine mammal whistle signals, the method comprising the following steps: Step 1, noise reduction preprocessing: perform low-pass filtering, median filtering, mean filtering, etc. on the data to reduce the interference of noise and interference signals on the target signal; Step 2, calculate the time-frequency spectrum diagram of the signal; for a segment of data y[e] containing the whistle signal, , Let \(N\) be the number of data points. Using the short-time Fourier transform, the time-frequency spectrogram of this segment of data is obtained; Step 3: Manually select points and locate them; for each contour curve on the time-frequency spectrogram, manually assist in selecting points separately. The minimum number of selected points is two, and the positions need to include the starting point, the ending point, and the places where the contour trend changes significantly; locate and obtain the coordinate values of the \(N\) manually selected points on the time-frequency spectrogram. The coordinate values include time \(t\), frequency \(f\), and power spectrum \(E\); Step 4: Obtain the candidate contour set; select the power extreme points at each moment on the time-frequency diagram to form the candidate contour set , is the number of extreme points at time \(t_i\), is the frequency coordinate value of the \(j\)-th power extreme point at time \(t_i\) on the time-frequency diagram, is the power spectrum coordinate value of the \(j\)-th power extreme point at time \(t_i\) on the time-frequency diagram; Step 5: Contour trend judgment and linear fitting; among the \(N\) manually selected points, every two adjacent points form a fitting interval. Perform trend judgment and linear fitting in the \(N - 1\) intervals to obtain the set of fitting contour points in the interval, denoted as , and finally obtain the time and frequency information of the total contour; Step 6: Extrapolation and correction of the starting point and the ending point: For the starting point , if the difference between the frequency coordinate of the second manually selected point and the frequency of the first selected point is less than \(0\), for the extreme points ( , , ) on , where \(\Delta t\) is the step size of the abscissa on the time-frequency diagram, remove the extreme points of and from the candidate contour set; if is greater than \(0\), then remove the extreme points of and from the candidate contour set and update ; if there exists and at the previous time point , update the contour set ; For the ending point , if is less than \(0\), for the extreme points on ( , , ) , remove , The extreme points of are removed from the candidate contour set; if , the extreme points of are removed from the candidate contour set, and and are updated; if there exist at the next time point. Update the contour set .

[0006] Furthermore, for the automatic adjustment of the manually selected points in step 3), it is ensured that the selected points correspond to the coordinate points on the time-frequency diagram. The specific method is as follows: Determine the time in that is closest to , ; Determine the frequency in the corresponding f at time that is closest to , ; Update all the points in this way to obtain , and the coordinates of the time-frequency spectrum diagram correspond to , , , where is the sampling frequency, and is the number of Fourier transform points ① Obtain the difference array of the i-th manually selected point , = 1,..., N , corresponding to the two points ; is judged as an upward trend, otherwise it is a downward trend; ② According to the trend and the manually selected points, screen the points in the candidate contour set: Taking as an example, for the candidate contour points at time ( , , ), , if it is an upward trend, then remove the points and from the candidate contour set; for the points with a downward trend, remove the points and from the candidate contour set, and update , is the number of extreme points at time ③ Select the next extrapolated contour point of the manually selected points except the last one in the candidate contour set: Search for the candidate contour point with the maximum power at a certain moment , and judge whether it is within the range; if so, determine as the contour point; if not, within the range, search for the candidate point closest to ; if none, double it, and search in the above - mentioned way until the first extrapolated contour point is selected at the moment , and construct the contour set as , where q = 1; ④ According to the existing contour set fit the remaining contour points cyclically, being the time of the last point in the contour set; if , first, screen the candidate contour set , ; for the points with an upward trend, remove the points with frequencies lower than and higher than from the candidate contour set; for the downward trend, remove the points with frequencies higher than and lower than from the candidate contour set; then, perform linear fitting based on the existing contour points to obtain the frequency ; if there are no candidate contour points at the moment, set the fitting point as the contour point ; if there are candidate contour points, find the point with the maximum power among them ; if , , then determine as the contour point ; if , search for the candidate contour point whose distance to the fitting line is less than and satisfies the change trend at this moment, that is and , if it is a decreasing trend, here it is ; , determine as the contour point ; if and no candidate contour point whose distance to the fitting line is less than and satisfies the change trend can be found, then delete the previous contour point , and this point will be marked as a green dot on the time-frequency diagram. If there is only at this time, then at time, reselect the first extrapolated contour point according to step ③ and denote it as . Search for contour points on according to step ④; if there are more than two contour points in the contour set at this time, then search for contour points on according to step ④ . Update the contour set to or or ; If , update the contour set to . If , then update the contour set to according to step ③; if , then jump to the next step.

[0007] Further, in step 6) described above, if there are points deleted during the fitting process on the time-frequency diagram, it is necessary to determine whether additional manual point selection is required. If so, add manual points at the positions where the fitting contour deviates, and then process from step 3) again.

[0008] An automatic extraction system for the time-frequency characteristics of marine mammal whistle signals, the system includes a data input module, a data processing module, a module for calculating the time-frequency spectrogram of the signal, a module for manual point selection and positioning, a module for obtaining a candidate contour set, a module for contour trend judgment and linear fitting, a module for extrapolation and correction of the starting and ending points, and a result output module. The modules are sequentially connected by data communication; The data processing module runs step 1 described above; The module for calculating the time-frequency spectrogram of the signal runs step 2 described above; The module for manual point selection and positioning runs step 3 described above; The module for obtaining a candidate contour set runs step 4 described above; The module for contour trend judgment and linear fitting runs step 5 described above; The module for extrapolation and correction of the starting and ending points runs step 6 described above.

[0009] An automatic extraction device for the time-frequency characteristics of marine mammal whistle signals, the device is equipped with the system described above.

[0010] The beneficial effects of the present invention compared with the prior art: There is often noise in the marine environment, which not only leads to a low signal-to-noise ratio but also interferes with the contour extraction of the target signal. The present invention has better anti-interference ability and effectively improves the accuracy of contour extraction. Description of the Drawings

[0011] Figure 1 is an algorithm flow chart; Figure 2 is a manually selected point diagram for the algorithm flow, where the manually selected points are the black points in the orange box; Figure 3 is a diagram for positioning and correcting manually selected points in the algorithm flow, where the black points in the orange box are the manually selected points and the red points are the corrected points; Figure 4 is a diagram of the intermediate process of fitting, where the red ones are the fitting contour points and the black ones are the manually selected points; Figure 5 is a diagram of the contour points to be deleted during fitting; Figure 6 is a fitting contour diagram after adding manually selected points; Figure 7 is an example diagram of extrapolating the start and end points. Red, green, and black points can be seen on the time-frequency diagram. Among them, the red points are the contour points fitted by the algorithm, the green points are the points deleted during the fitting process, and the black points are the manually selected N points, and the red points in the yellow box are the extrapolated start and end points; Figure 8 is an example diagram of extracting the characteristics of different whistle signals; a is before extracting the whistle signal with multiple contours, b is after extracting the whistle signal with multiple contours, c is before extracting the whistle signal with undulating contours, d is before extracting the whistle signal with undulating contours, e is before extracting the whistle signal with interference, and f is after extracting the whistle signal with interference. Specific embodiments

[0012] The present invention will be described in detail below in conjunction with specific embodiments, but the protection scope of the present invention is not limited by any form of the embodiments.

[0013] The present invention is an automatic extraction method for the time-frequency characteristics of marine mammal whistle signals. According to the characteristics of narrowband frequency modulation of the whistle signals, this method manually selects the start and end points of the contour to initially determine the trend of each contour curve of the signal; fine-tunes the manually selected points to ensure they fall on the contour line, and uses a combination of trend judgment and linear fitting to automatically obtain the frequency and time information of the contour line; performs extrapolation to determine the initial and end points of the contour.

[0014] The specific operation steps include: 1) Denoising preprocessing. Perform median filtering, mean filtering, etc. on the data to reduce the interference of noise.

[0015] 2) Obtain the time-frequency spectrum diagram of the signal. Select a section of data y[e] containing frequency modulation type signals, , is the number of data points, and in this embodiment It is 102,400. Using the short-time Fourier transform, the time-frequency spectrogram of this section of data is obtained. The set parameters include The number of Fourier transform points, which is 4,096 in this embodiment; The window length, which is ; is the number of overlapping samples of the window, which is half of the window length in this embodiment. The coordinates of the time-frequency spectrogram correspond to

[0016] , is the sampling frequency, which is 128 kHz in this embodiment.

[0017] 3) Manual point selection. For each contour curve on the time-frequency spectrogram, manual point selection is performed separately. The number of selected points is at least two, and the positions include but are not limited to the starting point, ending point, frequency extreme point, and contour intersection point, etc. For the intersection point, at least one point should be selected before and after according to the actual situation. Obtain the coordinates (time, frequency) of N manually selected points on the time-frequency spectrogram and sort them by time, denoted as ; 4) Automatic adjustment of manually selected points to ensure that the selected points correspond to the coordinate points on the time-frequency diagram. Determine the time in that is closest to , . Determine the frequency on the corresponding f at time that is closest to , . Update all the points in this way to obtain .

[0018] 5) Obtain the candidate contour set. Select the power extreme points at each moment on the time-frequency diagram to form the candidate contour set. Select the power extreme points at each moment on the time-frequency diagram to form the candidate contour set , is the number of extreme points at the moment.

[0019] 6) Contour trend judgment and linear fitting.

[0020] ① Obtain 's difference array , = 1,..., N , corresponds to two points. is judged as an upward trend, otherwise it is a downward trend.

[0021] ② According to the trend and manually selected points, screen the points in the candidate contour set. Take For example, for the candidate contour points on ( , , ), , if it is an upward trend, then and the points are removed from the candidate contour set; for the points with a downward trend, and the points are removed from the candidate contour set. Update .

[0022] ③ Select the next extrapolated contour point of the manually selected point except the last one in the candidate contour set. Search for the candidate contour point with the maximum power at , and judge whether it is within the range. If so, determine as the contour point; if not, within the range, search for the candidate point closest to ; if not, then double it, and search in the above manner until the first extrapolated contour point is selected at , and construct the contour set as , where q = 1.

[0023] ④ Fit the remaining contour points cyclically according to the existing contour set . is the time of the last point in the contour set.

[0024] If , first, screen the candidate contour set , . For the points with an upward trend, the points with a frequency lower than and higher than are removed from the candidate contour set; for the downward trend, the points with a frequency higher than and lower than are removed from the candidate contour set. Then, perform linear fitting based on the existing contour points to obtain the frequency . If there are no candidate contour points at , then set the fitting point as the contour point ; if there are candidate contour points, find the point with the maximum power among them . If For contour points ; If , find candidate contour points whose distance to the fitted line at this moment is less than and satisfy the change trend, that is and (if it is a decreasing trend, here it is ), , determine as contour point ; If and no candidate contour point with a distance less than to the fitted line and satisfying the change trend is found, then delete the previous contour point , and mark this point as a green point on the time-frequency diagram. If there is only in the contour set at this time, then reselect the first extrapolated contour point according to step ③ at the moment of and denote it as . Search for contour points according to step ④ on ; If there are more than two contour points in the contour set at this time, then search for contour points according to step ④ on . Update the contour set to or or .

[0025] If , update the contour set to . If , then update the contour set to according to step ③; If , then jump to the next step.

[0026] 7) Extrapolation correction of the starting point and the ending point.

[0027] For the starting point , if , for the extreme points on ( , , ), , remove the extreme points of and from the candidate contour set; If , then remove the extreme points of and from the candidate contour set and update . If there exists and at the previous time point, , update the contour set .

[0028] ​For the end point If For the extreme points on ( , , ), , remove the extreme points of and from the candidate contour set; if , then remove the extreme points of and from the candidate contour set and update . If there are and at the next time point, , update the contour set .

[0029] 8) If there are green marked points on the time-frequency diagram, it is necessary to determine whether additional manual point selection is required, usually adding point selection at the positions where the fitting contour deviates. Process again from step 3).

[0030] 9) The application examples of the method of the present invention are as Figures 2 - 7 shown. The extraction effects of multiple signal contours are as Figure 8 shown. Figure 8 It can be seen that the method of the present invention can accurately obtain signal contour features in various forms such as multi-line, undulating lines and accompanying interference. The left side shows the original signal situation, and the right side shows the situation after extraction using the method of the present invention.

Claims

1. An automatic extraction method for the time-frequency characteristics of marine mammal whistle signals, characterized in that, The method includes the following steps: Step 1, noise reduction preprocessing: perform low-pass filtering, median filtering, mean filtering, etc. on the data to reduce the interference of noise and interference signals on the target signal; Step 2: Calculate the time-frequency spectrum diagram; for a segment of data y[e] containing the whistle signal, , where is the number of data points, use the short-time Fourier transform to obtain the time-frequency spectrum diagram of this segment of data; Step 3, manual point selection and positioning; for each contour curve on the time-frequency spectrogram, perform manual assisted point selection respectively. The minimum number of selected points is two, and the positions need to include the starting point, the ending point, and the places where the contour trend changes greatly; locate on the time-frequency spectrogram and obtain the coordinate values of N manually selected points. The coordinate values include time t, frequency f, and power spectrum E; Step 4, obtain the candidate contour set; Select the power extreme points at each moment on the time-frequency diagram to form a candidate contour set , is the number of extreme points at the moment, is the frequency coordinate value of the j-th power extreme point at the moment on the time-frequency diagram, is the power spectrum coordinate value of the j-th power extreme point at the moment on the time-frequency diagram; Step 5: Profile trend judgment and line fitting; among the N points manually selected, every two adjacent points form a fitting interval. Trend judgment and line fitting are performed in the N - 1 intervals to obtain the set of fitting profile points within the interval, denoted as , and finally the time and frequency information of the total profile are obtained; Step 6. Extrapolation and correction of the starting point and the ending point: For the starting point , if the difference between the frequency coordinate of the second manually selected point and the frequency of the first selected point , for the extreme point on ( , , ), , is the step size of the abscissa on the time-frequency diagram, remove the extreme points of and from the candidate contour set; if , then remove the extreme points of and from the candidate contour set and update ; if there exists and at the previous time point , update the contour set ; For the end point If For the extreme points on ( , , ), Remove the extreme points of and from the candidate contour set; If , then remove the extreme points of and from the candidate contour set and update ; If there exists at a subsequent time point and , ; update the contour set .

2. The method according to claim 1, wherein The automatic adjustment of the manually selected points in step 3) is to ensure that the selected points correspond to the coordinate points on the time-frequency diagram. The specific method is as follows: In to determine the time that is closest to , ; At the time , determine the frequency that is closest to in the corresponding f, ; Update all the points in this way to obtain . The coordinates of the time-frequency spectrum diagram correspond to , , . is the number of Fourier transform points.

3. The method according to claim 1, wherein The said step 5) contour trend judgment and linear fitting: ① Obtain the i-th manually selected point 's difference array , = 1,..., N , corresponding to two points; It is judged as an upward trend, otherwise it is a downward trend; ② Screen the points in the candidate contour set according to the trend and manual point selection: Taking as an example, for the candidate contour points at time ( , , ), , if it is an upward trend, then remove the points and from the candidate contour set; for the points with a downward trend, remove the points and from the candidate contour set, update , is the number of extreme points at that time; ③ Select the next extrapolated contour point of the manually selected point except the last one in the candidate contour set: Search for the candidate contour point with the maximum power at a moment , and judge whether it is within ; if so, determine as the contour point. Otherwise, within find the candidate point closest to ; if not, double it and find it in the above way until the first extrapolated contour point is selected at the moment of ; then construct the contour set as where q = 1; and the contour set is ​ ④According to the existing contour set Circularly fit the remaining contour points is the time of the last point in the contour set; like , first, filter the candidate contour set , ; For the points of the upward trend, the frequency is lower than and above Points with a frequency higher than are removed from the candidate contour set; for a downward trend, and below Remove the candidate contour set; then, according to the existing contour points, a straight line fitting is used to obtain Frequency ;like If there is no candidate contour point at this moment, the fitting point Define as contour point ; If there are candidate contour points, find the point with the maximum power among them ;if , , then determine For contour points ;like , find the moment when the distance to the fitting line is less than And the candidate contour points that meet the change trend, that is, and , if it is a decreasing trend, here is ; ,Sure For contour points ;like And no distance from the fitted straight line is less than And when the candidate contour point meets the change trend, delete the previous contour point , and this point will be marked as a green point on the time-frequency graph; if there is only , then in Always follow step ③ to reselect the first extrapolated contour point and record it as ;exist Follow step ④ to find the contour points; if there are more than two contour points in the contour set, Follow step ④ to find the contour points ; Update the contour set to or or ; If , update the contour set to ; If , then according to step ③, update the contour set to ; If , then jump to the next step.

4. The method according to claim 1, characterized in that In the said step 6), if there are points deleted during the fitting process on the time-frequency diagram, it is necessary to judge whether it is necessary to increase manual point selection. If so, add manual points at the positions where the fitting contour deviates, and then process again from step 3).

5. An automatic extraction system for the time-frequency characteristics of marine mammal whistle signals, characterized in that, The system includes a data input module, a data processing module, a module for calculating the time-frequency spectrogram of the signal, a module for manual point selection and positioning, a module for obtaining the candidate contour set, a module for contour trend judgment and linear fitting, a module for extrapolation and correction of the starting point and the ending point, and a result output module. The said modules are sequentially connected by data communication; The data processing module runs step 1 described in any one of claims 1-4; The module for calculating the time-frequency spectrogram of the signal runs step 2 described in any one of claims 1-4; The module for manual point selection and positioning runs step 3 described in any one of claims 1-4; The module for obtaining the candidate contour set runs step 4 described in any one of claims 1-4; The module for contour trend judgment and linear fitting runs step 5 described in any one of claims 1-4; The module for extrapolation and correction of the starting point and the ending point runs step 6 described in any one of claims 1-4.

6. An automatic extraction device for the time-frequency characteristics of marine mammal whistle signals, characterized in that, The said device is equipped with the system described in claim 5.

Images