Automatic Extraction Method, System and Device for Time-Frequency Characteristics of Marine Mammal Whistle Signals

Abstract

The present invention relates to an automatic extraction method, system and device for the time-frequency characteristics of marine mammal whistle signals, belonging to the technical field of signal processing. The method of the present invention includes noise reduction preprocessing, calculating the time-frequency spectrogram of the signal, manually selecting points and positioning, and obtaining a candidate contour set; judging the contour trend and linear fitting; extrapolating and correcting the starting point and the ending point. The present invention also provides a system and a device for running the method. The method of the present invention has better anti-interference ability and effectively improves the accuracy of contour extraction.

Description

Technical Field

[0001] The present invention belongs to the technical field of signal processing, and particularly relates to an automatic extraction method, system and device for the time-frequency characteristics of marine mammal whistle signals. Background Art

[0002] Whistle signals are important social signals of marine mammals and an important way for them to communicate, interact and express emotions among individuals or groups. The typical characteristics of such signals are narrowband, long-duration and frequency modulation: the duration is usually between dozens of milliseconds and several seconds, the frequency range varies from dozens of hertz to hundreds of kilohertz, and multiple harmonic components are often accompanied. After Fourier transform, the whistle signal can be decomposed into a fundamental frequency and harmonics. The fundamental frequency refers to the lowest frequency component obtained by decomposition, and the harmonics refer to other components, which are integers of the fundamental frequency. On the time-frequency spectrum, the signal usually appears as one or more contour curves, where the curve with the lowest frequency corresponds to the fundamental frequency component, and the rest are harmonic components.

[0003] The feature extraction of whistle signals is the key to monitoring marine mammals using underwater acoustics, which can not only improve the accuracy of population identification, but also be the only basis for individual identification through voiceprint. Existing extraction methods usually focus on the fundamental frequency and ignore harmonic information; at the same time, they have high requirements for signal-to-noise ratio, and their anti-interference ability and robustness are poor. These limitations seriously restrict the development and application of underwater acoustic monitoring technology for marine mammals. Summary of the Invention

[0004] The present method innovatively adopts a technical solution of trend fitting. There is often noise in the marine environment, which not only leads to a low signal-to-noise ratio, but also interferes with the extraction of the target signal. Conventional energy maxima and minima will cause deviations, while trend fitting can achieve better results. Specifically, it includes:

[0005] First, on the time-frequency diagram, manually select the start and end points of the contour line (appropriately add selected points according to the prompts of the fitting process); secondly, locate the manually selected points; then, automatically obtain the contour curve according to the contour trend and the fitted straight line; finally, determine the start and end points of the contour according to the contour curve and save the contour parameters.

[0006] The present invention is generally implemented through the following steps:

[0007] An automatic extraction method for the time-frequency characteristics of marine mammal whistle signals, the method comprising the following steps:

[0008] Step 1, noise reduction preprocessing: perform low-pass filtering, median filtering, mean filtering, etc. on the data to reduce the interference of noise and interference signals on the target signal;

[0009] Step 2: Calculate the time-frequency spectrogram; For a segment of data y[e] containing the whistle signal, , where is the number of data points, perform short-time Fourier transform to obtain the time-frequency spectrogram of this segment of data;

[0010] Step 3: Manually select points and locate; For each contour curve on the time-frequency spectrogram, perform manual assisted point selection respectively. The minimum number of selected points is two, and the positions need to include the starting point, the ending point, and the places where the contour trend changes significantly; Locate on the time-frequency spectrogram and obtain the coordinate values of the N manually selected points. The said coordinate values include time t, frequency f, and power spectrum E;

[0011] Step 4: Obtain the candidate contour set; Select the power extreme points at each moment on the time-frequency diagram to form the candidate contour set , is the number of extreme points at the moment, is the frequency coordinate value of the j-th power extreme point on the time-frequency diagram at the moment, is the power spectrum coordinate value of the j-th power extreme point on the time-frequency diagram at the moment;

[0012] Step 5: Contour trend judgment and linear fitting; Among the N manually selected points, every two adjacent points form a fitting interval. Perform trend judgment and linear fitting in the N - 1 intervals to obtain the set of fitting contour points within the interval, denoted as , and finally obtain the time and frequency information of the total contour;

[0013] Step 6: Extrapolation and correction of the starting point and the ending point: For the starting point , if the difference between the frequency coordinate of the second manually selected point and the frequency of the first selected point , for the extreme points on ( , , ), , is the step size of the abscissa on the time-frequency diagram, remove the extreme points of and from the candidate contour set; If , then remove the extreme points of and from the candidate contour set and update ; If there exists and at the previous time point, , update the contour set ;

[0014] For the end point If For the extreme points on ( , , ), , remove the extreme points of and from the candidate contour set; if , then remove the extreme points of and from the candidate contour set and update ; if there exists and at the next time point . Update the contour set .

[0015] Furthermore, for the automatic adjustment of the manually selected points in step 3), ensure that the selected points correspond to the coordinate points on the time-frequency diagram. The specific method is as follows: Determine the time in that is closest to , ; Determine the frequency in the corresponding f that is closest to at time , ; Update all the points in this way to obtain , and the coordinates of the time-frequency spectrum diagram correspond to , , is the sampling frequency, is the number of Fourier transform points

[0016] Furthermore, for the contour trend judgment and linear fitting in step 5)

[0017] ① Obtain the difference array of the i-th manually selected point , = 1,..., N , corresponds to two points; is judged as an upward trend, otherwise it is a downward trend;

[0018] ② According to the trend and the manually selected points, screen the points in the candidate contour set: Take as an example, for the candidate contour points at time ( , , ), If it is an upward trend, 、 The points with a downward trend are removed from the candidate contour set; for the points with a downward trend, 、 The points are removed from the candidate contour set and updated , yes The number of extreme points at the moment;

[0019] ③ Select the next extrapolated contour point except the last one in the candidate contour set: Find The candidate contour point with the maximum power at the moment ,judge Is it Within the range; yes, then confirm is a contour point; otherwise, Find the closest Candidate points If not, Double, in Search as above until you find Always select the first extrapolated contour point , construct the contour set as , where q=1;

[0020] ④According to the existing contour set Loop fitting of the remaining contour points, Time to focus the final point for the outline;

[0021] like ,First, filter the candidate contour set , ; For the points of the upward trend, the frequency is lower than and above Points with a frequency higher than are removed from the candidate contour set; and for a downward trend, points with a frequency higher than and below Remove the candidate contour set; then, according to the existing contour points, a straight line fitting is used to obtain Frequency ;like There is no candidate contour point at this moment, then the fitting point Define as contour point ; If there are candidate contour points, find the point with the maximum power among them ;if , , then determine Contour points ;like , find the moment when the distance to the fitting straight line is less than And candidate contour points that satisfy the changing trend, that is, and , if it is a decreasing trend, here is ; , determine as a contour point ; If and no candidate contour point whose distance from the fitting line is less than and that satisfies the changing trend is found, then the previous contour point is deleted, and this point will be marked as a green point on the time-frequency diagram. If there is only in the contour set at this time, then at time, reselect the first extrapolated contour point according to step ③ and denote it as . Search for contour points on according to step ④; If there are more than two contour points in the contour set at this time, then search for contour points on according to step ④ . Update the contour set to or or ;

[0022] If , update the contour set to . If , then according to step ③, update the contour set to ; If , then jump to the next step.

[0023] Furthermore, in step 6) described above, if there are points deleted during the fitting process on the time-frequency diagram, it is necessary to determine whether it is necessary to add manual point selection. If so, add manual point selection at the position where the fitting contour deviates, and then process from step 3) again.

[0024] An automatic extraction system for the time-frequency characteristics of marine mammal whistle signals, the system includes a data input module, a data processing module, a module for calculating the time-frequency spectrum diagram of the signal, a module for manual point selection and positioning, a module for obtaining a candidate contour set, a module for contour trend judgment and linear fitting, a module for extrapolation and correction of the starting point and the ending point, and a result output module, and the modules are sequentially connected by data communication;

[0025] The data processing module runs step 1 described above;

[0026] The module for calculating the time-frequency spectrum diagram of the signal runs step 2 described above;

[0027] The module for manual point selection and positioning runs step 3 described above;

[0028] The module for obtaining a candidate contour set runs step 4 described above;

[0029] The contour trend judgment and line fitting module executes step 5 described above;

[0030] The extrapolation and correction module for the starting and ending points executes step 6 described above.

[0031] An automatic extraction device for the time-frequency characteristics of marine mammal whistle signals, and the device is equipped with the system described above.

[0032] Advantages of the present invention compared with the prior art: Noise often exists in the marine environment, which not only leads to a low signal-to-noise ratio but also interferes with the extraction of the contour of the target signal. The present invention has better anti-interference ability and effectively improves the accuracy of contour extraction. Brief Description of the Drawings

[0033] Figure 1 is a flowchart of the algorithm;

[0034] Figure 2 is a diagram of manually selected points in the algorithm process, where the manually selected points are the black points in the orange box;

[0035] Figure 3 is a diagram of positioning and correction of manually selected points in the algorithm process, where the black points in the orange box are the manually selected points and the red points are the corrected points;

[0036] Figure 4 is a diagram of the intermediate fitting process, where the red ones are the fitting contour points and the black ones are the manually selected points;

[0037] Figure 5 is a diagram of contour points to be deleted during fitting;

[0038] Figure 6 is a fitting contour diagram after adding manually selected points;

[0039] Figure 7 is an example diagram of extrapolation of the starting and ending points. Red, green, and black points can be seen on the time-frequency diagram. Among them, the red points are the contour points fitted by the algorithm, the green points are the points deleted during the fitting process, and the black points are the manually selected N points, and the red points in the yellow box are the extrapolated starting and ending points;

[0040] Figure 8 is an example diagram of feature extraction for different whistle signals; a is before the extraction of a whistle signal with multiple contours; b is after the extraction of a whistle signal with multiple contours; c is before the extraction of a whistle signal with a fluctuating contour; d is before the extraction of a whistle signal with a fluctuating contour; e is before the extraction of a whistle signal with interference; f is after the extraction of a whistle signal with interference. Detailed Embodiment

[0041] The present invention will be described in detail below in conjunction with specific embodiments, but the protection scope of the present invention is not limited by any form of the embodiments.

[0042] The present invention is an automatic extraction method for the time-frequency characteristics of the whistle signals of marine mammals. According to the characteristics of narrowband frequency modulation of the whistle signals, this method uses artificial selection of the starting and ending points of the contour to preliminarily determine the trend of each contour curve of the signal; fine-tunes the artificial selected points to ensure that they fall on the contour line, and adopts a method combining trend judgment and linear fitting to automatically obtain the frequency and time information of the contour line; performs extrapolation to determine the initial and ending points of the contour.

[0043] The specific operation steps include:

[0044] 1) Noise reduction preprocessing. Perform median filtering, mean filtering, etc. on the data to reduce the interference of noise.

[0045] 2) Obtain the time-frequency spectrogram of the signal. Select a section of data y[e] containing frequency modulation type signals, , where is the number of data points, and in this embodiment, is 102,400. Use the short-time Fourier transform to obtain the time-frequency spectrogram of this section of data. The set parameters include the number of Fourier transform points, which is 4,096 in this embodiment; the window length, which is in this embodiment; is the number of overlapping samples of the window, which is half of the window length in this embodiment. The coordinates of the time-frequency spectrogram correspond to

[0046] , where is the sampling frequency, which is 128 kHz in this embodiment.

[0047] 3) Artificial point selection. For each contour curve on the time-frequency spectrogram, perform artificial point selection respectively. The number of selected points is at least two, and the positions include but are not limited to the starting point, ending point, frequency extreme point, and contour intersection point, etc. For the intersection point, at least one point should be selected before and after according to the actual situation. Obtain the coordinates (time, frequency) of N artificial selected points on the time-frequency spectrogram and sort them by time, denoted as ;

[0048] 4) Automatic adjustment of the artificial selected points to ensure that the selected points correspond to the coordinate points on the time-frequency diagram. Determine the time in that is closest to . Determine the frequency in the corresponding f that is closest to at the time . Determine the frequency in that is closest to , . Update all points in this way and get .

[0049] 5) Obtain the candidate profile set. Select the power extreme points at each moment on the time-frequency graph to form the candidate profile set. Obtain the candidate profile set. Select the power extreme points at each moment on the time-frequency graph to form the candidate profile set. , yes The number of extreme points at the moment.

[0050] 6) Contour trend judgment and straight line fitting.

[0051] ① Acquisition The difference array of , =1,...,N , Corresponding to Two o'clock. It is judged as an upward trend, otherwise it is a downward trend.

[0052] ② According to the trend and manual selection, filter the points in the candidate contour concentration. For example. Candidate contour points on ( , , ), If it is an upward trend, 、 The points with a downward trend are removed from the candidate contour set; for the points with a downward trend, 、 The points are removed from the candidate contour set. Update .

[0053] ③ Select the next extrapolated contour point except the last manually selected point in the candidate contour set. The candidate contour point with the maximum power at the moment ,judge Is it If yes, confirm is a contour point; otherwise, Find the closest Candidate points If not, Double, in Search as above until you find Always select the first extrapolated contour point , construct the contour set as , where q=1.

[0054] ④According to the existing contour set Loop to fit the remaining contour points. Is the time of the last point in the contour set.

[0055] If , first, screen the candidate contour set , . For the points with an upward trend, move the points with a frequency lower than and higher than out of the candidate contour set; for the downward trend, move the points with a frequency higher than and lower than out of the candidate contour set. Then, perform linear fitting based on the existing contour points to obtain frequency . If there are no candidate contour points at the moment, set the fitting point as the contour point ; if there are candidate contour points, find the point with the maximum power value . If , then determine as the contour point ; if , find the candidate contour point whose distance to the fitting line is less than and satisfies the changing trend, that is and (if it is a decreasing trend, here it is ), , determine as the contour point ; if and no candidate contour point whose distance to the fitting line is less than and satisfies the changing trend can be found, then delete the previous contour point , and mark this point as a green point on the time-frequency diagram. If there is only in the contour set at this time, then re-select the first extrapolated contour point at according to step ③ and mark it as . Search for contour points on according to step ④; if there are more than two contour points in the contour set at this time, then search for contour points on according to step ④ . Update the contour set to or or .

[0056] If , update the contour set to . If , then update the contour set according to step ③ ; If , then jump to the next step.

[0057] 7) Extrapolation correction of the starting point and the ending point.

[0058] For the starting point , if , for the extreme points on ( , , ), , remove the extreme points of and from the candidate contour set; if , then remove the extreme points of and from the candidate contour set and update . If there exists and at the previous time point, , update the contour set .

[0059] For the ending point , if , for the extreme points on ( , , ), , remove the extreme points of and from the candidate contour set; if , then remove the extreme points of and from the candidate contour set and update . If there exists and at the next time point, , update the contour set .

[0060] 8) If there are green marked points on the time-frequency diagram, it is necessary to determine whether additional manual point selection is required, usually by adding point selection at the positions where the fitting contour deviates. Then process from step 3) again.

[0061] 9) Application examples of the method of the present invention are as Figures 2 - 7 shown. The extraction effects of multiple signal contours are as Figure 8 shown. Figure 8It can be seen that the method of the present invention can accurately obtain the signal contour features in various forms such as multi-line, undulating lines and accompanying interference. The left side shows the original signal situation, and the right side shows the situation after extraction using the method of the present invention.

Claims

1. An automatic extraction method for the time-frequency characteristics of marine mammal whistle signals, characterized in that, The method includes the following steps: Step 1, noise reduction preprocessing: Perform low-pass filtering, median filtering, mean filtering, etc. on the data to reduce the interference of noise and interference signals on the target signal; Step 2: Calculate the time-frequency spectrum diagram; for a section of data y[e] containing the whistle signal, , where is the number of data points, use the short-time Fourier transform to obtain the time-frequency spectrum diagram of this section of data; Step 3, manual point selection and positioning: For each contour curve on the time-frequency spectrogram, perform manual assisted point selection separately. The minimum number of selected points is two, and the positions need to include the starting point, the ending point, and the places where the contour trend changes greatly; Locate and obtain the coordinate values of N manually selected points on the time-frequency spectrogram. The coordinate values include time t, frequency f, and power spectrum E; Step 4, obtain the candidate contour set; Select the power extreme points at each moment on the time-frequency diagram to form a candidate contour set , is the number of extreme points at the moment, is the frequency coordinate value of the j-th power extreme point on the time-frequency diagram at the moment, is the power spectrum coordinate value of the j-th power extreme point on the time-frequency diagram at the moment; Step 5, contour trend judgment and linear fitting; among the N points manually selected, every two adjacent points form a fitting interval, trend judgment and linear fitting are performed in the N - 1 intervals to obtain the set of fitting contour points within the interval, denoted as , and finally the time and frequency information of the total contour are obtained; Step 6. Extrapolation and correction of the starting point and the ending point: For the starting point , if the difference between the frequency coordinate of the second manually selected point and the frequency of the first selected point , for the extreme point on ( , , ), , being the step size of the abscissa on the time-frequency diagram, remove the extreme points of and from the candidate contour set; if , then remove the extreme points of and from the candidate contour set and update ; if there exists and at the previous time point, , update the contour set ; For the end point If For the extreme points on ( , , ), Remove the extreme points of and from the candidate contour set; if , then remove the extreme points of and from the candidate contour set and update ; If there exists at a subsequent time point and , ; update the contour set .

2. The method according to claim 1, wherein The automatic adjustment of the manual point selection in step 3) is to ensure that the selected points correspond to the coordinate points on the time-frequency diagram. The specific method is as follows: In determine the time that is closest to , ; On the time , determine the frequency that is closest to in the corresponding f, ; Update all the points in this way to obtain . The coordinates of the time-frequency spectrum diagram correspond to , , . is the number of Fourier transform points.

3. The method according to claim 1, wherein The said step 5) contour trend judgment and linear fitting: ① Obtain the i-th manually selected point 's difference array , = 1, ..., N , corresponding to two points; It is judged as an upward trend, otherwise it is a downward trend; ②Filter the points in the candidate contour set according to the trend and manual point selection: Take as an example. For the candidate contour points at time ( , , ), , if it is an upward trend, then remove the points and from the candidate contour set; for the points with a downward trend, remove the points and from the candidate contour set, update , is the number of extreme points at the moment; ③ Select the next extrapolated contour point of the manually selected points except the last one in the candidate contour set: Search for The candidate contour point with the maximum power at the moment , and judge Whether it is within ; if so, determine As the contour point; No, then Find the closest Candidate points If not, Double, in Search as above until you find Always select the first extrapolated contour point , construct the contour set as , where q=1; ④According to the existing contour set Cyclically fit the remaining contour points, is the time of the last point in the contour set; like ,First, filter the candidate contour set , ; For the points of the upward trend, the frequency is lower than and above Points with a frequency higher than are removed from the candidate contour set; and for a downward trend, points with a frequency higher than and below Remove the candidate contour set; then, according to the existing contour points, a straight line fitting is used to obtain Frequency ;like There is no candidate contour point at this moment, then the fitting point Define as contour point ; If there are candidate contour points, find the point with the maximum power among them ;if , , then determine Contour points ;like , find the moment when the distance to the fitting straight line is less than And the candidate contour points that meet the change trend, that is, and , if it is a decreasing trend, here is ; ,Sure Contour points ;like And no distance from the fitting line is less than And when the candidate contour point meets the change trend, delete the previous contour point , and this point will be marked as a green point on the time-frequency graph; if there is only , then in Always follow step ③ to reselect the first extrapolated contour point and record it as ;exist Follow step ④ to find the contour points; if there are more than two contour points in the contour set, Follow step ④ to find the contour points ; Update the contour set to or or ; If , update the contour set to ; If , then according to step ③, update the contour set to ; If , then jump to the next step.

4. The method according to claim 1, characterized in that, In the said step 6), if there are points deleted during the fitting process on the time-frequency diagram, it is necessary to judge whether it is necessary to increase manual point selection. If so, add manual points at the positions where the fitting contour deviates, and then process from step 3) again.

5. An automatic extraction system for the time-frequency characteristics of the whistle signals of marine mammals, characterized in that, The system includes a data input module, a data processing module, a module for calculating the time-frequency spectrogram of the signal, a module for manual point selection and positioning, a module for obtaining the candidate contour set, a module for contour trend judgment and linear fitting, a module for extrapolation and correction of the starting point and the ending point, and a result output module. The said modules are sequentially connected by data communication; The data processing module runs step 1 described in any one of claims 1-4; The module for calculating the time-frequency spectrogram of the signal runs step 2 described in any one of claims 1-4; The module for manual point selection and positioning runs step 3 described in any one of claims 1-4; The module for obtaining the candidate contour set runs step 4 described in any one of claims 1-4; The module for contour trend judgment and linear fitting runs step 5 described in any one of claims 1-4; The module for extrapolation and correction of the starting point and the ending point runs step 6 described in any one of claims 1-4.

6. An automatic extraction device for the time-frequency characteristics of marine mammal whistle signals, characterized in that, The said device is equipped with the system described in claim 5.

Images