A method and apparatus for identifying a whale sound, and a storage medium
By constructing a whale sound recognition model using graph convolutional networks and graph neural networks, the structural correlation between sound segments in whale species acoustic recognition was solved, achieving high-precision and robust species recognition and improving recognition performance in complex marine environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUILIN UNIV OF ELECTRONIC TECH
- Filing Date
- 2026-03-24
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies struggle to capture the structural relationships between sound segments in acoustic recognition of cetaceans, resulting in insufficient recognition accuracy and robustness.
We employ Graph Convolutional Networks (GCN) combined with Graph Neural Networks (GNN) to extract features and perform model analysis on cetacean vocal signals. By constructing graph-structured data and utilizing the spectral characteristics and temporal continuity of cetacean vocal signals, we achieve end-to-end species identification.
It improves the recognition accuracy and robustness in complex marine environments, effectively explores the species-specific associations of sound wave segments, and solves the shortcomings of traditional methods in non-local association modeling and structural interaction expression, with a recognition accuracy of 96.37%.
Smart Images
Figure CN122157674A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of sound recognition technology, specifically to a method, device, and storage medium for whale sound recognition. Background Technology
[0002] Cetacean acoustic signals are one of the core data carriers in cetacean ecology research. Against the backdrop of global biodiversity conservation and marine ecological governance, acoustic identification of cetacean species has become an important topic at the intersection of marine biology, environmental monitoring, and bioinformatics. With the iteration of acoustic sensing technology and automated data acquisition and computing capabilities, the accuracy of cetacean acoustic acquisition and analysis has significantly improved. However, cetacean acoustic data from different species not only exhibit obvious species characteristics, but also show complex temporal correlations between acoustic signals of the same species, posing challenges to cetacean species identification.
[0003] In the early stages of research on acoustic identification of cetaceans, researchers generally relied on traditional acoustic analysis and machine learning methods. Traditional methods involved manually designing and extracting acoustic features, such as Mel Frequency Cepstral Coefficients (MFCCs) to characterize the spectral envelope, spectral centroids to describe energy distribution, and pulse interval statistics to capture temporal considerations. These features were then combined with classifiers such as Support Vector Machines (SVMs) and Random Forests to distinguish the sound signals of different species.
[0004] However, the aforementioned traditional methods have significant drawbacks: First, the design of artificial features relies on domain knowledge, making it difficult to cover the specific acoustic patterns of different cetacean species. Second, these methods treat sound segments as independent samples, ignoring the inherent structural relationships within cetacean sounds. With the rapid development of deep learning, data-driven feature learning methods are increasingly being applied to species identification in bioacoustics (including cetaceans and other animals). For example, Piczak used Convolutional Neural Networks (CNNs) to extract features from the Mel spectrograms of environmental sounds, successfully classifying 50 categories of environmental sounds (including some animal calls), demonstrating the advantages of CNNs in extracting spectral features from acoustic signals. Prengel E et al. used Long Short-Term Memory (LSTM) networks to construct recurrent neural network models to analyze the temporal sequences of bird calls, achieving a high accuracy rate in various bird sound classification tasks and effectively capturing the temporal dynamic features of bird calls. In cetacean research, DM Cholewiak et al. addressed the standardization issues in the classification and measurement of humpback whale song features, pointing out core problems such as inconsistent phrase type definitions and ignoring the variability of topic sequences while relying solely on song duration. They proposed guiding principles for phrase segmentation and an analytical paradigm based on phrase sequences, providing crucial references for the standardization of humpback whale song research. Lee H et al. attempted to use deep belief networks to perform unsupervised feature learning on humpback whale song fragments, providing a feature representation foundation for subsequent species identification. These methods, combining big data and deep learning, overcome the limitations of manual features, demonstrating significantly better recognition performance than traditional methods in single-species, low-noise scenarios. However, their ability to model structural information remains insufficient, and deep learning methods still have significant shortcomings in the acoustic identification of biological species. The advantage of CNNs lies in their efficient extraction of local spectral features, but they struggle to capture non-local structural correlation features in temporal signals such as sound. For example, the species-specific "pulse cluster-interval-pulse cluster" topology (such as the social whistle sequence of orcas) cannot be fully captured by the local receptive field of CNNs. Recurrent Neural Networks (RNNs) can capture whether there is a dependency between different time-series signals, but they are not sensitive to the degree of correlation between different frames of the same time-series signal.
[0005] In summary, existing algorithms have the drawback of failing to capture the structural relationships between sound segments in acoustic recognition among cetacean species. Summary of the Invention
[0006] The technical problem to be solved by the present invention is to provide a method, device and storage medium for whale sound recognition, which addresses the shortcomings of the prior art.
[0007] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A method for whale sound recognition, comprising the following steps: Multiple raw whale audio data were obtained from the WHOI whale sound database, and all the raw whale audio data were combined to obtain a raw whale audio dataset; The original whale audio dataset is preprocessed to obtain a preprocessed whale audio dataset; Feature analysis is performed on the preprocessed whale audio dataset to obtain multiple target whale audio features, and all the target whale audio features are combined to obtain the original node feature matrix; A training model is constructed, and the training model is analyzed using the original node feature matrix to obtain a sound recognition model. Import the whale audio data to be identified, and use the sound recognition model to identify the whale audio data to obtain whale sound recognition results.
[0008] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: A whale sound recognition device, comprising: The dataset collection module is used to obtain multiple raw whale audio data from the WHOI whale sound library and combine all the raw whale audio data to obtain a raw whale audio dataset; The preprocessing module is used to preprocess the original whale audio dataset to obtain a preprocessed whale audio dataset. The feature analysis module is used to perform feature analysis on the preprocessed whale audio dataset to obtain multiple target whale audio features, and to combine all the target whale audio features to obtain the original node feature matrix; The model analysis module is used to construct a training model and perform model analysis on the training model using the original node feature matrix to obtain a sound recognition model. The import module is used to import the audio data of whales to be identified; The recognition result module is used to recognize the whale audio data to be recognized through the sound recognition model to obtain whale sound recognition results.
[0009] Based on the above-mentioned method for recognizing whale sounds, the present invention also provides a whale sound recognition system.
[0010] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a whale sound recognition system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the whale sound recognition method as described above.
[0011] Based on the above-described method for whale sound recognition, this invention also provides a computer-readable storage medium.
[0012] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the whale sound recognition method as described above.
[0013] The beneficial effects of this invention are as follows: by preprocessing the original whale audio dataset, a preprocessed whale audio dataset is obtained; by analyzing the features of the preprocessed whale audio dataset, an original node feature matrix is obtained; by analyzing the training model using the original node feature matrix, a sound recognition model is obtained; and by recognizing the whale audio data to be recognized using the sound recognition model, whale sound recognition results are obtained. This allows the training model to not only utilize the spectral features of whale sounds, but also to characterize the temporal continuity, feature similarity, and non-local structural associations between different sound frames. This overcomes the limitation of traditional deep learning in its insufficient attention to the structural information of temporal signals, fully explores the degree of association in sound signals, improves the recognition accuracy, robustness, and generalization ability in complex marine environments, effectively explores the species-specific associations of sound wave segments, and solves the defects of traditional methods in non-local association modeling and structural interaction representation. Attached Figure Description
[0014] Figure 1 This is one of the flowcharts illustrating the whale sound recognition method provided in an embodiment of the present invention; Figure 2 The network structure diagram of the graph convolutional layer, global pooling layer, and classification layer of the whale sound recognition method provided in the embodiments of the present invention; Figure 3 The network structure diagram of the graph structure construction network for the whale sound recognition method provided in this embodiment of the invention; Figure 4 This is a block diagram of a whale sound recognition device provided in an embodiment of the present invention. Detailed Implementation
[0015] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.
[0016] Figure 1 This is a flowchart illustrating a whale sound recognition method provided in an embodiment of the present invention.
[0017] like Figure 1 As shown, a method for whale sound recognition includes the following steps: S1: Obtain multiple raw whale audio data from the WHOI whale sound library, and combine all the raw whale audio data to obtain a raw whale audio dataset; S2: Preprocess the original whale audio dataset to obtain a preprocessed whale audio dataset; S3: Perform feature analysis on the preprocessed whale audio dataset to obtain multiple target whale audio features, and combine all the target whale audio features to obtain the original node feature matrix; S4: Construct a training model, and perform model analysis on the training model using the original node feature matrix to obtain a sound recognition model; S5: Import the whale audio data to be identified, and use the sound recognition model to identify the whale audio data to obtain the whale sound recognition result.
[0018] Specifically, the dataset (i.e., the original cetacean audio dataset) comes from the WHOI Cetacean Sound Database, maintained by the Woods Hole Oceanographic Institution in the United States. This database is a globally authoritative resource platform for cetacean acoustic data, containing original acoustic records of cetaceans from multiple sea areas and species. Original acoustic records of the six cetaceans used in this experiment were extracted from this website, including WAV format audio files from six species: beluga whale, bowhead whale, false killer whale, fin whale, humpback whale, and killer whale. These were stored in corresponding subfolders under the database directory, categorized by species.
[0019] In the above embodiments, a preprocessed whale audio dataset is obtained by preprocessing the original whale audio dataset. The original node feature matrix is obtained by feature analysis of the preprocessed whale audio dataset. The sound recognition model is obtained by model analysis of the training model using the original node feature matrix. The whale sound recognition result is obtained by recognizing the whale audio data to be recognized using the sound recognition model. This allows the training model to not only utilize the spectral features of whale sounds, but also to characterize the temporal continuity, feature similarity, and non-local structural associations between different sound frames. This overcomes the limitation of traditional deep learning in its insufficient attention to the structural information of temporal signals, fully explores the degree of association in sound signals, improves the recognition accuracy, robustness, and generalization ability in complex marine environments, effectively explores the species-specific associations of sound wave segments, and solves the defects of traditional methods in non-local association modeling and structural interaction expression.
[0020] Optionally, as an embodiment of the present invention, the process of preprocessing the original whale audio dataset to obtain a preprocessed whale audio dataset includes: The original whale audio dataset is converted to a new format to obtain a converted whale audio dataset. Each converted whale audio data in the converted whale audio dataset is processed into frames to obtain multiple framed whale audio data, and all the framed whale audio data are combined to obtain a framed whale audio dataset. The segmented whale audio dataset is subjected to noise reduction processing to obtain a noise-reduced whale audio dataset. The denoised whale audio dataset is normalized to obtain a normalized whale audio dataset. The normalized whale audio dataset is then windowed to obtain a preprocessed whale audio dataset.
[0021] It should be understood that the original sound signal (i.e., the converted whale audio data) is processed by framing.
[0022] In the above embodiments, the original whale audio dataset is preprocessed to obtain a preprocessed whale audio dataset, which overcomes the limitation of traditional deep learning that lacks attention to the temporal signal structure information, fully explores the correlation in the sound signal, and improves the recognition accuracy in complex marine environments.
[0023] Optionally, as an embodiment of the present invention, the process of performing feature analysis on the preprocessed whale audio dataset to obtain multiple target whale audio features includes: The preprocessed whale audio dataset is subjected to short-time Fourier transform to obtain the original whale audio feature set; The original whale audio feature set is filtered by a Mel filter bank to obtain a filtered whale audio feature set. Logarithmic calculation is performed on the filtered whale audio feature set to obtain the whale audio feature set to be processed; The discrete cosine transform is performed on the whale audio feature set to be processed to obtain the transformed whale audio feature set. Multiple 13-dimensional MFCC features are extracted from the transformed whale audio feature set, and the 13-dimensional MFCC features are used as target whale audio features to obtain multiple target whale audio features.
[0024] It should be understood that by sequentially performing short-time Fourier transform, Mel filter bank filtering, and cepstral analysis on the input signal (i.e., the preprocessed whale audio dataset), MFCC features (i.e., target whale audio features) can be extracted.
[0025] Specifically, from a temporal perspective, the MFCC features of cetacean vocalizations exhibit significant temporal correlation. Due to the physical continuity of sound, the MFCC coefficients of adjacent time frames show a smooth transition in numerical distribution (e.g., the gradual variation of low-frequency coefficients). This strong temporal dependence is quantified by the edge weights of adjacent nodes in the graph structure, with the weight values positively correlated with the similarity of features between frames.
[0026] From the perspective of characteristic energy, changes in the energy of the previous frame signal will affect the energy of the subsequent frame signal, that is, an energy transfer phenomenon will occur.
[0027] In summary, the temporal continuity and feature intensity synergy among the MFCC features of cetacean signals constitute a complex structure in the non-Euclidean space of graph theory, which can provide the necessary structural foundation for graph neural networks.
[0028] In the above embodiments, feature analysis is performed on the preprocessed whale audio dataset to obtain multiple target whale audio features, clearly understand the time-domain and frequency-domain features of whale sounds, and provide the necessary structural foundation for subsequent processing.
[0029] Optionally, as an embodiment of the present invention, such as Figure 2 As shown, the training model includes a graph structure construction network, multiple graph convolutional layers, a global pooling layer, and a classification layer. The process of performing model analysis on the trained model using the original node feature matrix to obtain the sound recognition model includes: The original adjacency matrix is obtained by performing adjacency matrix analysis on the original node feature matrix through the network constructed by the graph structure. The original adjacency matrix is normalized to obtain the normalized adjacency matrix; The normalized adjacency matrix is convolved by multiple graph convolutional layers to obtain the feature matrix of the node to be processed. The feature matrix of the node to be processed is pooled by the global pooling layer to obtain the pooled node feature matrix. The pooled node feature matrix is classified by the classification layer to obtain multiple predicted probabilities; Import multiple real labels, calculate the loss function for all real labels and all predicted probabilities, and obtain the loss function. The parameters of the trained model are updated according to the loss function to obtain the sound recognition model.
[0030] Specifically, for graph-structured data in the acoustic recognition task of cetaceans, a Graph Convolutional Network (GCN) based on spectral domain graph convolution theory is constructed. The core lies in how to implement graph convolutional layers to adapt to the topological characteristics of cetacean sound nodes and edges. Its core is to fully utilize the receptive field of nodes by performing convolution operations on the input graph-structured data, leveraging a sufficiently deep network to maximize the scope of the nodes' receptive fields. The deeper the model, the more neighborhood information the nodes can capture. This design not only combines the associative characteristics of graph-structured data but also leaves sufficient room for subsequent adjustments to model complexity and fitting effects. In the initial graph convolutional layers, nodes capture only local features of the first-order neighborhood (such as the association between a single pulse cluster and its adjacent segments) through spectral domain convolution. As the network depth increases, features are integrated across orders through layer-by-layer propagation: in the second layer of graph convolution, the neighborhood features aggregated by the nodes already contain the first-order information processed in the first layer, indirectly representing second-order neighborhood features (such as the temporal dependence of pulse cluster sequences). This recursive propagation mechanism expands the receptive field of nodes as the network deepens, gradually learning the full-map feature information of the entire input signal, providing a more comprehensive structural basis for species identification.
[0031] It should be understood that the GCN model contains five core computational units: three graph convolutional layers, one global pooling layer, and one classification layer. The graph structure data (i.e., the normalized adjacency matrix) constructed using k-nearest neighbors is sequentially passed through the three graph convolutional layers to obtain aggregated feature information (i.e., the feature matrix of the nodes to be processed). Then, the global pooling layer and the classification layer are used for cetacean species sound recognition, realizing an end-to-end application from node feature extraction to species recognition.
[0032] Specifically, Graph Neural Networks (GNNs) are typically used to process graph-structured data. Their core idea is to learn the global topology and local features of the graph by using a message-passing mechanism to allow nodes to update their features through information exchange with their neighbors. The original node attributes are used as initial features, as shown in the following formula: , in Indicates the first The node of the first Layer feature representation, It is a node The original attribute vector. And in the first... In the middle, node From neighboring nodes The received information is: , in These are message functions (such as multilayer perceptrons and attention mechanisms). It is a node and The properties of the edges between them.
[0033] After message passing, the node Aggregating all neighbor messages yields the following aggregated features: , in, It is an aggregate function. It is a node The set of neighbors.
[0034] Given the features and aggregate features of the node in the previous layer, update the node features of the current layer using the following formula: , in These are update functions, such as residual joins and gating mechanisms.
[0035] In the above embodiments, the sound recognition model is obtained by performing model analysis on the training model through the original node feature matrix. This breaks through the limitation of traditional deep learning in that it lacks attention to the structural information of temporal signals, fully explores the degree of correlation in sound signals, improves the recognition accuracy in complex marine environments, effectively explores the species-specific correlation of sound wave segments, and solves the defects of traditional methods in non-local correlation modeling and structural interaction expression.
[0036] Optionally, as an embodiment of the present invention, such as Figure 3 As shown, the original node feature matrix includes multiple whale audio node features. The process of constructing a network using the graph structure to perform adjacency matrix analysis on the original node feature matrix to obtain the original adjacency matrix includes: The first formula is used to calculate the original similarity scores corresponding to each of the whale audio node features and any remaining whale audio node features. , in, For the first The characteristics of the first whale audio node and the first The original similarity of the audio node features of each whale species. This represents the total number of audio node features in whales. For the first Individual whale audio node features For the first Individual whale-like audio node features; The original similarity scores corresponding to each whale audio node feature are sorted according to their original similarity scores to obtain the sorted similarity scores corresponding to each whale audio node feature. The first N whale audio node features corresponding to the sorted similarity are used as neighbor node features, thereby obtaining multiple neighbor node features corresponding to each whale audio node feature. An edge matrix is constructed using all the whale audio node features and all the neighbor node features; Multiple edge nodes are obtained from the edge matrix; The second equation is used to calculate the adjacency matrix for each edge node and any remaining edge node, resulting in multiple adjacency elements corresponding to each edge node. The original adjacency matrix is then constructed using all these adjacency elements. The second equation is: , in, For the first The edge node and the first The adjacent elements of each edge node For the first The edge node and the first The association weight of each edge node.
[0037] Preferably, N can be 8.
[0038] Understandably, in the theoretical framework of graph theory, a graph is a mathematical tool for describing sets of entities and their interactions. A basic graph consists of two elements: nodes and edges. Nodes are abstract representations of entities (such as time-frequency frames of whale sounds, individual organisms, system components, etc.), describing the attribute information carrying the entity. Edges are quantitative expressions of the relationships between entities (such as temporal connections of signals, feature similarities, causal interactions, etc.), characterizing the interaction patterns between nodes through connections. The structured "node-edge" model of graphs is suitable for unstructured signals with complex feature relationships, such as whale sounds. Graphs can link the logical relationships between features with a structural foundation. The mathematical essence of graph structure data is a binary tuple, as shown below: , in: Represents a set of nodes; Represents the set of edges.
[0039] Specifically, in the process of constructing graph structured data, the adjacency matrix is a mathematical description of the strength of the associations between nodes. It transforms the topological relationships of "node-edge" into a computable numerical form through the numerical values of its elements, providing structured input for graph neural networks. The dimension of the adjacency matrix is the same as the number of nodes. Let the graph have a total of... If there are nodes, then the adjacency matrix for A square matrix (i.e., the original adjacency matrix), where elements Defined as: , In the formula, For nodes and The association weights between nodes are typically normalized to 0-1. Larger values indicate stronger associations. The definition of weights needs to be tailored to the specific scenario: in time-series data, they can be calculated based on the time interval between nodes (e.g., smaller intervals result in higher weights); in feature data, they can be calculated based on the similarity of node attributes (e.g., normalized values of cosine similarity or Euclidean distance).
[0040] Specifically, in constructing the graph structure of whale sounds, the K-Nearest Neighbor (KNN) graph construction method is used to establish a feature topological association model graph. Due to the correlation characteristics between whale sound segments, the similarity of node features can be quantified using k-nearest neighbor (KNN). For each node, its feature distance to all other nodes is calculated, and edges are established connecting the K nearest nodes. In the graph structure of this invention, the feature vectors of the whale sound signal serve as the node set V, and the node attributes are composed of one-dimensional feature vectors of the sound signal of the corresponding frame. The similarity between feature vectors... The original similarity (i.e., the initial similarity) is used as the basis for constructing edge E. The specific construction steps are as follows: (1) For the feature vectors in the sample set and (i.e., whale audio node features), where The feature dimension is denoted as MFCC in this invention. The Euclidean distance (i.e., the original similarity) between two MFCCs is calculated using the following formula: , The calculation of Euclidean distance is a quantification of the absolute difference between two points in a feature space. The smaller the value, the better the two feature points are. The closer the distance in the feature space of a dimension, the higher the overlap of their feature distributions, that is, the stronger the similarity; if The larger the value, the greater the deviation between the two in the feature space, and the weaker their similarity.
[0041] (2) For each feature vector (i.e., whale audio node features), first calculate its features with all other feature vectors. similarity (i.e., the original similarity), and sort these values in ascending order. Select the k closest feature vectors with the strongest similarity. And record With these k The edges between them form an edge (i.e., the edge matrix).
[0042] By analogy, for each feature vector in a sample, k associated feature vectors are found, ultimately forming a set of nodes. Sum of edges Feature sample map .
[0043] In the above embodiments, the original adjacency matrix is obtained by constructing a network using a graph structure and performing adjacency matrix analysis on the original node feature matrix. This improves the recognition accuracy in complex marine environments, effectively uncovers species-specific associations of acoustic fragments, and solves the shortcomings of traditional methods in non-local association modeling and structural interaction representation.
[0044] Optionally, as an embodiment of the present invention, the process of performing normalization analysis on the original adjacency matrix to obtain a normalized adjacency matrix includes: The original adjacency matrix is calculated using the third equation to obtain the adjacency matrix to be processed. The adjacency matrix to be processed includes multiple adjacency features to be processed. The third equation is: , in, The adjacency matrix to be processed. This is the original adjacency matrix. It is the identity matrix; The fourth equation is used to calculate each of the adjacent features to be processed, and a degree matrix is constructed using all the calculation results. The fourth equation is: , in, For the first Line number Degree features of the column To list the total, For the first Line number The adjacency characteristics of the column to be processed; The normalized adjacency matrix is obtained by calculating the degree matrix and the adjacency matrix to be processed using the fifth equation. The fifth equation is: , in, For the normalized adjacency matrix, For degree matrix, The adjacency matrix to be processed.
[0045] It should be understood that (Add a self-loop to allow the node to contain its own information). yes The degree matrix, It is a learnable weight matrix.
[0046] In the above embodiments, the original adjacency matrix is normalized to obtain a normalized adjacency matrix, which effectively uncovers the species-specific associations of acoustic fragments and solves the shortcomings of traditional methods in nonlocal association modeling and structural interaction representation.
[0047] Optionally, as an embodiment of the present invention, the process of performing convolution processing on the normalized adjacency matrix through multiple graph convolutional layers to obtain the feature matrix of the nodes to be processed includes: The normalized adjacency matrix is calculated using the sixth equation to obtain the feature matrix of the nodes to be processed. The sixth equation is: , in, For the first The feature matrix of the node to be processed corresponding to each graph convolutional layer. For activation function, For the normalized adjacency matrix, For the first The feature matrix of the node to be processed corresponding to each graph convolutional layer. For the normalized adjacency matrix, For the first The learnable weight matrix corresponding to each graph convolutional layer.
[0048] Specifically, Graph Convolutional Networks (GCNs) are the most basic application of GNNs. Their core is to extend traditional convolution operations from Euclidean space (such as image grids) to non-Euclidean graph-structured data. This invention achieves node feature aggregation through spectral domain graph convolution. Spectral domain graph convolution is based on the Laplacian matrix of graphs and spectral decomposition theory, defining graph convolution as a filtering operation of signals in the Fourier domain. Let the adjacency matrix of the graph... The degree matrix is and Then the Laplace matrix of the graph is ,right Perform eigenvalue decomposition ,in It is the eigenvector matrix. If the eigenvalues are a diagonal matrix, then the mathematical expression for convolution on the graph is: , in It is a filter function. These are the feature vectors of nodes in the graph. In practical applications, they are usually computed using an approximate graph convolutional layer, and their mathematical expression is: .
[0049] In the above embodiments, the normalized adjacency matrix is convolved by multiple graph convolutional layers to obtain the feature matrix of the node to be processed, thereby realizing the aggregation of node features and effectively mining the species-specific association of acoustic fragments. This solves the defects of traditional methods in non-local association modeling and structural interaction expression.
[0050] Optionally, as an embodiment of the present invention, the process of calculating the loss function for all the true labels and all the predicted probabilities to obtain the loss function includes: The loss function is obtained by calculating all the true labels and all the predicted probabilities using the seventh equation, which is: , in, For loss function, To predict the total probability, The total number of types, For the first The first of the first category A real label, For the first The first of the first category Each predicted probability.
[0051] It should be understood that the loss function is as follows: , in, The total number of samples, For the first The true label of each sample For the predicted first The sample belongs to the first The probability of a class (i.e., the predicted probability).
[0052] In the above embodiments, loss functions are calculated for all real labels and all predicted probabilities to obtain the loss function, which improves the recognition accuracy in complex marine environments, effectively explores the species-specific associations of sound wave segments, and solves the defects of traditional methods in non-local association modeling and structural interaction representation.
[0053] Optionally, as another embodiment of the present invention, to address the difficulty of existing acoustic recognition methods for cetacean species in uncovering structural correlations between sound segments, this invention takes cetacean sound signals as the research object, uses 13-dimensional Mel-frequency cepstral coefficients as model input features, and constructs a "node-edge" graph structure through the K-nearest neighbor algorithm to transform acoustic features into structured data with correlation characteristics. Based on a spectral domain graph neural network consisting of a three-layer graph convolutional network, a global pooling layer, and a classification layer, node and neighbor features are aggregated to achieve end-to-end species classification. Experimental results on publicly available cetacean sound datasets show that the accuracy of this invention on the test set reaches 96.37%, outperforming support vector machines, convolutional neural networks, and traditional graph neural network methods. Research indicates that graph structure data and graph neural networks can fully utilize the structure between sound data and uncover feature-structure correlation information in cetacean sounds, providing an efficient and feasible application solution for acoustic recognition among cetacean species.
[0054] Optionally, as another embodiment of the present invention, this invention uses a framework of feature extraction-graph structure modeling-graph convolutional classification, combining acoustic features with graph structure data structures; it employs a K-nearest neighbors (KNN) graph structure modeling method based on MFCC features, first transforming the spectral features of whale sounds into a node-edge graph structure—using the MFCC feature vectors of framed sound segments as nodes, calculating the feature similarity between segments and constructing edges through the KNN algorithm, thus obtaining node-edge graph structure data; finally, a custom graph convolutional network (GCN) is built to learn features from the graph structure data, aggregating the structural features of nodes and their neighbors through multi-layer graph convolution, and outputting species identification results. This method overcomes the limitation of traditional deep learning in its lack of attention to the structural information of temporal signals such as sound, fully exploring the correlation in sound signals, and can improve the recognition accuracy in complex marine environments.
[0055] Optionally, as another embodiment of the present invention, the passive acoustic monitoring (PAM) of the present invention can capture the sounds emitted by various cetaceans. Cetaceans use sound signals for communication, navigation, reproduction, and hunting. These sound signals not only have significant temporal dynamic changes and frequency distribution differences, but also have local correlation structures between sound wave segments (such as the temporal connection of different syllables and the coordinated changes of frequency components). In order to better understand the temporal and frequency domain characteristics of cetacean sounds and the reason why cetacean sound data can be applied to graph learning, the MFCC feature map of humpback whale sound signals is used for analysis.
[0056] Alternatively, as another embodiment of the present invention, the present invention has undergone... After layer iteration, the features of each node It contains its own attributes and also incorporates... The topological and feature information of layer neighbors can be well adapted to the graph structure data of cetacean sounds (nodes are MFCC feature vectors of whistle pulse clusters, and edges are the correlations between feature vectors), effectively capturing the temporal and feature correlations between different cetacean sound frames, thereby enabling species identification of cetaceans.
[0057] Optionally, as another embodiment of the present invention, as shown in Table 1, the experimental dataset of the present invention comes from the WHOI Cetacean Sound Database. This database is maintained by the Woods Hole Oceanographic Institution in the United States and is a globally authoritative cetacean acoustic data resource platform, containing original sound records of cetaceans from multiple sea areas and species. Original sound records of six cetaceans used in this experiment were extracted from this website, including WAV format sound files of six species: beluga whale, bowhead whale, false killer whale, fin whale, humpback whale, and killer whale, and stored in corresponding subfolders under the database directory according to species classification. Using a data crawling function, 2000 sound data points for each cetacean species were downloaded from the WHOI public website, and the dataset was randomly divided into training and test sets at a 9:1 ratio, stored in the train and test folders under the same directory, respectively. The data is preprocessed using the dioGraphDataset class: the original audio signal is segmented into frames, then 13-dimensional MFCC features are extracted as graph nodes, edge indices are constructed using KNN (k=8), and finally the data is encapsulated into graph structure data containing node features, edge indices and labels for model input. Table 1 is a statistical table of the dataset.
[0058] Table 1 Optionally, as another embodiment of the present invention, the training and testing process is completed on a server equipped with four NVIDIA GeForce RTX 4090 graphics cards, and the server's operating system is Windows 10 Professional. The specific hardware and software environment configuration is shown in Table 2.
[0059] Table 2 Table 3 shows the important parameter configurations involved in the training and testing of the model built in this invention. This table includes the MFCC processing of cetacean sound signals, graph construction rules, and the structural design of the spectral domain GCN model. Table 3 lists the important parameter settings.
[0060] Table 3 Optionally, as another embodiment of the present invention, in the task of recognizing the sound signals of cetaceans, in addition to the two core indicators of accuracy and loss function commonly used by neural network models, the special indicator of class balance F1 value can also reflect the performance of the model according to the task scenario and data characteristics.
[0061] First, accuracy and loss function are fundamental and universal metrics for evaluating model performance. Accuracy reflects the proportion of correct classifications made by the model overall, and can intuitively demonstrate the model's ability to identify cetacean species; the loss function characterizes the difference between the model's predicted values and the true labels, and its decreasing trend can verify the model's convergence and learning effect.
[0062] In this scenario of applying graph neural networks to cetacean sound recognition, the data is characterized by "multiple species categories and relatively small sample sizes for some species." Focusing solely on accuracy may not fully reflect the model's balanced recognition across different species. Therefore, the class balance F1 score becomes crucial. It is the average F1 score for each species category (macro-average F1), effectively measuring the model's overall performance in multi-category, unevenly distributed scenarios. A high F1 score indicates that the model not only achieves overall accuracy but also maintains good recognition precision for cetacean species with limited sample sizes. This demonstrates that graph neural networks, when fusing acoustic features with species topology, exhibit fairness and robustness in capturing features from different species categories, ultimately ensuring the reliability and rationality of this application in practical scenarios such as cetacean ecological monitoring and automatic species classification. The formula is as follows: , in, This represents the number of cetacean species in this experiment. To indicate the first The number of samples in which the species was correctly predicted. Other classes were misclassified as the first Number of samples in the class For non-first The number of samples that are correctly predicted. For the first The number of samples that were misclassified as other classes.
[0063] , in, For the accuracy of class 1, The recall rate for class 1 indicates that when the sample size of a certain class is small, its recognition performance will directly affect the final result of the model. In other words, it can effectively reflect the model's recognition balance for different cetacean species, thereby verifying the reliability of graph neural networks in the application of cetacean species sound recognition.
[0064] Optionally, as another embodiment of the present invention, this invention addresses the difficulty in capturing structural associations of sound segments in the acoustic recognition of cetaceans. It proposes a graph neural network-based recognition method, constructing an end-to-end framework of "feature extraction - graph structure modeling - graph convolutional classification." A graph structure is built using KNN (K=8) to effectively mine species-specific associations of sound wave segments. A GCN model based on spectral domain graph convolution achieves deep fusion of acoustic features and topological structure, overcoming the shortcomings of traditional methods in non-local association modeling and structural interaction representation. The present invention achieves an accuracy of 96.37% on the WHOI dataset, with recognition accuracy approaching or reaching 98% for most species. Its performance surpasses SVM, CNN, and traditional GCN methods. Furthermore, K=8 was determined to be the optimal parameter through K value comparison. The designed visualization interface enhances its practical applicability, providing a new path for cetacean acoustic recognition and laying the foundation for subsequent optimizations (such as dynamic edge weights and model lightweighting).
[0065] Figure 4 This is a block diagram of a whale sound recognition device provided in an embodiment of the present invention.
[0066] Alternatively, as another embodiment of the present invention, such as Figure 4 As shown, a whale sound recognition device includes: The dataset collection module is used to obtain multiple raw whale audio data from the WHOI whale sound library and combine all the raw whale audio data to obtain a raw whale audio dataset; The preprocessing module is used to preprocess the original whale audio dataset to obtain a preprocessed whale audio dataset. The feature analysis module is used to perform feature analysis on the preprocessed whale audio dataset to obtain multiple target whale audio features, and to combine all the target whale audio features to obtain the original node feature matrix; The model analysis module is used to construct a training model and perform model analysis on the training model using the original node feature matrix to obtain a sound recognition model. The import module is used to import the audio data of whales to be identified; The recognition result module is used to recognize the whale audio data to be recognized through the sound recognition model to obtain whale sound recognition results.
[0067] Optionally, another embodiment of the present invention provides a whale sound recognition system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the whale sound recognition method as described above. This system can be a computer or similar system.
[0068] Optionally, another embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the whale sound recognition method as described above.
[0069] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0070] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0071] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.
[0072] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.
[0073] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0074] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0075] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for whale sound recognition, characterized in that, Includes the following steps: Multiple raw whale audio data were obtained from the WHOI whale sound database, and all the raw whale audio data were combined to obtain a raw whale audio dataset; The original whale audio dataset is preprocessed to obtain a preprocessed whale audio dataset; Feature analysis is performed on the preprocessed whale audio dataset to obtain multiple target whale audio features, and all the target whale audio features are combined to obtain the original node feature matrix; A training model is constructed, and the training model is analyzed using the original node feature matrix to obtain a sound recognition model. Import the whale audio data to be identified, and use the sound recognition model to identify the whale audio data to obtain whale sound recognition results.
2. The whale sound recognition method according to claim 1, characterized in that, The process of preprocessing the original whale audio dataset to obtain the preprocessed whale audio dataset includes: The original whale audio dataset is converted to a new format to obtain a converted whale audio dataset. Each converted whale audio data in the converted whale audio dataset is processed into frames to obtain multiple framed whale audio data, and all the framed whale audio data are combined to obtain a framed whale audio dataset. The segmented whale audio dataset is subjected to noise reduction processing to obtain a noise-reduced whale audio dataset. The denoised whale audio dataset is normalized to obtain a normalized whale audio dataset. The normalized whale audio dataset is then windowed to obtain a preprocessed whale audio dataset.
3. The whale sound recognition method according to claim 1, characterized in that, The process of performing feature analysis on the preprocessed whale audio dataset to obtain multiple target whale audio features includes: The preprocessed whale audio dataset is subjected to short-time Fourier transform to obtain the original whale audio feature set; The original whale audio feature set is filtered by a Mel filter bank to obtain a filtered whale audio feature set. Logarithmic calculation is performed on the filtered whale audio feature set to obtain the whale audio feature set to be processed; The discrete cosine transform is performed on the whale audio feature set to be processed to obtain the transformed whale audio feature set. Multiple 13-dimensional MFCC features are extracted from the transformed whale audio feature set, and the 13-dimensional MFCC features are used as target whale audio features to obtain multiple target whale audio features.
4. The whale sound recognition method according to claim 1, characterized in that, The training model includes a graph structure network, multiple graph convolutional layers, a global pooling layer, and a classification layer. The process of performing model analysis on the trained model using the original node feature matrix to obtain the sound recognition model includes: The original adjacency matrix is obtained by performing adjacency matrix analysis on the original node feature matrix through the network constructed by the graph structure. The original adjacency matrix is normalized to obtain the normalized adjacency matrix; The normalized adjacency matrix is convolved by multiple graph convolutional layers to obtain the feature matrix of the node to be processed. The feature matrix of the node to be processed is pooled by the global pooling layer to obtain the pooled node feature matrix. The pooled node feature matrix is classified through the classification layer to obtain multiple predicted probabilities; Import multiple real labels, calculate the loss function for all real labels and all predicted probabilities, and obtain the loss function. The parameters of the trained model are updated according to the loss function to obtain the sound recognition model.
5. The whale sound recognition method according to claim 4, characterized in that, The original node feature matrix includes multiple whale audio node features. The process of constructing a network using the graph structure to perform adjacency matrix analysis on the original node feature matrix to obtain the original adjacency matrix includes: The first formula is used to calculate the original similarity scores corresponding to each of the whale audio node features and any remaining whale audio node features. ， in, For the first The characteristics of the first whale audio node and the first The original similarity of the audio node features of each whale species. This represents the total number of audio node features in whales. For the first Individual whale audio node features For the first Individual whale-like audio node features; The original similarity scores corresponding to each whale audio node feature are sorted according to their original similarity scores to obtain the sorted similarity scores corresponding to each whale audio node feature. The first N whale audio node features corresponding to the sorted similarity are used as neighbor node features, thereby obtaining multiple neighbor node features corresponding to each whale audio node feature. An edge matrix is constructed using all the whale audio node features and all the neighbor node features; Multiple edge nodes are obtained from the edge matrix; The second equation is used to calculate the adjacency matrix for each edge node and any remaining edge node, resulting in multiple adjacency elements corresponding to each edge node. The original adjacency matrix is then constructed using all these adjacency elements. The second equation is: ， in, For the first The edge node and the first The adjacent elements of each edge node For the first The edge node and the first The association weight of each edge node.
6. The whale sound recognition method according to claim 4, characterized in that, The process of performing normalization analysis on the original adjacency matrix to obtain the normalized adjacency matrix includes: The original adjacency matrix is calculated using the third equation to obtain the adjacency matrix to be processed. The adjacency matrix to be processed includes multiple adjacency features to be processed. The third equation is: ， in, The adjacency matrix to be processed. This is the original adjacency matrix. It is the identity matrix; The fourth equation is used to calculate each of the adjacent features to be processed, and a degree matrix is constructed using all the calculation results. The fourth equation is: ， in, For the first Line number Degree features of the column To list the total, For the first Line number The adjacency characteristics of the column to be processed; The normalized adjacency matrix is obtained by calculating the degree matrix and the adjacency matrix to be processed using the fifth equation. The fifth equation is: ， in, For the normalized adjacency matrix, For degree matrix, The adjacency matrix to be processed.
7. The whale sound recognition method according to claim 4, characterized in that, The process of performing convolution processing on the normalized adjacency matrix through multiple graph convolutional layers to obtain the feature matrix of the nodes to be processed includes: The normalized adjacency matrix is calculated using the sixth equation to obtain the feature matrix of the nodes to be processed. The sixth equation is: ， in, For the first The feature matrix of the node to be processed corresponding to each graph convolutional layer. For activation function, For the normalized adjacency matrix, For the first The feature matrix of the node to be processed corresponding to each graph convolutional layer. For the normalized adjacency matrix, For the first The learnable weight matrix corresponding to each graph convolutional layer.
8. The whale sound recognition method according to claim 4, characterized in that, The process of calculating the loss function for all the true labels and all the predicted probabilities includes: The loss function is obtained by calculating all the true labels and all the predicted probabilities using the seventh equation, which is: ， in, For loss function, To predict the total probability, The total number of types, For the first The first of the first category A real label, For the first The first of the first category Each predicted probability.
9. A whale sound recognition device, characterized in that, include: The dataset collection module is used to obtain multiple raw whale audio data from the WHOI whale sound library and combine all the raw whale audio data to obtain a raw whale audio dataset; The preprocessing module is used to preprocess the original whale audio dataset to obtain a preprocessed whale audio dataset. The feature analysis module is used to perform feature analysis on the preprocessed whale audio dataset to obtain multiple target whale audio features, and to combine all the target whale audio features to obtain the original node feature matrix; The model analysis module is used to construct a training model and perform model analysis on the training model using the original node feature matrix to obtain a sound recognition model. The import module is used to import the audio data of whales to be identified; The recognition result module is used to recognize the whale audio data to be recognized through the sound recognition model to obtain whale sound recognition results.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the whale sound recognition method as described in any one of claims 1 to 8.