A sound recognition method, device, electronic device and storage medium

Through deep learning methods, the recording data of Hainan gibbons are preprocessed and classified, and the lightweight model is used to realize real-time identification of Hainan gibbons audio, solving the problem of low recognition efficiency in the existing technology, and achieving efficient monitoring of Hainan gibbons sound.

CN114049899BInactive Publication Date: 2024-11-26RES INST OF FOREST RESOURCE INFORMATION TECHN CHINESE ACADEMY OF FORESTRY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111390367.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-23
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing technology is difficult to quickly and accurately identify the sound of Hainan gibbons, resulting in low monitoring efficiency of Hainan gibbons and inability to achieve efficient monitoring.

Method used

Deep learning method is adopted to obtain gibbon recording data for preprocessing, extract Mel spectral features, and classify them based on the trained lightweight gibbon sound classification model to achieve real-time recognition of gibbon audio clips.

Benefits of technology

Real-time and efficient monitoring of Hainan gibbon sounds is achieved, recognition speed and accuracy are improved, manpower and material investment is reduced, and data continuity and integrity are ensured.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114049899B_ABST
    Figure CN114049899B_ABST
Patent Text Reader

Abstract

The present application relates to the field of sound recognition technology, and specifically provides a sound recognition method, device, electronic device and storage medium, which pre-process the acquired gibbon recording data, extract Mel spectrum features, obtain the gibbon audio clips in the gibbon recording data to be classified by using the gibbon sound classification model trained with Mel spectrum features, and splice and crop the gibbon audio clips to obtain complete gibbon audio, and then record the calling time of the gibbon in the target area, so as to realize efficient monitoring of the gibbon. Because the adopted gibbon sound classification model is more lightweight, it can quickly identify the gibbon audio clips in the gibbon recording data, and realize real-time monitoring of the gibbon.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of sound recognition, and in particular to a sound recognition method, device, electronic device and storage medium. Background Art

[0002] The Hainan gibbon is one of the flagship species of the Hainan Tropical Rainforest National Park. It is a national first-class protected species and has been listed as one of the "world's most endangered primates" in the Red List of the World Conservation Union. It has important research significance and conservation value. Efficient monitoring of the Hainan gibbon is a prerequisite for its protection and a necessary means to strengthen protection.

[0003] Due to the unique calls of gibbons, the current monitoring method for gibbons is mainly based on sound monitoring. The key issue of this method is how to quickly and accurately identify the sounds of gibbons. At present, the sound identification method of gibbons is mainly based on sound interpretation by professionals, through tracking and recording by professionals, or through the use of deployed equipment to collect sound data and then manually interpret it. Although the method of tracking and recording by professionals can achieve precise data acquisition, it cannot ensure the continuity and integrity of the data; although the method of arranging equipment to collect sound data can make up for the shortcomings in data acquisition continuity and integrity, the data retrieved regularly still requires a lot of manpower and material resources to screen and identify. Overall, due to the limitations of the two methods, achieving efficient monitoring of Hainan gibbons is still a problem that needs to be solved. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a sound recognition method, device, electronic device and storage medium, which can realize efficient monitoring of gibbons.

[0005] The present invention provides a method for voice recognition, comprising the following steps:

[0006] Obtain gibbon recording data;

[0007] Preprocessing the gibbon recording data to obtain gibbon recording data to be classified;

[0008] Extracting Mel spectrum features of the gibbon recording data to be classified;

[0009] Based on the trained gibbon sound classification model, the gibbon recording data to be classified is classified according to the Mel spectrum features to obtain the gibbon audio clips in the gibbon recording data to be classified.

[0010] In some embodiments, the step of obtaining gibbon recording data comprises the following steps:

[0011] Gibbon audio recording data is collected in real time by using a recording device in a gibbon group, wherein the gibbon audio recording data includes segments containing gibbon audio and segments not containing gibbon audio.

[0012] In some embodiments, the preprocessing of the gibbon recording data to obtain the gibbon recording data to be classified comprises the following steps:

[0013] Sampling and quantizing the gibbon recording data;

[0014] Pre-emphasize the sampled and quantized gibbon recording data;

[0015] Frame the pre-emphasized gibbon recording data;

[0016] The framed gibbon recording data is windowed to obtain the gibbon recording data to be classified.

[0017] In some embodiments, the step of extracting the Mel spectrum features of the gibbon recording data to be classified comprises the following steps:

[0018] Performing fast Fourier transform on the gibbon recording data to be classified to obtain a signal spectrum;

[0019] Performing Mel frequency conversion on the signal spectrum to obtain Mel frequency;

[0020] The Mel frequency is subjected to Mel filtering to obtain Mel spectrum characteristics.

[0021] In some embodiments, the method of classifying the to-be-classified gibbon recording data based on the trained gibbon sound classification model according to the Mel spectrum features to obtain the gibbon audio in the to-be-classified gibbon recording data comprises the following steps:

[0022] Expanding the dimension of the Mel spectrum feature, and inputting the Mel spectrum feature after the dimension expansion into the convolution layer of the trained gibbon sound classification model to obtain feature data; wherein the trained gibbon sound classification model includes five convolution layers with convolution kernels of 3*3;

[0023] The feature data is subjected to data enhancement processing, and the feature data after data enhancement processing is input into the fully connected layer of the trained gibbon sound classification model to output the gibbon audio segment in the gibbon recording data to be classified.

[0024] In some embodiments, after outputting the obtained gibbon audio segment in the gibbon recording data to be classified, the following steps are also included:

[0025] The gibbon audio clips are spliced ​​and trimmed to obtain complete gibbon audio, and the calling time of the gibbon is recorded.

[0026] In some embodiments, the generalization ability of the trained gibbon sound classification model is evaluated according to the F-Score indicator.

[0027] An embodiment of the present application provides a sound recognition device, comprising:

[0028] An acquisition module is used to obtain gibbon recording data;

[0029] A preprocessing module, used for preprocessing the gibbon recording data to obtain the gibbon recording data to be classified;

[0030] An extraction module, used for extracting Mel spectrum features of the gibbon recording data to be classified;

[0031] The classification module is used to classify the gibbon recording data to be classified according to the Mel spectrum features based on the trained gibbon sound classification model, and obtain the gibbon audio clips in the gibbon recording data to be classified.

[0032] An electronic device provided in an embodiment of the present application includes a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the sound recognition method described in any one of the above items are performed.

[0033] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned sound recognition methods are executed.

[0034] The present application discloses a method, device, electronic device and storage medium for sound recognition, which pre-processes the acquired gibbon recording data to obtain the gibbon recording data to be classified; then extracts the Mel spectrum features of the gibbon recording data to be classified; and obtains the gibbon audio clips in the gibbon recording data to be classified by inputting the Mel spectrum features into the trained gibbon sound classification model. The gibbon sound classification model used is more lightweight, has a fast recognition speed, and has an accurate recognition effect after pre-processing in advance, which can realize real-time and efficient monitoring of gibbon sounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 A flow chart of a sound recognition method provided by an embodiment of the present application is shown;

[0037] Figure 2 A flowchart of preprocessing gibbon recording data provided by an embodiment of the present application is shown;

[0038] Figure 3 A flowchart of extracting Mel spectrum features of gibbon recording data to be classified provided in an embodiment of the present application is shown;

[0039] Figure 4 A flowchart of obtaining gibbon audio based on a trained gibbon sound classification model provided in an embodiment of the present application is shown;

[0040] Figure 5 The structure block diagram of the voice recognition device provided by the embodiment of the present application is shown;

[0041] Figure 6 A structural block diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0042] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0043] With the vigorous development of big data and artificial intelligence technology, deep learning methods have been widely used in the field of sound recognition. Deep learning can automatically extract features and has the advantages of strong learning ability, good robustness, and strong generalization ability. It is suitable for efficient processing of large amounts of data, while reducing manpower and material resources and ensuring the accuracy of the results. However, complex neural network models take a lot of time to calculate, and as the number of layers increases, they are often accompanied by gradient diffusion and gradient disappearance, so it is difficult to ensure the real-time results. Based on this, this paper proposes a sound recognition method, device, electronic device and storage medium, which can recognize the sound of gibbons in real time and efficiently based on the deep learning method of the lightweight network model.

[0044] As the instruction manual Figure 1 As shown, the embodiment of the present application provides a sound recognition method, comprising the following steps:

[0045] S1. Obtain gibbon recording data;

[0046] S2, preprocessing the gibbon recording data to obtain the gibbon recording data to be classified;

[0047] S3, extracting Mel spectrum features of the gibbon recording data to be classified;

[0048] S4. Based on the trained gibbon sound classification model, classify the gibbon recording data to be classified according to the Mel spectrum features to obtain the gibbon audio clips in the gibbon recording data to be classified.

[0049] In step S1, gibbon audio recording data is collected in real time by placing a recording device in the gibbon group, wherein the collected gibbon audio recording data includes segments containing gibbon audio and segments not containing gibbon audio.

[0050] Furthermore, in order to improve the recording quality of the recording device, the recording device can adopt multiple noise-reducing silicon microphones; further, the recording device has a communication function, such as a 4G / 5G module, so that the collected gibbon recording data can be obtained from the recording device in real time; further, the recording device has a positioning function, such as a GPS module, so that the target area of ​​the obtained gibbon recording data can be determined in real time, which is convenient for tracking and monitoring of gibbons.

[0051] In this embodiment, the gibbon recording data collection area is located in the gibbon C group (109°14′38.16″E, 19°5′49.48″N) and E group (109°14′18.53″E, 19°11′13.80″N) of the Hainan Tropical Rainforest National Park. The gibbon recording data collection time is 36.16 hours in total, and the collection time is distributed from March 24, 2021 to July 31, 2021. A GPS recording locator with 4G transmission function is used to collect data. The recording part of the equipment uses 3 noise-reducing silicon microphones to ensure the recording quality. The sampling rate of the gibbon recording data obtained by the equipment is 16KHz, the sampling accuracy is 16bit, and the file type is wav format.

[0052] It should be noted that obtaining gibbon recording data is not limited to the gibbon recording data collected in real time by the recording equipment placed in the gibbon group. The gibbon recording data can also be obtained from a third party, which does not constitute a limitation on the present application.

[0053] In step S2, after obtaining the gibbon recording data, it is necessary to pre-process the obtained gibbon recording data. Figure 2 As shown, preprocessing the acquired gibbon recording data to obtain the gibbon recording data to be classified includes the following steps:

[0054] S201, sampling and quantizing the gibbon recording data;

[0055] S202, pre-emphasize the sampled and quantized gibbon recording data;

[0056] S203, dividing the pre-emphasized gibbon recording data into frames;

[0057] S204, adding windows to the framed gibbon recording data to obtain the gibbon recording data to be classified.

[0058] In this embodiment, the acquired gibbon recording data is first sampled and quantized at a sampling rate of 16KHz; then input into a high-pass filter to complete the pre-emphasis of the sampled gibbon recording data, and the high-pass filter used is:

[0059] H(z)=1-αz -1

[0060] Among them, z -1 It indicates that the filter H(z) is a first-order filter, and α is a constant, which is taken as 0.95 here.

[0061] Then, the pre-emphasized gibbon recording data is framed, with 400 sampling points (25ms) as the frame length, 200 sampling points (12.5ms) as the frame shift, and the frame length frame shift overlap rate is 50% to ensure the continuity of the sound; finally, windowing is performed to solve the problem of discontinuity at both ends of the gibbon recording data after framing, and the gibbon recording data to be classified is obtained. The window function calculation formula used is:

[0062]

[0063] Among them, n represents the sampling point and N represents the frame length.

[0064] In step S3, after obtaining the recording data of the gibbon to be classified, it is necessary to extract the Mel spectrum features of the recording data of the gibbon to be classified. Figure 3 As shown, extracting the Mel spectrum features of the gibbon recording data to be classified includes the following steps:

[0065] S301, performing fast Fourier transform on the recorded data of the gibbon to be classified to obtain a signal spectrum;

[0066] S302, performing Mel frequency conversion on the signal spectrum to obtain Mel frequency;

[0067] S303: Perform Mel filtering on the Mel frequency to obtain Mel spectrum features.

[0068] In this embodiment, the recorded data of the gibbon to be classified is first subjected to a fast Fourier transform to obtain a signal spectrum, and the calculation formula is:

[0069]

[0070] Wherein, X(i,k) represents the signal spectrum, i represents the frame subscript, and 0≤k≤N-1.

[0071] Then, the linear frequency result obtained by fast Fourier transform, that is, the signal spectrum, is converted into Mel frequency. The calculation formula is:

[0072]

[0073] Wherein, Mel(f) represents the Mel frequency, and f represents the linear frequency.

[0074] Finally, the Mel frequency is input into the Mel filter bank for Mel filtering to obtain the Mel spectrum characteristics. The calculation formula is:

[0075]

[0076] Wherein, m represents the filter number, 0≤m≤M, M represents the number of filters, and 80 Mel filters are selected for filtering.

[0077] In step S4, in order to achieve the lightweight and recognition effect of the gibbon sound classification model, the gibbon sound classification model trained in this application is improved based on ResNet34.

[0078] Among them, in order to make the trained gibbon sound classification model more lightweight, the size and number of convolution kernels of the trained gibbon sound classification model are reduced compared with ResNet34. The trained gibbon sound classification model reduces the convolution kernel size of the first layer from 7x7 to 3x3, sets the number of convolution kernels of the second layer to one-fourth of the original, and the step size is set to (2, 1), and sets the number of convolution kernels of the third, fourth, and fifth layers to one-fourth of the original, and the step size is (2, 2); in order to make the trained gibbon sound classification model achieve better recognition effect, the mean and variance of the feature data obtained by convolution processing are calculated, and the calculated mean and variance of the feature data are cascaded as the input of the fully connected layer. Since the mean can reflect the average level of the feature data and the variance can reflect the degree of discreteness of the feature data, using the variance and mean of the feature data as the input of the fully connected layer can make the recognition effect better. Therefore, before entering the fully connected layer, the variance and mean of the feature data are calculated, and the mean and variance are vertically spliced ​​and then input into the fully connected layer for classification. Among them, the comparison between the trained gibbon sound classification model and the ResNet34 network structure is as follows:

[0079] As shown in Table 1.

[0080]

[0081] Table 1

[0082] When training the gibbon sound classification model, taking the 3s gibbon recording data as an example, after preprocessing and extracting the Mel spectrum features, a 80*24 Mel spectrum feature can be obtained, and then the Mel spectrum feature is dimensionally expanded. The size of the expanded Mel spectrum feature becomes 10*1*80*241. The expanded result is input into the gibbon sound classification model. First, through the first convolutional layer Conv1, a feature map of 10*16*42*123 can be obtained; then through the second convolutional layer Conv2_x, a feature map of 10*16*21*123 can be obtained; then through the third convolutional layer Conv3_x, a feature map of 10*32*11*62 can be obtained; then through the fourth convolutional layer Conv4_x, a feature map of 10*64*6*31 can be obtained; finally, through the fifth convolutional layer Conv5_x, a feature map of 10*128*3*16 can be obtained. The mean and variance of the third dimension of the result of the fifth convolutional layer Conv5_x are calculated, and the size of the feature map obtained is 10*128*1*16. The variance and the mean are concatenated in the fourth dimension to obtain a feature map size of 10x*128*1*32; the second and third dimensions of the feature map are swapped to obtain a feature map size of 10*128*1*32; the third dimension is reduced and the second and third dimensions are swapped to obtain a feature map of 10*32*128; finally, the second dimension of the above feature map of size 10*32*128 is summed and flattened and input into the fully connected layer to obtain the classification result.

[0083] In addition, when training the gibbon sound classification model, the sample data required can be obtained in the manner described in step S1, and the sample data set is divided into a training set and a test set. When dividing the training set and the test set, a trade-off is required. The smaller the test set data, the less accurate the estimation of the generalization error of the model. In this embodiment, the manually annotated data set is randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1 to obtain the optimal number of training rounds.

[0084] Since the trained gibbon sound classification model is a binary classification model, that is, it identifies segments containing gibbon audio and segments without gibbon audio from gibbon recording data, and a single accuracy index cannot well reflect the generalization ability of the model, the F-Score index is used in this application as an index to evaluate the generalization ability of the gibbon sound classification model. Among them, F-Score is a comprehensive evaluation index, which is affected by precision and recall, and is the harmonic average of the two. The calculation method is:

[0085]

[0086] Among them, Pre represents the precision, Re represents the recall, β represents the balance coefficient, and the calculation formulas of Pre and Re are:

[0087]

[0088]

[0089] Among them, TP means that the output of the gibbon sound classification model is true and the label is true; FP means that the output of the gibbon sound classification model is true and the label is false; FN means that the output of the gibbon sound classification model is false and the label is true. β is used to balance the importance of Pre and Re in the F-Score calculation. When β=1, Pre and Re are equally important; when β<1, Pre has a greater impact on the result; when β>1, Re has a greater impact on the result. In this embodiment, β=1.

[0090] The steps of training the gibbon sound classification model should be technical means well known to those skilled in the art and will not be elaborated here.

[0091] As the instruction manual Figure 4 As shown, based on the trained gibbon sound classification model, the gibbon recording data to be classified is classified according to the Mel spectrum features to obtain the gibbon audio in the gibbon recording data to be classified, including the following steps:

[0092] S401, dimensionally expand the Mel spectrum feature, and input the dimensionally expanded Mel spectrum feature into a convolution layer of a trained gibbon sound classification model to obtain feature data; wherein the trained gibbon sound classification model includes five convolution layers, each of which has a convolution kernel of 3*3;

[0093] S402, performing data enhancement processing on the feature data, and inputting the feature data after data enhancement processing into the fully connected layer of the trained gibbon sound classification model to output the gibbon audio segment in the gibbon recording data to be classified.

[0094] This step is the same as the step of processing Mel spectrum features by the gibbon sound classification model when training the gibbon sound classification model, and will not be repeated here.

[0095] Furthermore, after obtaining the gibbon audio clips in the gibbon recording data to be classified, the gibbon audio clips are spliced ​​and cropped to obtain complete gibbon audio, and the calling time of the gibbon is recorded.

[0096] The present application provides a sound recognition method, which pre-processes the acquired gibbon recording data, extracts Mel spectrum features, obtains the gibbon audio clips in the gibbon recording data to be classified by using the gibbon sound classification model trained with Mel spectrum features, and splices and cuts the gibbon audio clips to obtain complete gibbon audio, and then records the calling time of the gibbon in the target area, so as to realize efficient monitoring of the gibbon. Because the adopted gibbon sound classification model is more lightweight, it can quickly identify the gibbon audio clips in the gibbon recording data, and realize real-time monitoring of the gibbon.

[0097] Based on the same concept of the present invention, the embodiment of the present application also provides a sound recognition device. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned sound recognition method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0098] As the instruction manual Figure 5 As shown, a sound recognition device provided in an embodiment of the present application includes:

[0099] An acquisition module 501 is used to acquire gibbon recording data;

[0100] A preprocessing module 502 is used to preprocess the gibbon recording data to obtain the gibbon recording data to be classified;

[0101] An extraction module 503 is used to extract the Mel spectrum features of the gibbon recording data to be classified;

[0102] The classification module 504 is used to classify the gibbon recording data to be classified according to the Mel spectrum features based on the trained gibbon sound classification model, and obtain the gibbon audio segment in the gibbon recording data to be classified.

[0103] In a possible implementation, the acquisition module 501 acquires gibbon recording data, including: collecting gibbon recording data in real time in a gibbon group through a recording device, wherein the gibbon recording data includes segments containing gibbon audio and segments not containing gibbon audio.

[0104] In a possible implementation manner, the preprocessing module 502 preprocesses the gibbon recording data to obtain the gibbon recording data to be classified, including:

[0105] Sampling and quantizing the gibbon recording data;

[0106] Pre-emphasize the sampled and quantized gibbon recording data;

[0107] Frame the pre-emphasized gibbon recording data;

[0108] The framed gibbon recording data is windowed to obtain the gibbon recording data to be classified.

[0109] In a possible implementation, the extraction module 503 extracts the Mel spectrum features of the gibbon recording data to be classified, including:

[0110] Performing fast Fourier transform on the gibbon recording data to be classified to obtain a signal spectrum;

[0111] Performing Mel frequency conversion on the signal spectrum to obtain Mel frequency;

[0112] The Mel frequency is subjected to Mel filtering to obtain Mel spectrum characteristics.

[0113] In a possible implementation, the classification module 504 classifies the gibbon recording data to be classified according to the Mel spectrum features based on the trained gibbon sound classification model, and obtains the gibbon audio segment in the gibbon recording data to be classified, including:

[0114] Expanding the dimension of the Mel spectrum feature, and inputting the Mel spectrum feature after the dimension expansion into the convolution layer of the trained gibbon sound classification model to obtain feature data; wherein the trained gibbon sound classification model includes five convolution layers with convolution kernels of 3*3;

[0115] The feature data is subjected to data enhancement processing, and the feature data after data enhancement processing is input into the fully connected layer of the trained gibbon sound classification model to output the gibbon audio segment in the gibbon recording data to be classified.

[0116] In a possible implementation, the device further includes:

[0117] The splicing and cropping module is used to splice and crop the gibbon audio clips to obtain complete gibbon audio and record the calling time of the gibbon.

[0118] The embodiment of the present application provides a sound recognition device, which pre-processes the acquired gibbon recording data to obtain the gibbon recording data to be classified; then extracts the Mel spectrum features of the gibbon recording data to be classified; and obtains the gibbon audio segment in the gibbon recording data to be classified by inputting the Mel spectrum features into the trained gibbon sound classification model. The gibbon sound classification model used is more lightweight, has a fast recognition speed, and the recognition effect is accurate after pre-processing in advance, which can realize real-time and efficient monitoring of gibbon sounds.

[0119] Based on the same concept of the present invention, the specification is attached Figure 6As shown, an electronic device 600 provided in an embodiment of the present application includes: a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the above-mentioned sound recognition method.

[0120] Specifically, the memory 602 and the processor 601 can be general-purpose memories and processors, which are not specifically limited here. When the processor 601 runs the computer program stored in the memory 602, the sound recognition method can be executed.

[0121] Corresponding to the above-mentioned information processing method in the game, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned sound recognition method are executed.

[0122] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0123] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0125] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0126] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A sound recognition method, characterized in that: The following steps are involved: Obtain gibbon recording data; Preprocessing the gibbon recording data to obtain gibbon recording data to be classified; Extracting Mel spectrum features of the gibbon recording data to be classified; Based on the trained gibbon sound classification model, the gibbon recording data to be classified is classified according to the Mel spectrum features to obtain the gibbon audio clips in the gibbon recording data to be classified, and the following steps are included: the Mel spectrum features are dimensionally expanded, and the Mel spectrum features after dimension expansion are input into the convolution layer of the trained gibbon sound classification model to obtain feature data; wherein the trained gibbon sound classification model includes five convolution layers with convolution kernels of 3*3; the feature data is data enhanced, and the feature data after data enhancement is input into the fully connected layer of the trained gibbon sound classification model to output the gibbon audio clips in the gibbon recording data to be classified; wherein the feature data obtained by the convolution process is mean and variance calculated, and the calculated mean and variance of the feature data are cascaded as the input of the fully connected layer; Among them, the gibbon sound classification model is improved based on ResNet34; The step of preprocessing the gibbon recording data to obtain the gibbon recording data to be classified comprises the following steps: First, the acquired gibbon recording data is sampled and quantized at a sampling rate of 16KHz; then the high-pass filter is input to complete the pre-emphasis of the sampled gibbon recording data. The high-pass filter used is: Among them, z -1 Indicates that the filter H(z) is a first-order filter, α is a constant, and α is taken as 0.95 here; Then, the pre-emphasized gibbon recording data is divided into frames, where 400 sampling points and 25ms are used as the frame length, 200 sampling points and 12.5ms are used as the frame shift, and the frame length and frame shift overlap rate is 50%; finally, windowing processing is performed to obtain the gibbon recording data to be classified, and the window function calculation formula used is: Among them, n represents the sampling point and N represents the frame length.

2. A sound recognition method according to claim 1, characterized in that: The method of obtaining the gibbon recording data comprises the following steps: Gibbon audio recording data is collected in real time by using a recording device in a gibbon group, wherein the gibbon audio recording data includes segments containing gibbon audio and segments not containing gibbon audio.

3. A sound recognition method according to claim 2, characterized in that: The step of extracting the Mel spectrum features of the gibbon recording data to be classified comprises the following steps: Performing fast Fourier transform on the gibbon recording data to be classified to obtain a signal spectrum; Performing Mel frequency conversion on the signal spectrum to obtain Mel frequency; The Mel frequency is subjected to Mel filtering to obtain Mel spectrum characteristics.

4. A method for voice recognition according to claim 3, characterized in that: After the outputting of the obtained gibbon audio segment in the gibbon recording data to be classified, the following steps are also included: The gibbon audio clips are spliced ​​and trimmed to obtain complete gibbon audio, and the calling time of the gibbon is recorded.

5. A voice recognition method according to claim 1, characterized in that: The generalization ability of the trained gibbon sound classification model was evaluated according to the F-Score indicator.

6. A voice recognition device, characterized in that: include: An acquisition module is used to obtain gibbon recording data; The preprocessing module is used to preprocess the gibbon recording data to obtain the gibbon recording data to be classified; wherein the obtained gibbon recording data is first sampled and quantized at a sampling rate of 16KHz; and then input into a high-pass filter to complete the pre-emphasis of the sampled gibbon recording data, and the high-pass filter used is: Among them, z -1 Indicates that the filter H(z) is a first-order filter, α is a constant, and α is taken as 0.95 here; Then, the pre-emphasized gibbon recording data is divided into frames, where 400 sampling points and 25ms are used as the frame length, 200 sampling points and 12.5ms are used as the frame shift, and the frame length and frame shift overlap rate is 50%; finally, windowing processing is performed to obtain the gibbon recording data to be classified, and the window function calculation formula used is: Where n represents the sampling point and N represents the frame length; An extraction module, used for extracting Mel spectrum features of the gibbon recording data to be classified; A classification module is used to classify the gibbon recording data to be classified according to the Mel spectrum features based on the trained gibbon sound classification model to obtain the gibbon audio clips in the gibbon recording data to be classified, and includes: dimensional expansion of the Mel spectrum features, and inputting the Mel spectrum features after dimensional expansion into the convolution layer of the trained gibbon sound classification model to obtain feature data; wherein the trained gibbon sound classification model includes five convolution layers with convolution kernels of 3*3; data enhancement processing is performed on the feature data, and the feature data after data enhancement processing is input into the fully connected layer of the trained gibbon sound classification model to output the gibbon audio clips in the gibbon recording data to be classified; wherein the mean and variance of the feature data obtained by the convolution processing are calculated, and the calculated mean and variance of the feature data are cascaded as the input of the fully connected layer; the gibbon sound classification model is improved based on ResNet34.

7. An electronic device, characterized in that: The electronic device comprises a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the sound recognition method according to any one of claims 1 to 5 are performed.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the sound recognition method according to any one of claims 1 to 5 are executed.