Automatic device for collecting bird voiceprint information

Through the four-microphone circular array and multi-sound source separation model, combined with feature extraction and grid proximity recognition, the problem of low bird sound recognition accuracy is solved, and the high accuracy recognition of bird soundprints is achieved, and background noise and There are many kinds of sound interference.

CN118609577BActive Publication Date: 2024-11-01JIANGSU ENVIRONMENTAL MONITORING CENT
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411085172.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-01
Estimated Expiration
2044-08-08

AI Technical Summary

Technical Problem

In the prior art, bird sound recognition accuracy is low, and due to the influence of background noise and complex types of sound, it is difficult to accurately identify bird sounds. Especially in natural environments, the sound spectrum rate is high, and multiple short trebles are continuous or discontinuous. Making identification more difficult.

Method used

The four-microphone circular array is used to collect voiceprint data, combined with edge computing and background processing devices, and preliminary preprocessing, separation and matching of voiceprint features is performed through feature extraction, LBP feature conversion, similarity calculation and multi-sound source separation model. Use grid proximity recognition method to improve identification accuracy.

Benefits of technology

It realizes more accurate identification of overlapping and individual voiceprint data, improves the accuracy of bird voiceprint recognition, reduces background noise interference, and enhances the stability of identification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118609577B_ABST
    Figure CN118609577B_ABST
Patent Text Reader

Abstract

The present invention discloses an automatic device for collecting bird voiceprint information. The present invention uses a voiceprint collection device to collect bird voiceprint data using a four-microphone circular array; uses an edge computing device to preliminarily preprocess the collected bird voiceprint data, and transmits the preliminarily preprocessed bird voiceprint data to a background processing device; the background processing device is used to receive the preliminarily preprocessed bird voiceprint data transmitted by the edge computing device, and process and identify the preliminarily preprocessed bird voiceprint data. The background processing first determines whether the collected voiceprint features are overlapping voiceprint features. If they are overlapping voiceprint features, the overlapping voiceprint features are separated, and then the similarity is used to match them with the voiceprints in the feature library, thereby realizing the recognition of the voiceprint. The present invention can more accurately identify overlapping or separate voiceprint data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent collection and identification of bird voiceprints, and in particular to an automatic collection device for bird voiceprint information. Background Art

[0002] Bird songs are a medium for mutual communication. They have many functions, such as seeking mates, defending territories, avoiding predators, and social interaction. They are an important way to coordinate behaviors between individuals and groups. Through song research, we can understand bird song communication, such as individual identification and species identification, sexual selection, etc. Similarly, in taxonomy, bird songs also play an important role, which can not only discover new species, but also evaluate species status and analyze phylogenetic relationships. Therefore, bird song research is also widely used in neurobiology, bird behavior, evolutionary biology and other fields.

[0003] At present, international research on bird acoustic behavior mainly focuses on two aspects: reproduction and sexual selection; classification and systematic evolution. Domestic research on bird acoustic behavior started late, and most of it focused on sound monitoring and simple acoustic behavior description and classification. Therefore, we should fully analyze the differences in bird acoustic signal calls, accumulate materials for bird song research, and lay the foundation for in-depth research on song.

[0004] However, the existing acoustic-based bird species identification method has poor detection and identification accuracy. Therefore, the background noise of the environmental samples in the existing technology is relatively large, and it is difficult to ensure the accuracy of sound feature recognition. Usually, the identification method is mainly to record sound samples-sample background noise reduction-sound feature recognition method for identification. In this case, there are usually problems such as large background noise and complex sound types, resulting in low recognition rate of bird sounds. At the same time, the sound spectrum of bird sounds is relatively high. Bird sounds have the characteristics of multiple short high-pitched continuous or discontinuous high-frequency sounds, and when monitoring bird sounds, multiple bird sounds may be obtained, which brings greater difficulties to bird sound recognition. Summary of the invention

[0005] The purpose of the present invention is to provide an automatic bird voiceprint information collection device to separate overlapping voiceprints and then match and identify the separated voiceprints using a feature similarity method.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] The device for automatically collecting bird voiceprint information includes:

[0008] The voiceprint collection device uses a four-microphone circular array to collect bird voiceprint data; the four-microphone circular array includes a microphone array containing 4 evenly distributed omnidirectional microphones, and the distance between two adjacent microphones is at the millimeter level, which can better collect voiceprint data of different phases and lay the foundation for subsequent voiceprint data analysis;

[0009] The edge computing device pre-processes the collected bird voiceprint data and transmits the pre-processed bird voiceprint data to the background processing device; the pre-processing includes temporarily storing the collected data to provide a certain guarantee for the interruption of data transmission; judging whether the data is valid and deleting invalid data, which alleviates the pressure of background data processing to a certain extent;

[0010] A background processing device, used to receive the bird voiceprint data after preliminary preprocessing transmitted by the edge computing device, and process and identify the bird voiceprint data after preliminary preprocessing;

[0011] The background processing device comprises:

[0012] A feature extraction module performs secondary preprocessing on the bird voiceprint data after the initial preprocessing, and performs feature extraction on the bird voiceprint data block after the secondary preprocessing;

[0013] Voiceprint conversion module, which converts the extracted voiceprint features into visual images and extracts the corresponding LBP features;

[0014] The voiceprint overlap judgment module calculates the similarity of each voiceprint feature based on the voiceprint feature and the corresponding LBP feature using the feature similarity calculation method to determine whether the voiceprints overlap;

[0015] Voiceprint matching module, if there is no overlap in voiceprints, the extracted voiceprint features are directly matched with the voiceprint features in the feature library to determine the voiceprint matching result; if there are overlapping voiceprints, the overlapping voiceprints are separated, and then the voiceprint features extracted from the separated voiceprint features are matched with the voiceprint features in the feature library to determine the voiceprint matching set; based on the voiceprint matching set, the voiceprint feature recognition is performed using the lattice proximity recognition method to determine the type of bird.

[0016] According to the above technical solution, the secondary preprocessing of the bird voiceprint data includes: pre-emphasis, high-pass filtering, bird call framing and windowing, and endpoint detection. Since the storage format of the chirping signal in the natural environment is a dual-channel format, and the chirping sound is relatively complex and changeable, in order to select a suitable index to represent the diversity of organisms, it is first necessary to perform pre-processing operations such as pre-emphasis, high-pass filtering, bird call framing and windowing, and endpoint detection on the chirping signal.

[0017] According to the above technical solution, the voiceprint feature extraction step includes:

[0018] The secondary preprocessed voiceprint data is filtered for background noise to obtain multiple valid frame data of suspected sound sources;

[0019] After processing multiple valid frame data of suspected sound sources by fast Fourier transform and Mel filter, taking logarithm and performing discrete cosine transform, a voiceprint feature vector is obtained; the voiceprint feature extraction step includes not only the feature extraction method mentioned above;

[0020] The voiceprint feature vectors extracted by each microphone at the same time are standardized, and the standardized voiceprint feature vectors are converted into visual images using an audio image algorithm to convert the bird sound audio signal, and the LBP features of the visual image are extracted; using an audio image algorithm to convert the bird sound audio signal into a visual image belongs to the prior art and will not be introduced in detail here. Different voiceprint features correspond to different LBP features of the visual image. Even if the perimeter features in the LBP features are the same, the possibility of having the same area is extremely small. Therefore, the LBP feature can be regarded as an important feature of the bird voiceprint feature, and the LBP feature can be used to identify the bird voiceprint;

[0021] Based on the voiceprint feature vector of each microphone standardized at the same time and the LBP feature corresponding to the visual image, the feature similarity is calculated using the feature similarity calculation method to determine whether the voiceprint data collected at the same time is overlapping voiceprint data; since a four-microphone circular array is used for identification of different phase directions, when there are multiple birds singing, each microphone in the four-microphone circular array collects voiceprint data of different phases, so the voiceprint data collected by each microphone in the four-microphone circular array is overlapping, and the feature data of the collected voiceprint data is different;

[0022] When all feature similarities at the same time are greater than or equal to the threshold , it proves that there is no overlapping voiceprint data at this moment, on the contrary, it proves that there is overlapping in the voiceprint data obtained at this moment, and the corresponding pre-processed voiceprint data in this state is marked as multi-source voiceprint data.

[0023] According to the above technical solution, the standardization process:

[0024] ;

[0025] In the formula, After standardization, The new voiceprint feature sequence extracted by the microphone, Indicates The voiceprint feature sequence extracted by the microphone, express The average value of the voiceprint feature sequence, Represents standard deviation.

[0026] Because the collected data may have certain differences in accuracy when collected or preprocessed, it is necessary to standardize the data;

[0027] According to the above technical solution, the feature similarity calculation method is:

[0028]

[0029]

[0030] in, Represents the voiceprint feature vector and voiceprint feature vector The cosine similarity of Represents the voiceprint feature vector Corresponding LBP features of visual images and voiceprint feature vectors The similarity of the LBP features of the corresponding visual images, Represents the voiceprint feature vector Corresponding LBP features and voiceprint feature vector Corresponding LBP features The cosine similarity of Represents the similarity of low-dimensional features, Represents the voiceprint feature vector The area characteristics of Represents the voiceprint feature vector The perimeter characteristics of Represents the voiceprint feature vector The area characteristics of Represents the voiceprint feature vector circumference characteristics.

[0031] According to the above technical solution, the voiceprint matching module uses a neural network model and a spatial clustering method to construct a multi-source separation model, and inputs overlapping voiceprints into the multi-source separation model to separate voiceprint features.

[0032] According to the above technical solution, the steps of constructing the multi-sound source separation model are:

[0033] Extract features of different scales from multi-source voiceprint data using an encoder, and fuse the features of different scales to obtain fused feature data;

[0034] The fused feature data is input into the separator to generate independent source signals using the attention mechanism and perform weighting;

[0035] The decoder network is used to reconstruct the data of each weighted independent source signal feature to obtain each separated independent signal, and then the feature of each separated signal is obtained, and the feature of the separated signal is recorded as separation feature 1 to determine the preliminary multi-source separation model;

[0036] The voiceprint obtained by each microphone is separated using the spatial clustering method to obtain the voiceprint separation feature, which is recorded as separation feature 2;

[0037] Convert the separation feature 1 and separation feature 2 data into visual images, and extract the LBP features corresponding to the visual images.

[0038] Using a feature similarity calculation method, separation feature 1 and separation feature 2 are matched with the voiceprint features in the feature library to determine the probability that the voiceprint features separated in separation feature 1 and separation feature 2 are the same and the probability that separation feature 1 matches the voiceprint features in the feature library;

[0039] The probability that the voiceprint features separated in separation feature 1 and separation feature 2 are the same is called separation similarity probability, and the probability that separation feature 1 matches the voiceprint features in the feature library is called matching similarity probability;

[0040] The separation similarity probability and the matching similarity probability are used as the judgment criteria for the network optimization of the multi-source separation model, and the preliminary multi-source separation model corresponding to the highest sum of the separation similarity probability and the matching similarity probability is taken as the final multi-source separation model.

[0041] The calculation method of separation similarity probability refers to the ratio of the number of identical voiceprint features separated in separation feature 1 and separation feature 2 to the total number of separation feature 1;

[0042] The calculation method of matching similarity probability refers to the similarity technology between the separated feature 1 and the voiceprint feature in the feature library is greater than or equal to the set threshold The ratio of the number of separation features to the total number of feature 1;

[0043] The highest sum of separation similarity probability and matching similarity probability means that the separation similarity probability is greater than or equal to the threshold And the matching similarity probability is greater than or equal to the threshold In the case of , the highest value of the sum of the separation similarity probability and the matching similarity probability is taken. If there are multiple combinations with the same highest value of the sum of the two probabilities, the combination corresponding to the highest matching similarity probability is the best combination.

[0044] According to the above technical solution, if the voiceprints overlap, the voiceprint matching set construction step includes:

[0045] Convert the separated voiceprint data into visual images, extract the LBP features corresponding to the visual images, and use the similarity calculation method to determine similar voiceprint data;

[0046] Assume that there are m voiceprint features stored in the feature library, and traverse each voiceprint feature in the feature library , use the feature similarity calculation method to calculate each voiceprint feature Fi to be identified and the voiceprint feature The similarity is greater than the threshold Voiceprint features Classified into the sequence C corresponding to the voiceprint feature Fi to be identified i In the sequence C i Considered as a voiceprint matching set;

[0047] If in C i If the sequence is 0, the voiceprint feature Fi to be identified is considered an invalid target or an unknown target, and is marked as an unknown target for expert judgment;

[0048] If the voiceprints do not overlap, the voiceprint matching set construction step includes:

[0049] The directly extracted voiceprint features are converted into visual images, and the LBP features corresponding to the visual images are extracted, and similar voiceprint data are determined using a similarity calculation method;

[0050] Assume that there are m voiceprint features stored in the feature library, and traverse each voiceprint feature in the feature library , use the feature similarity calculation method to calculate each voiceprint feature Fi to be identified and the voiceprint feature The similarity is greater than the threshold Voiceprint features Classified into the sequence C corresponding to the voiceprint feature Fi to be identified i In the sequence C i Considered as a voiceprint matching set;

[0051] If in C i If the sequence is 0, the voiceprint feature Fi to be identified is considered an invalid target or an unknown target, and is marked as an unknown target for judgment by an expert.

[0052] According to the above technical features, the steps of the grid proximity recognition method include:

[0053] Match the voiceprint to set C i As a standard library, calculate the closeness between the voiceprint feature Fi to be identified and each voiceprint feature in the standard library;

[0054] Select the voiceprint feature corresponding to the largest grid proximity in the standard library as the identification feature of the voiceprint feature Fi to be identified;

[0055] The bird species corresponding to the voiceprint feature with the largest grid proximity in the standard library is the bird species information corresponding to the voiceprint feature Fi to be identified.

[0056] According to the above technical features, the grid proximity calculation method is:

[0057]

[0058] In the formula, Represents the voiceprint feature Fi to be identified and the voiceprint features in the standard library The closeness of Represents the difference between the voiceprint feature Fi to be identified and the voiceprint feature in the standard library The inner product of Represents the difference between the voiceprint feature Fi to be identified and the voiceprint feature in the standard library The outer product of .

[0059] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention first determines whether the collected voiceprint features are overlapping voiceprint features. If they are overlapping voiceprint features, the overlapping voiceprint features are separated, and then the similarity is used to match the voiceprints in the feature library, thereby realizing the recognition of the voiceprint. In the recognition process, the voiceprint takes the LBP feature corresponding to the visual image as an important recognition feature, which increases the accuracy of voiceprint recognition to a certain extent. At the same time, since the overlapping voiceprints need to be separated, the separated voiceprint features are not accurate enough. Therefore, when matching with the voiceprints in the feature library, multiple similar voiceprint features may appear. Therefore, it is necessary to use the grid proximity recognition method to select the best matching feature, thereby realizing the recognition of the collected voiceprint. The present invention can more accurately identify overlapping or separate voiceprint data. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0061] Figure 1 This is a flow chart of the working steps of the automatic bird voiceprint information collection device of the present invention;

[0062] Figure 2 It is a flow chart of the steps for constructing the multi-sound source separation model of Example 1. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] Example 1

[0065] The voiceprint matching module uses a neural network model and a spatial clustering method to build a multi-source separation model. The specific steps of building the multi-source separation model include:

[0066] The collected multi-source voiceprint data is used to extract features of different scales using an encoder, and the features of different scales are fused to obtain fused feature data;

[0067] The fused feature data is input into the separator to generate independent source signals using the attention mechanism and perform weighting;

[0068] The decoder network is used to reconstruct the data of each weighted independent source signal feature to obtain each separated independent signal, and then the feature of each separated signal is obtained, and the feature of the separated signal is recorded as separation feature 1 to determine the preliminary multi-source separation model;

[0069] The voiceprint obtained by each microphone is separated using the spatial clustering method to obtain the voiceprint separation feature, which is recorded as separation feature 2;

[0070] Convert the separation feature 1 and separation feature 2 data into visual images, and extract the LBP features corresponding to the visual images.

[0071] Using a feature similarity calculation method, separation feature 1 and separation feature 2 are matched with the voiceprint features in the feature library to determine the probability that the voiceprint features separated in separation feature 1 and separation feature 2 are the same and the probability that separation feature 1 matches the voiceprint features in the feature library;

[0072] The probability that the voiceprint features separated in separation feature 1 and separation feature 2 are the same is called separation similarity probability, and the probability that separation feature 1 matches the voiceprint features in the feature library is called matching similarity probability;

[0073] The separation similarity probability and matching similarity probability are used as the judgment criteria for the network optimization of the multi-source separation model. And the matching similarity probability is greater than or equal to the threshold In the case of , the preliminary multi-source separation model corresponding to the time when the sum of the separation similarity probability and the matching similarity probability is the highest is taken as the final multi-source separation model.

[0074] Example 2

[0075] The steps for identifying non-overlapping voiceprints include:

[0076] L1. Use a voiceprint collection device with a four-microphone circular array to collect bird voiceprint data.

[0077] L2. Use an edge computing device to perform preliminary preprocessing on the collected bird voiceprint data, and transmit the preprocessed bird voiceprint data to a background processing device; the preliminary preprocessing includes temporarily storing the collected data, determining whether the data is valid, and deleting invalid data.

[0078] L3, a background processing device, is used to receive the bird voiceprint data after preliminary preprocessing transmitted by the edge computing device, and process and identify the bird voiceprint data after preliminary preprocessing, specifically performing:

[0079] S1. Perform secondary preprocessing on the bird voiceprint data after the preliminary preprocessing by using a feature extraction module, and perform feature extraction on the bird voiceprint data block after the secondary preprocessing. The specific steps include: filtering the secondary preprocessed voiceprint data for background noise to obtain multiple valid frame data of suspected sound sources;

[0080] After processing multiple valid frame data of suspected sound sources using fast Fourier transform and Mel filter, taking the logarithm and performing discrete cosine transform, the voiceprint feature vector is obtained.

[0081] S2. Use the voiceprint conversion module to convert the extracted voiceprint features into visual images and extract the corresponding LBP features.

[0082] S3, using the voiceprint overlap judgment module to calculate the similarity of each voiceprint feature based on the voiceprint feature and the corresponding LBP feature using a feature similarity calculation method to determine whether the voiceprints overlap. The specific steps include:

[0083] S301, standardizing the voiceprint feature vectors extracted by each microphone at the same time, and converting the standardized voiceprint feature vectors into visual images using an audio-to-image algorithm to convert the bird's chirping audio signal, and extracting LBP features of the visual image;

[0084] S302, calculating feature similarity using a feature similarity calculation method based on the voiceprint feature vectors processed by each microphone at the same time and the LBP features corresponding to the visual image, and determining whether the voiceprint data collected at the same time is overlapping voiceprint data;

[0085] S303: All feature similarities at the same time are greater than or equal to the threshold =0.8, which proves that there is no overlapping voiceprint data at this moment.

[0086] S4. Using the voiceprint matching module to match the extracted voiceprint features with the feature library by using matching rules, a voiceprint matching set is determined, and based on the voiceprint matching set, a lattice proximity recognition method is used to perform voiceprint feature recognition to determine the type of bird. The specific implementation includes:

[0087] S401, based on the extracted visual image LBP features corresponding to the voiceprint data, similar voiceprint data is determined using a similarity calculation method;

[0088] S402: Assume that there are m voiceprint features stored in the feature library, and traverse each voiceprint feature in the feature library. , use the feature similarity calculation method to calculate each voiceprint feature Fi to be identified and the voiceprint feature The similarity is greater than the threshold Voiceprint features Classified into the sequence C corresponding to the voiceprint feature Fi to be identified i In the sequence C i Considered as a voiceprint matching set;

[0089] S403, if in C i If the sequence is 0, the voiceprint feature Fi to be identified is considered an invalid target or an unknown target, and is marked as an unknown target for expert judgment;

[0090] S404: Match the voiceprint to set C i As a standard library, calculate the degree of closeness between the voiceprint feature Fi to be identified and each voiceprint feature in the standard library, select the voiceprint feature corresponding to the largest degree of closeness in the standard library as the identification feature of the voiceprint feature Fi to be identified, and the bird species corresponding to the voiceprint feature with the largest degree of closeness in the standard library is the bird species information corresponding to the voiceprint feature Fi to be identified.

[0091] Among them, the feature similarity calculation method is:

[0092]

[0093]

[0094] in, Represents the voiceprint feature vector and voiceprint feature vector The cosine similarity of Represents the voiceprint feature vector Corresponding LBP features of visual images and voiceprint feature vectors The similarity of the LBP features of the corresponding visual images, Represents the voiceprint feature vector Corresponding LBP features and voiceprint feature vector Corresponding LBP features The cosine similarity of Represents the similarity of low-dimensional features, Represents the voiceprint feature vector The area characteristics of Represents the voiceprint feature vector The perimeter characteristics of Represents the voiceprint feature vector The area characteristics of Represents the voiceprint feature vector Circumference characteristics;

[0095] The grid proximity calculation method is:

[0096]

[0097] In the formula, Represents the voiceprint feature Fi to be identified and the voiceprint features in the standard library The closeness of Represents the difference between the voiceprint feature Fi to be identified and the voiceprint feature in the standard library The inner product of Represents the difference between the voiceprint feature Fi to be identified and the voiceprint feature in the standard library The outer product of .

[0098] Example 3

[0099] The steps for identifying overlapping voiceprints include:

[0100] L1. Use a voiceprint collection device with a four-microphone circular array to collect bird voiceprint data.

[0101] L2. Use an edge computing device to perform preliminary preprocessing on the collected bird voiceprint data, and transmit the preprocessed bird voiceprint data to a background processing device; the preliminary preprocessing includes temporarily storing the collected data, determining whether the data is valid, and deleting invalid data.

[0102] L3, a background processing device, is used to receive the bird voiceprint data after preliminary preprocessing transmitted by the edge computing device, and process and identify the bird voiceprint data after preliminary preprocessing, specifically performing:

[0103] Q1. Perform secondary preprocessing on the bird voiceprint data after the preliminary preprocessing using the feature extraction module, and perform feature extraction on the bird voiceprint data block after the secondary preprocessing. The specific steps include: filtering the secondary preprocessed voiceprint data for background noise to obtain multiple valid frame data of suspected sound sources;

[0104] After processing multiple valid frame data of suspected sound sources using fast Fourier transform and Mel filter, taking the logarithm and performing discrete cosine transform, the voiceprint feature vector is obtained.

[0105] Q2. Use the voiceprint conversion module to convert the extracted voiceprint features into visual images and extract the corresponding LBP features.

[0106] Q3. Using the voiceprint overlap judgment module to calculate the similarity of each voiceprint feature based on the voiceprint features and the corresponding LBP features using the feature similarity calculation method, to determine whether the voiceprints overlap. The specific steps include:

[0107] Q301. Standardize the voiceprint feature vectors extracted by each microphone at the same time, and use the audio-image algorithm to convert the standardized voiceprint feature vectors into visual images, and extract the LBP features of the visual images;

[0108] Q302, based on the voiceprint feature vector of each microphone processed in a standardized manner at the same time and the LBP feature corresponding to the visual image, the feature similarity is calculated using the feature similarity calculation method to determine whether the voiceprint data collected at the same time is overlapping voiceprint data;

[0109] Q303: At the same time, the similarity of multiple voiceprint features is less than the threshold =0.8, it is proved that the voiceprint data acquired at this moment has overlapping phenomenon, and the corresponding pre-processed voiceprint data of this state is marked as multi-source voiceprint data.

[0110] Q4. Use the voiceprint matching module to match the extracted voiceprint features with the feature library by using matching rules to determine the voiceprint matching set. Based on the voiceprint matching set, use the grid proximity recognition method to perform voiceprint feature recognition and determine the type of bird. The specific implementation includes:

[0111] Q401, inputting the multi-source voiceprint data into the multi-source separation model constructed in Example 1 to perform voiceprint separation;

[0112] Q402, converting the separated voiceprint data into a visual image, extracting the LBP features corresponding to the visual image, and determining similar voiceprint data using a similarity calculation method;

[0113] Q403, suppose there are m voiceprint features stored in the feature library, traverse each voiceprint feature in the feature library , use the feature similarity calculation method to calculate each voiceprint feature Fi to be identified and the voiceprint feature The similarity is greater than the threshold Voiceprint features Classified into the sequence C corresponding to the voiceprint feature Fi to be identified i In the sequence C i Considered as a voiceprint matching set;

[0114] Q404, if in C i If the sequence is 0, the voiceprint feature Fi to be identified is considered an invalid target or an unknown target, and is marked as an unknown target for expert judgment;

[0115] Q405, match the voiceprint to set C i As a standard library, calculate the degree of closeness between the voiceprint feature Fi to be identified and each voiceprint feature in the standard library, select the voiceprint feature corresponding to the largest degree of closeness in the standard library as the identification feature of the voiceprint feature Fi to be identified, and the bird species corresponding to the voiceprint feature with the largest degree of closeness in the standard library is the bird species information corresponding to the voiceprint feature Fi to be identified.

[0116] Among them, the feature similarity calculation method is:

[0117]

[0118]

[0119] in, Represents the voiceprint feature vector and voiceprint feature vector The cosine similarity of Represents the voiceprint feature vector Corresponding LBP features of visual images and voiceprint feature vectors The similarity of the LBP features of the corresponding visual images, Represents the voiceprint feature vector Corresponding LBP features and voiceprint feature vector Corresponding LBP features The cosine similarity of Represents the similarity of low-dimensional features, Represents the voiceprint feature vector The area characteristics of Represents the voiceprint feature vector The perimeter characteristics of Represents the voiceprint feature vector The area characteristics of Represents the voiceprint feature vector Circumference characteristics;

[0120] Calculation method of grid proximity:

[0121]

[0122] In the formula, Represents the voiceprint feature Fi to be identified and the voiceprint features in the standard library The closeness of Represents the difference between the voiceprint feature Fi to be identified and the voiceprint feature in the standard library The inner product of Represents the difference between the voiceprint feature Fi to be identified and the voiceprint feature in the standard library The outer product of .

[0123] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0124] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. The device for automatically collecting bird voiceprint information is characterized by: include: The voiceprint collection device uses a four-microphone circular array to collect bird voiceprint data; The edge computing device performs preliminary preprocessing on the collected bird voiceprint data, and transmits the preprocessed bird voiceprint data to the background processing device; The preliminary preprocessing includes temporarily storing the collected data, determining whether the data is valid, and deleting invalid data; A background processing device, used to receive the bird voiceprint data after preliminary preprocessing transmitted by the edge computing device, and process and identify the bird voiceprint data after preliminary preprocessing; The background processing device comprises: A feature extraction module performs secondary preprocessing on the bird voiceprint data after the initial preprocessing, and performs feature extraction on the bird voiceprint data block after the secondary preprocessing; Voiceprint conversion module, which converts the extracted voiceprint features into visual images and extracts the corresponding LBP features; The voiceprint overlap judgment module calculates the similarity of each voiceprint feature based on the voiceprint feature and the corresponding LBP feature using a feature similarity calculation method to determine whether the voiceprints overlap; the feature similarity calculation method is: Among them, C(.) represents the cosine similarity between the voiceprint feature vector F1 and the voiceprint feature vector F2, score(F1, F2) represents the similarity between the LBP feature of the visual image corresponding to the voiceprint feature vector F1 and the LBP feature of the visual image corresponding to the voiceprint feature vector F2, Cr(.) represents the LBP feature F corresponding to the voiceprint feature vector F1 1lbp The LBP feature F corresponding to the voiceprint feature vector F1 2lbp The cosine similarity of Represents the similarity of low-dimensional features, F 1area represents the area feature of the voiceprint feature vector F1, F 1Perimeter represents the perimeter feature of the voiceprint feature vector F1, F 2area represents the area feature of the voiceprint feature vector F2, F 2Perimeter Represents the perimeter feature of the voiceprint feature vector F2; Voiceprint matching module, if there is no overlap in voiceprints, the extracted voiceprint features are directly matched with the voiceprint features in the feature library to determine the voiceprint matching result; if there are overlapping voiceprints, the overlapping voiceprints are separated, and then the voiceprint features extracted from the separated voiceprint features are matched with the voiceprint features in the feature library to determine the voiceprint matching set; based on the voiceprint matching set, the voiceprint feature recognition is performed using the lattice proximity recognition method to determine the type of bird.

2. The automatic bird voiceprint information collection device according to claim 1, characterized in that: The secondary preprocessing of the bird voiceprint data includes: pre-emphasis, high-pass filtering, bird call frame windowing and endpoint detection.

3. The automatic bird voiceprint information collection device according to claim 1, characterized in that: The voiceprint feature extraction step comprises: The secondary preprocessed voiceprint data is filtered for background noise to obtain multiple valid frame data of suspected sound sources; After processing multiple valid frame data of suspected sound sources using fast Fourier transform and Mel filter, taking the logarithm and performing discrete cosine transform to obtain the voiceprint feature vector; The voiceprint feature vectors extracted by each microphone at the same time are standardized, and the standardized voiceprint feature vectors are converted into visual images using an audio-to-image algorithm to extract the LBP features of the visual images; Based on the voiceprint feature vector of each microphone processed in a standardized manner at the same time and the LBP feature corresponding to the visual image, the feature similarity is calculated using the feature similarity calculation method to determine whether the voiceprint data collected at the same time is overlapping voiceprint data; When all feature similarities at the same moment are greater than or equal to the threshold α, it proves that there is no overlapping voiceprint data at this moment. Otherwise, it proves that the voiceprint data obtained at this moment has overlapping phenomenon, and the corresponding pre-processed voiceprint data in this state is marked as multi-source voiceprint data.

4. The automatic bird voiceprint information collection device according to claim 3, characterized in that: The standardization process: In the formula, y i represents the new voiceprint feature sequence extracted by the i-th microphone after standardization, x i represents the voiceprint feature sequence extracted by the i-th microphone, Represents x i The average value of the voiceprint feature sequence, s represents the standard deviation.

5. The automatic bird voiceprint information collection device according to claim 1, characterized in that: The voiceprint matching module uses a neural network model and a spatial clustering method to build a multi-source separation model, and inputs overlapping voiceprints into the multi-source separation model to separate voiceprint features.

6. The automatic bird voiceprint information collection device according to claim 5, characterized in that: The steps of constructing the multi-source sound separation model are as follows: Extract features of different scales from multi-source voiceprint data using an encoder, and fuse the features of different scales to obtain fused feature data; The fused feature data is input into the separator to generate independent source signals using the attention mechanism and perform weighting; The decoder network is used to reconstruct the data of each weighted independent source signal feature to obtain each separated independent signal, and then the feature of each separated signal is obtained, and the feature of the separated signal is recorded as separation feature 1 to determine the preliminary multi-source separation model; The voiceprint obtained by each microphone is separated using the spatial clustering method to obtain the voiceprint separation feature, which is recorded as separation feature 2; The separation feature 1 and the separation feature 2 data are converted into visual images, and the LBP features corresponding to the visual images are extracted. The separation feature 1 and the separation feature 2 are matched with the voiceprint features in the feature library using the feature similarity calculation method, and the probability that the voiceprint features separated in the separation feature 1 and the separation feature 2 are the same and the probability that the separation feature 1 matches the voiceprint features in the feature library are determined; The probability that the voiceprint features separated in separation feature 1 and separation feature 2 are the same is called separation similarity probability, and the probability that separation feature 1 matches the voiceprint features in the feature library is called matching similarity probability; The separation similarity probability and the matching similarity probability are used as the judgment criteria for the network optimization of the multi-source separation model, and the preliminary multi-source separation model corresponding to the highest sum of the separation similarity probability and the matching similarity probability is taken as the final multi-source separation model.

7. The automatic bird voiceprint information collection device according to claim 1, characterized in that: If the voiceprints overlap, the voiceprint matching set construction step includes: Convert the separated voiceprint data into visual images, extract the LBP features corresponding to the visual images, and use the similarity calculation method to determine similar voiceprint data; Assume that there are m voiceprint features stored in the feature library, and traverse each voiceprint feature Q in the feature library i , use the feature similarity calculation method to calculate each voiceprint feature Fi to be identified and the voiceprint feature Q i The similarity of the voiceprint feature Q with a similarity greater than the threshold β i Classified into the sequence C corresponding to the voiceprint feature Fi to be identified i In the sequence C i Considered as a voiceprint matching set; If in C i If the sequence is 0, the voiceprint feature Fi to be identified is considered an invalid target or an unknown target, and is marked as an unknown target for expert judgment; If the voiceprints do not overlap, the voiceprint matching set construction step includes: The directly extracted voiceprint features are converted into visual images, and the LBP features corresponding to the visual images are extracted, and similar voiceprint data are determined using a similarity calculation method; Assume that there are m voiceprint features stored in the feature library, and traverse each voiceprint feature Q in the feature library i , use the feature similarity calculation method to calculate each voiceprint feature Fi to be identified and the voiceprint feature Q i The similarity of the voiceprint feature Q with a similarity greater than the threshold β i Classified into the sequence C corresponding to the voiceprint feature Fi to be identified i In the sequence C i Considered as a voiceprint matching set; If in C i If the sequence is 0, the voiceprint feature Fi to be identified is considered an invalid target or an unknown target, and is marked as an unknown target for judgment by an expert.

8. The device for automatically collecting bird voiceprint information according to claim 1, characterized in that: The steps of the grid proximity recognition method include: Match the voiceprint to set C i As a standard library, calculate the closeness between the voiceprint feature Fi to be identified and each voiceprint feature in the standard library; Select the voiceprint feature corresponding to the largest grid proximity in the standard library as the identification feature of the voiceprint feature Fi to be identified; The bird species corresponding to the voiceprint feature with the largest grid proximity in the standard library is the bird species information corresponding to the voiceprint feature Fi to be identified.

9. The automatic bird voiceprint information collection device according to claim 8, characterized in that: The grid proximity calculation method is: In the formula, δ0(Fi,Bi) represents the degree of closeness between the voiceprint feature Fi to be identified and the voiceprint feature Bi in the standard library. represents the inner product of the voiceprint feature Fi to be identified and the voiceprint feature Bi in the standard library, and Fi⊙Bi represents the outer product of the voiceprint feature Fi to be identified and the voiceprint feature Bi in the standard library.