Bird active identification method, device and system and storage medium

Abstract

The invention provides a bird active identification method, device and system and a storage medium, and relates to the technical field of bird identification. The method comprises the following steps: sending an instruction for playing a bird sound signal; wherein the bird sound signal comprises at least one preset sound of a specific bird; when it is determined that the sound pickup captures the target sound, the position of the target bird away from the sound pickup is determined; wherein the target sound is made by the target bird; adjusting the shooting angle of the PTZ camera based on the position of the target bird from the sound pickup, so as to carry out photographing and video recording on the target bird, and obtaining an image set and a video set; performing feature extraction on the target sound, the image set and the video set to obtain target sound features, image features and video features; and identifying the target bird based on the target sound feature, the image feature and the video feature. According to the invention, the identification accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of bird identification, and in particular to a bird active identification method, device, system and storage medium. BACKGROUND

[0002] Birds are an important part of nature. Bird identification is not only a technical activity of "identifying species", but also an important link between scientific research, ecological protection and public life. Its role penetrates into multiple levels of nature and human society, and has important significance for sustainable development.

[0003] Current identification methods are mainly passive identification, relying on cameras to wait for the appearance of birds. The camera will capture a large number of invalid images or videos, which not only occupies too much storage space, but also requires a lot of time to identify which frames appear birds in the later identification process.

[0004] In addition, even if the camera can capture birds, it will also be affected by the camera being blocked, changes in light and background noise, which will also reduce the recognition rate. SUMMARY

[0005] The embodiments of the present application provide a bird active identification method, device, system and storage medium to solve the problem of poor identification effect of the current passive identification method.

[0006] In a first aspect, the embodiments of the present application provide a bird active identification method, comprising:

[0007] issuing an instruction to play a bird sound signal; wherein the bird sound signal includes a preset call of at least one specific bird;

[0008] When it is determined that the sound pickup device captures the target call, the position of the target bird from the sound pickup device is determined; wherein the target call is emitted by the target bird;

[0009] Based on the position of the target bird from the sound pickup device, the shooting angle of the pan-tilt camera is adjusted to take pictures and record videos of the target bird, to obtain an image set and a video set;

[0010] The target call, the image set and the video set are respectively subjected to feature extraction to obtain target sound features, image features and video features;

[0011] Based on the target sound features, image features and video features, the target bird is identified.

[0012] In a possible implementation manner, a plurality of sound pickup devices are arranged at the target position, and all the sound pickup devices are arranged on a circumference;

[0013] Once it is determined that the microphone has captured the target call, the distance of the target bird from the microphone is determined, including:

[0014] Once it is determined that the microphone has captured the target call, a hyperbolic equation is established based on the distance difference between each microphone and the time difference between each microphone receiving the target call, to determine the position of the target bird from the microphone.

[0015] In one possible implementation, feature extraction is performed on the target call, image set, and video set separately to obtain target sound features, image features, and video features, including:

[0016] Extract the Mel frequency cepstral coefficients and / or spectral features of the target call to obtain the target sound features;

[0017] Image features are obtained by extracting deep features from an image set using a convolutional neural network;

[0018] Dynamic features are extracted from the video set using a 3D convolutional neural network or optical flow method to obtain video features.

[0019] In one possible implementation, before performing feature extraction on the target vocalizations, image sets, and video sets respectively, the following steps are also included:

[0020] The target call is processed by frame segmentation and pre-emphasis to obtain the processed target call.

[0021] The images in the image set are subjected to grayscale conversion, noise reduction, and image enhancement to obtain the processed image set;

[0022] Extract keyframes from the video set to obtain a keyframe set;

[0023] Feature extraction was performed on the target's vocalizations, image sets, and video sets, including:

[0024] Feature extraction was performed on the processed target vocalizations, the processed image set, and the keyframe set, respectively.

[0025] In one possible implementation, target birds are identified based on target sound features, image features, and video features, including:

[0026] Time alignment processing is performed on the target sound features, image features, and video features to obtain the speaker feature vector, image feature vector, and video feature vector;

[0027] The target birds are identified based on voiceprint feature vectors, image feature vectors, and video feature vectors.

[0028] In one possible implementation, target birds are identified based on voiceprint feature vectors, image feature vectors, and video feature vectors, including:

[0029] The voiceprint feature vector, image feature vector, and video feature vector are concatenated to obtain a fused feature vector;

[0030] The fused feature vector is input into a pre-trained bird species identification model to obtain the species of the target bird.

[0031] In one possible implementation, target birds are identified based on voiceprint feature vectors, image feature vectors, and video feature vectors, including:

[0032] The voiceprint feature vector is input into the voice recognition model to obtain the first classification result of the target bird.

[0033] The image feature vector is input into the image recognition model to obtain the second classification result of the target bird;

[0034] The video feature vector is input into the video recognition model to obtain the third classification result of the target bird.

[0035] Based on the results of the first, second, and third classifications, the species of the target bird are obtained.

[0036] Secondly, embodiments of the present invention provide a bird active identification device, comprising:

[0037] A release signal module is used to issue a command to play bird sound signals; wherein, the bird sound signals include at least one preset call of a specific bird.

[0038] The location determination module is used to determine the distance between the target bird and the microphone when the microphone detects the target call; wherein, the target call is emitted by the target bird.

[0039] The shooting module is used to adjust the camera's shooting angle based on the distance of the target bird from the microphone, so as to take pictures and record videos of the target bird, and obtain an image set and a video set;

[0040] The feature extraction module is used to extract features from the target's vocalizations, image sets, and video sets respectively, to obtain the target's sound features, image features, and video features.

[0041] The recognition module is used to identify target birds based on their sound features, image features, and video features.

[0042] Thirdly, embodiments of the present invention provide an active bird identification system, including multiple speakers, multiple microphones, a pan-tilt camera, and an active bird identification device;

[0043] The speaker is used to play bird sound signals at preset playback cycles after receiving instructions;

[0044] The microphone is used to capture bird calls;

[0045] Pan-tilt cameras are used to photograph or record videos of birds;

[0046] The active bird identification device is used to identify birds based on the methods described in the first aspect above or any possible implementation thereof.

[0047] Fourthly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect or any possible implementation thereof.

[0048] Fifthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect or any possible implementation thereof.

[0049] In a sixth aspect, embodiments of the present invention provide a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect or any possible implementation thereof.

[0050] In this embodiment of the invention, to actively attract birds and improve the efficiency of bird identification, a command to play bird sound signals is first sent to the speaker. Upon receiving the command, the speaker plays the bird sound signals stored within it to lure specific birds. Next, once the microphone has captured the target call, the distance between the target bird and the microphone needs to be determined to adjust the pan-tilt camera's shooting angle for clear photographing and video recording. Finally, after obtaining the target bird's call, image set, and video set, feature extraction is performed on these data for accurate identification, thereby improving the accuracy of the identification process. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the bird active recognition system provided in an embodiment of the present invention;

[0052] Figure 2 This is a flowchart illustrating the implementation of the active bird identification method provided in this embodiment of the invention.

[0053] Figure 3 This is a schematic diagram showing the placement of the microphone provided in an embodiment of the present invention;

[0054] Figure 4This is a schematic diagram of the structure of the bird active recognition device provided in an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0056] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0057] As described in the background section, current bird identification primarily relies on passive camera recognition. However, in complex environments, obstructions or lighting conditions can easily lead to inaccurate identification. Furthermore, the inventors have found that current methods of passively photographing birds, especially rare species, have a low probability of success and cannot meet the needs of bird identification. To address these issues, this invention provides an active bird identification method, apparatus, system, and storage medium.

[0058] In order to proactively identify birds, an active bird identification system was built, such as... Figure 1 As shown, the system includes multiple high-fidelity speakers 110, multiple microphones 120, a pan-tilt camera 130, and a bird active recognition device 400.

[0059] After selecting a deployment site, multiple high-fidelity speakers 110 can be installed at the site, such as four speakers 110 spaced at 90° intervals. Upon receiving a command, the multiple high-fidelity speakers 110 will play bird sound signals at preset playback cycles. The bird sound signals are preset calls specific to a particular bird species in the deployment site, which is at least one rare bird species found in the site. The preset calls are courtship and / or flocking calls. These calls are extracted from a bird call database. Since the rare bird species in each deployment site are known, preset calls for specific birds can be extracted from the database and saved. When a preset call needs to be played, it will be played through the speakers after receiving a command to play the bird sound signal.

[0060] For example, preset calls can be stored on a hard drive. For each specific deployment location, since the rare bird species in each location are known, in order to improve the efficiency of rare bird identification, the preset calls stored on the hard drive are played through the speaker when active identification is required, so as to actively attract specific rare birds.

[0061] Multiple microphones 120 are deployed at the target location. To capture bird calls from all directions, all microphones 120 can be arranged evenly on a circle. For example, four microphones 120 can be evenly distributed at 90° intervals on a circle with a radius of R. Alternatively, the microphones 120 can be array microphones. Array microphones utilize multiple microphones to simultaneously capture sound signals from different directions and use signal processing algorithms to synthesize these signals into a single recording, thus enabling clearer capture of bird calls.

[0062] The PTZ camera 130 has a device that allows the camera to rotate in both horizontal and vertical directions, enabling it to capture images from multiple angles. The PTZ camera 130 adjusts its shooting angle according to position commands to capture more accurate and clear photos or videos. After deploying the active bird recognition system, this system can be used to identify birds.

[0063] The camera captures images with a resolution of no less than M*N pixels, and the video frame rate is set to (fps) frames / second; the microphone sampling frequency is f_s Hz, and the quantization bit depth is b bits.

[0064] Currently, bird identification relies on passive methods, meaning it relies on randomly waiting for birds to appear and cannot quickly locate specific species, resulting in very low identification efficiency. Therefore, the active bird identification system provided by this invention, upon receiving a command to play bird sound signals, plays preset calls of specific birds through a speaker, facilitating active attraction and guidance of specific birds and improving the efficiency of identifying similar birds. In addition to actively playing preset calls of specific birds, a microphone and a camera are also deployed, enabling multi-directional bird identification and improving accuracy.

[0065] See Figure 2 The flowchart illustrating the implementation of the bird active identification method provided in this embodiment of the invention is described in detail below:

[0066] S210, issue the instruction to play bird sound signals.

[0067] The bird sound signal includes a preset call of at least one rare bird species. This preset call can be a courtship and / or gregarious call, used to actively attract specific birds. The specific bird species refers to at least one rare bird species located in the area where the active bird identification system is deployed. The specific bird species can be determined based on the application scenario; for example, at least N species can be specified, where N is greater than or equal to 10.

[0068] It should be noted that the preset calls are the calls of specific birds in the deployment area of ​​the active bird identification system, which are designed to attract birds of the same species. Since the rare birds in each deployment area are known, the preset calls of the specific birds to be identified can be extracted from the bird call database and stored on the hard drive.

[0069] The bird sounds are emitted by multiple high-fidelity speakers. Upon receiving a command to play bird sounds, the speakers play the sounds at preset intervals, attracting birds of the same species and thus achieving active identification.

[0070] S220. When it is determined that the microphone has captured the target call, determine the distance between the target bird and the microphone.

[0071] The target call is made by the target bird. After the multiple microphones are deployed and capture the target call, they will send a signal to the bird active identification device that the target call has been captured. The bird active identification device will then determine the distance between the target call and the microphone based on the target call.

[0072] In this embodiment, four microphones are deployed, evenly distributed on a circle of radius R, spaced 90° apart, and labeled A, B, C, and D. The coordinates of the circle's center are set as the origin O(0,0), and the microphone coordinates are as follows: Figure 3 As shown: A(R,0), B(0,R), C(-R,0), D(0,-R), the speed of sound in air is a constant v.

[0073] Let the target bird's location be P(x,y), and the times when the microphones receive the sound be tA, tB, tC, and tD. According to the time-difference positioning principle, the time difference between the sound's propagation from P to each microphone is proportional to the distance difference. A system of equations can be established using the time differences between each pair of microphones to solve for (x,y).

[0074] The relationship between the time difference and distance difference between pickups A and B is as follows:

[0075] dAB=∣PA-PB∣=v∣tA-tB∣;

[0076] According to the definition of a hyperbola, the locus is a hyperbola with the equation: (dAB / 2) 2 (x-2R) 2 -(R 2 / 2-(dAB / 2) 2 (y-2R) 2 =1 (A real solution exists only if dAB < 2R)

[0077] The relationship between the time difference and distance difference between pickups A and C is as follows:

[0078] dAC = |PA - PC| = v|tA - tC|;

[0079] The locus is a hyperbola on the x-axis, and the equation is: (dAC / 2) 2 x 2 -(R 2 -(dAC / 2) 2 )y 2 = 1 (real solutions exist only when dAC < 2R)

[0080] The relationship between the time difference and the distance difference of the pickups A and D is:

[0081] dAD = |PA - PD| = v|tA - tD|;

[0082] The locus is a hyperbola.

[0083] The process of solving by simultaneously solving equations is as follows:

[0084] Let tA be the reference time, calculate the time differences: ΔtAB = tB - tA, ΔtAC = tC - tA, ΔtAD = tD - tA. The corresponding distance differences are: dAB = v|ΔtAB|, dAC = v|ΔtAC|, dAD = v|ΔtAD|.

[0085] Taking AB and AC as examples, simultaneously solve the hyperbola equations:

[0086] The hyperbola equation of AB is:

[0087] The sign is determined by the magnitudes of tA and tB.

[0088] The hyperbola equation of AC is:

[0089] The sign is determined by the magnitudes of tA and tC.

[0090] Taking the case of tA < tC in the hyperbola equation of AC (the sign is negative) as an example:

[0091]

[0092] After simplifying this expression and substituting it into the hyperbola equation of AB, solve for (x, y) by the elimination method or the numerical method.

[0093] S230. Based on the position of the target bird relative to the pickup, adjust the shooting angle of the pan-tilt camera to take pictures and record videos of the target bird, obtaining an image set and a video set.

[0094] Once the distance between the target bird and the microphone is determined, the pan-tilt camera's shooting angle can be adjusted according to the target bird's position, allowing for multi-angle shooting of the target bird to avoid obstructing certain angles.

[0095] In addition, the camera gimbal can be calibrated in advance using fixed-point calibration method or laser-assisted calibration method, and the shooting angle can be quickly adjusted with the camera.

[0096] S240. Extract features from the target's vocalizations, image set, and video set respectively to obtain the target's vocal features, image features, and video features.

[0097] After obtaining the target call, image set, and video set, the category of the target bird can be identified based on the acquired target call, image set, and video set.

[0098] In some embodiments, in order to obtain the features of the target call, image set and video set more quickly and accurately, it is also necessary to preprocess the target call, image set and video set first to avoid the influence of some abnormal factors.

[0099] In this embodiment, the target call can be processed by frame segmentation and pre-emphasis to obtain the processed target call. Pre-emphasis can enhance high-frequency components.

[0100] In terms of sound modality, unlike general audio processing, bird call recognition requires sophisticated preprocessing such as frame segmentation and pre-emphasis to effectively separate weak call signals that are often mixed with environmental noise, and to optimize the extraction effect of subsequent voiceprint features (such as MFCCs).

[0101] It can perform grayscale conversion, noise reduction, and image enhancement on each image in the image set to obtain the processed image set.

[0102] Keyframes can be extracted from the video set, redundant frames can be removed, and a keyframe set can be obtained, which can reduce the amount of computation.

[0103] In terms of visual modalities, keyframe extraction technology is particularly crucial when dealing with birds moving rapidly or partially obscured. It can accurately filter out the clearest and most discernible pose moments from long video streams, greatly improving the efficiency of image feature utilization.

[0104] In some embodiments, after obtaining the processed target call, the processed image set, and the keyframe set, feature extraction is also required.

[0105] In this embodiment, the Mel frequency cepstral coefficients and / or spectral features of the processed target call can be extracted to obtain the target sound features.

[0106] Specifically, the processed target call can be passed through a Mel filter bank to calculate the logarithmic energy, and then subjected to a discrete cosine transform (DCT) to obtain MFCC features.

[0107] Alternatively, the short-time Fourier transform (STFT) of the sound signal can be calculated to obtain a spectrum diagram. Features such as peak frequency and bandwidth can be extracted from the spectrum diagram to form spectral features.

[0108] Mel-frequency cepstral coefficients (MFCCs) are the coefficients that make up the Mel-frequency cepstral spectrum.

[0109] Image features are obtained by extracting depth features from the processed image set using a convolutional neural network.

[0110] In addition, color histograms and SIFT features can be extracted from the processed image set to form image features.

[0111] Specifically, ResNet can be used. The processed image set is input into ResNet, and the image features are extracted by ResNet.

[0112] Video features are obtained by extracting dynamic features from a set of keyframes using a 3D convolutional neural network or optical flow method.

[0113] 3D-CNN can be used to extract dynamic features from a set of keyframes.

[0114] S250 identifies target birds based on their sound features, image features, and video features.

[0115] After obtaining the target sound features, image features, and video features, it is necessary to perform time alignment on the target sound features, image features, and video features because there may be slight deviations in the clocks of the sound and image / video acquisition devices.

[0116] First, based on the timestamps of the microphone and the PTZ camera, the target sound features, image features and video features collected at the same time are matched to obtain the voiceprint feature vector, image feature vector and video feature vector.

[0117] Then, the target birds are identified based on voiceprint feature vectors, image feature vectors, and video feature vectors.

[0118] In some embodiments, the voiceprint feature vector, image feature vector, and video feature vector can be concatenated first to obtain a fused feature vector.

[0119] Then, the fused feature vector is input into a pre-trained bird species recognition model to obtain the species of the target bird.

[0120] The key to bird identification lies in multimodal fusion recognition: by deploying microphones for high-precision sound localization, and combining detailed texture features captured by static images with dynamic behavioral information provided by video sequences, a collaborative recognition framework integrating sound, image, and video features is constructed. This deep fusion strategy not only mutually corroborates and compensates for the shortcomings of single modalities, such as the failure of vision in low light / occlusion conditions, or the confusion of sound in strong noise, but also significantly improves the accuracy and robustness of bird identification in complex wild environments, making it crucial for accurate and efficient bird identification.

[0121] In this embodiment, the bird species recognition model is obtained by training and testing on a training set and a test set. The training set is used to train the bird species recognition model. The training set includes multiple training samples, each of which includes a fused feature vector formed by fusing voiceprint feature vectors, image feature vectors, and video feature vectors, and also includes the bird species label corresponding to the fused feature vector.

[0122] In this embodiment, the bird species identification model can be a random forest model or a neural network model; no limitation is made here.

[0123] In some embodiments, three models can be constructed to identify the three features respectively, and the final identification result can be determined based on the identification results of the three models.

[0124] In this embodiment, the voiceprint feature vector can be input into a voice recognition model to obtain a first classification result for the target bird. The image feature vector can be input into an image recognition model to obtain a second classification result for the target bird. The video feature vector can be input into a video recognition model to obtain a third classification result for the target bird. Finally, based on the first, second, and third classification results, the species of the target bird is determined.

[0125] Specifically, the three classification results can be fused using a weighted summation method, with different weights assigned to each of the three classification results, so as to determine the species of the target bird based on the final result.

[0126] Based on the confidence levels (AVG) of the three identification results, the bird with the highest confidence level is obtained by summarizing and weighting. The confirmation methods are different for birds with different protection levels. Generally, birds are directly judged by the model version, while birds with level 1 or 2 protection are confirmed by the back-end maintenance personnel.

[0127] In this embodiment, the sound recognition model, image recognition model, and video recognition model can be a support vector machine (SVM) or a neural network.

[0128] Furthermore, the identified bird species, along with their associated sounds, images, and videos, are stored in a database. The database table structure can include bird ID, species name, location time, voiceprint feature data, image / video feature data, and confidence level parameters such as highest confidence, lowest confidence, and average confidence, facilitating subsequent queries and analysis. Additionally, the bird sounds stored in the speaker can be periodically evaluated, and the sounds in the sound database can be updated and replaced according to recognition needs, improving the induction rate.

[0129] To demonstrate the effectiveness of the active identification method provided by this invention in bird attraction and identification, data statistics were conducted in a protected area based on conventional passive identification methods and the active identification method provided by this invention. The results are shown in Tables 1-3. The data in the tables were obtained from the trial results of a national wetland park in Zhangjiakou, Hebei Province.

[0130] Table 1. Recognition results of traditional bird identification devices

[0131]

[0132]

[0133] Table 2. Identification results of the identification method provided by this invention.

[0134]

[0135]

[0136]

[0137] Table 3 Comparison results of the two recognition methods

[0138]

[0139]

[0140] As can be seen from the data above, the active recognition method provided by this invention improves confidence by 17.07% and the number of sounds by 10.7% compared with the traditional passive recognition method.

[0141] By quantitatively evaluating the induction effect, and using the target birds' response confidence and the number of effective responses as core indicators, the voiceprint playback strategy is dynamically adjusted: for inefficient voiceprints, such as those with an induction success rate of <50% for three consecutive cycles, step-by-step frequency amplification playback and multi-scene voiceprint supplementation can be implemented; for efficient voiceprints, such as those with an induction success rate >50%, the activation intensity is attenuated and intermittent playback is performed in different time periods; at the same time, playback is stopped for voiceprints that have been ineffective for a long time, forming a closed loop of "monitoring-evaluation-adjustment", which significantly improves the induction efficiency and avoids the birds developing acoustic adaptations.

[0142] The identification method provided by this invention, in order to actively attract birds, first requires sending a command to a speaker to play bird sound signals. Upon receiving the command, the speaker plays the bird sound signals to lure the birds. Next, once the microphone has captured the target call, the distance between the target bird and the microphone needs to be determined to adjust the pan-tilt camera's shooting angle for clear photographing and video recording. Finally, after obtaining the target bird's call, image set, and video set, feature extraction is performed on these data to improve identification accuracy.

[0143] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0144] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0145] Figure 4 A schematic diagram of the bird active identification device provided in an embodiment of the present invention is shown. For ease of explanation, only the parts related to the embodiment of the present invention are shown, and are described in detail below:

[0146] like Figure 4 As shown, the bird active identification device 400 includes:

[0147] The release signal module 410 is used to issue a command to play bird sound signals; wherein the bird sound signals include at least one preset call of a specific bird.

[0148] The location determination module 420 is used to determine the distance between the target bird and the microphone when the microphone detects the target call; wherein the target call is emitted by the target bird.

[0149] The shooting module 430 is used to adjust the shooting angle of the camera based on the distance of the target bird from the microphone, so as to take pictures and record videos of the target bird, and obtain an image set and a video set;

[0150] The feature extraction module 440 is used to extract features from the target's vocalizations, image set, and video set respectively, to obtain the target's sound features, image features, and video features.

[0151] The recognition module 450 is used to identify target birds based on target sound features, image features, and video features.

[0152] In one possible implementation, multiple microphones are placed at the target location, and all microphones are arranged on a circumference.

[0153] The location determination module 420 is used to determine the position of the target bird from the microphone when the microphone captures the target call, based on the distance difference between each microphone and the time difference between each microphone receiving the target call.

[0154] In one possible implementation, the feature extraction module 440 is used to extract the Mel frequency cepstral coefficients and / or spectral features of the target call to obtain the target sound features;

[0155] Image features are obtained by extracting deep features from an image set using a convolutional neural network;

[0156] Dynamic features are extracted from the video set using a 3D convolutional neural network or optical flow method to obtain video features.

[0157] In one possible implementation, the feature extraction module 440 is used to perform frame-segmentation and pre-emphasis processing on the target call to obtain the processed target call.

[0158] The images in the image set are subjected to grayscale conversion, noise reduction, and image enhancement to obtain the processed image set;

[0159] Extract keyframes from the video set to obtain a keyframe set;

[0160] Feature extraction was performed on the processed target vocalizations, the processed image set, and the keyframe set, respectively.

[0161] In one possible implementation, the recognition module 450 is used to perform time alignment processing on the target sound features, image features and video features to obtain the voiceprint feature vector, image feature vector and video feature vector;

[0162] The target birds are identified based on voiceprint feature vectors, image feature vectors, and video feature vectors.

[0163] In one possible implementation, the recognition module 450 is used to concatenate the voiceprint feature vector, image feature vector and video feature vector to obtain a fused feature vector.

[0164] The fused feature vector is input into a pre-trained bird species identification model to obtain the species of the target bird.

[0165] In one possible implementation, the recognition module 450 is used to input the voiceprint feature vector into the sound recognition model to obtain the first classification result of the target bird;

[0166] The image feature vector is input into the image recognition model to obtain the second classification result of the target bird;

[0167] The video feature vector is input into the video recognition model to obtain the third classification result of the target bird.

[0168] Based on the results of the first, second, and third classifications, the species of the target bird are obtained.

[0169] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. For example... Figure 5 As shown, the electronic device 5 of this embodiment includes a processor 50 and a memory 51. The memory 51 stores a computer program 52. When the processor 50 executes the computer program 52, it implements the steps in the various method embodiments described above. Alternatively, when the processor 50 executes the computer program 52, it implements the functions of each module / unit in the various device embodiments described above.

[0170] For example, computer program 52 may be divided into one or more modules / units, which are stored in memory 51 and executed by processor 50 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 52 in electronic device 5.

[0171] Electronic device 5 may include, but is not limited to, processor 50 and memory 51. Those skilled in the art will understand that... Figure 5 This is merely an example of electronic device 5 and does not constitute a limitation on electronic device 5. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 5 may also include input / output devices, network access devices, buses, etc.

[0172] The processor 50 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0173] The memory 51 can be an internal storage unit of the electronic device 5, such as a hard disk or RAM. The memory 51 can also be an external storage device of the electronic device 5, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 51 can include both internal and external storage units of the electronic device 5. The memory 51 is used to store the computer program 52 and other programs and data required by the electronic device 5. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0174] For the sake of simplicity and clarity, only the above-described functional modules / units are used as examples. In practical applications, the functions described above can be assigned to different functional modules / units as needed. These modules / units can be implemented in hardware, software, or a combination of both.

[0175] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the methods described in the above-described method embodiments.

[0176] This invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the methods described in the above-described method embodiments.

[0177] Computer programs include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0178] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not detailed or described in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Unless otherwise specified or in conflict with logic, the terminology and / or descriptions between different embodiments are consistent and can be referenced interchangeably. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0179] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for active bird identification, characterized in that, include: Issue a command to play bird sound signals; wherein the bird sound signals include at least one preset call of a specific bird; When it is determined that the microphone has captured the target call, the distance between the target bird and the microphone is determined; wherein, the target call is emitted by the target bird; Based on the distance of the target bird from the microphone, the shooting angle of the pan-tilt camera is adjusted to take pictures and record videos of the target bird, resulting in an image set and a video set; Feature extraction is performed on the target call, the image set, and the video set respectively to obtain target sound features, image features, and video features; The target bird is identified based on its sound features, image features, and video features.

2. The active bird identification method according to claim 1, characterized in that, Multiple microphones are placed at the target location, and all microphones are arranged in a circle. When it is determined that the microphone has captured the target call, determining the distance between the target bird and the microphone includes: When it is determined that the microphone has captured the target call, a hyperbolic equation is established based on the distance difference between each microphone and the time difference between each microphone receiving the target call, to determine the position of the target bird from the microphone.

3. The active bird identification method according to claim 1, characterized in that, The step of extracting features from the target call, the image set, and the video set to obtain target sound features, image features, and video features includes: Extract the Mel frequency cepstral coefficients and / or spectral features of the target call to obtain the target sound features; The image features are obtained by extracting the depth features of the image set based on a convolutional neural network; The video features are obtained by extracting dynamic features from the video set based on a three-dimensional convolutional neural network or optical flow method.

4. The active bird identification method according to claim 1 or 3, characterized in that, Before performing feature extraction on the target call, the image set, and the video set, respectively, the method further includes: The target call is subjected to frame-segmentation and pre-emphasis processing to obtain the processed target call. Each image in the image set is subjected to grayscale conversion, noise reduction, and image enhancement to obtain the processed image set; Extract keyframes from the video set to obtain a keyframe set; The feature extraction process for the target call, the image set, and the video set includes: Feature extraction is performed on the processed target cry, the processed image set, and the keyframe set, respectively.

5. The active bird identification method according to claim 1, characterized in that, The identification of the target bird based on the target's sound features, image features, and video features includes: The target sound features, image features, and video features are time-aligned to obtain voiceprint feature vectors, image feature vectors, and video feature vectors. The target bird is identified based on the voiceprint feature vector, image feature vector, and video feature vector.

6. The active bird identification method according to claim 5, characterized in that, The identification of the target bird based on the voiceprint feature vector, image feature vector, and video feature vector includes: The voiceprint feature vector, image feature vector, and video feature vector are concatenated to obtain a fused feature vector. The fused feature vector is input into a pre-trained bird species recognition model to obtain the species of the target bird.

7. The active bird identification method according to claim 5, characterized in that, The identification of the target bird based on the voiceprint feature vector, image feature vector, and video feature vector includes: The voiceprint feature vector is input into the sound recognition model to obtain the first classification result of the target bird. The image feature vector is input into the image recognition model to obtain the second classification result of the target bird; The video feature vector is input into the video recognition model to obtain the third classification result of the target bird; Based on the first classification result, the second classification result, and the third classification result, the species of the target bird is obtained.

8. A bird active identification device, characterized in that, include: A release signal module is used to issue a command to play bird sound signals; wherein, the bird sound signals include at least one preset call of a specific bird; A location determination module is used to determine the distance between the target bird and the microphone when the microphone detects a target call; wherein the target call is emitted by the target bird. The shooting module is used to adjust the shooting angle of the camera based on the distance of the target bird from the microphone, so as to take pictures and record videos of the target bird, and obtain an image set and a video set; The feature extraction module is used to extract features from the target call, the image set, and the video set respectively, to obtain target sound features, image features, and video features; The identification module is used to identify the target bird based on the target's sound features, image features, and video features.

9. A bird active identification system, characterized in that, It includes multiple speakers, multiple microphones, a pan-tilt camera, and an active bird recognition device; The speaker is used to play bird sound signals at preset playback cycles after receiving an instruction; The microphone is used to capture bird calls; The pan-tilt camera is used to take pictures or record videos of birds; The bird active identification device is used to identify birds based on the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images