Mobile monitoring method for bird sound source tracking identification

Abstract

The invention discloses a mobile monitoring method for bird sound source tracking and identification, which comprises the following steps: acquiring a bird image sample, constructing a bird identification model, and training the bird identification model based on the bird image sample; synchronously acquiring bird sound signals of multiple channels through a microphone array; calculating sound wave propagation time difference of each receiving channel of the microphone array based on the pre-processed bird sound signal, and further obtaining a sound source positioning result; transmitting the sound source positioning result to the intelligent carrier, and driving the intelligent carrier to autonomously move towards the sound source; bird images are collected in real time based on video collection equipment, image recognition is conducted on the bird images collected in real time through the trained bird recognition model, a bird recognition result is obtained, the bird recognition result and a sound source positioning result are combined and displayed, and birds are monitored in real time. According to the invention, automatic identification and information transmission of the target bird species are realized, and a movable and intelligent solution is provided for bird monitoring.

Description

Technical Field

[0001] The present invention belongs to the field of bird ecology and conservation biology technology, and in particular relates to a mobile monitoring method for tracking and identifying bird sound sources. Background Art

[0002] Socioeconomic development and global climate change have led to a sharp decline in wildlife populations worldwide. This decline not only threatens biodiversity but also has profound impacts on the integrity and stability of ecosystems. Therefore, accurately assessing wildlife populations and their changing trends is crucial for developing effective conservation strategies.

[0003] However, although traditional wildlife survey methods, such as line transects, point sampling, and infrared camera methods, have played an important role in ecological research, these methods have limitations. A recent study published in Cuestiones de Fisioterapia pointed out that traditional methods require a lot of manpower and material resources. For example, researchers found that some hidden birds are often more difficult to find during bird surveys. Moreover, the accuracy of traditional methods is often affected by human factors. For example, investigators need to have professional knowledge, but due to different experiences of investigators, errors will occur in the identification of certain individual animals. In addition, when faced with large areas with complex and changing environments, traditional wildlife population survey methods often seem powerless and cannot meet the needs of modern ecological protection for efficient and accurate data.

[0004] However, the rapid development of artificial intelligence is gradually changing the traditional methods of population ecological monitoring. AI technology can efficiently process and analyze large amounts of data through technologies such as machine learning, deep learning, computer vision, and soundscape analysis. Through AI's deep learning algorithm, the species and number of animals in the image can be quickly identified, which not only greatly improves the efficiency of the survey, but also reduces human errors and improves data accuracy. In addition, AI technology has also shown great potential in sound monitoring. Traditional animal sound monitoring mainly relies on recording equipment and manual identification, but for scenes with large amounts of data and more complex environments, traditional methods are time-consuming and labor-intensive. Müller et al. (2023) used bioacoustics and eDNA technology and found that soundscape analysis and deep learning can effectively track the recovery of tropical forest animal communities.

[0005] In this context, there is an urgent need for a mobile monitoring method for bird sound source tracking and identification that combines traditional surveys with AI, so as to make up for the shortcomings of existing technologies with intelligent, mobile, and real-time monitoring methods. Summary of the Invention

[0006] To solve the above technical problems, the present invention proposes a mobile monitoring method for bird sound source tracking and identification. This method uses a high-resolution camera and microphone array to collect bird images and sound data in real time, and performs local processing through embedded devices to achieve automatic identification and information transmission of target bird species. It focuses on solving the problems of poor real-time transmission and large interference to birds in the existing technology, and provides a mobile and intelligent solution for bird monitoring to solve the problems existing in the above-mentioned existing technology.

[0007] To achieve the above objectives, the present invention provides a mobile monitoring system for bird sound source tracking and identification, comprising:

[0008] An image recognition module is used to build a bird recognition model based on the Keras-YOLOv3 network and collect bird image samples to train the bird recognition model;

[0009] The sound source localization module is used to synchronously collect multi-channel bird sound signals through a microphone array and perform pre-processing;

[0010] An edge computing module, connected to the sound source localization module, is used to receive the preprocessed bird sound signal and calculate the sound wave propagation time difference of each receiving channel of the microphone array using a generalized cross-correlation algorithm to obtain the sound source localization result;

[0011] An intelligent mobile module, connected to the edge computing module, is used to transmit the sound source localization result to the intelligent carrier, drive the intelligent carrier to move autonomously toward the sound source; at the same time, the sound source localization result is transmitted to the remote monitoring module by using the intelligent carrier;

[0012] The remote monitoring module uses video acquisition and wireless transmitter equipment to collect and transmit bird images in real time. The trained bird recognition model is used to perform image recognition on the bird images collected in real time to obtain bird recognition results. The bird recognition results are combined with the sound source localization results for display, and birds are monitored in real time.

[0013] Optionally, the image recognition module includes:

[0014] A data collection unit, configured to collect bird image samples and perform data annotation and data enhancement on the bird image samples;

[0015] A model building unit for building a bird recognition model based on a Keras-YOLOv3 network; the Keras-YOLOv3 network framework uses Darknet-53 as the backbone network. The Darknet-53 network includes 52 convolutional layers and 23 residual modules with cross-layer connections, performs feature extraction through five downsamplings, and includes a feature pyramid network at three detection scales, which achieves multi-scale feature fusion through upsampling and feature concatenation;

[0016] The model training unit is used to train the bird recognition model based on the bird image samples after data annotation and data enhancement to obtain a trained bird recognition model.

[0017] Optionally, the sound source localization module includes:

[0018] A sound source acquisition unit, used for synchronously acquiring multi-channel bird sound signals through a microphone array;

[0019] The sound source processing unit is used to perform filtering, noise reduction, gain compensation and synchronization calibration on the collected bird sound signals.

[0020] Optionally, the edge computing module includes:

[0021] a signal receiving unit, configured to receive the pre-processed bird sound signal;

[0022] The sound source localization unit is used to calculate the sound wave propagation time difference of each receiving channel of the microphone array based on the preprocessed bird sound signal using a generalized cross-correlation algorithm to obtain the sound source localization result;

[0023] The signal enhancement unit is used to perform signal enhancement on the sound source localization result by using an optimal beamforming algorithm to obtain an enhanced sound source localization result.

[0024] The present invention also provides a mobile monitoring method for tracking and identifying bird sound sources, based on the system, comprising the following steps:

[0025] Collect bird image samples, build a bird recognition model based on the Keras-YOLOv3 network, and train the bird recognition model based on the bird image samples;

[0026] Multi-channel bird sound signals are collected synchronously through a microphone array and pre-processed;

[0027] Based on the pre-processed bird sound signals, the generalized cross-correlation algorithm is used to calculate the sound wave propagation time difference of each receiving channel of the microphone array, and then the sound source localization result is obtained;

[0028] Transmitting the sound source localization result to an intelligent carrier, driving the intelligent carrier to autonomously move toward the sound source; at the same time, using the intelligent carrier to transmit the sound source localization result in real time;

[0029] Bird images are collected in real time based on video acquisition equipment, and image recognition is performed on the real-time collected bird images through a trained bird recognition model to obtain bird recognition results. The bird recognition results are combined with sound source localization results for display, and birds are monitored in real time.

[0030] Optionally, the Keras-YOLOv3 network framework uses Darknet-53 as the backbone network. The Darknet-53 network includes 52 convolutional layers and 23 residual modules with cross-layer connections, and realizes feature extraction through 5 downsampling. It also includes a feature pyramid network of 3 detection scales, and realizes multi-scale feature fusion through upsampling and feature splicing.

[0031] Optionally, the process of synchronously collecting multi-channel bird sound signals through a microphone array and preprocessing the signals includes:

[0032] Multi-channel bird sound signals are synchronously collected through a microphone array, and the collected bird sound signals are filtered, denoised, gain compensated and synchronously calibrated.

[0033] Optionally, after obtaining the sound source localization result, the method further includes:

[0034] An optimal beamforming algorithm is used to perform signal enhancement on the sound source localization result to obtain an enhanced sound source localization result.

[0035] Optionally, after collecting the bird image samples, the method further includes:

[0036] Data annotation and data enhancement are performed on the bird image samples.

[0037] Optionally, performing image recognition on the bird images collected in real time using the trained bird recognition model to obtain the bird recognition result further includes:

[0038] In the post-processing stage, non-maximum suppression is used to remove redundant frames to obtain the final bird recognition results.

[0039] Compared with the prior art, the present invention has the following advantages and technical effects:

[0040] The present invention provides a bird image recognition model based on deep learning, which can achieve the effect of real-time detection of bird categories in the wild through model training based on data from an existing bird image database; based on a microphone array device, by measuring the arrival time difference of sound waves received by multiple microphones, a time delay estimation algorithm is used to infer the azimuth of the bird sound source; during movement, the intelligent carrier can receive the sound source azimuth information calculated based on the microphone array in real time, and dynamically adjust the direction of travel accordingly to achieve autonomous tracking of the sound source.

[0041] The present invention uses a high-resolution camera and microphone array to collect bird images and sound data in real time, and performs local processing through embedded devices to achieve automatic identification and information transmission of target bird species. It focuses on solving the problems of poor real-time transmission and large interference to birds in the existing technology, and provides a mobile and intelligent solution for bird monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0043] Figure 1 Schematic diagram of a method flow in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0045] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] Example 1

[0047] This embodiment provides a mobile monitoring system for tracking and identifying bird sound sources, including:

[0048] An image recognition module is used to build a bird recognition model based on the Keras-YOLOv3 network and collect bird image samples to train the bird recognition model;

[0049] The sound source localization module is used to synchronously collect multi-channel bird sound signals through a microphone array and perform pre-processing;

[0050] An edge computing module, connected to the sound source localization module, is used to receive the preprocessed bird sound signal and calculate the sound wave propagation time difference of each receiving channel of the microphone array using a generalized cross-correlation algorithm to obtain the sound source localization result;

[0051] An intelligent mobile module, connected to the edge computing module, is used to transmit the sound source localization result to the intelligent carrier, drive the intelligent carrier to move autonomously toward the sound source; at the same time, the sound source localization result is transmitted to the remote monitoring module by using the intelligent carrier;

[0052] The remote monitoring module is connected to the image recognition module and the intelligent mobile module respectively, and is used to collect bird images in real time based on the video acquisition device, perform image recognition on the bird images collected in real time through the trained bird recognition model, obtain bird recognition results, combine the bird recognition results with the sound source localization results for display, and monitor the birds in real time.

[0053] The image recognition module can be implemented to include: a data collection unit for collecting bird image samples and performing data annotation and data enhancement on the bird image samples; a model construction unit for constructing a bird recognition model based on the Keras-YOLOv3 network; the Keras-YOLOv3 network framework uses Darknet-53 as the backbone network, the Darknet-53 network includes 52 convolutional layers and 23 residual modules with cross-layer connections, and realizes feature extraction through 5 downsamplings, and also includes a feature pyramid network of 3 detection scales, and realizes multi-scale feature fusion through upsampling and feature splicing; a model training unit for training the bird recognition model based on the bird image samples after data annotation and data enhancement to obtain a trained bird recognition model.

[0054] As a specific implementation method, the image recognition module of this embodiment is based on the AXU2CGB-E model FPGA development board platform and adopts the Keras-YOLOv3 framework to implement the bird target detection algorithm.

[0055] First, the LabelImg tool was used to annotate the bird image dataset according to the PASCAL VOC standard, generating an XML file containing bounding box coordinates and species classification labels. Data augmentation techniques such as mosaic enhancement and random HSV perturbation were then used to improve the model's generalization ability.

[0056] The model network architecture uses Darknet-53 as the backbone network, which consists of 52 convolutional layers and 23 residual modules with cross-layer connections. Feature extraction is achieved through five downsampling steps. For feature fusion, a Feature Pyramid Network (FPN) with three detection scales is constructed. Multi-scale feature fusion is achieved through upsampling and feature concatenation, effectively improving the detection capabilities of birds of different sizes.

[0057] During the model training phase, model parameters are continuously adjusted through multiple iterations to reduce the error between prediction results and annotations, thereby improving model detection performance. After training, the model is quantized and deployed on an FPGA development board. High-definition cameras are used to collect real-time image data of wild birds for feature recognition. In the post-processing phase, non-maximum suppression is used to remove redundant frames and output the final detection results.

[0058] The sound source localization module can be implemented to include: a sound source collection unit for synchronously collecting multi-channel bird sound signals through a microphone array; and a sound source processing unit for filtering, noise reduction, gain compensation, and synchronous calibration of the collected bird sound signals.

[0059] As a specific implementation method, the microphone array synchronously collects multi-channel sound signals and optimizes signal quality and reduces the impact of noise interference and hardware differences through preprocessing steps such as filtering and noise reduction, gain compensation and synchronous calibration.

[0060] The edge computing module can be implemented to include: a signal receiving unit for receiving preprocessed bird sound signals; a sound source localization unit for calculating the sound wave propagation time difference of each receiving channel of the microphone array based on the preprocessed bird sound signals using a generalized cross-correlation algorithm, thereby obtaining a sound source localization result; and a signal enhancement unit for using an optimal beamforming algorithm to perform signal enhancement on the sound source localization result to obtain an enhanced sound source localization result.

[0061] As a specific implementation method, the edge computing module of this embodiment uses the generalized cross-correlation (GCC) algorithm to calculate the sound wave propagation time difference of each receiving channel of the microphone array, and then derives the sound source azimuth information. Combined with the optimal beamforming algorithm, the target signal is enhanced, the background noise and acoustic reverberation are effectively suppressed, and the sound source positioning is improved. The sound source positioning result (azimuth angle) is transmitted to the intelligent carrier through the edge computing module, driving it to move autonomously close to the sound source. At the same time, the azimuth angle is transmitted to the remote monitoring platform in real time through the Bluetooth module of the intelligent carrier to realize real-time monitoring of the system status.

[0062] As a specific implementation method, the remote receiving module of this embodiment uses a video capture device and a wireless transmitter to transmit the image recognized by the FPGA development board to the computer, receives information through Python, develops a web page using html+JavaScript+CSS under the Django framework, and then displays the recognized results in the browser.

[0063] like Figure 1 As shown, on the other hand, based on the same inventive concept as the above embodiment, this embodiment also provides a mobile monitoring method for tracking and identifying bird sound sources. The method and the system provided in the above embodiment can be mutually referenced in terms of effect, and include the following steps:

[0064] Collect bird image samples, build a bird recognition model based on the Keras-YOLOv3 network, and train the bird recognition model based on the bird image samples;

[0065] Multi-channel bird sound signals are collected synchronously through a microphone array and pre-processed;

[0066] Based on the pre-processed bird sound signals, the generalized cross-correlation algorithm is used to calculate the sound wave propagation time difference of each receiving channel of the microphone array, and then the sound source localization result is obtained;

[0067] Transmitting the sound source localization result to an intelligent carrier, driving the intelligent carrier to autonomously move toward the sound source; at the same time, using the intelligent carrier to transmit the sound source localization result in real time;

[0068] Bird images are collected in real time based on video acquisition equipment, and image recognition is performed on the real-time collected bird images through a trained bird recognition model to obtain bird recognition results. The bird recognition results are combined with sound source localization results for display, and birds are monitored in real time.

[0069] The Keras-YOLOv3 network framework uses Darknet-53 as the backbone network. The Darknet-53 network includes 52 convolutional layers and 23 residual modules with cross-layer connections. Feature extraction is achieved through 5 downsamplings. It also includes a feature pyramid network of 3 detection scales, and multi-scale feature fusion is achieved through upsampling and feature splicing.

[0070] The process of synchronously collecting multi-channel bird sound signals through a microphone array and performing preprocessing can be implemented, including: synchronously collecting multi-channel bird sound signals through a microphone array, filtering and noise reduction, gain compensation and synchronous calibration of the collected bird sound signals.

[0071] Optionally, after obtaining the sound source localization result, the method further includes: performing signal enhancement on the sound source localization result using an optimal beamforming algorithm to obtain an enhanced sound source localization result.

[0072] It is feasible that after collecting the bird image samples, the method further includes: performing data annotation and data enhancement processing on the bird image samples.

[0073] It is feasible to perform image recognition on bird images collected in real time through the trained bird recognition model, and the process of obtaining the bird recognition result also includes: in the post-processing stage, using non-maximum suppression to remove redundant frames to obtain the final bird recognition result.

[0074] Example 2

[0075] This embodiment uses the grassland ecosystem in Inner Mongolia as an example. It can be understood that the application proposed in this embodiment can be applied not only to grassland ecosystems, but also to other ecosystems such as forest ecosystems.

[0076] Step 1: Build an image recognition model:

[0077] Focusing on the grassland ecosystem in Inner Mongolia, we understand the bird species in Inner Mongolia based on multiple sources such as experiments, books, and literature. We collect the required bird image samples through open source databases such as iNaturalist for annotation and training of bird image recognition models, thereby constructing a bird image recognition model for Inner Mongolia.

[0078] Step 2: Development of bird sound source localization system:

[0079] Data acquisition and preprocessing: The microphone array synchronously collects multi-channel sound signals and optimizes signal quality and reduces the impact of noise interference and hardware differences through preprocessing steps such as filtering and noise reduction, gain compensation, and synchronous calibration.

[0080] Delay Estimation and Signal Enhancement: In the edge computing module, the generalized cross-correlation (GCC) algorithm is used to calculate the difference in sound wave propagation time between each receiving channel of the microphone array, thereby deriving the direction of the sound source. Combined with the optimal beamforming algorithm, the target signal is enhanced, effectively suppressing background noise and acoustic reverberation, and improving the accuracy of sound source localization.

[0081] Step 3: Development of the mobility control algorithm for the intelligent carrier:

[0082] Smart Carrier Control and Remote Monitoring: The sound source localization result (azimuth angle) is transmitted to the smart carrier via the edge computing module. A PID control algorithm is used to control the rotation of the four-wheel motors, driving them to autonomously move closer to the sound source. Simultaneously, the azimuth angle is transmitted in real time to the remote monitoring platform via the smart carrier's Bluetooth module, enabling real-time monitoring of the system status.

[0083] Step 4: Build a remote monitoring platform:

[0084] The remote receiving module uses a video capture device and a wireless transmitter to transmit images detected by the FPGA development board to a computer, where they are received using Python and its third-party libraries. Simultaneously, the computer uses Django as a web development framework, employing front-end technologies such as HTML, JavaScript, and CSS to build a bird sound source location and image recognition system, creating a remote detection platform within the browser.

[0085] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A mobile monitoring system for bird sound source tracking and identification, characterized in that: include: An image recognition module is used to build a bird recognition model based on the Keras-YOLOv3 network and collect bird image samples to train the bird recognition model; The sound source localization module is used to synchronously collect multi-channel bird sound signals through a microphone array and perform pre-processing; An edge computing module, connected to the sound source localization module, is used to receive the preprocessed bird sound signal and calculate the sound wave propagation time difference of each receiving channel of the microphone array using a generalized cross-correlation algorithm to obtain the sound source localization result; An intelligent mobile module, connected to the edge computing module, is used to transmit the sound source localization result to the intelligent carrier, drive the intelligent carrier to move autonomously toward the sound source; at the same time, the sound source localization result is transmitted to the remote monitoring module by using the intelligent carrier; The remote monitoring module is connected to the image recognition module and the intelligent mobile module respectively, and is used to collect bird images in real time based on the video acquisition device, perform image recognition on the bird images collected in real time through the trained bird recognition model, obtain bird recognition results, combine the bird recognition results with the sound source localization results for display, and monitor the birds in real time.

2. The system according to claim 1, wherein: The image recognition module includes: A data collection unit, configured to collect bird image samples and perform data annotation and data enhancement on the bird image samples; A model building unit for building a bird recognition model based on a Keras-YOLOv3 network; the Keras-YOLOv3 network framework uses Darknet-53 as the backbone network. The Darknet-53 network includes 52 convolutional layers and 23 residual modules with cross-layer connections, performs feature extraction through five downsamplings, and includes a feature pyramid network at three detection scales, which achieves multi-scale feature fusion through upsampling and feature concatenation; The model training unit is used to train the bird recognition model based on the bird image samples after data annotation and data enhancement to obtain a trained bird recognition model.

3. The system according to claim 1, wherein: The sound source localization module includes: A sound source acquisition unit, used for synchronously acquiring multi-channel bird sound signals through a microphone array; The sound source processing unit is used to perform filtering, noise reduction, gain compensation and synchronization calibration on the collected bird sound signals.

4. The system according to claim 1, wherein: The edge computing module includes: a signal receiving unit, configured to receive the pre-processed bird sound signal; The sound source localization unit is used to calculate the sound wave propagation time difference of each receiving channel of the microphone array based on the preprocessed bird sound signal using a generalized cross-correlation algorithm to obtain the sound source localization result; The signal enhancement unit is used to perform signal enhancement on the sound source localization result by using an optimal beamforming algorithm to obtain an enhanced sound source localization result.

5. A mobile monitoring method for bird sound source tracking and identification, characterized in that: The system according to any one of claims 1 to 4 comprises the following steps: Collect bird image samples, build a bird recognition model based on the Keras-YOLOv3 network, and train the bird recognition model based on the bird image samples; Multi-channel bird sound signals are collected synchronously through a microphone array and pre-processed; Based on the pre-processed bird sound signals, the generalized cross-correlation algorithm is used to calculate the sound wave propagation time difference of each receiving channel of the microphone array, and then the sound source localization result is obtained; Transmitting the sound source localization result to an intelligent carrier, driving the intelligent carrier to autonomously move toward the sound source; at the same time, using the intelligent carrier to transmit the sound source localization result in real time; Bird images are collected in real time based on video acquisition equipment, and image recognition is performed on the real-time collected bird images through a trained bird recognition model to obtain bird recognition results. The bird recognition results are combined with sound source localization results for display, and birds are monitored in real time.

6. The method according to claim 5, characterized in that The Keras-YOLOv3 network framework uses Darknet-53 as the backbone network. The Darknet-53 network includes 52 convolutional layers and 23 residual modules with cross-layer connections. It achieves feature extraction through 5 downsamplings. It also includes a feature pyramid network with 3 detection scales and realizes multi-scale feature fusion through upsampling and feature splicing.

7. The method according to claim 5, characterized in that The process of synchronously collecting multi-channel bird sound signals through a microphone array and preprocessing them includes: Multi-channel bird sound signals are synchronously collected through a microphone array, and the collected bird sound signals are filtered, denoised, gain compensated and synchronously calibrated.

8. The method according to claim 5, characterized in that After obtaining the sound source localization result, it also includes: An optimal beamforming algorithm is used to perform signal enhancement on the sound source localization result to obtain an enhanced sound source localization result.

9. The method according to claim 5, characterized in that After collecting bird image samples, we also include: Data annotation and data enhancement are performed on the bird image samples.

10. The method according to claim 9, characterized in that The trained bird recognition model is used to perform image recognition on the real-time collected bird images. The process of obtaining the bird recognition results also includes: In the post-processing stage, non-maximum suppression is used to remove redundant frames to obtain the final bird recognition results.

Images