A bird sound event detection method and system for monitoring state key protected birds

Abstract

The application discloses a bird sound event detection method and system for monitoring state-protected birds, and comprises the following steps: collecting bird sound data and environmental noise data of state-protected birds, and constructing a bird sound spectrogram data set and a noisy bird sound data set; constructing a bird sound noise reduction model and a target detection model; training and jointly optimizing the bird sound noise reduction model and the target detection model according to the bird sound spectrogram data set and the noisy bird sound data set; constructing an image-audio mapping module for converting the image detection result of the target detection model into an audio sound event detection result; acquiring bird sound data to be detected, searching and sound source positioning the target bird sound according to the bird sound noise reduction model, the target detection model and the image-audio mapping module, and determining the species, time-frequency and direction information of the target bird sound. The application can improve the detection effect of field bird sound, and further improve the classification accuracy, and can be widely applied to the technical field of acoustic information processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of acoustic information processing, and particularly relates to a bird sound event detection method and system for monitoring state key protected birds. BACKGROUND

[0002] State key protected birds are usually key species in the ecosystem, and their existence and health status directly affect the stability and diversity of the ecosystem. In recent years, bird sound monitoring plays a crucial role in bird research, especially in non-invasive monitoring, continuous monitoring possibilities and large-scale data collection. The application of this technology has greatly promoted the development of bird ecology and conservation biology. As a non-invasive technology, bird sound monitoring has minimal disturbance to birds and their habitats compared to traditional research methods such as capture and tagging. Using sound monitoring equipment, researchers can collect data without direct contact with birds, thereby reducing the impact on the natural behavior of birds. Through audio detection and identification of these state key protected birds, it can help monitor and protect these species. However, there are usually a lot of background noise in the field environment, such as wind noise, rain noise, human activity noise, etc., which may interfere with the detection and identification of bird sounds. Especially, many state key protected birds are very rare in number, and the calling interval of some birds is very long, so it is relatively difficult to obtain the corresponding bird call.

[0003] Bird sound detection and classification is a key field in bioacoustics research. Bird sounds usually occur in complex natural environments, and these environmental characteristics can cause the overlap of sound events, making it more difficult to detect and classify bird sounds. Many traditional signal processing-based methods rely on extracting basic time-frequency features from audio signals, such as Mel-frequency cepstral coefficients (MFCC), pitch, energy, etc., and then detecting sound events through rule-based or template matching methods. This kind of method is simple to implement and does not require complex algorithms and model training. However, this kind of method is heavily dependent on pre-defined rules or templates and can only capture some simple and obvious features, and cannot handle complex time series patterns and spectral changes, and has poor flexibility. In recent years, deep learning has rapidly developed in the field of bird sound detection and classification. CNN and CRNN-based bird sound detection methods can automatically extract complex time-frequency features from raw audio data, reducing the dependence on manually designed features, and can capture more rich sound patterns and details, and usually achieve good detection accuracy in handling detection tasks containing multiple sound events. In current deep learning bird sound detection methods, most CNN-based networks usually first convert audio data into other features such as Mel spectrograms, and then input them into the network. CNN extracts local features through multiple convolutional and pooling layers, and finally classifies or detects through fully connected layers. This method is good at capturing local patterns in audio and is suitable for detecting time-localized, i.e., segment-level, sound events. CRNN combines the advantages of CNN and RNN. First, the audio data is converted into time-frequency features through the convolutional layer, which is similar to the pure CNN model. Then, these features are passed through a recurrent neural network (RNN), usually a long short-term memory network (LSTM) or a gated recurrent unit (GRU), to capture the time series dependency in the audio. CRNN can therefore handle both the time-frequency features and the temporal dynamics of audio, making it suitable for detecting frame-level sound events.

[0004] The traditional signal processing bird sound detection method has insufficient flexibility and adaptability in the face of complex and variable real scenes, and is difficult to capture and process complex sound event characteristics. The bird sound detection method based on CNN often only focuses on the bird sound classification information of fixed length audio segments and ignores the specific variable bird song time range information. The method based on CRNN often only focuses on the classification and time range information of the bird sound event in the audio segment, but ignores the specific frequency range information and the position information of the actual sound source signal. In addition, many bird identification models often only focus on the bird sounds that can be seen everywhere in daily life, and ignore the state key protected birds. Moreover, the actual field recording signal often contains a large amount of various noises, which will greatly affect the reasoning effect of the detection model. The current detection method has the disadvantages of insufficient frequency analysis, noise interference prevention, missed detection prevention and environmental adaptability, which limits its effect in actual application scenarios. Especially for the state protected birds that need to be monitored for a long time and widely, the limitations of the existing method make it difficult to guarantee the accuracy and comprehensiveness of the monitoring data. SUMMARY

[0005] The main purpose of the embodiment of the present application is to provide a bird sound event detection method and system for state key protected bird monitoring, which can improve the detection effect of field bird sound and further improve the classification accuracy.

[0006] To achieve the above-mentioned purpose, one aspect of the embodiment of the present application provides a bird sound event detection method for state key protected bird monitoring, comprising the following steps:

[0007] Collecting bird sound data and environmental noise data of state key protected birds;

[0008] According to the bird sound data and the environmental noise data, a bird sound spectrogram data set and a noisy bird sound data set are constructed;

[0009] A bird sound noise reduction model and a target detection model are constructed;

[0010] According to the bird sound spectrogram data set and the noisy bird sound data set, the bird sound noise reduction model and the target detection model are trained and jointly optimized. The bird sound noise reduction model is used for noise reduction processing of noisy bird sound data, and the target detection model is used for detecting target images;

[0011] An image-audio mapping module is constructed, which is used to convert the image detection result of the target detection model into a sound event detection result of audio;

[0012] Acquire bird sound data to be detected, and search and locate the sound source of target bird sound according to the bird sound denoising model, the target detection model and the image-audio mapping module, to determine the species, time-frequency and direction information of the target bird sound.

[0013] In some embodiments, the bird sound data and environmental noise data of the state-protected bird are collected, including the following steps:

[0014] According to the list of state-level and second-level protected birds, the sound data of these birds is collected, including data downloaded from a bird sound data website or field recording data; wherein the sampling rate is not less than 32 kHz and the quantization accuracy is not less than 16 bits.

[0015] The collected data is uniformly processed and denoised to obtain clean bird sound data audio files after screening;

[0016] According to the bird species, the screened audio files are connected to obtain audio files of each bird species;

[0017] According to the noise data sampling rate requirement, quantization accuracy requirement and noise type, environmental noise data is acquired; wherein the noise type includes target noise in the actual monitoring area of the state-protected bird, including wind sound, automobile sound, rain sound, stream sound, cricket sound, katydid sound, cicada sound and frog sound.

[0018] In some embodiments, the bird sound spectrogram dataset and the noisy bird sound dataset are constructed according to the bird sound data and the environmental noise data, including the following steps:

[0019] The acquired bird sound data is converted from time domain to frequency domain to generate bird sound spectrograms;

[0020] Adjust the image size and color depth of the bird sound spectrogram;

[0021] Use an image annotation tool to draw a bounding box for each target bird sound on the bird sound spectrogram, label the category and save the annotation file;

[0022] The bird sound spectrogram and the corresponding annotation file constitute the bird sound spectrogram dataset;

[0023] The clean bird sound data is added with noise data to obtain the noisy bird sound dataset.

[0024] In some embodiments, the acquired bird sound data is converted from time domain to frequency domain to generate bird sound spectrograms, including the following steps:

[0025] The data sampling point number P of a bird sound signal is recorded, the frame length flen is set as 2048, the frame shift hlen is set as 512, the frame number fn of the bird sound signal after framing is obtained, and each frame signal after framing is recorded as x(p, λ), wherein p is the sampling point number, and λ is the frame number; the calculation formula of the frame number fn of the bird sound signal after framing is:

[0026] The current frame x(p, λ) and the last frame x(p, λ-1) are overlapped by a frame with a length of overlap=512 to obtain a target frame x o (p, λ); the total length of the frame overlap is the frame length flen.

[0027] The target frame x o (p, λ) is windowed, the type of the window is a Hamming window w(p, ψ), wherein ψ is 0.46, and the window length is equal to the frame length point number flen=2048, to obtain all the windowed frames x w (p, λ).

[0028] The DFT operation of P=2048 points is performed on each frame x w (p, λ) to obtain the amplitude spectrum X(f, λ) of each frame signal, f represents a frequency point, and λ is the frame number, the analysis of the first P f frequency points of the frequency spectrum is completed, and the bird sound spectrogram is generated.

[0029] The image size and color depth of the bird sound spectrogram are adjusted, including the following steps:

[0030] The calculation parameters of the generated spectrogram are uniformly processed, wherein the STFT window size is 2048, the frame shift is 512, and the image resolution DPI is 300.

[0031] The vertical axis scale of the spectrogram is set as a linear scale, and the color mapping is set as an 'inferno' mapping table; by setting the mapping table, different intensity frequency components in the bird sound spectrogram are mapped to specific colors.

[0032] The noise data is added to the obtained clean bird sound data to obtain the noisy bird sound data set, including the following steps:

[0033] The length of a bird sound signal is set as z seconds, the sampling rate is 32 kHz, and the corresponding sampling point number is z*32000; when the sampling point number of the selected noise signal is greater than z*32000, z*32000 continuous sampling points are randomly selected from the noise signal.

[0034] When clean bird sound signals and noise signals of the same length are obtained, noise is added to the clean bird sound signals; wherein each time the noise is selected so that the proportion of each type of noise type in the entire noisy bird sound signal is uniform, and the number of noisy bird sound signals of different noise types is ensured to be the same, and the signal mixing is carried out according to the following formula:

[0035]

[0036] Wherein t represents the time of the signal, z(t) is the noisy bird sound signal, z1(t) and n(t) are the clean bird sound signal and the noise signal, ξ is the gain coefficient of n(t) in the mixing process, and snr represents the signal-to-noise ratio of the clean bird sound signal and the noise signal.

[0037] In some embodiments, the bird sound noise reduction model comprises a data input layer, an initial causal convolution layer, an expansion residual block, a jump connection summary and a final convolution output layer; the construction of the bird sound noise reduction model comprises the following steps:

[0038] The input shape of the data input layer is (IN, 1), wherein IN represents the length of the input signal, and is used to receive the noisy bird sound signal;

[0039] The initial causal convolution layer uses a causal convolution kernel to process the input signal, so that each output depends on the current and previous input values, the number of convolution kernels of the initial causal convolution layer is 128, and the convolution kernel size is 3, and the process can be represented as:

[0040] h causal =σ(W causal *x+b causal )

[0041] Wherein W causal and b causal are the causal convolution kernel weight and bias, h causal is the output of the causal convolution, and σ is the ReLU activation function;

[0042] The expansion residual block has a total of 30 expansion residual blocks, and each expansion residual block includes an expansion convolution layer, an activation function, a gating mechanism, a 1x1 convolution layer, a jump connection and a residual connection; wherein the number of convolution kernels of the expansion convolution layer is 256, the convolution kernel size is 3, and the expansion rate increases by power from 1 to 512 (1, 2, 4, 8, 16, 32, 64, 128, 256, 512), a total of 10 expansion rates, and the stacking number is 3; the activation function includes hyperbolic tangent tanh function and sigmoid function, wherein the hyperbolic tangent tanh function is applied to the first part of the output of the expansion convolution layer to generate h tanh , and the sigmoid function is applied to the second part of the output of the expansion convolution layer to generate hsigma The gating mechanism generates h tanh and h sigma in combination through an element-wise multiplication operation. gate Then, h gate generates h 1x1 through a 1x1 convolutional layer, the main function of which is to adjust the dimension of the feature map through linear transformation and generate residual output and skip connection output, the number of convolutional kernels of which is 256, among which the number of residual channels and skip connections is 128 respectively; the skip connection part extracts the skip connection output from h 1x1 for the summary of subsequent layers, and the residual connection part adds the residual output of the 1x1 convolutional layer to the input signal to generate the output of the expanded residual block.

[0043] The skip connection summary generates a comprehensive feature representation by summing all the skip connection outputs of the expanded residual blocks; wherein the comprehensive feature representation contains the features of all layers, providing rich feature representation for the final output layer through feature information at different scales and different expansion rates, and the formula of the summing operation is:

[0044]

[0045] where s d represents the skip connection output of the dth expanded residual block, and D is the total number of expanded residual blocks; s represents the sum result.

[0046] The last convolutional output layer is composed of three convolutional layers, among which the first convolutional layer has a convolutional kernel number of 2048 and a convolutional kernel size of 3, which is used to extract deeper features from the comprehensive feature representation of the skip connection; the second convolutional layer has a convolutional kernel number of 256 and a convolutional kernel size of 3 to compress the feature map to a smaller dimension; the third convolutional layer is a 1x1 convolutional layer with a convolutional kernel number and size of 1, which is used to linearly transform the feature map to generate the final denoised bird sound signal.

[0047] In some embodiments, the method of constructing a target detection model comprises the following steps:

[0048] A Backbone backbone network is constructed, which includes a Stem Layer starting layer, convolutional modules from Stage 1 to Stage 4, a CSP layer, and an SPPF layer; wherein the Stem Layer starting layer is composed of a convolutional module, which includes a two-dimensional convolutional layer, batch normalization after convolutional operation, and a SiLU activation function; the Stem Layer starting layer is used to preliminarily extract low-level features of the image, and its operation process can be represented by the following formula:

[0049] y=SiLU(BatchNorm(Conv2d(x,W,b)))

[0050] Where x is the input image, and W and b are the convolution kernel weights and biases, respectively;

[0051] Four Stage Layers are used to progressively extract and integrate features. Each Stage contains a convolutional module and a CSP layer. The CSP layer is used for feature integration and reducing computational cost. It includes two convolutional modules, a residual connection, and two Darknet Bottleneck modules. Each convolutional module includes a convolutional layer followed by a batch normalization layer and a SiLU activation function.

[0052] The DarknetBottleneck module consists of two convolutional layers for efficient extraction of complex features. First, there is a 1×1 convolutional layer to reduce the number of channels and improve computational efficiency. Then, there is a 3×3 convolutional layer to extract local features. Finally, the input and output are connected through residual connections to enhance gradient propagation.

[0053] The SPPF layer is used for multi-scale feature integration, which enhances the model's ability to detect targets at different scales. It includes three pooling operations with pooling kernel sizes of 5, 9 and 13, which connect pooling features at different scales to form richer feature representations.

[0054] The Neck section includes TopDown and BottomUp paths, used for top-down feature fusion and bottom-up feature extraction, respectively. The TopDown path first performs an upsampling operation, upsampling the deeper feature maps to match their size with the shallower feature maps, and then connects the downsampled feature maps with the corresponding deeper feature maps through a concatenation operation. The BottomUp path, on the other hand, uses a downsampling operation to fuse the shallower feature maps with the deeper feature maps.

[0055] The Head section includes basic convolutional layers and convolutional layers dedicated to bounding box regression and classification; the basic convolutional layers are used to initially process the input feature maps and extract more compact and useful features.

[0056] A soft nonmaximum suppression algorithm with Gaussian decay is used to decay the confidence of overlapping candidate boxes.

[0057] In some embodiments, training and jointly optimizing the bird sound denoising model and the target detection model based on the bird sound spectrogram dataset and the noisy bird sound dataset includes the following steps:

[0058] The bird sound denoising network and the object detection network were pre-trained based on the noisy bird sound dataset and the bird sound spectrogram dataset, respectively.

[0059] A joint optimization framework for bird sound denoising and detection is constructed, cascading the bird sound denoising network and the target detection network, specifically including:

[0060] Define a combined denoising-detection loss function, where the denoising model uses mean squared error loss, and the mean squared error loss is... MSE The calculation process is as follows:

[0061]

[0062] Where M represents the total number of samples, It is the value of the k-th sample predicted by the model, y k It is the true value of the k-th sample;

[0063] The noise reduction-detection combined loss function is defined as:

[0064] Loss total =μLoss yolo +γLoss MSE

[0065] Among them, Loss yolo =Loss EWA +CIoU+DFL, μ and γ are weighting parameters; Loss EWA Represents energy-weighted attention loss; CIoU represents complete intersection-union ratio loss; DFL represents distribution focus loss;

[0066] Freeze the initial causal convolutional layer and the first 9 expanded residual blocks of the bird sound denoising network, and freeze the Stem Layer start layer and Stage 1 convolutional module of the target detection network;

[0067] Set the number of epochs and the batch size to 50 and 32 respectively;

[0068] The Adam optimizer was used as the stochastic batch gradient descent optimizer, and the initial learning rate was set to lnitial_lr of 0.0001.

[0069] During training, model training is completed when the loss value on the validation set does not decrease after 10 epochs.

[0070] In some embodiments, the image-audio mapping module is used to convert the image detection results of the target detection model into sound event detection results for audio, including the following steps:

[0071] Let the sampling rate of a bird sound audio signal be Sr, the total duration of the audio be Dur, and the width and height of the corresponding spectrogram image be width and height, respectively. Based on the duration and sampling rate of the audio signal, the total number of sampling points L can be calculated:

[0072] L = Dur * Sr

[0073] Where L represents the total number of sampling points of the audio signal, Sr of the audio signal refers to the number of samples collected per second, which is used for the quantization of digital audio, and Dur of the audio signal is the total duration of the audio signal.

[0074] The number of sampling points S corresponding to each pixel on the time axis pixel for:

[0075]

[0076] Where width represents the width of the image;

[0077] Temporal resolution represents the duration of time corresponding to each pixel on the time axis, and it is calculated as follows:

[0078]

[0079] Among them, t pixel This represents the time length corresponding to each pixel on the time axis;

[0080] Frequency resolution represents the frequency bandwidth corresponding to each pixel on the frequency axis, which can be obtained from the Nyquist sampling theorem. pixel :

[0081]

[0082] Among them, f pixel This represents the frequency bandwidth corresponding to each pixel on the frequency axis;

[0083] Based on the input image pixel coordinate range (x min ,x max This allows us to calculate the corresponding audio signal time range:

[0084] t start =x min *t pixel

[0085] t end =x max *t pixel

[0086] Among them, t start and t end These represent the start and end points of time, respectively.

[0087] Based on the input image pixel coordinate range (y min ,y max ), calculate the corresponding audio signal frequency range:

[0088] f low =(height-y max )*f pixel

[0089] f high =(height-y min )*f pixel

[0090] Among them, f low and f high These represent the lower and upper limits of the frequency, respectively; image pixel coordinates (y min ,y max () indicates the frequency range selected on the image, which is converted into the actual frequency range by the frequency resolution.

[0091] In some embodiments, acquiring the bird sound data to be detected, retrieving and locating the target bird sound based on the bird sound denoising model, the target detection model, and the image-audio mapping module, and determining the species, time-frequency, and direction information of the target bird sound, includes the following steps:

[0092] The multi-channel bird sound signals collected on-site were first converted into single-channel signals, then sliced ​​into fixed-length segments and converted into spectrogram images. During slicing, the relative time position information of each audio segment relative to the original long audio was recorded. A trained target detection model was then used for detection, yielding corresponding results including bird category and the image pixel coordinates of the rectangular detection boxes. When the model detected a shape containing the target bird's sound in the image, the image pixel coordinate range of each rectangular detection box was converted into a corresponding time and frequency range. The start and end time periods, maximum and minimum frequency ranges, and dominant frequency of the target bird's sound were determined and output as results. The calculation process for the dominant frequency is as follows: first, the rectangular detection box B... i Maximum internal energy value E max,i Then calculate the maximum energy value E. max,i coordinates (x) e ,y e ), the coordinates of the maximum energy value (x e ,y e Convert the data into corresponding time points and frequency points, and determine the frequency point as the main frequency.

[0093] Based on the relative time position information of each audio segment relative to the original long audio, the start and end time periods of the target bird sound relative to the audio segment are converted into the start and end time periods of the original long audio, thereby completing the target bird sound retrieval of the long recording file.

[0094] Each channel of the four-channel bird sound signal collected on-site was converted into a corresponding spectrogram. A trained model was then used to detect the bird sound signal in each channel, obtaining the start and end times t of the target bird's sound in each channel. start and t end and the maximum and minimum frequencies f low and f high The start and end times and maximum and minimum frequencies of each channel are compared. The minimum start time, maximum end time, minimum minimum frequency, and maximum maximum frequency of all channels are taken as the new [t]. start , t end f low f high For each channel of the four-channel bird sound signal, based on the start and end times t start and t end The sample is truncated and then filtered using a linear-phase FIR bandpass filter. The bandpass filter's bandwidth is set to the minimum and maximum frequencies f. low and f high The frequency range between;

[0095] The time difference of arrival was calculated using a generalized cross-correlation algorithm on the target bird data of each channel after truncation and bandpass filtering;

[0096] After cross-correlation of the signals from each channel of target bird data that have undergone truncation and bandpass filtering, three time delay estimates D are obtained. 12 D 13 D 14 Then perform azimuth angle The calculation formula is as follows:

[0097] Another aspect of this invention provides a bird sound event detection system for monitoring nationally protected birds, comprising:

[0098] The first module is used to collect bird sound data and environmental noise data of nationally protected birds;

[0099] The second module is used to construct a bird sound spectrogram dataset and a noisy bird sound dataset based on the bird sound data and the environmental noise data.

[0100] The third module is used to build bird sound noise reduction models and target detection models;

[0101] The fourth module is used to train and jointly optimize the bird sound denoising model and the target detection model based on the bird sound spectrogram dataset and the noisy bird sound dataset; wherein, the bird sound denoising model is used to denoise the noisy bird sound data, and the target detection model is used to detect target images;

[0102] The fifth module is used to construct an image-audio mapping module, which converts the image detection results of the target detection model into audio event detection results.

[0103] The sixth module is used to acquire bird sound data to be detected, and to retrieve and locate the target bird sound based on the bird sound noise reduction model, the target detection model and the image-audio mapping module, and to determine the species, time frequency and direction information of the target bird sound.

[0104] To achieve the above objectives, another aspect of the present invention provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0105] To achieve the above objectives, another aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0106] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0107] The embodiments of this invention include at least the following beneficial effects: This invention provides a method and system for detecting bird sounds in monitoring nationally protected birds. The method collects bird sound data of nationally protected birds and environmental noise data; constructs a bird sound spectrogram dataset and a noisy bird sound dataset based on the bird sound data and the environmental noise data; constructs a bird sound denoising model and a target detection model; trains and jointly optimizes the bird sound denoising model and the target detection model based on the bird sound spectrogram dataset and the noisy bird sound dataset; wherein, the bird sound denoising model is used to denoise the noisy bird sound data, and the target detection model is used to detect target images; an image-audio mapping module is constructed to convert the image detection results of the target detection model into audio sound event detection results; acquires the bird sound data to be detected, and performs target bird sound retrieval and sound source localization based on the bird sound denoising model, the target detection model, and the image-audio mapping module to determine the species, time frequency, and direction information of the target bird sound. This invention can improve the detection effect of wild bird sound audio, thereby improving classification accuracy. Attached Figure Description

[0108] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present invention;

[0109] Figure 2 This is a flowchart of the overall steps provided in the embodiments of the present invention;

[0110] Figure 3 This is a diagram of the bird noise reduction network structure provided in an embodiment of the present invention;

[0111] Figure 4 This is a diagram of the Backbone and CSP layer structure provided in an embodiment of the present invention;

[0112] Figure 5 This is a flowchart of the target bird sound retrieval process provided in an embodiment of the present invention;

[0113] Figure 6 This is a flowchart illustrating the application of detection results provided in this embodiment of the invention to sound source localization.

[0114] Figure 7 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0115] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0116] It is understood that the terms “first,” “second,” etc., used in this invention may be used herein to describe various concepts, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another. For example, first information may also be referred to as second information without departing from the scope of embodiments of the invention, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to determination” as used herein may be interpreted as “when…” or “when…” or “in response to determination.”

[0117] The terms “at least one,” “multiple,” “each,” “any,” etc., used in this invention, “at least one” includes one, two, or more than two; “multiple” includes two or more than two; “each” refers to each of the corresponding multiple; and “any” refers to any one of the multiple.

[0118] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0119] To overcome the shortcomings of existing bird call event detection technologies for monitoring nationally protected birds, this invention provides a target detection-based method for bird call event detection in monitoring nationally protected birds. This method converts the image detection results of the target detection algorithm into the start and end times, maximum and minimum frequency ranges, and dominant frequencies of sound events within the audio signal. This method demonstrates excellent detection performance even for bird call audio with low signal-to-noise ratios in actual field recordings, maintaining both high recall and precision, thus preventing missed detections of rare birds. This method simultaneously focuses on detecting the temporal location and frequency range information of bird call events. The detected frequency range information dynamically changes according to the different call patterns of different birds, improving the classification accuracy of different bird sounds. These detection results can be effectively applied to the retrieval of target birds in long-term, large-volume actual field recordings. Furthermore, these detection results also include the horizontal angle information for sound source localization.

[0120] The bird sound event detection method and system for monitoring nationally protected birds provided in this invention relates to the field of acoustic information processing technology. The bird sound event detection method for monitoring nationally protected birds provided in this invention can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the bird sound event detection method for monitoring nationally protected birds, but is not limited to the above forms.

[0121] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0122] like Figure 1 The diagram shown is a schematic representation of an implementation environment provided by an embodiment of the present invention. (Refer to...) Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected via a network, either wirelessly or via a wired connection, to complete data transmission and exchange.

[0123] Server 101 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0124] Additionally, server 101 can also be a node server in a blockchain network. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0125] Terminal 102 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc. It can also be a vehicle-mounted terminal of the various device types described above, but is not limited to these. Terminal 102 and server 101 can be directly or indirectly connected via wired or wireless communication, and this embodiment of the invention does not impose any limitations.

[0126] Exemplary based on Figure 1The implementation environment shown in this embodiment of the invention provides a bird sound event detection method for monitoring nationally protected birds. The following description uses the application of this bird sound event detection method for monitoring nationally protected birds in server 101 as an example. It can be understood that this method can also be applied to terminal 102.

[0127] Reference Figure 2 , Figure 2 This is a flowchart illustrating a bird sound event detection method for monitoring nationally protected birds, applied to a server, provided in an embodiment of the present invention. The execution subject of this method can be any of the aforementioned computer devices (including servers or terminals). (Refer to...) Figure 2 The method may include the following steps:

[0128] S1. Collect bird call data and environmental noise data for nationally protected birds;

[0129] S2. Construct a bird sound spectrogram dataset and a noisy bird sound dataset;

[0130] S3. Construct a bird sound noise reduction model;

[0131] S4. Construct a target detection model;

[0132] S5. Train and jointly optimize the bird sound noise reduction and target detection models, and set the optimizer, learning rate, and learning rate strategy.

[0133] S6. Construct the image-audio mapping module;

[0134] S7. Target bird sound retrieval and sound source localization.

[0135] 1. The specific details of step S1 are as follows:

[0136] (1) Collect sound data of these birds based on the national list of Class I and Class II protected birds. This data can be downloaded from bird sound data websites (such as xeno-canto) or actual field recordings.

[0137] (2) Birdsong data requirements:

[0138] The sampling rate must be no less than 32kHz and the quantization precision no less than 16 bits. There are no geographical restrictions on the bird species included in the data.

[0139] (3) Assume the total number of bird call categories is V, and the total duration of bird call data for each category is T. v Given v = 1, 2, ..., V, find the total duration T of each category of data in the bird calls dataset. v The duration must be no less than 600 seconds. Each category can contain multiple audio files, and each file must be at least 10 seconds long.

[0140] (4) Standardize the format of the collected data: audio format: wav, sampling frequency: 32kHz, quantization precision: 16 bits, number of audio channels: mono. Use audio processing software (such as Adobe Audition) to perform noise reduction processing to obtain clean bird sound data, and then manually check to ensure that the signal-to-noise ratio of the clean bird sound data is not less than 30dB.

[0141] (5) Connect the audio files selected in the previous step according to bird species, with each species corresponding to a long audio file. Slice the audio file, with each slice lasting between 5 and 15 seconds. The bird song structure of each slice should be complete, and any bird song segments that are cut off during slicing should be discarded.

[0142] (6) Acquiring environmental noise data: The minimum sampling rate for noise data is 32kHz, the minimum quantization accuracy is 16 bits, and the noise type is common noise in the actual monitoring area of ​​key national birds (such as wind, car noise, rain, stream noise, human voice, cricket chirping, grasshopper chirping, cicada chirping, frog croaking). The total duration of each category of noise data is required to be no less than 1200 seconds. Each category of data can include several audio files, and the duration of each audio file in each category is no less than 120 seconds.

[0143] 2. The specific details of step S2 are as follows:

[0144] (1) Constructing a bird sound spectrogram dataset: First, the clean bird sound data obtained in S1 is converted from the time domain to the frequency domain. The specific calculation process is as follows:

[0145] 1) Let P be the number of data sampling points of a bird sound signal. Set the frame length flen to 2048 and the frame shift hlen to 512. Then, get the number of frames fn of the bird sound signal. Let each frame signal after segmentation be x(p,λ), where p is the sampling point number and λ is the frame number.

[0146]

[0147] 2) Overlay the current frame x(p,λ) with the previous frame x(p,λ-1) for a length of overlap = 512 to obtain x. o (p,λ), the total length after frame stacking is still flen, p=0,1,…,flen-1.

[0148]

[0149] 3) For x o Windowing is applied to (p,λ) using a Hamming window w(p,ψ), where ψ is 0.46 and the window length is equal to the frame length points flen = 2048. This yields all windowed frames x.w (p,λ).

[0150]

[0151] x w (p,λ)=x0(p,λ)*w(p,ψ),0≤p≤flen-1#(4)

[0152] 4) For each frame x w Perform a P=2048-point DFT operation on (p,λ), and take the modulus of the DFT result to obtain the amplitude spectrum X(f,λ) of each frame signal, where f represents the frequency point and λ is the frame number. Due to the symmetry of the Fourier transform, only the first P points of the spectrum are processed. f Analyze each frequency point, among which

[0153]

[0154] (2) Convert the generated bird sound spectrogram to a standard image format and adjust the image size and color depth to suit subsequent processing. This includes the following steps:

[0155] 1) The calculation parameters for generating the spectrogram were standardized: STFT window size: 2048, frame shift: 512, image resolution DPI: 300.

[0156] 2) The image size should be no less than 640*640 to avoid excessive loss of image details, but also not too large to avoid slowing down the training speed. The vertical axis scale of the spectrogram should be set to a linear scale, and the color mapping should be set to 'inferno'. 'inferno' is a color mapping table that typically transitions from black (low intensity) to yellow, red, and then to white (high intensity), making the details very clear. By setting this mapping table, different intensities of frequency components in the bird spectrogram are mapped to specific colors, making the image more intuitive and easier to analyze.

[0157] 3) Use the image annotation tool LabelImg to draw bounding boxes for each target bird sound on the bird sound spectrogram and label the category (labeled as bird name). Save the annotation file in yolo_txt format, that is, each image corresponds to a txt file, and each line of the file contains information about one target, including class, x_center, y_center, width, and height.

[0158] 4) The above bird sound spectrograms and corresponding annotation files are used to form a bird sound spectrogram dataset. The dataset is divided into a training set and a validation set with a ratio of 7:3 using a stratified sampling strategy.

[0159] 5) Add the noisy data to the clean bird sound signal obtained in S1 to obtain the noisy bird sound dataset. The noise addition process is as follows:

[0160] 1) Assuming a bird sound signal has a length of z seconds, and considering a sampling rate of 32kHz, the required number of sampling points is z*32000. Select a noise signal. If the number of sampling points of the selected noise signal is greater than z*32000 (i.e., the noise duration exceeds z seconds), randomly select z*32000 consecutive sampling points from it.

[0161] 2) After obtaining clean bird call signals and noise signals of equal length, add the noise to the clean bird call signal. Choose one type of noise each time to ensure that the proportion of each type of noisy bird call signal is even in the overall noisy bird call signal, and that the number of noisy bird call signals of different noise types is the same. Mix the signals according to the following formula:

[0162]

[0163] Where t represents the signal time, z(t) is the noisy bird sound signal, z1(t) and n(t) are the clean bird sound signal and the noise signal, respectively, ξ is the gain coefficient of n(t) during the mixing process, and snr represents the signal-to-noise ratio (SNR) of the clean bird sound signal and the noise signal, measured in decibels (dB). When synthesizing the noisy bird sound signal for the training set, snr is selected from the range {-15, -10, -7.5, -5, -2.5, 0, 2.5, 5, 7.5, 10, 12.5, 15}. When synthesizing the noisy bird sound signal for the validation set, snr is {-15, -10, -5, 0, 5, 10, 15}. The number of noisy bird sound signals with each SNR is equal.

[0164] (6) The above noisy bird sound data and the corresponding clean bird sound data are used to form a noisy bird sound dataset, in which the clean bird sound data is used as the label, and the dataset is divided into a training set and a validation set with a ratio of 7:3 using a stratified sampling strategy.

[0165] 3. The specific details of step S3 are as follows:

[0166] (1) As Figure 3 As shown, the bird sound denoising network consists of a data input layer, an initial causal convolutional layer, a dilated residual block, a skip connection aggregation layer, and a final output layer. First, the input shape of the data input layer is (IN, 1), where IN represents the length of the input signal, used to receive noisy bird sound signals. The initial causal convolutional layer uses causal convolutional kernels to process the input signal, ensuring that each output depends only on the current and previous input values. It has 128 kernels of size 3, and its process can be represented as follows:

[0167] h causal =σ(Wcausal *x+b causal )#(7)

[0168] Among them, W causal and b causal These represent the causal convolution kernel weights and biases, h, respectively. causal σ is the output of the causal convolution, and σ is the ReLU activation function.

[0169] The dilated residual blocks are the core of the denoising network, consisting of 27 blocks. Each block includes a dilated convolutional layer, an activation function, a gating mechanism, a 1x1 convolutional layer, skip connections, and residual connections. The dilated convolutional layers have 256 kernels, a kernel size of 3, and dilation rates increasing exponentially from 1 to 512 (1, 2, 4, 8, 16, 32, 64, 128, 256, 512), with 10 dilation rates and a stack size of 3. The activation functions include the hyperbolic tangent (tanh) function and the sigmoid function. The hyperbolic tangent (tanh) function is applied to the first part of the output of the dilated convolutional layer, generating h. tanh The sigmoid function is applied to the second part of the output of the dilated convolutional layer to generate h. sigma The gating mechanism uses element-wise multiplication to control h. tanh and h sigma Combine to generate h gate Then, h gate h is generated through a 1x1 convolutional layer. 1x1 The main function of the 1x1 convolutional layer is to adjust the dimension of the feature map through linear transformation and generate residual output and skip connection output. It has 256 convolutional kernels, with 128 residual channels and 128 skip connections. The skip connection part starts from h... 1x1 The skip connection output is extracted and used for summarization in subsequent layers. The residual connection part adds the residual output of the 1x1 convolutional layer to the input signal to generate the output of the dilated residual block.

[0170] (2) Skip Connection Summarization: A comprehensive feature representation is generated by summing the skip connection outputs of all dilated residual blocks. This comprehensive feature representation contains features from all layers, providing rich feature representations for the final output layer through feature information at different scales and dilation rates. The summation formula is as follows:

[0171]

[0172] Among them, s d This represents the jump connection output of the d-th expanded residual block, where D is the total number of expanded residual blocks.

[0173] The final convolutional output layer consists of three convolutional layers. The first convolutional layer has 2048 kernels and a kernel size of 3, used to extract deeper features from the comprehensive feature representation of skip connections. The second convolutional layer has 256 kernels and a kernel size of 3, further compressing the feature map to a smaller dimension. The third convolutional layer is a 1x1 convolutional layer with a kernel size of 1, used to perform a linear transformation on the feature map, generating the final denoised bird sound signal.

[0174] 4. The specific details of step S4 are as follows:

[0175] (1), reference Figure 4 Build the Backbone network, such as Figure 4 As shown in (a), the backbone consists of a StemLayer initiation layer, convolutional modules from Stage 1 to Stage 4, a CSP layer, and an SPPF layer. The StemLayer initiation layer is composed of a convolutional module (ConvModule). This module includes a two-dimensional convolutional layer (Conv2d) with a kernel size of 3x3, a stride of 2, and padding of 1. The convolution operation is followed by batch normalization (BatchNorm) and the SiLU activation function. This layer is mainly used for the initial extraction of low-level features of the image. Its operation process can be represented by the following formula:

[0176] y=SiLU(BatchNorm(Conv2d(x,W,b)))#(9)

[0177] Where x is the input image, and W and b are the convolution kernel weights and biases, respectively.

[0178] The four StageLayer layers are a key part of Backbone for progressively extracting and integrating features. Each Stage contains a Convolutional Module (ConvModule) and a CSP layer.

[0179] like Figure 4 As shown in (b), the CSP layer is used for feature integration and computational reduction, and includes two convolutional modules (ConvModules), one residual connection, and two Darknet Bottleneck modules. Each convolutional module includes a convolutional layer with a kernel size of 3×3 and a stride of 1, followed by a batch normalization layer (BatchNorm) and the SiLU activation function.

[0180] The DarknetBottleneck module consists of two convolutional layers for more efficient extraction of complex features. First, there is a 1×1 convolutional layer to reduce the number of channels and improve computational efficiency, followed by a 3×3 convolutional layer to extract local features. Finally, the input and output are connected through residual connections to enhance gradient propagation.

[0181] The SPPF layer is used for multi-scale feature integration, enhancing the model's ability to detect targets at different scales. It includes three pooling operations with kernel sizes of 5, 9, and 13, which connect pooled features at different scales to form richer feature representations.

[0182] (2) The Neck section includes TopDown and BottomUp paths, used for top-down feature fusion and bottom-up feature extraction, respectively. The TopDown path first performs an upsampling operation, upsampling the deeper feature maps to match their size with the shallower ones. Then, it connects the downsampled feature maps with the corresponding deeper feature maps using a concatenation operation. The BottomUp path fuses the shallower and deeper feature maps using a downsampling operation. The concatenation operation stitches the feature maps along the channel dimension, forming a new feature map containing multi-scale information. Next, a CSP layer (CSPLayer) is used to process the concatenated feature maps. The CSP layer (CSPLayer) contains two convolutional modules, each including a convolutional layer (3×3 kernel size, stride 1), a batch normalization layer (BatchNorm2d), and a SiLU activation function. The TopDown and BottomUp paths can be represented as follows:

[0183] TopDown(x1,x2)=CSPLayer(Concat(Upsample(x1),x2))#(10)

[0184] BottomUp(x2,x1)=CSPLayer(Concat(DownSample(x2),x1))#(11)

[0185] Here, x1 is a deeper feature map, that is, a feature map from a later layer in the backbone, which usually has a smaller spatial resolution and higher semantic information, while x2 is a shallower feature map, that is, a feature map from a earlier layer in the backbone, which usually has a larger spatial resolution and lower semantic information.

[0186] (3) The Head section includes a basic convolutional layer (ConvModule) and a dedicated convolutional layer (Conv2d) for bounding box regression and classification. The basic convolutional layer (ConvModule) is used for preliminary processing of the input feature map to extract more compact and useful features. This layer includes a convolution operation, batch normalization (BatchNorm), and an activation function (SiLU). The kernel size of the convolution operation is set to 3×3, and the stride is set to 1. The dedicated convolutional layer (Conv2d) for bounding box regression and classification is responsible for the final detection output. This layer includes two different branches, one for bounding box regression and the other for object classification. The kernel size of the convolution operation in each branch is set to 1×1, and the stride is set to 1. Through the processing of this layer, the bounding box regression branch outputs the predicted bounding box position and size, while the classification branch outputs the predicted object category.

[0187] During training, the bounding box regression loss and classification loss need to be calculated separately. The Energy Weighted Attention Loss (EWA Loss) function is defined. First, the relevant parameters of each detection box i need to be extracted, including the center coordinates, width, and height of the bounding box, as well as the confidence score C. i and category prediction score P i In addition, it is also necessary to calculate the average energy E within the detection frame. i The calculation process for the mean energy is as follows:

[0188]

[0189] Among them, |B i | Represents detection box B i The number of pixels within the spectrum, S(x,y) is the value of the spectrogram at position (x,y).

[0190] Confidence level C i Energy mean E i and category prediction score P i The input feature vector is concatenated and used as input to a weighted network. The weighted network first performs a non-linear transformation on the input feature vector through a fully connected layer to obtain an intermediate feature vector. This non-linear transformation is applied to the output of the fully connected layer using a Corrected Linear Unit (ReLU) activation function. In the second fully connected layer, the network calculates attention weights for each input feature. These weights are normalized using a softmax function to ensure their sum is 1. Next, the attention weights are applied to the original input features, and the features are weighted using element-wise multiplication to obtain a weighted feature vector. Subsequently, the weighted confidence C is calculated through the third and fourth fully connected layers. iw and category prediction score P iwThe output of the third fully connected layer is mapped to a confidence range using a sigmoid activation function, and the output of the fourth fully connected layer is mapped to a class prediction score range using the same activation function. The calculation process of the energy-weighted attention loss function (EWA Loss) can be expressed as follows:

[0191]

[0192] Loss EWA =Loss C +Loss p #(15)

[0193] Where N represents the total number of detection boxes, and C represents the total number of categories. Let be the true label of the j-th category of the i-th detection box.

[0194] The bounding box regression loss is defined as the Complete Intersection over Union (CioU) Loss and the Distribution Focus Loss (DFLLoss). The calculation process of CiU Loss is as follows:

[0195]

[0196] Where IoU is the Intersection over Union ratio, A is the intersection area of ​​the predicted bounding box and the ground truth bounding box, and B is their union area. d is the Euclidean distance between the center points of the predicted and ground truth bounding boxes, c is the diagonal length of the smallest closure region containing both the predicted and ground truth bounding boxes, α is a coefficient measuring the consistency of the aspect ratios of the bounding boxes, v is a measure of the difference in aspect ratios between the predicted and ground truth bounding boxes, and W... G and H G These are the width and height of the actual bounding box, W. P and H P These are the predicted width and height of the bounding box, respectively.

[0197] The calculation process for DFL Loss is as follows:

[0198] DFL(s a ,s a+1 )=-((y a+1 -y)log(s a )+(yy a log(s) a+1 ))#(20)

[0199] Among them, s a and s a+1 Here, y is the predicted probability distribution value, and y is the target value. a It is the left-hand integer part of the target value, y a+1 It is the integer part to the right of the target value.

[0200] (5) Using the Gaussian decay soft nonmaximum suppression algorithm (Soft-NMS), more detection information is retained by decaying the confidence of overlapping candidate boxes, which is beneficial to improving the model's recall while maintaining high precision, thereby improving the overall detection performance.

[0201]

[0202] Among them, S j ′ is candidate box B j The updated confidence score, S j It is candidate box B j The original confidence score, IoU(B) i B j ) represents candidate box B i and B j The crossover ratio between them, g is a parameter that controls the decay rate, and its value is set to 0.5.

[0203] 5. The specific details of step S5 are as follows:

[0204] (1) The noisy bird sound dataset and bird sound spectrogram dataset constructed in S2 are used to pre-train the bird sound denoising network and the object detection network respectively. During the pre-training process, an early stopping strategy is used. The model training is completed when the loss value of the validation set does not decrease after 10 epochs.

[0205] (2) Construct a joint optimization framework for bird sound denoising and detection, cascading the bird sound denoising network and the target detection network. Specifically, the output of the bird sound denoising network serves as the input to the target detection network, where the denoising network outputs a processed, clean bird sound signal. This signal is converted into a corresponding spectrogram before being input into the target detection network, and the conversion process is consistent with the process of constructing the bird sound spectrogram dataset in S2. The time-domain to frequency-domain parameter settings and the size of the generated spectrogram are also consistent with those in S2, so the labels of the bird sound spectrogram dataset in S2 can be shared. Then, the generated spectrogram is input into the target detection network for training. The specific training process is as follows:

[0206] 1) Define the combined denoising-detection loss function, where the denoising model uses mean squared error (MSE) loss. The calculation process for MSE loss is as follows:

[0207]

[0208] Where M represents the total number of samples, It is the value of the k-th sample predicted by the model, y k It is the true value of the k-th sample.

[0209] The noise reduction-detection combined loss function is defined as follows:

[0210] Loss total =μLoss yolo +γLoss MSE #(twenty three)

[0211] Among them, Loss yolo =Loss EWA +CIoU+DFL, where μ and γ are weight parameters. The initial values ​​of μ and γ are set to 1, and are respectively set to L after 10 training iterations. yolo and L MSE The reciprocal of the relative value.

[0212] 2) Freeze the initial causal convolutional layer and the first 9 dilated residual blocks of the bird sound denoising network, and freeze the Stem Layer start layer and Stage 1 convolutional module of the object detection network. These frozen layers have learned stable low-level features during the pre-training stage, and keeping them unchanged will help to focus training resources on optimizing higher-level features.

[0213] 3) Set the number of epochs and the batch size to 50 and 32 respectively.

[0214] 4) Optimizer selection: The Adam optimizer is used as the stochastic batch gradient descent optimizer, and the initial learning rate is set to lnitial_lr of 0.0001.

[0215] 5) During training, model training is completed when the loss value on the validation set does not decrease after 10 epochs.

[0216] 6. The specific details of step S6 are as follows:

[0217] (1) The sound event detection task is to identify which sound events occur and when in an audio segment. It requires not only the start and end times of the events but also the classification of the detected events. By constructing an image-audio mapping module, the target detection results of the image can be converted into sound event detection results of the audio. First, it is necessary to obtain the parameters of the audio signal and its corresponding spectrogram image. The specific process is as follows: Let the sampling rate of a bird sound audio signal be Sr, the total duration of the audio be Dur, and the width and height of the corresponding spectrogram image be width and height, respectively. Based on the duration and sampling rate of the audio signal, the total number of sampling points can be calculated:

[0218] L=Dur*Sr#(24)

[0219] Where L represents the total number of sampling points in the audio signal. The sampling rate Sr of the audio signal refers to the number of samples collected per second, used for quantization of digital audio. In digital audio processing, a higher sampling rate captures more sound details, but also increases the amount of data. The audio duration Dur is the total duration of the audio signal, in seconds. The width and height of the image determine the resolution of the spectrogram used to represent the audio signal.

[0220] The number of sampling points corresponding to each pixel on the time axis is:

[0221]

[0222] Here, S pixel This represents the number of sampling points corresponding to each pixel on the time axis, determining the resolution on the time axis. The larger the image width, the shorter the time corresponding to each pixel, and the higher the time resolution.

[0223] (2) Calculation of time and frequency resolution

[0224] Temporal resolution represents the duration of time corresponding to each pixel on the time axis, and it is calculated as follows:

[0225]

[0226] Among them, t pixel This represents the duration of each pixel on the timeline, measured in seconds. It determines the level of detail in the image over time. By combining the sampling rate and image width, it is possible to precisely locate the temporal details of the audio signal.

[0227] Frequency resolution represents the frequency bandwidth corresponding to each pixel on the frequency axis, which can be obtained from the Nyquist sampling theorem:

[0228]

[0229] Among them, f pixel This represents the frequency bandwidth corresponding to each pixel on the frequency axis, measured in Hertz (Hz). The Nyquist sampling theorem states that the sampling rate should be at least twice the highest frequency of the signal to avoid aliasing. By using the resolution of the image height, the frequency components of the audio signal can be accurately represented.

[0230] (3) Converting image pixel coordinates to time range:

[0231] Based on the input image pixel coordinate range (x min ,x max This allows us to calculate the corresponding audio signal time range:

[0232] t start=x min *t pixel #(28)

[0233] t end =x max *t pixel #(29)

[0234] Among them, t start and t end These represent the start and end points of the time, respectively. By combining image pixel coordinates with temporal resolution, these values ​​can precisely pinpoint the specific range of the audio signal on the time axis. Image pixel coordinates (x... min ,x max This represents the selected time interval on the image, which, combined with the time length corresponding to each pixel, is converted into the actual audio signal time range.

[0235] (4) Converting image pixel coordinates to frequency range

[0236] Based on the input image pixel coordinate range (y min ,y max This allows us to calculate the corresponding audio signal frequency range.

[0237] f low =(height-y max )*f pixel #(30)

[0238] f high =(height-y min )*f pixel #(31)

[0239] Among them, f low and f high These represent the lower and upper limits of the frequency, respectively. Image pixel coordinates (y min ,y max () indicates the frequency range selected on the image, which is converted into the actual frequency range by the frequency resolution.

[0240] 7. The specific details of step S7 are as follows:

[0241] (1)Reference Figure 5 The four-channel bird sound signals collected on-site were first converted into single-channel signals, then sliced ​​into fixed-length segments and converted into spectrogram images. During slicing, the relative time position information of each audio segment relative to the original long audio was recorded. Then, the target detection model trained in steps S3, S4, and S5 was used for detection, yielding corresponding detection results. The detection results include the bird category, the image pixel coordinates (x, y) of the rectangular detection box, and the corresponding bird type. min ,y min ,xmax ,y max Information such as [list of information] is used. When the model detects the shape of a target bird's sound in an image, it will frame the shape of the target bird's sound with a rectangle. The size of the rectangle will dynamically change according to the size of the target bird's sound shape. Based on the above steps, the image pixel coordinate range of each rectangular detection box is converted into the corresponding time and frequency range. That is, the start and end time period, the maximum and minimum frequency range, and the dominant frequency of the target bird's sound are calculated and output as results. The calculation process of the dominant frequency is as follows:

[0242] 1) Calculate the rectangular detection box B i Maximum internal energy value E max,i

[0243]

[0244] Where S(x,y) is the value of the spectrogram at position (x,y).

[0245] 2) Then calculate the maximum energy value E. max,i coordinates (x) e ,y e Based on the above steps, the coordinates of the maximum energy value (x) are determined. e ,y e The audio is converted into corresponding time and frequency points, with the frequency point being the main frequency. Then, based on the relative time position information of each audio segment with respect to the original long audio, the start and end time periods of the target bird sound relative to the audio segment are converted into the start and end time periods of the original long audio, thereby achieving the purpose of retrieving target bird sounds from long audio files.

[0246] (2)Reference Figure 6 The four-channel bird sound signals (channels 1, 2, 3, and 4) collected on-site were converted into corresponding spectrograms for each channel. Then, the trained model was used to detect the bird sound signals in each channel, obtaining the start and end times t of the target bird sound in each channel. start and t end and the maximum and minimum frequencies f low and f high Compare the start and end times and maximum and minimum frequencies of each channel, and take the minimum start time, maximum end time, minimum minimum frequency, and maximum maximum frequency of all channels as the new [t]. start , t end f low f high Based on the frequency characteristics of the target bird sounds, a linear-phase FIR bandpass filter is used to enhance the target signal and suppress non-target signals. Specifically, for each channel of the four-channel bird sound signal, a start-end time t is applied.start and t end The sample is truncated and then filtered using a linear-phase FIR bandpass filter. The bandpass filter's bandwidth is set to the minimum and maximum frequencies f. low and f high The frequency range between.

[0247] (3) The time difference of arrival (TDOA) is calculated using the generalized cross-correlation algorithm (GCC) on the target bird data of each channel after truncation and bandpass filtering.

[0248] (4) In summary, after cross-correlation of the signals from the target bird data of each channel after truncation and bandpass filtering, three time delay estimates are obtained, which are D. 12 D 13 D 14 Then perform azimuth angle The calculation is as follows:

[0249]

[0250] This invention successfully detected the start and end times, maximum and minimum frequency ranges, and dominant frequencies of various bird calls in a multi-category mixed national key protected bird call data test set. It can effectively retrieve the sounds of target birds, achieving high-efficiency detection even in low signal-to-noise ratio environments. Furthermore, the average error of the target bird's azimuth angle calculated from the four-channel recording signals collected on-site is significantly reduced. Details are shown in Table 1 below:

[0251] Table 1. Azimuth localization results of the sound source signal when R=50m

[0252]

[0253] As shown in Table 1, when the sound source is 50m away, the average error of the azimuth angle estimation for the song of the fork-tailed sunbird is 0.78°.

[0254] In summary, compared with the prior art, the present invention has the following advantages:

[0255] 1. Accurately monitor the issue of nationally protected birds;

[0256] 2. This method addresses the problems of traditional bird call detection methods, such as lack of accurate time range information, frequency range information, dominant frequency information, poor detection performance in low signal-to-noise ratio environments, missed detections, and lack of directional information.

[0257] Another aspect of this invention provides a bird sound event detection system for monitoring nationally protected birds, comprising:

[0258] The first module is used to collect bird sound data and environmental noise data of nationally protected birds;

[0259] The second module is used to construct a bird sound spectrogram dataset and a noisy bird sound dataset based on the bird sound data and the environmental noise data.

[0260] The third module is used to build bird sound noise reduction models and target detection models;

[0261] The fourth module is used to train and jointly optimize the bird sound denoising model and the target detection model based on the bird sound spectrogram dataset and the noisy bird sound dataset; wherein, the bird sound denoising model is used to denoise the noisy bird sound data, and the target detection model is used to detect target images;

[0262] The fifth module is used to construct an image-audio mapping module, which converts the image detection results of the target detection model into audio event detection results.

[0263] The sixth module is used to acquire bird sound data to be detected, and to retrieve and locate the target bird sound based on the bird sound denoising model, the target detection model, and the image-audio mapping module, thereby determining the species, time frequency, and direction information of the target bird sound.

[0264] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0265] This invention also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned bird sound event detection method for monitoring nationally protected birds. This electronic device can be any smart terminal, including tablet computers and in-vehicle computers.

[0266] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0267] Please see Figure 7 , Figure 7 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0268] The processor 701 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0269] The memory 702 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 702 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 702 and is called and executed by the processor 701 to execute the bird sound event detection method for monitoring nationally protected birds according to the embodiments of this invention.

[0270] The input / output interface 703 is used to implement information input and output;

[0271] The communication interface 704 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0272] Bus 705 transmits information between various components of the device (e.g., processor 701, memory 702, input / output interface 703, and communication interface 704);

[0273] The processor 701, memory 702, input / output interface 703, and communication interface 704 are connected to each other within the device via bus 705.

[0274] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described bird sound event detection method for monitoring nationally protected birds.

[0275] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0276] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0277] It should be noted that in various specific embodiments of the present invention, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of the present invention require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the user's separate permission or consent is the necessary user-related data for the normal operation of the embodiments of the present invention acquired.

[0278] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0279] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0280] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0281] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0282] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0283] It should be understood that in this invention, "at least one (item)" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0284] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0285] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0286] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0287] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0288] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of the present invention should be within the scope of the claims of the present invention.

Claims

1. A method for detecting bird sound events for monitoring nationally protected birds, characterized in that, Includes the following steps: Collect bird call data and environmental noise data for nationally protected bird species; Based on the bird sound data and the environmental noise data, construct a bird sound spectrogram dataset and a noisy bird sound dataset; Construct a bird sound noise reduction model and a target detection model; Based on the bird sound spectrogram dataset and the noisy bird sound dataset, the bird sound denoising model and the target detection model are trained and jointly optimized; wherein, the bird sound denoising model is used to denoise the noisy bird sound data, and the target detection model is used to detect target images; An image-audio mapping module is constructed to convert the image detection results of the target detection model into audio event detection results. The bird sound data to be detected is acquired, and the target bird sound is retrieved and the sound source is located according to the bird sound noise reduction model, the target detection model and the image-audio mapping module to determine the species, time frequency and direction information of the target bird sound; The process of acquiring bird sound data to be detected, retrieving and locating the target bird sound based on the bird sound denoising model, the target detection model, and the image-audio mapping module, and determining the species, time-frequency, and direction information of the target bird sound, includes the following steps: The four-channel bird sound signals collected on-site were first converted into single-channel signals, then sliced ​​into fixed-length segments and converted into spectrogram images. During slicing, the relative time position information of each audio segment relative to the original long audio was recorded. A trained target detection model was then used for detection, yielding corresponding results including bird category and the image pixel coordinates of rectangular detection boxes. When the model detects a shape containing the target bird's sound in an image, it frames that shape with a rectangle. The size of the rectangle dynamically changes according to the size of the target bird's sound shape. The image pixel coordinate range of each rectangular detection box is converted into a corresponding time and frequency range. The start and end time periods, maximum and minimum frequency ranges, and dominant frequency of the target bird's sound are determined and output as results. The calculation process for the dominant frequency is as follows: first, the rectangular detection box... Maximum internal energy value Then calculate the maximum energy value. coordinates ( ), the coordinates of the maximum energy value ( Convert the data into corresponding time points and frequency points, and determine the frequency point as the main frequency. Based on the relative time position information of each audio segment relative to the original long audio, the start and end time periods of the target bird sound relative to the audio segment are converted into the start and end time periods of the original long audio, thereby completing the target bird sound retrieval of the long recording file. Each channel of the four-channel bird sound signal collected on-site was converted into a corresponding spectrogram. A trained model was then used to detect the bird sound signal in each channel, obtaining the start and end times of the target bird's sound in each channel. and and maximum and minimum frequencies and The start and end times and maximum and minimum frequencies of each channel are compared. The minimum start time, maximum end time, minimum minimum frequency, and maximum maximum frequency of all channels are taken as the new [value]. For each channel of the four-channel bird sound signal, based on the start and end times... and The sample is truncated and then filtered using a linear-phase FIR bandpass filter. The bandpass filter's bandwidth is set to the minimum and maximum frequencies. and The frequency range between; The time difference of arrival was calculated using a generalized cross-correlation algorithm on the target bird data of each channel after truncation and bandpass filtering; After cross-correlation of the signals from each channel of target bird data that have undergone truncation and bandpass filtering, three time delay estimates are obtained. , , Then perform azimuth angle The calculation formula is as follows: .

2. The bird sound event detection method for monitoring nationally protected birds according to claim 1, characterized in that, The collection of bird call data and environmental noise data for nationally protected birds includes the following steps: Based on the national list of Class I and Class II protected birds, sound data of these birds was collected. The collection methods included data downloaded from bird sound data websites or recordings made in the wild; the sampling rate was no less than 32kHz and the quantization precision was no less than 16 bits. The collected data is standardized and noise is reduced to obtain clean bird sound audio files after filtering. Based on the bird species, the selected audio files are concatenated to obtain the audio file for each bird species; Environmental noise data is acquired based on the noise data sampling rate requirements, quantification accuracy requirements, and noise types. Among them, the noise types include target noise in the actual monitoring area of ​​key national bird species, including: wind noise, car noise, rain noise, stream noise, human voices, cricket chirping, grasshopper chirping, cicada chirping, and frog croaking.

3. The bird sound event detection method for monitoring nationally protected birds according to claim 1, characterized in that, The process of constructing a bird sound spectrogram dataset and a noisy bird sound dataset based on the bird sound data and the environmental noise data includes the following steps: The acquired bird sound data is converted from the time domain to the frequency domain to generate a bird sound spectrogram; Adjust the size and color depth of the bird spectral image; Use image annotation tools to draw bounding boxes for each target bird sound on the bird sound map, label the category, and save the annotation file; The bird sound spectrogram dataset is constructed by combining bird sound spectrograms and corresponding annotation files. Noise data is added to the acquired clean bird sound data to obtain the noisy bird sound dataset.

4. The bird sound event detection method for monitoring nationally protected birds according to claim 3, characterized in that, The process of converting the acquired bird sound data from the time domain to the frequency domain and generating a bird sound spectrogram includes the following steps: Record the number of data sampling points for a bird sound signal. Set frame length 2048, frame shift The value is 512, which gives the number of frames in the bird sound signal. Each frame signal after the scoring frame is ,in Number the sampling points. The frame number is used to represent the number of frames in a bird sound signal; the formula for calculating the number of frames is: ; Current frame Compared to the previous frame Perform a length of =512 frames are stacked to obtain the target frame The total length after frame stacking is the frame length. ; For the target frame Add windows; the type of window is a Hamming window. ,in Take 0.46, the window length equals the frame length in pixels. =2048, obtain all windowed frames. ; For each frame conduct A 2048-point DFT operation is performed, and the magnitude of the DFT result is used to obtain the amplitude spectrum of each frame of the signal. , Indicates frequency point, Number the frames to complete the analysis of the spectrum. Analysis of individual frequency points generates bird sound spectrograms; Adjusting the size and color depth of the bird spectrogram image includes the following steps: The calculation parameters for generating the spectrogram are standardized, with the STFT window size being 2048, the frame shift being 512, and the image resolution DPI being 300. Set the vertical axis scale of the spectrogram to a linear scale and the color mapping to the 'inferno' mapping table; by setting the mapping table, frequency components of different intensities are mapped to corresponding colors in the bird spectrogram; The process of adding noise data to the acquired clean bird sound data to obtain the noisy bird sound dataset includes the following steps: Set the length of a bird sound signal to z seconds, the sampling rate to 32kHz, and the required number of sampling points to z*32000. When the number of sampling points of the selected noise signal is greater than z*32000, randomly select z*32000 consecutive sampling points. After obtaining clean bird call signals and noise signals of equal length, the noise is added to the clean bird call signal. Each time, one type of noise is added, ensuring that the proportion of each type of noisy bird call signal is even in the overall noisy bird call signal, and that the number of noisy bird call signals of different noise types is the same. The signals are mixed according to the following formula: Where t represents the time of the signal, and z(t) is the noisy bird sound signal. and These represent clean bird sound signals and noise signals, respectively, with 𝜉 indicating the mixing process. The gain coefficient, snr, represents the signal-to-noise ratio between the clean bird sound signal and the noise signal.

5. The bird sound event detection method for monitoring nationally protected birds according to claim 1, characterized in that, The bird sound noise reduction model includes a data input layer, an initial causal convolutional layer, an expanded residual block, a skip connection aggregation layer, and a final convolutional output layer; the construction of the bird sound noise reduction model includes the following steps: The input shape of the data input layer is (IN, 1), where IN represents the length of the input signal and is used to receive noisy bird sound signals; The initial causal convolutional layer uses causal convolutional kernels to process the input signal, such that each output depends on the current and previous input values. The initial causal convolutional layer has 128 kernels and a kernel size of 3, and its process is represented as follows: in, and These are the causal convolution kernel weights and biases, respectively. The output of causal convolution, for Activation function; There are 30 dilated residual blocks in total. Each dilated residual block includes a dilated convolutional layer, an activation function, a gating mechanism, a 1x1 convolutional layer, skip connections, and residual connections. The dilated convolutional layers have 256 kernels, a kernel size of 3, and dilation rates increasing exponentially from 1 to 512 (1, 2, 4, 8, 16, 32, 64, 128, 256, 512), with 10 dilation rates and a stack size of 3. The activation functions include the hyperbolic tangent (tanh) function and the sigmoid function. The hyperbolic tangent (tanh) function is applied to the first part of the output of the dilated convolutional layer to generate... The sigmoid function is applied to the second part of the output of the dilated convolutional layer to generate... Gating mechanisms use element-wise multiplication operations to... and Combine, generate ;Then, Generated through a 1x1 convolutional layer The 1x1 convolutional layer adjusts the dimension of the feature map through linear transformation and generates residual output and skip connection output. It has 256 kernels, with 128 residual channels and 128 skip connections. The skip connection part... The skip connection output is extracted and used for summarization in subsequent layers. The residual connection part adds the residual output of the 1x1 convolutional layer to the input signal to generate the output of the dilated residual block. Skip connection summation generates a comprehensive feature representation by summing the skip connection outputs of all dilated residual blocks. This comprehensive feature representation incorporates features from all layers, providing rich feature representations for the final output layer through feature information at different scales and dilation rates. The formula for the summation operation is as follows: in, Indicates the first The jump connection output of an expanded residual block This represents the total number of expanded residual blocks; Indicates the summation result; The final convolutional output layer consists of three convolutional layers. The first convolutional layer has 2048 kernels and a kernel size of 3, used to extract deeper features from the comprehensive feature representation of skip connections. The second convolutional layer has 256 kernels and a kernel size of 3 to compress the feature map to a smaller dimension. The third convolutional layer is a 1x1 convolutional layer with a kernel size of 1, used to perform a linear transformation on the feature map to generate the final denoised bird sound signal.

6. The bird sound event detection method for monitoring nationally protected birds according to claim 1, characterized in that, The construction of the target detection model includes the following steps: The backbone network is constructed, consisting of a Stem Layer initiation layer, convolutional modules from Stage 1 to Stage 4, a CSP layer, and an SPPF layer. The Stem Layer initiation layer comprises a convolutional module, which includes a 2D convolutional layer, batch normalization following the convolution operation, and a SiLU activation function. The Stem Layer initiation layer is used to initially extract low-level features of the image, and its operation process is represented by the following formula: in, For the input image, and These are the kernel weights and biases, respectively. Four Stage Layers are used to progressively extract and integrate features. Each Stage contains a convolutional module and a CSP layer. The CSP layer is used for feature integration and reducing computational cost. It includes two convolutional modules, a residual connection, and two Darknet Bottleneck modules. Each convolutional module includes a convolutional layer followed by a batch normalization layer and a SiLU activation function. The DarknetBottleneck module consists of two convolutional layers for efficient extraction of complex features. First, there is a 1×1 convolutional layer to reduce the number of channels and improve computational efficiency. Then, there is a 3×3 convolutional layer to extract local features. Finally, the input and output are connected through residual connections to enhance gradient propagation. The SPPF layer is used for multi-scale feature integration, which enhances the model's ability to detect targets at different scales. It includes three pooling operations with pooling kernel sizes of 5, 9 and 13, which connect pooling features at different scales to form richer feature representations. The Neck section includes TopDown and BottomUp paths, used for top-down feature fusion and bottom-up feature extraction, respectively. The TopDown path first performs an upsampling operation, upsampling the deeper feature maps to match their size with the shallower feature maps, and then connects the downsampled feature maps with the corresponding deeper feature maps through a concatenation operation. The BottomUp path, on the other hand, uses a downsampling operation to fuse the shallower feature maps with the deeper feature maps. The Head section includes basic convolutional layers and convolutional layers dedicated to bounding box regression and classification; the basic convolutional layers are used to initially process the input feature maps and extract more compact and useful features. A soft nonmaximum suppression algorithm with Gaussian decay is used to decay the confidence of overlapping candidate boxes.

7. The bird sound event detection method for monitoring nationally protected birds according to claim 1, characterized in that, The step of training and jointly optimizing the bird sound denoising model and the target detection model based on the bird sound spectrogram dataset and the noisy bird sound dataset includes the following steps: The bird sound denoising network and the object detection network were pre-trained based on the noisy bird sound dataset and the bird sound spectrogram dataset, respectively. A joint optimization framework for bird sound denoising and detection is constructed, cascading the bird sound denoising network and the target detection network, specifically including: Define a combined denoising-detection loss function, where the denoising model uses mean squared error loss. The calculation process is as follows: in, This represents the total number of samples. It is the model's prediction of the first The value of each sample It is the first The true value of each sample; The noise reduction-detection combined loss function is defined as: in, , and These are weight parameters; This represents energy-weighted attention loss; Represents the complete intersection and comparison of losses; Represents the focal point loss of the distribution; Freeze the initial causal convolutional layer and the first 9 expanded residual blocks of the bird sound denoising network, and freeze the StemLayer initial layer and Stage 1 convolutional module of the target detection network; Set the number of epochs and the batch size to 50 and 32 respectively; The Adam optimizer was used as the stochastic batch gradient descent optimizer, and the initial learning rate was set to lnitial_lr of 0.0001. During training, model training is completed when the loss value on the validation set does not decrease after 10 epochs.

8. The bird sound event detection method for monitoring nationally protected birds according to claim 1, characterized in that, The image-audio mapping module is used to convert the image detection results of the target detection model into audio event detection results, including the following steps: Let the sampling rate of a bird sound audio signal be... The total duration of the audio is The width and height of the corresponding spectrogram image are set to 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 ... and The total number of sampling points is calculated based on the duration and sampling rate of the audio signal. : in, This represents the total number of sampling points in the audio signal, and the sampling rate of the audio signal. This refers to the number of samples collected per second, used for the quantization of digital audio and the duration of the audio. It is the total duration of the audio signal; The number of sampling points corresponding to each pixel on the time axis for: in, Represents the width of the image; Temporal resolution represents the duration of time corresponding to each pixel on the time axis, and it is calculated as follows: in, This represents the time length corresponding to each pixel on the time axis; Frequency resolution represents the frequency bandwidth corresponding to each pixel on the frequency axis, which is derived from the Nyquist sampling theorem. : in, This represents the frequency bandwidth corresponding to each pixel on the frequency axis; Based on the input image pixel coordinate range Calculate the corresponding audio signal time range: in, and These represent the start and end points of time, respectively. Based on the input image pixel coordinate range Calculate the corresponding audio signal frequency range: in, and These represent the lower and upper limits of the frequency, respectively; image pixel coordinates. This indicates the frequency range selected on the image, which is converted into the actual frequency range using frequency resolution.

9. A system for implementing the bird sound event detection method for monitoring nationally protected birds as described in any one of claims 1-8, characterized in that, include: The first module is used to collect bird sound data and environmental noise data of nationally protected birds; The second module is used to construct a bird sound spectrogram dataset and a noisy bird sound dataset based on the bird sound data and the environmental noise data. The third module is used to build bird sound noise reduction models and target detection models; The fourth module is used to train and jointly optimize the bird sound denoising model and the target detection model based on the bird sound spectrogram dataset and the noisy bird sound dataset; wherein, the bird sound denoising model is used to denoise the noisy bird sound data, and the target detection model is used to detect target images; The fifth module is used to construct an image-audio mapping module, which converts the image detection results of the target detection model into audio event detection results. The sixth module is used to acquire bird sound data to be detected, and to retrieve and locate the target bird sound based on the bird sound noise reduction model, the target detection model and the image-audio mapping module, and to determine the species, time frequency and direction information of the target bird sound.

Images