Bird identification method and system based on dynamic multi-modal fusion and self-evolution lightweight model
By using dynamic multimodal fusion and a self-evolving lightweight model, the vulnerability of single modality and model rigidity in bird recognition are solved, achieving high-precision, robust, and adaptive bird recognition, which is suitable for low-power deployment on edge devices.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CENTRAL SOUTH UNIVERSITY OF FORESTRY AND TECHNOLOGY
- Filing Date
- 2026-02-02
- Publication Date
- 2026-05-12
AI Technical Summary
Existing technologies for bird identification suffer from problems such as single-modal vulnerability, rigid multimodal fusion strategies, contradiction between model solidification and high deployment costs, and difficulty in achieving both lightweight design and recognition accuracy, resulting in insufficient recognition accuracy and adaptability in complex environments.
A dynamic multimodal fusion and self-evolving lightweight model is adopted. Voiceprint and image features are fused through cross-modal attention mechanism and environment-aware dynamic weight strategy. The lightweight recognition model is trained through a hybrid evolution strategy to achieve adaptive updating and optimization of the model.
It improves the accuracy and robustness of bird identification, meets the low-power real-time deployment requirements of edge devices, has continuous evolution capabilities, extends the effective service cycle of the system, and achieves the best trade-off between lightweight and high precision.
Smart Images
Figure CN122020550A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of bird identification technology, specifically to a bird identification method and system based on dynamic multimodal fusion and a self-evolving lightweight model. Background Technology
[0002] Automated bird monitoring is of great significance for biodiversity conservation and ecological research. Current mainstream technologies rely on deep learning-based image recognition or voiceprint recognition. Although both are developing rapidly, they both face inherent limitations: (1) Vulnerability of single modality: Image recognition fails when there is insufficient light, target occlusion, or excessive distance; voiceprint recognition accuracy drops sharply when there is high environmental noise or multiple sound sources. Neither can independently cope with complex outdoor environments.
[0003] (2) Rigid multimodal fusion strategies: Existing research attempts to combine audio and video data mostly adopt static methods such as early feature splicing and late decision fusion. These methods cannot be dynamically adjusted according to the real-time quality of each modal data in a specific scenario, and fail to maximize the complementary advantages of multimodal data. Their generalization ability in real and changing environments is limited.
[0004] (3) The contradiction between model fixation and high deployment cost: High-precision models are usually computationally complex and have a large number of parameters, making them difficult to deploy on resource-constrained edge devices. Once deployed, the model cannot be updated and cannot adapt to differences in bird species in different geographical areas or bird behavior that changes with the seasons, resulting in a short system usability cycle and high maintenance costs.
[0005] (4) It is difficult to achieve both lightweight design and recognition accuracy: Directly applying general lightweight models (such as MobileNet) to professional bird recognition tasks often results in insufficient feature discrimination power due to insufficient model capacity, making it difficult to meet the high accuracy requirements of practical applications.
[0006] To address the aforementioned issues, there is an urgent need for a bird identification method and system based on dynamic multimodal fusion and a self-evolving lightweight model to solve the problems associated with traditional methods. Summary of the Invention
[0007] The purpose of this invention is to provide a bird identification method and system based on dynamic multimodal fusion and a self-evolving lightweight model. By dynamically fusing multimodal information and a self-evolving lightweight model, the accuracy, robustness and adaptability of bird identification in complex environments are significantly improved, while meeting the low-power real-time deployment requirements of edge devices.
[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A bird identification method based on dynamic multimodal fusion and a self-evolving lightweight model includes: Step 1: Collect multimodal data and preprocess it to obtain voiceprint feature maps and image tensors. The multimodal dataset includes raw audio data containing bird voiceprints and raw image data containing bird targets. Step 2: Extract high-level semantic feature vectors of voiceprint and image from the voiceprint feature map and image tensor respectively based on a lightweight convolutional neural network, and fuse them through a dynamic fusion module to obtain a fused feature vector; Step 3: Construct a lightweight recognition model, wherein the lightweight recognition model integrates a self-evolution module, the self-evolution module includes a meta-controller and a model structure search space, and the lightweight recognition model is trained by a hybrid evolution strategy through the self-evolution module; Step 4: Deploy the trained lightweight recognition model to the application terminal; Step 5: On the application terminal, the trained lightweight recognition model is used to identify birds and output the results.
[0009] Further, in step 1, the raw audio data containing bird voiceprints is preprocessed to obtain a voiceprint feature map, specifically as follows: The raw audio data containing bird voiceprints is pre-emphasized, framed, windowed, and subjected to short-time Fourier transform processing. Its Mel spectrogram or Mel frequency cepstral coefficients are calculated to obtain the voiceprint feature map.
[0010] Further, in step 1, the original image data containing bird targets is preprocessed to obtain image tensors, specifically as follows: The original image data containing bird targets is subjected to size normalization, center cropping, and color channel normalization to obtain the image tensor.
[0011] Furthermore, in step 2, the features are fused using a dynamic fusion module to obtain a fused feature vector, specifically as follows: The attention weights and context vectors of high-level semantic feature vectors of voiceprints to high-level semantic feature vectors of images, as well as the attention weights and context vectors of high-level semantic feature vectors of images to high-level semantic feature vectors of voiceprints, are calculated based on a cross-modal attention mechanism. Based on real-time collected environmental metadata, fusion weight coefficients are dynamically generated through an environmental perception subnetwork. The attention weights and context vectors are weighted and fused based on the fusion weight coefficients to obtain the final fused feature vector.
[0012] Furthermore, in step 3, the backbone network of the lightweight recognition model is GhostNet.
[0013] Furthermore, in step 3, the lightweight recognition model is trained using a hybrid evolutionary strategy through the self-evolutionary module, specifically as follows: The model structure is explored within the model structure search space based on the genetic algorithm. Part of the structural configuration of the lightweight recognition model is encoded as an individual, the fitness of each individual is calculated, and selection, crossover and mutation operations are performed based on the fitness. Policy optimization is based on reinforcement learning, treating the decisions of the meta-controller as actions and the improvement of the accuracy of the lightweight recognition model on the validation set as a reward. The parameters of the meta-controller are updated through a proximal policy optimization algorithm to maximize the cumulative reward. A multimodal dataset is constructed, and a lightweight recognition model is trained end-to-end based on the multimodal dataset, where the loss function is cross-entropy loss.
[0014] Furthermore, in step 4, the trained lightweight recognition model is deployed to the application terminal, specifically as follows: The trained lightweight recognition model is deployed to the application terminal. The application terminal monitors the performance of the lightweight recognition model in real time. If the average recognition confidence of the lightweight recognition model in the sliding event window is lower than the preset threshold or new labeled data is received, the evolution update process of the self-evolution module is triggered to update the training of the lightweight recognition model and redeploy the updated lightweight recognition model to the application terminal.
[0015] This invention also provides a bird recognition method based on dynamic multimodal fusion and a self-evolving lightweight model, applied to the aforementioned bird recognition system based on dynamic multimodal fusion and a self-evolving lightweight model, comprising: The data acquisition and preprocessing module is used to acquire raw audio data containing bird voiceprints and raw image data containing bird targets. It performs pre-emphasis, framing, windowing and short-time Fourier transform processing on the raw audio data to obtain voiceprint feature maps, and performs size normalization, center cropping and color channel normalization processing on the raw image data to obtain image tensors. The dynamic fusion module is used to extract high-level semantic feature vectors of voiceprint and image from voiceprint feature map and image tensor respectively based on lightweight convolutional neural network, and to fuse the feature vectors using cross-modal attention mechanism and environment-aware dynamic weight strategy to obtain fused feature vector. The model training and self-evolution module is used to build a lightweight recognition model. The lightweight recognition model integrates a self-evolution module, which includes a meta-controller and a model structure search space. The model training and self-evolution module is also used to train the lightweight recognition model using a hybrid evolutionary strategy based on genetic algorithms and reinforcement learning. The deployment and update module is used to deploy the trained lightweight recognition model to the application terminal, monitor its performance, and activate the self-evolution module to update and redeploy the model when the trigger conditions are met. The recognition execution module is used to process the input multimodal data on the application terminal through a lightweight recognition model and output bird recognition results.
[0016] In summary, the present invention has at least one of the following beneficial technical effects: 1. High recognition accuracy and strong environmental robustness: Traditional multimodal fusion methods employ fixed strategies, resulting in unstable performance when the environment changes. This invention utilizes a dynamic fusion method based on attention mechanisms and environmental perception, which dynamically adjusts the fusion weights of voiceprint and image features according to real-time environmental data (lighting, noise). For example, under low-light conditions at dusk, the system automatically reduces the weight of image features and relies more on voiceprint features, thus maintaining high and stable recognition accuracy in varied natural scenes. This advantage stems directly from the introduction of environmental perception weight α and the deep mining of feature associations through cross-modal attention.
[0017] (2) The model possesses continuous evolution and adaptive capabilities, and has a long lifespan: Existing models have fixed performance after deployment and quickly fail when faced with data distribution drift. This invention, through a hybrid self-evolutionary mechanism (genetic algorithm + reinforcement learning), enables the model to automatically optimize during the training phase and continuously adapt to new data through triggered updates after deployment. This means that the system can proactively learn newly emerging bird species or adapt to seasonal behavioral changes, significantly extending the effective service period. This advantage is the result of the combined effect of the self-evolutionary module and the triggered update mechanism.
[0018] (3) It achieves an optimal trade-off between lightweight design and high precision, which is beneficial for edge deployment: Simple model compression sacrifices accuracy. This invention explicitly designs the fitness function fi of the self-evolution process as a weighted sum of accuracy and computational efficiency. This makes the evolutionary direction automatically biased towards identifying the best-performing model structure within a given computing resource (such as the upper limit of the computing power of edge devices). Therefore, the final model can maintain excellent recognition accuracy while significantly reducing the number of parameters and computational cost (FLOPs), meeting the stringent requirements of low latency and low power consumption for real-time field monitoring. This advantage is a direct manifestation of the self-evolution mechanism being guided by multi-objective optimization.
[0019] (4) More efficient and intelligent utilization of multimodal information: Compared with simple feature splicing or post-voting, this invention achieves the modeling of deep semantic associations between voiceprints and image features through a cross-modal attention mechanism, so that the fused features contain richer joint information. This is essentially a data-driven, content-aware deep fusion, which significantly improves the recognition limit that is difficult to achieve by single-modal or shallow fusion methods. Attached Figure Description
[0020] Figure 1 This is a flowchart of the present invention; Figure 2 This is a schematic diagram of the training process for the self-evolving module; Figure 3 This is a schematic diagram of the hardware architecture of the system of the present invention. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0022] like Figure 1 As shown, this invention provides a bird identification method based on dynamic multimodal fusion and a self-evolving lightweight model, comprising: Step 1: Collect multimodal data and preprocess it to obtain voiceprint feature maps and image tensors. The multimodal dataset includes raw audio data containing bird voiceprints and raw image data containing bird targets. Step 2: Extract high-level semantic feature vectors of voiceprint and image from the voiceprint feature map and image tensor respectively based on a lightweight convolutional neural network, and fuse them through a dynamic fusion module to obtain a fused feature vector; Step 3: Construct a lightweight recognition model, wherein the lightweight recognition model integrates a self-evolution module, the self-evolution module includes a meta-controller and a model structure search space, and the lightweight recognition model is trained by a hybrid evolution strategy through the self-evolution module; Step 4: Deploy the trained lightweight recognition model to the application terminal; Step 5: On the application terminal, the trained lightweight recognition model is used to identify birds and output the results.
[0023] In step 1, multimodal data is collected and preprocessed to obtain voiceprint feature maps and image tensors. The multimodal dataset includes raw audio data containing bird voiceprints and raw image data containing bird targets, specifically: The field monitoring terminal collects raw audio data containing bird voiceprints using acoustic sensors. Raw image data containing bird targets are acquired through optical sensors. Among them, the acoustic sensor can be a microphone, and the optical sensor can be a camera. The collected data is preprocessed, specifically including: 1. Preprocess the raw audio data containing bird voiceprints to obtain a voiceprint feature map, specifically: Raw audio data containing bird voiceprints After pre-emphasis, framing, windowing, and short-time Fourier transform (STFT) processing, the Mel-spectrogram or Mel-frequency cepstral coefficients (MFCCs) are calculated to obtain the voiceprint feature map. .
[0024] 2. Preprocess the raw image data containing bird targets to obtain image tensors, specifically: Raw image data containing bird targets Perform size normalization (e.g., scaling to 224×224 pixels), center cropping, and color channel normalization (e.g., scaling pixel values to the [0, 1] range) to obtain the image tensor. .
[0025] In step 2, high-level semantic feature vectors of the voiceprint and the image are extracted from the voiceprint feature map and image tensor respectively based on a lightweight convolutional neural network, and then fused through a dynamic fusion module to obtain a fused feature vector, specifically: Field monitoring terminals utilize lightweight convolutional neural networks to analyze voiceprint feature maps. and image tensors Extracting high-level semantic feature vectors from voiceprints respectively and image high-level semantic feature vector The lightweight convolutional neural network course uses either a CNN network or a variant of MobileNetV3. The features are then fused using a dynamic fusion module to obtain a fused feature vector, specifically: 1. Cross-modal attention computation: Calculate the high-level semantic feature vector of the voiceprint. and image high-level semantic feature vector To capture semantic associations between modalities, including: (1) is the high-level semantic feature vector of voiceprint. and image high-level semantic feature vector Construct query, key, and value vectors respectively, as follows: (1) (2) In the formula, The weight matrix is a learnable matrix; (2) Calculate the attention weights and context vectors of the voiceprint features on the image features, as follows: (3) (4) In the formula, Key vector The dimension is used for scaling; similarly, the attention weights of image features on voiceprint features are calculated. and context vector ; 2. Dynamic weight generation and feature fusion for environmental perception: Introducing a dynamic fusion weight coefficient. This coefficient is determined by the environmental perception subnetwork based on real-time environmental metadata (such as light intensity). Environmental noise level The predicted generation is as follows: (5) In the formula, It is the Sigmoid activation function. For a multilayer perceptron, Indicates vector concatenation, ensuring Finally, the feature vectors are fused. for: (6) in, This indicates a feature splicing operation.
[0026] In step 3, a lightweight recognition model is constructed. This lightweight recognition model integrates a self-evolutionary module, which includes a meta-controller and a model structure search space. The lightweight recognition model is trained using a hybrid evolutionary strategy through the self-evolutionary module, specifically as follows: A lightweight bird classification model was built using a field monitoring terminal. Its parameters are The backbone network uses GhostNet, which integrates a self-evolving module containing a meta-controller. (Parameters are) ) and a model structure search space ; like Figure 2 As shown, a hybrid evolutionary strategy is used to train the lightweight recognition model through a self-evolutionary module, specifically as follows: 1. Structure exploration based on genetic algorithm (GA): This involves configuring parts of the model's structure (such as the channel set of certain convolutional layers). Encode individuals, and in each generation, calculate the individual. fitness for: (7) In the formula, To verify the accuracy of the validation set, For computational load, To balance hyperparameters, based on Perform selection, crossover, and mutation operations; 2. Reinforcement Learning (RL) for Policy Optimization: The meta-controller... Decisions (such as adjusting the dynamic weight generation strategy) are considered actions. This will improve the model's accuracy on the validation set. Considered a reward Updated using algorithms such as Proximal Policy Optimization (PPO). To maximize accumulated rewards; 3. Model Training: Using the constructed multimodal dataset to train the model. End-to-end training is performed, with the loss function being cross-entropy loss. The self-evolution module operates concurrently with model training, continuously optimizing the model structure. Harmony controller strategy .
[0027] In step 4, the trained lightweight recognition model is deployed to the application terminal, specifically as follows: The trained lightweight recognition model is deployed to the application terminal, and the application terminal monitors the performance of the lightweight recognition model in real time. 1. When the model is detected to be in the sliding time window Average recognition confidence within Below the preset threshold Or receive a batch of newly labeled data At that time, the evolution update process is triggered; 2. Use of computing equipment The model is fine-tuned while the self-evolutionary module is activated in the local search space. The system performs rapid structure search and strategy adjustment to generate optimized model parameters. and controller parameters ; 3. Update the model Incremental deployment to terminals.
[0028] In step 5, the lightweight recognition model trained on the application terminal is used to identify birds and output the results. Specifically: For the target data to be identified, the computing device sequentially executes steps 1 and 2 to obtain the fused features. This is then input into the currently deployed lightweight recognition model. (or evolved) The model outputs a probability distribution vector. ,in The number of known bird species, Indicates that the input belongs to the first... The probability of a class. The computing device will... The corresponding category is output as the recognition result, and its confidence level can also be output simultaneously. .
[0029] This invention also provides a bird recognition method based on dynamic multimodal fusion and a self-evolving lightweight model, applied to the aforementioned bird recognition system based on dynamic multimodal fusion and a self-evolving lightweight model, comprising: The data acquisition and preprocessing module is used to acquire raw audio data containing bird voiceprints and raw image data containing bird targets. It performs pre-emphasis, framing, windowing and short-time Fourier transform processing on the raw audio data to obtain voiceprint feature maps, and performs size normalization, center cropping and color channel normalization processing on the raw image data to obtain image tensors. The dynamic fusion module is used to extract high-level semantic feature vectors of voiceprint and image from voiceprint feature map and image tensor respectively based on lightweight convolutional neural network, and to fuse the feature vectors using cross-modal attention mechanism and environment-aware dynamic weight strategy to obtain fused feature vector. The model training and self-evolution module is used to build a lightweight recognition model. The lightweight recognition model integrates a self-evolution module, which includes a meta-controller and a model structure search space. The model training and self-evolution module is also used to train the lightweight recognition model using a hybrid evolutionary strategy based on genetic algorithms and reinforcement learning. The deployment and update module is used to deploy the trained lightweight recognition model to the application terminal, monitor its performance, and activate the self-evolution module to update and redeploy the model when the trigger conditions are met. The recognition execution module is used to process the input multimodal data on the application terminal through a lightweight recognition model and output bird recognition results.
[0030] This invention also provides a specific hardware architecture diagram, such as... Figure 3 As shown.
[0031] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0032] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0033] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0034] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0035] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.
Claims
1. A bird identification method based on dynamic multimodal fusion and a self-evolving lightweight model, characterized in that, include: Step 1: Collect multimodal data and preprocess it to obtain voiceprint feature maps and image tensors. The multimodal dataset includes raw audio data containing bird voiceprints and raw image data containing bird targets. Step 2: Extract high-level semantic feature vectors of voiceprint and image from the voiceprint feature map and image tensor respectively based on a lightweight convolutional neural network, and fuse them through a dynamic fusion module to obtain a fused feature vector; Step 3: Construct a lightweight recognition model, wherein the lightweight recognition model integrates a self-evolution module, the self-evolution module includes a meta-controller and a model structure search space, and the lightweight recognition model is trained by a hybrid evolution strategy through the self-evolution module; Step 4: Deploy the trained lightweight recognition model to the application terminal; Step 5: On the application terminal, the trained lightweight recognition model is used to identify birds and output the results.
2. The bird identification method based on dynamic multimodal fusion and a self-evolving lightweight model according to claim 1, characterized in that, In step 1, the raw audio data containing bird voiceprints is preprocessed to obtain a voiceprint feature map, specifically: The raw audio data containing bird voiceprints is pre-emphasized, framed, windowed, and subjected to short-time Fourier transform processing. Its Mel spectrogram or Mel frequency cepstral coefficients are calculated to obtain the voiceprint feature map.
3. The bird identification method based on dynamic multimodal fusion and a self-evolving lightweight model according to claim 2, characterized in that, In step 1, the original image data containing bird targets is preprocessed to obtain image tensors, specifically: The original image data containing bird targets is subjected to size normalization, center cropping, and color channel normalization to obtain the image tensor.
4. The bird identification method based on dynamic multimodal fusion and a self-evolving lightweight model according to claim 3, characterized in that, In step 2, the features are fused using a dynamic fusion module to obtain a fused feature vector, specifically as follows: The attention weights and context vectors of high-level semantic feature vectors of voiceprints to high-level semantic feature vectors of images, as well as the attention weights and context vectors of high-level semantic feature vectors of images to high-level semantic feature vectors of voiceprints, are calculated based on a cross-modal attention mechanism. Based on real-time collected environmental metadata, fusion weight coefficients are dynamically generated through an environmental perception subnetwork. The attention weights and context vectors are weighted and fused based on the fusion weight coefficients to obtain the final fused feature vector.
5. The bird identification method based on dynamic multimodal fusion and self-evolving lightweight model according to claim 4, characterized in that, In step 3, the backbone network of the lightweight recognition model is GhostNet.
6. The bird identification method based on dynamic multimodal fusion and a self-evolving lightweight model according to claim 5, characterized in that, In step 3, the lightweight recognition model is trained using a hybrid evolutionary strategy through the self-evolutionary module, specifically as follows: The model structure is explored within the model structure search space based on the genetic algorithm. Part of the structural configuration of the lightweight recognition model is encoded as an individual, the fitness of each individual is calculated, and selection, crossover and mutation operations are performed based on the fitness. Policy optimization is based on reinforcement learning, treating the decisions of the meta-controller as actions and the improvement of the accuracy of the lightweight recognition model on the validation set as a reward. The parameters of the meta-controller are updated through a proximal policy optimization algorithm to maximize the cumulative reward. A multimodal dataset is constructed, and a lightweight recognition model is trained end-to-end based on the multimodal dataset, where the loss function is cross-entropy loss.
7. The bird identification method based on dynamic multimodal fusion and a self-evolving lightweight model according to claim 6, characterized in that, In step 4, the trained lightweight recognition model is deployed to the application terminal, specifically as follows: The trained lightweight recognition model is deployed to the application terminal. The application terminal monitors the performance of the lightweight recognition model in real time. If the average recognition confidence of the lightweight recognition model in the sliding event window is lower than the preset threshold or new labeled data is received, the evolution update process of the self-evolution module is triggered to update the training of the lightweight recognition model and redeploy the updated lightweight recognition model to the application terminal.
8. A bird recognition method based on dynamic multimodal fusion and a self-evolving lightweight model, applied to the bird recognition system based on dynamic multimodal fusion and a self-evolving lightweight model as described in any one of claims 1-7, characterized in that, include: The data acquisition and preprocessing module is used to acquire raw audio data containing bird voiceprints and raw image data containing bird targets. It performs pre-emphasis, framing, windowing and short-time Fourier transform processing on the raw audio data to obtain voiceprint feature maps, and performs size normalization, center cropping and color channel normalization processing on the raw image data to obtain image tensors. The dynamic fusion module is used to extract high-level semantic feature vectors of voiceprint and image from voiceprint feature map and image tensor respectively based on lightweight convolutional neural network, and to fuse the feature vectors using cross-modal attention mechanism and environment-aware dynamic weight strategy to obtain fused feature vector. The model training and self-evolution module is used to build a lightweight recognition model. The lightweight recognition model integrates a self-evolution module, which includes a meta-controller and a model structure search space. The model training and self-evolution module is also used to train the lightweight recognition model using a hybrid evolutionary strategy based on genetic algorithms and reinforcement learning. The deployment and update module is used to deploy the trained lightweight recognition model to the application terminal, monitor its performance, and activate the self-evolution module to update and redeploy the model when the trigger conditions are met. The recognition execution module is used to process the input multimodal data on the application terminal through a lightweight recognition model and output bird recognition results.