Crested ibis identification and ecological monitoring method based on multi-modal data fusion

Abstract

The application discloses a crested ibis recognition and ecological monitoring method based on multi-modal data fusion, relates to the technical field of wild animal ecological monitoring, and comprises the following steps: installing a micro Beidou positioning tracker on a crested ibis individual to collect activity trajectory data, arranging visible light and infrared thermal imaging equipment in the main habitat and foraging area of the crested ibis, arranging an acoustic sensor array around the night habitat and breeding ground, and acquiring visual and acoustic feature data sets.Through a cross-modal attention mechanism, three types of features are adaptively weighted and fused to generate a multi-modal joint feature vector containing time and space context information, which is associated and matched with a preset ecological database, an behavior-environment correlation model is established, and a foraging preference index, habitat suitability score and activity path planning scheme are output.The method integrates multi-modal data for joint analysis, completes the quantitative output of crested ibis ecological monitoring, and improves the integrity of crested ibis recognition and ecological monitoring.

Description

Technical Field

[0001] This invention belongs to the field of wildlife ecological monitoring technology, specifically a method for crested ibis identification and ecological monitoring based on multimodal data fusion. Background Technology

[0002] The crested ibis (Nipponia nippon) is a Class I protected wild animal in my country and an endangered bird species globally. Current crested ibis identification and ecological monitoring efforts largely employ single-modal monitoring methods. Some technologies use BeiDou satellite positioning trackers to collect crested ibis activity trajectory data, others rely on single visible light imaging devices to obtain surface features, and still others use independent acoustic sensors to collect the ibis's call sound waves. All types of monitoring data are collected and analyzed independently, without establishing a multi-dimensional, collaborative monitoring system. Ecological analysis of the crested ibis often focuses solely on processing single-modal data, failing to combine spatiotemporal trajectories, visual, and acoustic features for joint analysis. The correlation analysis between behavior and the environment relies on manual statistical analysis, lacking a standardized data matching mechanism.

[0003] Single-modal monitoring data can only reflect partial aspects of the crested ibis's ecological characteristics. Location data cannot be combined with visual and acoustic features for accurate individual identification. Visual monitoring equipment is hampered by nighttime conditions and cannot acquire effective feature information. Acoustic monitoring data is difficult to correlate with the crested ibis's spatial activity. Different modalities are not fused using adaptive methods; feature weights are fixed and cannot be adjusted based on the monitoring scenario. The matching method between the ecological database and monitoring features is rudimentary, failing to build a professional behavioral-environment correlation model, output quantitative indicators related to crested ibis foraging preferences and habitat suitability, or formulate activity path planning schemes.

[0004] It is necessary to achieve adaptive weighted fusion processing of three types of features: Beidou activity trajectory, visible light infrared vision, and acoustic features. It is also necessary to build a behavioral environment association model based on the correlation and matching of multimodal joint features and ecological database, and output corresponding quantitative indices and activity path planning related content. Summary of the Invention

[0005] This invention aims to solve at least one of the technical problems existing in the prior art;

[0006] Therefore, this invention proposes a method for crested ibis identification and ecological monitoring based on multimodal data fusion, including:

[0007] A miniaturized Beidou satellite positioning tracker is fixedly installed on the target crested ibis to collect the target crested ibis's activity trajectory data at fixed time intervals. The activity trajectory data includes latitude and longitude coordinates, flight altitude, and movement speed.

[0008] Visible light imaging equipment and infrared thermal imaging equipment were deployed in the habitat and main foraging areas of the target crested ibis to simultaneously capture the surface texture features, body shape and outline of the crested ibis and the nighttime thermal radiation features, forming a visual feature dataset.

[0009] High-sensitivity acoustic sensor arrays were deployed around the overnight roosting and breeding grounds of the target crested ibis to collect the call sound waves of the crested ibis under different behavioral states. The Mel frequency cepstral coefficients and time-frequency domain energy distribution of the call sound waves were extracted to form an acoustic feature dataset.

[0010] The activity trajectory data containing latitude and longitude coordinates, the visual feature dataset, and the acoustic feature dataset are input into the multimodal data fusion framework. The three types of features are adaptively weighted and fused through a cross-modal attention mechanism to generate a multimodal joint feature vector containing spatiotemporal context information.

[0011] The system calls a pre-set ecological database, associates and matches the multimodal joint feature vector with the ecological database, establishes a behavior-environment association model, and outputs the crested ibis's foraging preference index, habitat suitability score, and activity path planning scheme.

[0012] Furthermore, the step of fixing a miniaturized BeiDou satellite positioning tracker onto the target crested ibis and collecting its activity trajectory data at fixed time intervals includes:

[0013] Each miniaturized BeiDou satellite positioning tracker is configured with a unique device identification code, and the device identification code is bound to the individual number of the target crested ibis;

[0014] The data acquisition cycle of the Beidou satellite positioning tracker is set to a variable cycle, with the acquisition interval shortened during the crested ibis breeding season and extended during the non-breeding season, in order to balance data accuracy and energy consumption.

[0015] The ground receiving station receives the original positioning message transmitted back by the Beidou satellite positioning tracker and parses out the latitude and longitude coordinates, flight altitude and speed.

[0016] The parsed latitude and longitude coordinates are subjected to Kalman filtering to remove drift points caused by satellite signal obstruction, generating smoothed activity trajectory data.

[0017] Furthermore, visible light imaging devices and infrared thermal imaging devices are deployed in the target crested ibis's habitat and main foraging areas to simultaneously capture the crested ibis's surface texture features, body shape contours, and nighttime thermal radiation features, forming a visual feature dataset, including:

[0018] Based on the known activity range of the target crested ibis, a visual monitoring grid covering the entire habitat is drawn, wherein the node spacing of the visual monitoring grid does not exceed the minimum warning distance of the crested ibis;

[0019] At each node of the visual monitoring grid, the visible light imaging device and the infrared thermal imaging device are activated simultaneously to acquire dual-channel images of the target within the field of view at the same timestamp.

[0020] The RGB image acquired by the visible light imaging device is segmented to extract the plumage distribution pattern, beak shape, and leg posture of the crested ibis, thereby generating the body surface texture features.

[0021] The thermal image acquired by the infrared thermal imaging device is segmented by temperature threshold, the isotherm contour of the crested ibis body surface is extracted, and the three-dimensional dimensions of the body contour are calculated by combining the binocular visual ranging principle.

[0022] Background noise suppression processing is performed on the thermal images collected at night, and the thermal radiation intensity value of the core body temperature region of the crested ibis is extracted to generate the nighttime thermal radiation characteristics;

[0023] The surface texture features, body shape contours, and nighttime thermal radiation features at the same timestamp are packaged into a single visual feature data set, which is then aggregated to form the visual feature dataset.

[0024] Furthermore, a high-sensitivity acoustic sensor array is deployed around the target crested ibis's roosting and breeding grounds to collect the call sounds of the crested ibis under different behavioral states. The Mel-frequency cepstral coefficients and time-frequency energy distribution of the call sounds are extracted to form an acoustic feature dataset, including:

[0025] Centered on the target crested ibis's roosting site and nest, at least three of the aforementioned high-sensitivity acoustic sensors are deployed at different azimuth angles within a radius range to form an acoustic triangulation array;

[0026] When the acoustic triangulation array detects a sound signal with a sound pressure level exceeding a preset threshold, it triggers a synchronous recording mechanism to record the original audio waveform for a duration of time.

[0027] The original audio waveform is subjected to bandpass filtering to remove ambient background noise and retain the unique vocal frequency band of the crested ibis;

[0028] The filtered audio waveform is subjected to frame-by-frame windowing processing, and the Mel frequency cepstral coefficients of each frame signal are calculated as static features of the voiceprint.

[0029] Time-frequency analysis is performed on the filtered audio waveform to calculate its spectral centroid, bandwidth, and zero-crossing rate, generating the time-frequency domain energy distribution as the dynamic feature of the voiceprint.

[0030] The cepstral coefficients of the Mel frequency corresponding to each valid call are combined with the time-frequency domain energy distribution to generate an acoustic feature data, which is then summarized to form the acoustic feature dataset.

[0031] Further, the process of inputting the activity trajectory data containing latitude and longitude coordinates, the visual feature dataset, and the acoustic feature dataset into a multimodal data fusion framework, and adaptively weighting and fusing the three types of features through a cross-modal attention mechanism to generate a multimodal joint feature vector containing spatiotemporal context information includes:

[0032] The latitude and longitude coordinates in the activity trajectory data are analyzed, mapped to a two-dimensional plane coordinate system, and aligned with the timestamps of the visual feature dataset and the acoustic feature dataset to generate a data sequence with a unified spatiotemporal reference.

[0033] The cosine similarity between the surface texture features in the visual feature dataset and the Mel frequency cepstral coefficients in the acoustic feature dataset is calculated separately and used as a measure of intermodal correlation.

[0034] Based on the correlation metric, the fusion weight coefficients of the visual feature dataset, the acoustic feature dataset, and the activity trajectory data are dynamically adjusted, with the modalities with high correlation receiving higher weight coefficients.

[0035] The three types of data are linearly superimposed using the adjusted weight coefficients, and feature transformation is performed using a nonlinear activation function to eliminate the differences in data dimensions between different modalities.

[0036] The multi-source data after feature transformation are concatenated and the geographical location information indicated by the latitude and longitude coordinates is injected to generate the multimodal joint feature vector containing spatiotemporal context information.

[0037] Furthermore, the step of calling a pre-set ecological database and associating and matching the multimodal joint feature vector with the ecological database to establish a behavior-environment association model includes:

[0038] The ecological database contains data on the crested ibis's food chain distribution, climate temperature and humidity, and data on sources of human disturbance.

[0039] The real-time behavioral state labels of the crested ibis are decoupled from the multimodal joint feature vector, and the behavioral state labels include foraging, resting, vigilance and flight;

[0040] Using the behavior status label as the query key, the environmental parameter range corresponding to the behavior status is retrieved from the ecological database, including food source abundance index, ambient temperature threshold and noise decibel threshold;

[0041] The real-time collected food chain distribution data, climate temperature and humidity data, and human disturbance source data are compared with the environmental parameter range to calculate the environmental fit score.

[0042] When the environmental fit score is lower than the preset qualified threshold, the dominant feature dimension that leads to the low score in the multimodal joint feature vector is analyzed, and the corresponding environmental factor is marked as the limiting factor in the behavior-environment association model.

[0043] By traversing all combinations of behavioral states and environmental parameters, a multidimensional mapping table consisting of behavioral states, environmental parameters, and limiting factors is constructed, thus solidifying the behavioral-environment association model.

[0044] Furthermore, the output of the crested ibis's foraging preference index, habitat suitability score, and activity path planning scheme includes:

[0045] In the behavior-environment association model, the habitat type pointed to by the visual feature dataset of the crested ibis when it is foraging is extracted, and combined with the food chain distribution data in the ecological database, the food availability of the habitat type is calculated to generate the foraging preference index.

[0046] By combining the activity trajectory data, climate temperature and humidity data, and human disturbance source data recorded in the multimodal joint feature vector, the permanent habitat of the crested ibis is scored. The scoring criteria include activity frequency, temperature and humidity suitability, and disturbance distance, and a habitat suitability score is generated.

[0047] By tracing the continuous changes in latitude and longitude coordinates in the activity trajectory data, the seasonal activity routes of the crested ibis are identified. Combined with the climate temperature and humidity data in the ecological database, the safety of the rest stops along the activity routes is assessed, and the activity path planning scheme is generated.

[0048] Further, the step of dynamically adjusting the fusion weight coefficients of the visual feature dataset, the acoustic feature dataset, and the activity trajectory data based on the correlation metric includes:

[0049] A modal attention score matrix is ​​constructed using the correlation metric, wherein the cosine similarity between the visual feature dataset and the acoustic feature dataset is recorded as the first correlation value, the matching degree between the visual feature dataset and the activity trajectory data under the same spatiotemporal reference is recorded as the second correlation value, and the matching degree between the acoustic feature dataset and the activity trajectory data under the same spatiotemporal reference is recorded as the third correlation value.

[0050] The first, second, and third association values ​​are input into the Softmax function for normalization to obtain the normalized attention weights between each modality pair.

[0051] The average of the normalized attention weights of each modality itself and the other two modalities is used as the initial dynamic weight coefficient of the modality.

[0052] Based on the requirements of the multimodal joint feature vector generation task, preset modal weight biases are set for the behavior recognition task, individual recognition task, and anomaly detection task;

[0053] The initial dynamic weight coefficients are weighted and summed with the preset modality weight bias corresponding to the current task, and then normalized twice to generate the fusion weight coefficients used for final feature fusion.

[0054] Furthermore, it also includes steps for specific monitoring of the chick-rearing behavior of crested ibises:

[0055] When the body contour data in the visual feature dataset shows a lying position, it is determined to be in the infancy period, and the sampling frequency of the infrared thermal imaging device is automatically increased;

[0056] During the brooding period, the begging calls of chicks in the acoustic feature dataset are extracted and paired with the feeding calls of the parent birds to determine the timing pattern of the parent birds returning to their nests.

[0057] The timing pattern of parent birds returning to their nests is superimposed with the changes in latitude and longitude coordinates in the activity trajectory data to analyze the radius of distance the parent birds travel from their nests to forage.

[0058] If the radius of the distance by which the parent birds leave the nest to forage exceeds the safe foraging radius set in the ecological database, a parenting risk warning event will be generated in the behavior-environment association model.

[0059] Furthermore, it also includes the step of dynamically updating the ecological database based on historical data:

[0060] The number of crested ibis individuals identified in the multimodal joint feature vector is periodically counted, and the difference is calculated with the data of the same period of the previous year to obtain the change in population size;

[0061] By analyzing the latitude and longitude coordinates in the activity trajectory data, new activity areas of the crested ibis are identified, and the geographical coordinates of the new activity areas are written into the ecological database.

[0062] Monitor whether a new call type appears in the acoustic feature dataset. If a new call type exists, record its Mel frequency cepstral coefficients into the voiceprint library of the ecological database.

[0063] Based on changes in population size, new activity territories, and new call types, the threshold values ​​of various environmental parameters in the ecological database are adjusted to complete the version iteration of the ecological database.

[0064] Compared with the prior art, the beneficial effects of the present invention are:

[0065] A cross-modal attention mechanism is employed to adaptively weight and fuse activity trajectory data (including latitude and longitude coordinates, flight altitude, and movement speed) collected by BeiDou satellite positioning, visual feature datasets acquired by visible light and infrared thermal imaging equipment, and acoustic feature datasets (derived from Mel frequency cepstral coefficients and time-frequency domain energy distribution extracted by a high-sensitivity acoustic sensor array). The fusion weights of the three types of features can be dynamically adjusted according to different monitoring scenarios. The generated multimodal joint feature vector integrates spatiotemporal context information, weakens the scenario limitations of single-modal data, and eliminates the feature bias problems caused by nighttime visual monitoring, isolated positioning data, and independent acoustic data. This allows the individual characteristics and activity status of the crested ibis to be fully presented through joint features, adapting to the feature extraction needs of the crested ibis in different activity scenarios.

[0066] By linking multimodal joint feature vectors with a pre-built ecological database and constructing a behavior-environment correlation model, the monitoring characteristics of crested ibises can be accurately correlated with ecological environment data. This results in quantitative outputs such as foraging preference index and habitat suitability score. Combined with the spatiotemporal activity characteristics of crested ibises, activity path planning schemes can be generated, replacing the analysis method of manual statistical analysis. This transforms the analysis results of crested ibis ecological monitoring from qualitative description to quantitative output, adapting to the monitoring and analysis needs of different behavioral stages of crested ibises such as breeding, foraging, and activity, refining the dimensions of ecological monitoring results, and improving the analysis system of crested ibis ecological monitoring. Attached Figure Description

[0067] Figure 1 This is a flowchart illustrating the steps of the crested ibis identification and ecological monitoring method based on multimodal data fusion described in this invention.

[0068] Figure 2 A flowchart for deploying imaging equipment and generating a visual feature dataset;

[0069] Figure 3 This is a comparison of visual features before and after ReLU feature transformation;

[0070] Figure 4 Heatmap of crested ibis behavior and environmental constraints;

[0071] Figure 5 A comparative chart showing the monthly changes in the crested ibis population. Detailed Implementation

[0072] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] See Figure 1 A miniaturized BeiDou satellite positioning tracker was fixedly installed on the target crested ibis. This tracker collected the crested ibis's activity trajectory data at fixed time intervals, including latitude and longitude coordinates, flight altitude, and movement speed. Visible light imaging equipment and infrared thermal imaging equipment were deployed in the crested ibis's habitat and main foraging areas. These two types of equipment simultaneously captured the crested ibis's surface texture features, body shape contours, and nighttime thermal radiation characteristics, forming a visual feature dataset. A high-sensitivity acoustic sensor array was deployed around the crested ibis's roosting and breeding grounds. This array collected the call sound waves of the crested ibis in different behavioral states, and extracted the Mel-frequency cepstral coefficients and time-frequency domain energy distribution of the call sound waves to form an acoustic feature dataset. The activity trajectory data containing latitude and longitude coordinates, the visual feature dataset, and the acoustic feature dataset were jointly input into a multimodal data fusion framework. This framework adaptively weighted and fused the three types of features through a cross-modal attention mechanism, ultimately generating a multimodal joint feature vector containing spatiotemporal context information. By calling a pre-set ecological database, the generated multimodal joint feature vector is associated and matched with the ecological database to establish a behavior-environment association model. This model outputs the crested ibis's foraging preference index, habitat suitability score, and activity path planning scheme.

[0074] In one embodiment of the invention, each miniaturized BeiDou satellite positioning tracker is equipped with a unique device identification code, which is bound to the individual number of the target crested ibis. The data acquisition cycle of the BeiDou satellite positioning tracker is set to a variable cycle, with the acquisition interval shortened during the crested ibis breeding season and extended during the non-breeding season. The original positioning messages transmitted back by the BeiDou satellite positioning tracker are received by a ground receiving station, and the latitude and longitude coordinates, flight altitude, and movement speed are parsed out. The parsed latitude and longitude coordinates are then subjected to Kalman filtering to remove drift points caused by satellite signal obstruction, generating smoothed activity trajectory data. (See also...) Figure 2 Based on the known activity range of the target crested ibis, a visual monitoring grid covering the entire habitat is drawn. The node spacing of the visual monitoring grid does not exceed the minimum warning distance of the crested ibis. At each node of the visual monitoring grid, a visible light imaging device and an infrared thermal imaging device are activated simultaneously to acquire dual-channel images of the target within the field of view with the same timestamp.

[0075] RGB images acquired by visible light imaging devices are segmented to extract the plumage distribution pattern, beak shape, and leg posture of the crested ibis, generating surface texture features. Thermal images acquired by infrared thermal imaging devices are segmented using temperature thresholding to extract isotherm contours of the crested ibis's body surface. The three-dimensional dimensions of the body contour are calculated using binocular visual ranging principles. Background noise is suppressed in thermal images acquired at night, and the thermal radiation intensity values ​​of the core body temperature region of the crested ibis are extracted to generate nighttime thermal radiation features. Surface texture features, body contours, and nighttime thermal radiation features from the same time stamp are packaged into a single visual feature dataset.

[0076] In practical implementation, the miniaturized BeiDou satellite positioning tracker is equipped with a unique device identification code, which is bound to the individual number of the target crested ibis. The data acquisition cycle of the BeiDou satellite positioning tracker is set to a variable cycle, with the acquisition interval shortened during the crested ibis breeding season and extended during the non-breeding season. The raw positioning messages transmitted back by the BeiDou satellite positioning tracker are received by the ground receiving station, and the latitude and longitude coordinates, flight altitude, and movement speed are parsed out. The parsed latitude and longitude coordinates are then subjected to Kalman filtering for noise reduction. The state prediction step of the Kalman filtering noise reduction process uses the following formula:

[0077]

[0078] in: This represents the state prediction vector at time k based on time k-1. Represents the state transition matrix. The state estimation vector at time k-1 is represented. Based on the state prediction vector, the drift points caused by satellite signal obstruction are removed and smoothed activity trajectory data is generated.

[0079] In some embodiments, a visual monitoring grid covering the entire habitat is drawn based on the known activity range of the target crested ibis. The node spacing of the visual monitoring grid does not exceed the minimum warning distance of the crested ibis. At each node of the visual monitoring grid, a visible light imaging device and an infrared thermal imaging device are simultaneously activated to acquire dual-channel images of the target within the field of view at the same timestamp. The RGB images acquired by the visible light imaging device are segmented to extract the crested ibis's plumage distribution pattern, beak shape, and leg posture to generate surface texture features. The thermal images acquired by the infrared thermal imaging device are segmented by temperature threshold to extract the isotherm contours of the crested ibis's body surface. The three-dimensional dimensions of the body shape contour are calculated using the binocular visual ranging principle. The thermal images acquired at night are processed to suppress background noise and extract the thermal radiation intensity value of the core body temperature region of the crested ibis to generate nighttime thermal radiation features. The surface texture features, body shape contours, and nighttime thermal radiation features at the same timestamp are packaged and encapsulated into a single visual feature data set, which is then aggregated to form a visual feature dataset. Optionally, the data acquisition cycle of the miniaturized BeiDou satellite positioning tracker can be adjusted according to the seasonal changes in crested ibis behavior. A shorter acquisition interval is set during the breeding season to capture finer movements, while a longer interval is set during the non-breeding season to reduce energy consumption. It is understood that the deployment of nodes in the visual monitoring grid must avoid areas obstructed by vegetation, ensuring overlapping field of view coverage between visible light imaging and infrared thermal imaging equipment. The timestamp synchronization accuracy of dual-channel image acquisition must reach the millisecond level to guarantee the temporal alignment of surface texture features and nighttime thermal radiation features.

[0080] In some embodiments, the Kalman filtering denoising process further includes a measurement update step, which uses satellite positioning observations to correct the state prediction vector and iteratively processes it until the latitude and longitude coordinate sequence is smooth. Optionally, the thermal image temperature threshold segmentation of the infrared thermal imaging device sets a dynamic threshold based on the temperature difference between the crested ibis's body temperature and the environment. After extracting the isotherm contours, the three-dimensional dimensions are calculated using the binocular visual ranging principle, which involves the calculation of disparity between two imaging devices. It can be understood that the background noise suppression process uses a temporal filtering method to remove environmental heat source interference, focusing on extracting the thermal radiation intensity value of the core body temperature region of the crested ibis. Nighttime thermal radiation features and surface texture features are packaged and encapsulated under the same timestamp to form a record in the visual feature dataset.

[0081] In one embodiment of the present invention, at least three high-sensitivity acoustic sensors are deployed at different azimuth angles within a radius centered on the target crested ibis's roosting site and nest, forming an acoustic triangulation array. When the acoustic triangulation array detects a sound signal with a sound pressure level exceeding a preset threshold, a synchronous recording mechanism is triggered to record the original audio waveform for a specified duration. The original audio waveform is bandpass filtered to remove environmental background noise and retain the crested ibis's unique vocal frequency band. The filtered audio waveform is then framed and windowed, and the Mel-frequency cepstral coefficients of each frame are calculated as static features of the voiceprint. Time-frequency analysis is performed on the filtered audio waveform to calculate its spectral centroid, bandwidth, and zero-crossing rate, generating a time-frequency domain energy distribution as dynamic features of the voiceprint. The Mel-frequency cepstral coefficients corresponding to each valid call are combined with the time-frequency domain energy distribution to generate an acoustic feature data set, which is then compiled to form an acoustic feature dataset.

[0082] In practice, at least three high-sensitivity acoustic sensors are deployed at different azimuth angles within a 200-meter radius of the crested ibis's roosting site and nest, forming an acoustic triangulation array. When any high-sensitivity acoustic sensor in the array detects a sound pressure level exceeding 65 dB, a synchronous recording mechanism is triggered across all sensors, recording a 2-second raw audio waveform. The 2-second raw audio waveform is then bandpass filtered, with a passband frequency range of 500 Hz to 5000 Hz, to remove background noise and retain the crested ibis's unique vocalization frequency band. The filtered audio waveform is then framed and windowed, with a frame length of 25 milliseconds and a frame shift of 10 milliseconds. The Mel-frequency cepstral coefficients (CFCs) of each frame are calculated using discrete cosine transform and serve as the static features of the voiceprint. Time-frequency analysis is performed on the filtered audio waveform to calculate its spectral centroid, bandwidth, and zero-crossing rate, generating a time-frequency domain energy distribution as the dynamic feature of the voiceprint. The Mel-frequency cepstral coefficients corresponding to each valid call are combined with the time-frequency domain energy distribution to generate an acoustic feature data set, which is then compiled into an acoustic feature dataset.

[0083] In some embodiments, the extraction process of Mel-frequency cepstral coefficients involves mapping the frequency domain signal to a Mel scale and obtaining the cepstral coefficients through discrete cosine transform. The formula for calculating the Mel-frequency cepstral coefficients is expressed as:

[0084]

[0085] in: Represents the cepstral coefficients of the m-th Mel frequency. This represents the energy at the k-th Mel filter bank, where K represents the total number of Mel filter banks and m represents the order index of the cepstral coefficients. Optionally, the spectral centroid in the time-frequency domain energy distribution represents the sound brightness, the bandwidth represents the frequency range of the sound, and the zero-crossing rate represents the frequency at which the time-domain waveform crosses zero. The spectral centroid, bandwidth, and zero-crossing rate are calculated based on windowed and framed audio signal segments, with each signal segment corresponding to a time-frequency domain energy distribution vector.

[0086] In some embodiments, the deployment radius of the acoustic triangulation array is set based on the typical warning distance during the crested ibis breeding season, and the sensitivity threshold of the high-sensitivity acoustic sensors is set to 65 dB to exclude common wind noise and insect chirps. After triggering the synchronous recording mechanism, the three high-sensitivity acoustic sensors synchronously record the original audio waveform at a sampling rate of 48 kHz. The recording duration of 2 seconds includes the complete onset and attenuation process of the chirping sound wave. It can be understood that the bandpass filtering process uses a finite-length unit impulse response filter, with the passband set to 500 Hz to 5000 Hz to match the main energy frequency band of the crested ibis chirping, effectively filtering out low-frequency fan noise below 500 Hz and high-frequency electronic noise above 5000 Hz.

[0087] Optionally, the frame-by-frame windowing process uses a Hamming window to reduce spectral leakage. The first 12 orders of the Mel-frequency cepstral coefficients from each frame are typically extracted to form a static feature vector. Time-frequency analysis is performed based on the short-time Fourier transform; the spectral centroid, bandwidth, and zero-crossing rate together constitute a three-dimensional dynamic feature vector. In essence, the static feature vector of the Mel-frequency cepstral coefficients is concatenated with the dynamic feature vector of the time-frequency energy distribution along the feature dimension to form a complete acoustic feature dataset. The acoustic feature dataset consists of multiple such datasets arranged chronologically.

[0088] In one embodiment of the present invention, the latitude and longitude coordinates in the activity trajectory data are parsed, mapped to a two-dimensional plane coordinate system, and aligned with the timestamps of the visual feature dataset and acoustic feature dataset to generate a data sequence with a unified spatiotemporal reference. The cosine similarity between the surface texture features in the visual feature dataset and the Mel frequency cepstral coefficients in the acoustic feature dataset is calculated as a measure of intermodal correlation. Based on the correlation measure, the fusion weight coefficients of the visual feature dataset, acoustic feature dataset, and activity trajectory data are dynamically adjusted, with higher weight coefficients assigned to modalities with higher correlation. The adjusted weight coefficients are used to linearly superimpose the three types of data, and a nonlinear activation function is used for feature transformation to eliminate the differences in data dimensions between different modalities. The multi-source data after feature transformation are concatenated, and the geographical location information indicated by the latitude and longitude coordinates is injected to generate a multimodal joint feature vector containing spatiotemporal context information.

[0089] A modal attention score matrix is ​​constructed using correlation metrics. The cosine similarity between the visual feature dataset and the acoustic feature dataset is denoted as the first correlation value; the matching degree between the visual feature dataset and the activity trajectory data under the same spatiotemporal reference is denoted as the second correlation value; and the matching degree between the acoustic feature dataset and the activity trajectory data under the same spatiotemporal reference is denoted as the third correlation value. The first, second, and third correlation values ​​are input into a Softmax function for normalization to obtain the normalized attention weights between each modality pair. The average of the normalized attention weights of each modality itself and the other two modalities is used as the initial dynamic weight coefficient for that modality. Based on the requirements of the multimodal joint feature vector generation task, preset modal weight biases are set for the behavior recognition task, individual recognition task, and anomaly detection task. The initial dynamic weight coefficients are weighted and summed with the preset modal weight biases corresponding to the current task, and then normalized twice to generate the fusion weight coefficients used for the final feature fusion.

[0090] In practice, the latitude and longitude coordinates in the activity trajectory data are analyzed and mapped to a two-dimensional plane coordinate system using Gauss-Kruger projection. These coordinates are then strictly aligned with the timestamps of the visual and acoustic feature datasets to generate a data sequence with a unified spatiotemporal reference. The cosine similarity between the surface texture feature vector in the visual feature dataset and the Mel frequency cepstral coefficient vector in the acoustic feature dataset is calculated, and the result serves as a measure of intermodal correlation. Based on this correlation measure, the fusion weight coefficients of the visual feature dataset, acoustic feature dataset, and activity trajectory data are dynamically adjusted, with higher fusion weight coefficients assigned to modalities with higher correlation. The adjusted fusion weight coefficients are then used to linearly superimpose the three types of data, followed by feature transformation using a nonlinear activation function. The ReLU function is employed to eliminate the dimensional differences between different modalities. The multi-source data after feature transformation are then concatenated along the feature dimension, and geographical location information indicated by the latitude and longitude coordinates is injected to generate a multimodal joint feature vector containing spatiotemporal context information.

[0091] In some embodiments, a three-row, three-column modal attention score matrix is ​​constructed using correlation metrics. The cosine similarity between the visual feature dataset and the acoustic feature dataset is denoted as the first correlation value. The matching degree between the visual feature dataset and the activity trajectory data under the same spatiotemporal reference is denoted as the second correlation value. The matching degree between the acoustic feature dataset and the activity trajectory data under the same spatiotemporal reference is denoted as the third correlation value. The matching degree is obtained by calculating the reciprocal of the spatial distance between the feature vectors at the same timestamp.

[0092] The first, second, and third association values ​​are input into the Softmax function for normalization, resulting in the normalized attention weights between each modality pair. The formula for the Softmax function is as follows:

[0093]

[0094] in: This represents the normalized attention weight of mode i with respect to mode j. This represents the correlation metric between modality i and modality j, where N represents the total number of modalities (N=3 here), and e represents the natural constant. The initial dynamic weight coefficient for each modality is the arithmetic mean of its normalized attention weights relative to the other two modalities. Optionally, based on the specific requirements of the multimodal joint feature vector generation task, preset modal weight biases are assigned to the action recognition task, the individual recognition task, and the anomaly detection task. The action recognition task assigns a higher preset modal weight bias to the visual feature dataset, the individual recognition task assigns a higher preset modal weight bias to the acoustic feature dataset, and the anomaly detection task assigns a higher preset modal weight bias to the activity trajectory data. Essentially, the calculated initial dynamic weight coefficients are weighted and summed with the preset modal weight biases corresponding to the current task. The weights in the summation are preset empirical values, and a second normalization is performed to ensure that the sum of all weight coefficients is 1, generating the fusion weight coefficients used for the final feature fusion.

[0095] In some embodiments, the feature transformation employs the ReLU function applied to the linearly superimposed feature vector. The ReLU function sets negative values ​​to zero and retains positive values, thereby eliminating dimensional differences and introducing nonlinearity. It can be understood that multi-source data stitching involves concatenating the transformed vectors of visual features, acoustic features, and trajectory features end-to-end, with geographic location information added as an additional coordinate dimension to the end of the stitched vector. Optionally, the data sequences with a unified spatiotemporal reference are aligned with millisecond-level precision to ensure that the timestamps of image acquisition in the visual feature dataset, the start timestamps of audio in the acoustic feature dataset, and the timestamps of location points in the activity trajectory data are consistent; inconsistent data segments are discarded. The dynamic adjustment of the fusion weight coefficients is an iterative process. Each time a new synchronized data segment is received, the first, second, and third association values ​​are recalculated, and the modal attention score matrix is ​​updated, thereby generating new fusion weight coefficients for the generation of the multimodal joint feature vector at that moment.

[0096] See Figure 3This is a comparison chart of visual features before and after ReLU feature transformation, used to demonstrate the difference in the distribution of visual features of the crested ibis before and after the ReLU nonlinear activation function transformation. The core effect is to demonstrate the effectiveness of dimension elimination and nonlinear feature extraction in multimodal data fusion. The red line represents the visual feature values ​​before the transformation, ranging from approximately -1.5 to 1.5, including both positive and negative values, with a relatively dispersed data distribution. The blue line represents the visual feature values ​​after the ReLU transformation, with the value range compressed to 0 to 0.6, all negative values ​​set to 0, and only non-negative features retained. Negative value suppression sets all feature values ​​less than 0 to 0, eliminating negative interference and making the features more focused on the effective activation parts. Dimension compression compresses the original large range of feature values ​​into a smaller positive number interval, eliminating the difference in data dimensions between different modalities. Nonlinearity introduction introduces nonlinearity through piecewise linear transformation, providing a more discriminative feature representation for subsequent multimodal feature fusion.

[0097] In one embodiment of the present invention, the ecological database includes food chain distribution data, climate temperature and humidity data, and human disturbance source data of the crested ibis. Real-time behavioral state labels of the crested ibis are decoupled from the multimodal joint feature vector, including foraging, resting, vigilance, and flight. Using the behavioral state label as the query key, the environmental parameter ranges typically corresponding to that behavioral state are retrieved from the ecological database, including food source abundance index, environmental temperature threshold, and noise decibel threshold. The real-time collected food chain distribution data, climate temperature and humidity data, and human disturbance source data are compared with the environmental parameter ranges to calculate an environmental fit score. When the environmental fit score is lower than a preset acceptable threshold, the dominant feature dimensions in the multimodal joint feature vector that lead to the low score are analyzed, and the corresponding environmental factors are marked as limiting factors in the behavior-environment association model. All combinations of behavioral states and environmental parameters are traversed to construct a multidimensional mapping table composed of behavioral states, environmental parameters, and limiting factors, thus solidifying the behavior-environment association model.

[0098] In the behavior-environment association model, the habitat type indicated by the visual feature dataset of crested ibises in foraging state is extracted. Combined with food chain distribution data from the ecological database, food availability for that habitat type is calculated, generating a foraging preference index. By integrating activity trajectory data, climate temperature and humidity data, and human disturbance source data recorded in the multimodal joint feature vector, the crested ibis's permanent habitat is scored. The scoring criteria include activity frequency, temperature and humidity suitability, and disturbance distance, generating a habitat suitability score. By tracing the continuous changes in latitude and longitude coordinates in the activity trajectory data, the seasonal migration routes of the crested ibis are identified. Combined with climate temperature and humidity data along the route from the ecological database, safety assessments are conducted on rest stops along the migration route, generating activity path planning schemes.

[0099] In practical implementation, the ecological database contains food chain distribution data, climate temperature and humidity data, and human disturbance source data for the crested ibis. Real-time behavioral state labels of the crested ibis are decoupled from the multimodal joint feature vector. These labels include foraging, resting, vigilance, and flight. Using these behavioral state labels as query keys, the corresponding environmental parameter ranges for each behavioral state are retrieved from the ecological database. These environmental parameter ranges include food source abundance index, ambient temperature threshold, and noise decibel threshold. The real-time collected food chain distribution data, climate temperature and humidity data, and human disturbance source data are compared with these environmental parameter ranges to calculate an environmental fit score. When the environmental fit score is lower than a preset acceptable threshold, the dominant feature dimensions in the multimodal joint feature vector that lead to the low score are analyzed, and the corresponding environmental factors are marked as limiting factors in the behavior-environment association model. All combinations of behavioral states and environmental parameters are traversed to construct a multidimensional mapping table composed of behavioral states, environmental parameters, and limiting factors, thus solidifying the behavior-environment association model. Environmental fit score. The calculation uses the following formula for quantitative evaluation:

[0100]

[0101] in: This indicates the score for environmental fit. This represents the preset weighting coefficient of the i-th environmental parameter. This represents the actual measured value of the i-th environmental parameter collected in real time. This represents the reference range value of the i-th environmental parameter retrieved from the ecological database. It is a comparison function, when the actual measured value Falling within the reference range value The output value is 1 if the condition is met, otherwise the output value is 0, where n represents the total number of environmental parameters participating in the comparison.

[0102] In the behavior-environment association model, the habitat type indicated by the visual feature dataset of crested ibises in a foraging state is extracted. Combined with food chain distribution data from the ecological database, food availability for that habitat type is calculated. Food availability is quantified by statistically analyzing the frequency and biomass of the crested ibis's main food organisms per unit area of ​​that habitat, generating a foraging preference index. By integrating activity trajectory data, climate temperature and humidity data, and human disturbance source data recorded in the multimodal joint feature vector, the crested ibis's permanent habitat is scored. The scoring criteria include activity frequency, temperature and humidity suitability, and disturbance distance, generating a habitat suitability score. Activity frequency is calculated based on the density of coordinate points appearing in the habitat from the activity trajectory data. Temperature and humidity suitability is determined based on whether the climate temperature and humidity data are within the crested ibis's physiological comfort range. Distance to disturbance is determined based on the straight-line distance between the nearest disturbance source and the center of the habitat from the human disturbance source data.

[0103] By tracing the continuous changes in latitude and longitude coordinates in the activity trajectory data, the seasonal activity routes of the crested ibis are identified. Combined with climate temperature and humidity data along the route from the ecological database, safety assessments are conducted on rest stops along the activity route, and activity path planning schemes are generated. The safety assessment takes into account the food source abundance index around the rest stops, the probability of extreme weather occurrence, and the distribution of known human disturbance sources.

[0104] In some embodiments, the decoupling of behavioral state labels is achieved through a pre-trained classification model. This model takes a multimodal joint feature vector as input and outputs the probability distributions of four states: foraging, resting, alert, and flying. The state corresponding to the maximum probability is determined as the real-time behavioral state label. The environmental parameter ranges stored in the ecological database are organized in the form of data tables, with different behavioral state labels associated with different parameter value ranges, as shown in Table 1.

[0105] It can be understood that the process of marking limiting factors is as follows: when the environmental fit score is lower than the acceptable threshold, the actual measured value of each environmental parameter is checked one by one. Does it exceed its reference range value? The parameters that exceed the limits are marked as limiting factors. The behavior state dimension of the multidimensional mapping table contains four behaviors, the environmental parameter dimension contains three categories: food, temperature and humidity, and noise, and the limiting factor dimension records the names of the marked parameters.

[0106] Optionally, the calculation of the foraging preference index further incorporates the frequency of pecking actions of crested ibises identified centrally from the visual feature dataset. The frequency of pecking actions is obtained by analyzing the number of times the crested ibis's beak contacts the ground in consecutive image frames. The formula for calculating the habitat suitability score is:

[0107]

[0108] in: Indicates habitat suitability score, This represents the normalized activity frequency value. This represents the Boolean value indicating the suitability of temperature and humidity. Indicates the interference distance. The weighting coefficients are preset. The generation of the activity route planning scheme includes identifying key rest stops on the activity route, assessing their safety during the activity period based on historical climate temperature and humidity data of key rest stops in the ecological database, marking key rest stops with safety below the threshold as high-risk points, and suggesting detours or providing alternative rest stop coordinates in the planned route scheme, as shown in Table 1.

[0109] Table 1: Correspondence between behavioral states and environmental parameter ranges

[0110]

[0111] In some embodiments, the pass threshold is set to 0.7 when the environmental fit score is... A threshold below 0.7 triggers the limiting factor analysis process. Food availability calculations require access to the food chain distribution sub-database within the ecological database, which stores survey data on crested ibis food resources across different habitat grids. Migration route identification is understood to be based on the clustering and path fitting of latitude and longitude coordinates in the activity trajectory data over time; seasonality is manifested in the concentration of coordinate points towards fixed geographical areas in specific months. Safety assessment calculates a risk value for each stopover point, taking into account the probability of extreme low temperatures, high temperatures, and heavy rainfall occurring at that point during the same historical period, as well as the density of known human disturbance sources. The final output of the activity route planning scheme is a recommended route consisting of a series of safe coordinate points, along with the suggested stop duration at each point along the route.

[0112] See Figure 4 This is a heatmap of crested ibis behavior and environmental constraints, used to quantify the degree of constraint imposed on crested ibises by various environmental factors under different behavioral states. Among the three behaviors—foraging, resting, and vigilance—human interference is the highest limiting factor, indicating that human activity is the core stressor affecting the daily behavior of crested ibises. Foraging behavior is highly dependent on food supply and has low sensitivity to temperature, humidity, and noise; resting behavior is sensitive to temperature changes and prefers quiet, low-interference environments; vigilance behavior is mainly triggered by noise and human interference and is a typical stress response behavior; flight behavior is only limited by temperature and has high tolerance to other environmental factors. Humidity did not show a limiting effect in any behavioral state, indicating that crested ibises are well adapted to the humidity environment of the current study area. Prioritizing the reduction of human interference intensity, targeted optimization of food supply in foraging areas, temperature control in resting areas, and noise shielding is recommended.

[0113] In one embodiment of the present invention, when the body contour data in the visual feature dataset shows a lying position, it is determined to be in the brooding period, and the sampling frequency of the infrared thermal imaging device is automatically increased. During the brooding period, the begging calls of chicks in the acoustic feature dataset are extracted and paired with the feeding calls of parent birds for analysis to determine the timing pattern of parent birds returning to their nests. The timing pattern of parent birds returning to their nests is superimposed with the changes in latitude and longitude coordinates in the activity trajectory data to analyze the radius of distance the parent birds travel from the nest to forage. If the radius of distance the parent birds travel from the nest to forage exceeds the safe forage radius set in the ecological database, a brooding risk warning event is generated in the behavior-environment association model. The number of crested ibises identified in the multimodal joint feature vector is periodically counted and the difference is calculated with the data of the same period of the previous year to obtain the change in population size. The latitude and longitude coordinates in the activity trajectory data are analyzed to identify new activity territories of crested ibises, and the geographical coordinates of the new activity areas are written into the ecological database. The system monitors whether new call types appear in the acoustic feature dataset. If a new call type exists, its Mel-frequency cepstral coefficients are recorded in the acoustic signature database of the ecological database. Based on population changes, new activity territories, and new call types, the threshold values ​​of various environmental parameters in the ecological database are adjusted to complete the version iteration of the ecological database.

[0114] In practice, when the body contour data in the visual feature dataset shows the bird lying down, it is determined to be in the juvenile stage. The reduction in body contour data is manifested by the average area enclosed by the isotherm contours of the crested ibis's body surface in multiple consecutive frames of infrared thermal images being less than 80% of the baseline area of ​​the individual's historical non-juvenile stage. The sampling frequency of the infrared thermal imaging equipment is automatically increased from the conventional one frame per minute to one frame per ten seconds. During the juvenile stage, the begging calls of chicks are extracted from the acoustic feature dataset. The chicks' begging calls are identified by the high-frequency energy proportion of their Mel frequency cepstral coefficients and paired with the feeding calls of the parents in the acoustic feature dataset. The pairing analysis, based on the temporal proximity of the calling events and the similarity of the voiceprint features, determines the timing pattern of the parents' return to the nest. The timing pattern of the parents' return to the nest is characterized by the time sequence of the parents' feeding calls within a 24-hour period. By overlaying the timing patterns of parent birds returning to their nests with the changes in latitude and longitude coordinates in their activity trajectory data, the radius of distance the parent birds travel from the nest to forage can be analyzed. This radius is calculated using the following formula:

[0115]

[0116] in: This represents the maximum radius of distance the parent birds can travel from the nest to forage during a single departure event. and This represents the i-th planar projection coordinate in the activity trajectory data of the parent birds during their time away from the nest. and The coordinates represent the planar projection of the nest's center, and the max function represents taking the maximum value among all calculated distances. If the radius of the distance the parent birds travel from the nest to forage exceeds the safe foraging radius set in the ecological database, a parenting risk warning event is generated in the behavior-environment association model. The parenting risk warning event records the exceeded distance radius value, the time of occurrence, and the corresponding parent bird individual number.

[0117] In some embodiments, the number of crested ibis individuals identified in the multimodal joint feature vector is periodically counted, with a statistical period of one calendar month. The population change is calculated by differencing the data from the same period of the previous year using the following formula:

[0118]

[0119] in: This indicates the change in population size. This indicates the number of crested ibis individuals identified within the current statistical period. This indicates the number of crested ibis individuals identified within the same statistical period of the previous year. Analyzing the latitude and longitude coordinates in the activity trajectory data, new activity territories of the crested ibis are identified. The identification method involves spatially clustering all activity trajectory points from the past year, removing historically known habitat ranges, and identifying the remaining clustered areas as the new activity territories. The geographical coordinate boundaries of these new activity territories are then written into the geographic information sub-database of the ecological database. The acoustic feature dataset is monitored for the emergence of new call types. A new call type is defined as a sound feature whose Mel-frequency cepstral coefficient vector has a minimum cosine distance greater than a preset threshold between it and all existing template vectors in the speaker database of the ecological database. If a new call type exists, its Mel-frequency cepstral coefficient vector is added as a new template to the speaker database of the ecological database. Based on population changes, new activity territories, and new call types, the threshold values ​​of various environmental parameters in the ecological database are adjusted, completing the version iteration of the ecological database. Version iteration is reflected in the updating of fields in the environmental parameter threshold table and the incrementing of the version number. Optionally, determining whether body contour data has shrunk requires a duration threshold; for example, the juvenile stage determination is triggered only if the average body contour data over 24 consecutive hours is below 80% of the baseline area. It is understood that increasing the sampling frequency of infrared thermal imaging equipment only applies to equipment deployed near nests already identified as juvenile nests; equipment in other areas maintains its normal sampling frequency. Further analysis of parent birds' homing time patterns includes calculating the average time interval between two consecutive homing visits and the total daily homing frequency. Analysis of the distance radius for foraging away from the nest needs to exclude sudden long-distance movement coordinates caused by avoiding predators or severe weather; these coordinates are filtered out based on abnormal movement speed and trajectory smoothness.

[0120] In some embodiments, population size change Positive values ​​indicate population growth, while negative values ​​indicate population decline; the absolute value reflects the intensity of the change. Identification of new activity territories requires combining behavioral state labels from the multimodal joint feature vector. A new territory is only confirmed as valid if foraging or resting behavior labels are frequently observed in a given area. Monitoring and recording new call types can be understood as an offline batch processing procedure, typically executed monthly, comparing all acoustic feature data collected in the past month with the voiceprint database. Adjusting environmental parameter thresholds in the ecological database includes updating suitable temperature ranges based on climate temperature and humidity data from new activity territories, and adjusting habitat carrying capacity-related parameters based on population change trends. The ecological database version iteration is an automated process; threshold adjustment and version number updates are triggered when any of the following conditions—population change, new activity territory coordinates, or new call type template—meet the update rules.

[0121] See Figure 5 This is a comparative chart of monthly changes in the crested ibis population, used to analyze the monthly trends of the crested ibis population in the current year and the previous year, primarily reflecting the population growth and seasonal rhythm characteristics. In the current year, the population size in each month is higher than in the previous year, with a 51.4% increase at the end of the year compared to the beginning of the year, and a 23.3% increase compared to the same period of the previous year, indicating significant achievements in crested ibis conservation. Both years show a pattern of "spring growth → summer peak → autumn decline," with the peak occurring in August, consistent with the biological characteristics of the crested ibis reaching its highest point of the year after the breeding season (March-July) when young birds leave the nest. The current year's curve is more gently fluctuating, with a higher peak, indicating a significant improvement in population viability and reproductive success rate compared to the previous year. The peak time point is used to confirm the end of the breeding season, optimizing the monitoring cycle.

[0122] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A method for crested ibis identification and ecological monitoring based on multimodal data fusion, characterized in that, The method includes: A miniaturized Beidou satellite positioning tracker is fixedly installed on the target crested ibis to collect the target crested ibis's activity trajectory data at fixed time intervals. The activity trajectory data includes latitude and longitude coordinates, flight altitude and movement speed. The collection interval is shortened during the crested ibis's breeding season and extended during the non-breeding season. Visible light imaging equipment and infrared thermal imaging equipment were deployed in the main habitats and foraging areas of the crested ibis to simultaneously capture the surface texture features, body shape and outline of the crested ibis and the nighttime thermal radiation features, forming a visual feature dataset. A high-sensitivity acoustic sensor array was deployed around the target crested ibis's roosting and breeding grounds. Centered on the target crested ibis's roosting and nests, at least three high-sensitivity acoustic sensors were deployed at different azimuth angles within the radius to form an acoustic triangulation array. The call sound waves of the crested ibis under different behavioral states were collected, and the Mel frequency cepstral coefficients and time-frequency domain energy distribution of the call sound waves were extracted to form an acoustic feature dataset. The activity trajectory data containing latitude and longitude coordinates, the visual feature dataset, and the acoustic feature dataset are input into the multimodal data fusion framework. The three types of features are adaptively weighted and fused through a cross-modal attention mechanism to generate a multimodal joint feature vector containing spatiotemporal context information. The cosine similarity between the surface texture features in the visual feature dataset and the Mel frequency cepstral coefficients in the acoustic feature dataset is calculated separately and used as a correlation measure between modalities. Based on the correlation measure, the fusion weight coefficients of the visual feature dataset, acoustic feature dataset, and activity trajectory data are dynamically adjusted. The system calls a pre-set ecological database, associates and matches the multimodal joint feature vector with the ecological database, establishes a behavior-environment association model, and outputs the crested ibis's foraging preference index, habitat suitability score, and activity path planning scheme.

2. The method for crested ibis identification and ecological monitoring based on multimodal data fusion according to claim 1, characterized in that, The process of fixing a miniaturized BeiDou satellite positioning tracker onto the target crested ibis and collecting its activity trajectory data at fixed time intervals includes: Each miniaturized BeiDou satellite positioning tracker is configured with a unique device identification code, and the device identification code is bound to the individual number of the target crested ibis; The data acquisition cycle of the Beidou satellite positioning tracker is set to a variable cycle, with the acquisition interval shortened during the crested ibis breeding season and extended during the non-breeding season, in order to balance data accuracy and energy consumption. The ground receiving station receives the original positioning message transmitted back by the Beidou satellite positioning tracker and parses out the latitude and longitude coordinates, flight altitude and speed. The parsed latitude and longitude coordinates are subjected to Kalman filtering to remove drift points caused by satellite signal obstruction, generating smoothed activity trajectory data.

3. The method for crested ibis identification and ecological monitoring based on multimodal data fusion according to claim 2, characterized in that, The plan involves deploying visible light imaging and infrared thermal imaging equipment in the crested ibis's habitat and main foraging areas to simultaneously capture the ibis's surface texture features, body shape contours, and nighttime thermal radiation characteristics, forming a visual feature dataset, including: Based on the known activity range of the target crested ibis, a visual monitoring grid covering the entire habitat is drawn, wherein the node spacing of the visual monitoring grid does not exceed the minimum warning distance of the crested ibis; At each node of the visual monitoring grid, the visible light imaging device and the infrared thermal imaging device are activated simultaneously to acquire dual-channel images of the target within the field of view at the same timestamp. The RGB image acquired by the visible light imaging device is segmented to extract the plumage distribution pattern, beak shape, and leg posture of the crested ibis, thereby generating the body surface texture features. The thermal image acquired by the infrared thermal imaging device is segmented by temperature threshold, the isotherm contour of the crested ibis body surface is extracted, and the three-dimensional dimensions of the body contour are calculated by combining the binocular visual ranging principle. Background noise suppression processing is performed on the thermal images collected at night, and the thermal radiation intensity value of the core body temperature region of the crested ibis is extracted to generate the nighttime thermal radiation characteristics; The surface texture features, body shape contours, and nighttime thermal radiation features at the same timestamp are packaged into a single visual feature data set, which is then aggregated to form the visual feature dataset.

4. The method for crested ibis identification and ecological monitoring based on multimodal data fusion according to claim 3, characterized in that, The method involves deploying a high-sensitivity acoustic sensor array around the target crested ibis's roosting and breeding grounds to collect the call sounds of the crested ibis under different behavioral states. The Mel-frequency cepstral coefficients and time-frequency energy distribution of these call sounds are extracted to form an acoustic feature dataset, including: Centered on the target crested ibis's roosting site and nest, at least three of the aforementioned high-sensitivity acoustic sensors are deployed at different azimuth angles within a radius range to form an acoustic triangulation array; When the acoustic triangulation array detects a sound signal with a sound pressure level exceeding a preset threshold, it triggers a synchronous recording mechanism to record the original audio waveform for a duration of time. The original audio waveform is subjected to bandpass filtering to remove ambient background noise and retain the unique vocal frequency band of the crested ibis; The filtered audio waveform is subjected to frame-by-frame windowing processing, and the Mel frequency cepstral coefficients of each frame signal are calculated as static features of the voiceprint. Time-frequency analysis is performed on the filtered audio waveform to calculate its spectral centroid, bandwidth, and zero-crossing rate, generating the time-frequency domain energy distribution as the dynamic feature of the voiceprint. The cepstral coefficients of the Mel frequency corresponding to each valid call are combined with the time-frequency domain energy distribution to generate an acoustic feature data, which is then summarized to form the acoustic feature dataset.

5. The method for crested ibis identification and ecological monitoring based on multimodal data fusion according to claim 4, characterized in that, The process involves inputting the activity trajectory data containing latitude and longitude coordinates, the visual feature dataset, and the acoustic feature dataset into a multimodal data fusion framework. An adaptive weighted fusion of the three types of features is performed through a cross-modal attention mechanism to generate a multimodal joint feature vector containing spatiotemporal context information, including: The latitude and longitude coordinates in the activity trajectory data are analyzed, mapped to a two-dimensional plane coordinate system, and aligned with the timestamps of the visual feature dataset and the acoustic feature dataset to generate a data sequence with a unified spatiotemporal reference. The cosine similarity between the surface texture features in the visual feature dataset and the Mel frequency cepstral coefficients in the acoustic feature dataset is calculated separately and used as a measure of intermodal correlation. Based on the correlation metric, the fusion weight coefficients of the visual feature dataset, the acoustic feature dataset, and the activity trajectory data are dynamically adjusted, with the modalities with high correlation receiving higher weight coefficients. The three types of data are linearly superimposed using the adjusted weight coefficients, and feature transformation is performed using a nonlinear activation function to eliminate the differences in data dimensions between different modalities. The multi-source data after feature transformation are concatenated and the geographical location information indicated by the latitude and longitude coordinates is injected to generate the multimodal joint feature vector containing spatiotemporal context information.

6. The method for crested ibis identification and ecological monitoring based on multimodal data fusion according to claim 5, characterized in that, The step of calling a pre-set ecological database, associating and matching the multimodal joint feature vector with the ecological database, and establishing a behavior-environment association model includes: The ecological database contains data on the crested ibis's food chain distribution, climate temperature and humidity, and data on sources of human disturbance. The real-time behavioral state labels of the crested ibis are decoupled from the multimodal joint feature vector, and the behavioral state labels include foraging, resting, vigilance and flight; Using the behavior status label as the query key, the environmental parameter range corresponding to the behavior status is retrieved from the ecological database, including food source abundance index, ambient temperature threshold and noise decibel threshold; The real-time collected food chain distribution data, climate temperature and humidity data, and human disturbance source data are compared with the environmental parameter range to calculate the environmental fit score. When the environmental fit score is lower than the preset qualified threshold, the dominant feature dimension that leads to the low score in the multimodal joint feature vector is analyzed, and the corresponding environmental factor is marked as the limiting factor in the behavior-environment association model. By traversing all combinations of behavioral states and environmental parameters, a multidimensional mapping table consisting of behavioral states, environmental parameters, and limiting factors is constructed, thus solidifying the behavioral-environment association model.

7. The method for crested ibis identification and ecological monitoring based on multimodal data fusion according to claim 6, characterized in that, The output of the crested ibis's foraging preference index, habitat suitability score, and activity route planning scheme includes: In the behavior-environment association model, the habitat type pointed to by the visual feature dataset of the crested ibis when it is foraging is extracted, and combined with the food chain distribution data in the ecological database, the food availability of the habitat type is calculated to generate the foraging preference index. By combining the activity trajectory data, climate temperature and humidity data, and human disturbance source data recorded in the multimodal joint feature vector, the permanent habitat of the crested ibis is scored. The scoring criteria include activity frequency, temperature and humidity suitability, and disturbance distance, and a habitat suitability score is generated. By tracing the continuous changes in latitude and longitude coordinates in the activity trajectory data, the seasonal migration route of the crested ibis is identified. Combined with the climate temperature and humidity data along the route in the ecological database, the safety of the rest stops on the migration route is assessed, and the activity path planning scheme is generated.

8. The method for crested ibis identification and ecological monitoring based on multimodal data fusion according to claim 7, characterized in that, The step of dynamically adjusting the fusion weight coefficients of the visual feature dataset, the acoustic feature dataset, and the activity trajectory data based on the correlation metric includes: A modal attention score matrix is ​​constructed using the correlation metric, wherein the cosine similarity between the visual feature dataset and the acoustic feature dataset is recorded as the first correlation value, the matching degree between the visual feature dataset and the activity trajectory data under the same spatiotemporal reference is recorded as the second correlation value, and the matching degree between the acoustic feature dataset and the activity trajectory data under the same spatiotemporal reference is recorded as the third correlation value. The first, second, and third association values ​​are input into the Softmax function for normalization to obtain the normalized attention weights between each modality pair. The average of the normalized attention weights of each modality itself and the other two modalities is used as the initial dynamic weight coefficient of the modality. Based on the requirements of the multimodal joint feature vector generation task, preset modal weight biases are set for the behavior recognition task, individual recognition task, and anomaly detection task; The initial dynamic weight coefficients are weighted and summed with the preset modality weight bias corresponding to the current task, and then normalized twice to generate the fusion weight coefficients used for final feature fusion.

9. The method for crested ibis identification and ecological monitoring based on multimodal data fusion according to claim 8, characterized in that, It also includes steps for specifically monitoring the chick-rearing behavior of crested ibises: When the body contour data in the visual feature dataset shows a lying position, it is determined to be in the infancy period, and the sampling frequency of the infrared thermal imaging device is automatically increased; During the brooding period, the begging calls of chicks in the acoustic feature dataset are extracted and paired with the feeding calls of the parent birds to determine the timing pattern of the parent birds returning to their nests. The timing pattern of parent birds returning to their nests is superimposed with the changes in latitude and longitude coordinates in the activity trajectory data to analyze the radius of distance the parent birds travel from their nests to forage. If the radius of the distance by which the parent birds leave the nest to forage exceeds the safe foraging radius set in the ecological database, a parenting risk warning event will be generated in the behavior-environment association model.

10. The method for crested ibis identification and ecological monitoring based on multimodal data fusion according to claim 9, characterized in that, It also includes the step of dynamically updating the ecological database based on historical data: The number of crested ibis individuals identified in the multimodal joint feature vector is periodically counted, and the difference is calculated with the data of the same period of the previous year to obtain the change in population size; By analyzing the latitude and longitude coordinates in the activity trajectory data, new activity areas of the crested ibis are identified, and the geographical coordinates of the new activity areas are written into the ecological database. Monitor whether a new call type appears in the acoustic feature dataset. If a new call type exists, record its Mel frequency cepstral coefficients into the voiceprint library of the ecological database. Based on changes in population size, new activity territories, and new call types, the threshold values ​​of various environmental parameters in the ecological database are adjusted to complete the version iteration of the ecological database.

Images