Tibetan antelope behavior analysis system based on multimodal image fusion and optimized transmission method

Abstract

The present invention discloses a Tibetan antelope behavior analysis system and an optimized transmission method based on multimodal image fusion, which belongs to the field of image fusion technology. By synchronously collecting infrared images and visible light images, and performing alignment, denoising and normalization processing on them, the temperature texture collaborative anomaly index and the spatial dynamic anomaly index are extracted, a polynomial regression model for identifying non-target heat sources is constructed, and an analysis value of high-temperature non-target objects is output; the system divides the high-temperature area in the infrared image into credible targets, uncertainty and interference levels, and calculates the fusion interference risk value for the uncertain area based on the response intensity and weight distribution of the image fusion algorithm; finally, Tibetan antelope behavior recognition is performed based on the fused image, and the recognition confidence is dynamically adjusted in combination with the interference risk, so as to achieve high-precision and interference-resistant recognition of the true behavior of the Tibetan antelope; the invention improves the behavior recognition accuracy and the intelligent level of ecological monitoring in multimodal image fusion under complex environments.

Description

Technical Field

[0001] The present invention relates to the technical field of image fusion, and in particular to a Tibetan antelope behavior analysis system and an optimized transmission method based on multimodal image fusion. Background Art

[0002] Tibetan antelope behavioral analysis involves the systematic study and analysis of their behavioral characteristics and activity patterns in their natural environment. This analysis typically encompasses their migration routes, foraging habits, reproductive behavior, social structure, and responses to external disturbances (such as human activity or climate change). The goal is to uncover the ecological adaptation mechanisms and survival strategies of the Tibetan antelope, providing a scientific basis for species conservation and ecological management.

[0003] The existing technology has the following shortcomings:

[0004] During multimodal image fusion, non-target high-temperature objects often appear in infrared images, such as bare rocks heated by sunlight on a summer afternoon. These areas appear bright in thermal images and can easily be misidentified as target animals such as Tibetan antelopes by fusion algorithms (especially attention-based models), leading to misdirected fusion weight allocation. As a result, the true animal features may be weakened, obscured, or even completely lost in the fused image, seriously interfering with the accuracy of behavior recognition and monitoring. Summary of the Invention

[0005] The purpose of the present invention is to provide a Tibetan antelope behavior analysis system based on multimodal image fusion and an optimized transmission method to address the shortcomings of the background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a Tibetan antelope behavior analysis system based on multimodal image fusion, comprising an image acquisition module, an image preprocessing module, a feature extraction module, an interference analysis model construction module, and a classification module:

[0007] Image acquisition module, synchronously collecting multimodal image data including infrared images and visible light images;

[0008] An image preprocessing module, for preprocessing the multimodal image data;

[0009] A feature extraction module extracts temperature texture collaborative features and spatial dynamic anomaly features from the infrared image;

[0010] An interference analysis model construction module trains a machine learning model based on the temperature texture collaborative features and spatial dynamic anomaly features, and outputs a high-temperature non-target object analysis value corresponding to each high-temperature area;

[0011] The grading module grades and marks the high-temperature area in the infrared image according to the analysis value of the high-temperature non-target object, divides it into a credible target level, an uncertain level and an interference target level, and performs corresponding processing.

[0012] Preferably, the temperature texture synergy anomaly index is calculated after analyzing the extracted temperature texture synergy features, including: Extract the high temperature area and set the threshold as , and obtain the preliminary target mask: ; For infrared images Extract fine-grained edge / texture features: ; PST is an edge detection method implemented by Fourier domain convolution and nonlinear phase operation. represents the scale control parameter in PST, represents the phase nonlinear control parameter in PST, Represents the pixel value of the image at position (x, y) after PST processing, and constructs the temperature texture collaborative feature map , which represents the coupling relationship between thermal intensity and texture response, is defined as: ; Represents the pixel value of the infrared image after normalization; Represents the texture response image after PST transformation; in the high temperature mask area Up, right Find the statistical mean and standard deviation , the temperature texture synergy anomaly index is defined as the discrete degree and intensity coupling of the texture temperature synergy feature, and the expression is: Where, Represents the temperature texture synergy anomaly index.

[0013] Preferably, the spatial dynamic anomaly index is generated after analyzing the extracted spatial dynamic anomaly features, specifically including: Extract the high temperature area and obtain the preliminary target mask , a total of T frames; Indicates thermal mask, which is based on infrared image The pixel temperature exceeds the set threshold Extract the binary image; select a spatial location as candidate areas;

[0014] For the pixels in region R, the structural similarity value is calculated between each pair of consecutive frames (It, It+1), and a set of temporal structural similarity values ​​is generated to calculate the change in SSIM between adjacent frames. , the expression is: Where, represents the structural similarity value between frames t and t+1 in region R, It represents the structural similarity value between frames t+1 and t+2 in region R; T represents the number of frames in the infrared image sequence. The spatial dynamic anomaly index is defined by the combination of SSIM volatility and mean inverse ratio, and the expression is: Where, represents the average of all SSIM values, and SDI represents the spatial dynamic anomaly index.

[0015] Preferably, the interference analysis model construction module specifically includes:

[0016] The temperature texture synergistic anomaly index and the spatial dynamic anomaly index are converted into a comprehensive feature vector, and the comprehensive feature vector is used as the input of the machine learning model;

[0017] The machine learning model uses each set of comprehensive feature vectors to predict the high-temperature non-target object analysis value label corresponding to each high-temperature area as the prediction target, and minimizes the sum of the prediction errors of the high-temperature non-target object analysis value labels corresponding to all high-temperature areas as the training target. The machine learning model is trained until the sum of the prediction errors reaches convergence and the model training is stopped.

[0018] The high-temperature non-target object analysis value corresponding to each high-temperature area is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

[0019] Preferably, the level classification module specifically includes:

[0020] Comparing the obtained high-temperature non-target object analysis value with a gradient threshold, the gradient threshold including a first threshold and a second threshold, wherein the first threshold is less than the second threshold, and comparing the high-temperature non-target object analysis value with the first threshold and the second threshold respectively;

[0021] If the high-temperature non-target object analysis value is greater than the second threshold, it is determined to be an interference target level; the area is downgraded in the fused image;

[0022] If the high-temperature non-target object analysis value is greater than or equal to the first threshold and less than or equal to the second threshold, it is determined to be an uncertain level; the area is marked as an area requiring further evaluation;

[0023] If the analysis value of the high-temperature non-target object is less than the first threshold, it is judged as a credible target level, and a high fusion weight is assigned to the area, which directly participates in the subsequent Tibetan antelope behavior analysis and processing.

[0024] The present invention also provides a multimodal image fusion Tibetan antelope behavior analysis optimization transmission method, comprising:

[0025] Collecting multimodal image data including infrared images and visible light images, and preprocessing the multimodal image data;

[0026] extracting temperature texture collaborative features and spatial dynamic anomaly features from the infrared image;

[0027] A machine learning model is trained based on the temperature texture collaborative features and the spatial dynamic anomaly features to output the high-temperature non-target object analysis value corresponding to each high-temperature area;

[0028] According to the high-temperature non-target object analysis value, the high-temperature area in the infrared image is graded and marked, divided into a credible target level, an uncertain level, and an interference target level, and processed accordingly;

[0029] For high-temperature areas in the uncertainty level, the response strength and weight distribution of the fusion algorithm are combined during the image fusion process to analyze the degree of interference with the target feature expression in the fused image and output the fusion interference risk value;

[0030] Combined with the fused interference risk value, optimized transmission and intelligent scheduling are performed for high temperature areas of uncertain levels.

[0031] Preferably, for high temperature areas in the uncertainty level, during the image fusion process, the intermediate response map information is extracted: the key feature map of the image before fusion ; Feature map of the target area of ​​the fused image ; Extract regions from fusion algorithm The fusion weight value In the area Calculate the degree of change of the key feature map before and after fusion , the expression is: Where, Indicates the number of pixels in the area; the overall degree of distortion and regional fusion weights , define the fusion interference risk value , the expression is: .

[0032] Preferably, the optimized transmission and intelligent scheduling processing of image data are performed in combination with the fused interference risk value, specifically including:

[0033] Compare the fused interference risk value of the high temperature area of ​​each uncertainty level with the preset scheduling threshold;

[0034] If the fused interference risk value is lower than the first scheduling threshold, the data of the corresponding image area is marked as low priority and transmitted later or compressed;

[0035] If the fusion interference risk value is between the first scheduling threshold and the second scheduling threshold, the region is marked as medium priority and enters the asynchronous fusion cache queue;

[0036] If the fusion interference risk value is higher than the second scheduling threshold, the area is marked as high priority, immediately transmitted to the central analysis end, and the enhanced fusion mode is enabled for focused processing.

[0037] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0038] 1. The multimodal image fusion Tibetan antelope behavior analysis system and optimized transmission method provided by the present invention address the problem that non-target high-temperature interference objects in infrared images are easily misjudged. It innovatively proposes a dual-feature modeling method of temperature texture collaborative anomaly index and spatial dynamic anomaly index, combines a polynomial regression model to achieve quantitative analysis of the interference risk in high-temperature areas, and accurately divides credible, interference and uncertain areas through a hierarchical labeling mechanism, effectively improving the fidelity of target animal features in the fused image and the accuracy of behavior recognition.

[0039] 2. This approach guides optimized transmission and intelligent scheduling in uncertain areas by integrating interference risk values. Dynamically assigning data priorities within limited bandwidth or edge computing power ensures prioritized fusion and enhancement in key areas, improving the system's resource scheduling efficiency and environmental adaptability. This overall solution not only enhances the robustness of image fusion and target recognition, but also enhances the system's practicality and intelligence in complex field ecological monitoring scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0041] Figure 1 This is a mind map of the system modules of the present invention.

[0042] Figure 2 This is a mind map of the method of the present invention. DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0044] Example 1, please refer to Figure 1 As shown, the Tibetan antelope behavior analysis system based on multimodal image fusion described in this embodiment includes an image acquisition module, an image preprocessing module, a feature extraction module, an interference analysis model construction module, and a level classification module:

[0045] Image acquisition module, synchronously collecting multimodal image data including infrared images and visible light images;

[0046] An image preprocessing module, for preprocessing the multimodal image data;

[0047] A feature extraction module extracts temperature texture collaborative features and spatial dynamic anomaly features from the infrared image;

[0048] An interference analysis model construction module trains a machine learning model based on the temperature texture collaborative features and spatial dynamic anomaly features, and outputs a high-temperature non-target object analysis value corresponding to each high-temperature area;

[0049] The grading module grades and marks the high-temperature area in the infrared image according to the analysis value of the high-temperature non-target object, divides it into a credible target level, an uncertain level and an interference target level, and performs corresponding processing.

[0050] The image acquisition module is used to synchronously acquire multi-channel image information of the target area in a natural field environment, specifically including:

[0051] Multimodal sensor deployment: The module includes infrared thermal imaging sensors and visible light image sensors installed at fixed observation points or unmanned monitoring equipment, and can optionally further include depth cameras, night vision enhancers, or radar sensors to enhance imaging capabilities in low light and complex terrain.

[0052] Time synchronization control mechanism: The acquisition of infrared and visible light images is controlled by a unified timestamp and uses a high-precision synchronization signal trigger mechanism to ensure that images of different modalities are acquired in the same time window, thereby reducing the impact of timing deviation on subsequent registration and fusion accuracy;

[0053] Spatial perspective consistency design: Various sensors are installed in fixed relative positions and uniform orientations. Through calibration, the extrinsic parameter matrix between sensors is obtained to achieve spatial perspective consistency between images, providing a geometric basis for subsequent image registration.

[0054] Environmental adaptability: The acquisition module is equipped with automatic exposure control, anti-reflection filters and temperature compensation devices to cope with extreme natural conditions such as strong light on the plateau, low temperatures, high wind speeds, etc., ensuring the stability of image quality;

[0055] Data caching and uploading mechanism: The collected multimodal image data is temporarily stored in the local cache unit and uploaded to the back-end processing system through the wireless communication module (such as LoRa, 4G / 5G, satellite link) under network conditions.

[0056] The image preprocessing module is used to perform structured processing on the multimodal image data acquired by the image acquisition module to improve the effectiveness of subsequent fusion and recognition, specifically including:

[0057] Aligning images of different modalities (e.g., infrared and visible light images) in the spatial dimension so that corresponding pixels have the same geographic or target meaning. The registration process includes:

[0058] Geometric transformation calculation: Calculate the affine transformation matrix or projection matrix based on sensor calibration parameters or key points extracted from the scene (such as SIFT, SURF, ORB);

[0059] Intermodal feature matching: Multimodal alignment algorithms, such as image registration based on mutual information, are used to match image pairs with significant information differences, such as infrared and visible light.

[0060] Deep learning-assisted registration: Optional use of neural network-based end-to-end image registration models (such as RegNet or Deep Registration CNN) to improve registration accuracy, especially for images without obvious texture areas.

[0061] Eliminate noise signals generated during the acquisition process and improve image clarity and edge feature expression, including:

[0062] Spatial filtering methods: such as median filtering, Gaussian filtering, bilateral filtering, etc., used to remove random noise;

[0063] Frequency domain denoising methods: such as wavelet transform denoising, which is suitable for retaining image structural information while suppressing high-frequency noise;

[0064] Modal Adaptive Strategy: To address the problem of low contrast between high-heat areas and background in infrared images, the denoising strength is dynamically adjusted to retain key thermal feature areas.

[0065] Unifying the differences in numerical range, dynamic range, and scale expression of multimodal images for fusion processing. The normalization processing includes:

[0066] Grayscale normalization: normalize the pixel values ​​of different modal images to a unified grayscale range (such as [0,1] or [0,255]);

[0067] Thermal intensity normalization: Perform contrast stretching or histogram equalization on the thermal values ​​of infrared images to enhance the contrast of high-heat areas;

[0068] Unified spatial resolution: Image resampling methods (such as bilinear interpolation and nearest neighbor interpolation) are used to ensure that images of different modalities have consistent resolution scales.

[0069] Structural consistency alignment: Edge-preserving normalization methods (such as the Retinex algorithm) are used to ensure that the images have good structural perception characteristics before fusion.

[0070] Through the above preprocessing steps, the image preprocessing module achieves high consistency in spatial position, feature expression and data scale of images of different modalities, providing a standardized and comparable input basis for subsequent feature extraction, fusion and recognition.

[0071] In the feature extraction module, the Temperature Texture Coordination (TTC) feature is extracted to address the potential target recognition errors caused by relying solely on thermal intensity in traditional infrared images, especially when non-animal high-temperature objects (such as sunlit rocks) are present. By simultaneously considering the infrared intensity distribution of high-temperature areas and their texture edge structure in the image, TTC can effectively distinguish between true target areas with biological structural characteristics (such as clear outlines and closed edges) and non-target heat source areas with irregular textures and scattered edges. This improves the recognition accuracy of high-temperature interference targets and provides structural discrimination basis for subsequent fusion judgment.

[0072] The extracted temperature texture synergy features are analyzed and the temperature texture synergy anomaly index is calculated, including:

[0073] From infrared images Extract the high temperature area and set the threshold as , and obtain the preliminary target mask: ; For infrared images Extract fine-grained edge / texture features: ;PST is an edge detection method implemented by Fourier domain convolution and nonlinear phase operation, which can enhance weak texture edges and is suitable for low-contrast infrared images; Represents the scale control parameter in PST, which adjusts the width of the filter in the frequency domain and affects the accuracy of detail extraction; smaller values ​​are used to enhance subtle edges; Represents the phase nonlinear control parameter in PST, which controls the nonlinear degree of the enhancement effect and determines the amplitude and response mode of the edge response; It represents the pixel value at position (x, y) of the image after PST processing, reflecting the texture / edge response strength at that location.

[0074] Constructing temperature-texture collaborative feature maps , which represents the coupling relationship between thermal intensity and texture response, is defined as: ; Represents the pixel value of the infrared image after normalization (such as linear normalization to [0, 1]), which is used to unify the numerical scale of thermal intensity; Represents the texture response image after PST transformation, and the pixel value at (x, y) after normalization. The two images are first normalized by standard processing (for example, 0-1 normalization) to ensure that the numerical scale is consistent.

[0075] In the high temperature mask area Up, right Find the statistical mean and standard deviation The temperature texture synergy anomaly index is defined as the discrete degree and intensity coupling of the texture temperature synergy feature, and the expression is: Where, Represents the temperature texture synergy anomaly index.

[0076] If the TTCAI is too high, it means that the texture of the high-temperature area is inconsistent and the distribution is abnormal, which may be a non-animal target (such as rock); if the TTCAI is too low, it means that the boundary of the heat source area is clear and consistent with the heat intensity, and it has typical animal target characteristics.

[0077] The goal of extracting spatial dynamic motion features (SDM) is to leverage the dynamic changes in image frames within a time series to model the spatial behavior patterns of high-temperature areas and identify whether they possess the typical motion characteristics of biological targets. Because animals like Tibetan antelopes typically exhibit directional, continuous, and deformable motion trajectories in image sequences, while static heat sources (such as rocks or ground hot spots) maintain relatively stable form and position, SDM serves as a key discriminant feature in the dynamic dimension, assisting in identifying true target areas with distinct motion consistency while suppressing the weighting influence of static interference areas during the fusion process.

[0078] After analyzing the extracted spatial dynamic anomaly features, a spatial dynamic anomaly index is generated, which specifically includes:

[0079] From infrared images Extract the high temperature area and obtain the preliminary target mask , a total of T frames; Indicates thermal mask, which is based on infrared image The pixel temperature exceeds the set threshold The extracted binary image is used to mark the high temperature area. Select a spatial location as a candidate region (such as a sliding window or a target region).

[0080] For the pixels in region R, the structural similarity value is calculated between each pair of consecutive frames (It, It+1), and a set of temporal structural similarity values ​​is generated to calculate the change in SSIM between adjacent frames. , the expression is: Where, represents the structural similarity value between frames t and t+1 in region R, It represents the structural similarity value between frames t+1 and t+2 in region R; T represents the number of frames in the infrared image sequence. The spatial dynamic anomaly index is defined by the combination of SSIM volatility and mean inverse ratio, and the expression is: Where, It represents the average of all SSIM values, indicating the degree of overall structural preservation, and SDI represents the spatial dynamic anomaly index.

[0081] High SDI (e.g. > 0.5): The regional structure varies greatly over time and is highly dynamic → it may be a target such as the Tibetan antelope; low SDI (close to 0): The structure remains stable → it may be a stationary heat source such as rock.

[0082] The interference analysis model construction module trains a machine learning model based on the temperature texture collaborative features and spatial dynamic anomaly features, and outputs the high-temperature non-target object analysis value corresponding to each high-temperature area, specifically including:

[0083] The temperature texture synergistic anomaly index and the spatial dynamic anomaly index are converted into a comprehensive feature vector, and the comprehensive feature vector is used as the input of the machine learning model;

[0084] The machine learning model uses each set of comprehensive feature vectors to predict the high-temperature non-target object analysis value label corresponding to each high-temperature area as the prediction target, and minimizes the sum of the prediction errors of the high-temperature non-target object analysis value labels corresponding to all high-temperature areas as the training target. The machine learning model is trained until the sum of the prediction errors reaches convergence and the model training is stopped.

[0085] The high-temperature non-target object analysis value corresponding to each high-temperature area is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

[0086] In practical applications, the trained polynomial regression model is used to predict the comprehensive feature vector of the high-temperature area in the new input image, and the high-temperature non-target object analysis value corresponding to each high-temperature area is output. This value is used for subsequent high-temperature area classification and interference risk assessment.

[0087] The classification module classifies the high-temperature area in the infrared image into a credible target level, an uncertain level, and an interference target level according to the high-temperature non-target object analysis value, and performs corresponding processing, specifically including:

[0088] Comparing the obtained high-temperature non-target object analysis value with a gradient threshold, the gradient threshold including a first threshold and a second threshold, wherein the first threshold is less than the second threshold, and comparing the high-temperature non-target object analysis value with the first threshold and the second threshold respectively;

[0089] If the high-temperature non-target object analysis value is greater than the second threshold, it is determined to be an interference target level, indicating that the area is highly likely to be a non-animal heat source; this area is downgraded in the fused image or excluded from the fusion feature extraction to avoid interference with the fusion decision;

[0090] If the high-temperature non-target object analysis value is greater than or equal to the first threshold and less than or equal to the second threshold, it is judged as uncertain level, indicating that the area has a certain interference risk but the characteristics are not fully significant;

[0091] Mark the area as requiring further assessment;

[0092] If the analysis value of the high-temperature non-target object is less than the first threshold, it is judged to be a credible target level, indicating that the area is highly likely to be a real biological target; a higher fusion weight is given to the area, and it directly participates in the subsequent Tibetan antelope behavior analysis and processing process.

[0093] Example 2, please refer to Figure 2 As shown, the Tibetan antelope behavior analysis and optimization transmission method based on multimodal image fusion described in this embodiment includes:

[0094] Collecting multimodal image data including infrared images and visible light images, and preprocessing the multimodal image data;

[0095] extracting temperature texture collaborative features and spatial dynamic anomaly features from the infrared image;

[0096] A machine learning model is trained based on the temperature texture collaborative features and the spatial dynamic anomaly features to output the high-temperature non-target object analysis value corresponding to each high-temperature area;

[0097] According to the high-temperature non-target object analysis value, the high-temperature area in the infrared image is graded and marked, divided into a credible target level, an uncertain level, and an interference target level, and processed accordingly;

[0098] For high-temperature areas in the uncertainty level, the response strength and weight distribution of the fusion algorithm are combined during the image fusion process to analyze the degree of interference with the target feature expression in the fused image and output the fusion interference risk value;

[0099] Combined with the fused interference risk value, optimized transmission and intelligent scheduling are performed for high temperature areas of uncertain levels.

[0100] Among them, for high temperature areas in the uncertainty level, during the image fusion process (such as deep fusion network or attention mechanism fusion), the intermediate response map information is extracted:

[0101] Key feature map of the image before fusion , such as edge maps, texture direction maps;

[0102] Feature map of the target area of ​​the fused image , especially with high temperature areas The parts where the spatial positions overlap.

[0103] Extracting regions from the fusion algorithm The fusion weight value , which can be derived from: attention weight map; fusion feature map channel response intensity; pixel weighted average / transformation weight value.

[0104] In the area Calculate the degree of change of the key feature map before and after fusion (feature distortion), the expression is: Where, Indicates the number of pixels in the region; the difference can be measured using L1 norm, structural similarity descent, gradient direction change, etc. Comprehensive distortion degree and regional fusion weights , define the fusion interference risk value , the expression is: .

[0105] Receive the output of the fused interference risk value (FIR) of each uncertainty level high temperature area;

[0106] The system presets two levels of scheduling thresholds, namely the first scheduling threshold TL and the second scheduling threshold TH. <TH;

[0107] The FIR of each high-temperature area is compared with TL and TH, and divided into the following three scheduling levels:

[0108] Low priority (FIR < TL);

[0109] Medium priority (TL≤ FIR≤T_H);

[0110] High priority (FIR > TH).

[0111] Low-priority processing: Data in the corresponding area is marked as a delayed transmission area and can be compressed, sampled at a low frequency, or uploaded when network resources are idle; the system can use the edge caching mechanism to temporarily store such data to reduce resource usage.

[0112] Medium priority processing: The regional image data enters the fusion asynchronous processing queue, and the system dynamically allocates the processing time according to the fusion model load in the main thread or batch scheduling; at the same time, the fusion parameters are retained for reuse in subsequent fusion model adjustments.

[0113] High-priority processing: Regional data is immediately marked as key image blocks and uploaded to the central server or high-performance node first; the system triggers enhanced fusion mode, which may include the use of higher-precision image registration algorithms, stronger feature extraction models, or explicit fusion enhancement strategies (such as weighting strategies and local reconstruction) to reduce interference effects.

[0114] Based on historical scheduling results, the values ​​of TL and TH are dynamically adjusted through reinforcement learning or sliding window statistical models to adapt to the system response requirements under different climate, light or target density conditions, thereby achieving adaptive optimization of the scheduling strategy.

[0115] By implementing risk-aware scheduling management for uncertain areas before image fusion, the optimization scheduling module achieves the goals of resource priority allocation, dynamic adjustment of fusion processing, and self-optimization of system performance, significantly improving the continuous monitoring efficiency and recognition accuracy of the behavior of targets such as Tibetan antelopes in complex ecological environments.

[0116] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0117] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0118] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0119] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A multimodal image fusion Tibetan antelope behavior analysis system, characterized by: It includes image acquisition module, image preprocessing module, feature extraction module, interference analysis model building module and level classification module: Image acquisition module, synchronously collecting multimodal image data including infrared images and visible light images; An image preprocessing module, for preprocessing the multimodal image data; A feature extraction module extracts temperature texture collaborative features and spatial dynamic anomaly features from the infrared image; The temperature texture synergy anomaly index is calculated after analyzing the extracted temperature texture synergy features, including: Extract the high temperature area and set the threshold as , and obtain the preliminary target mask: ; For infrared images Extract fine-grained edge or texture features: ; PST is an edge detection method implemented by Fourier domain convolution and nonlinear phase operation. represents the scale control parameter in PST, represents the phase nonlinear control parameter in PST, Represents the pixel value of the image at position (x, y) after PST processing, and constructs the temperature texture collaborative feature map , which represents the coupling relationship between thermal intensity and texture response, is defined as: ; Represents the pixel value of the infrared image after normalization; Represents the texture response image after PST transformation; in the preliminary target mask Up, right Find the statistical mean and standard deviation , the temperature texture synergy anomaly index is defined as the discrete degree and intensity coupling of the texture temperature synergy feature, and the expression is: Where, represents the temperature texture synergy anomaly index; The spatial dynamic anomaly index is generated after analyzing the extracted spatial dynamic anomaly features, including: Extract the high temperature area and obtain the preliminary target mask , a total of T frames; select a spatial position as candidate areas; For pixels in region R, in each pair of consecutive frames Calculate the structural similarity value between the two frames and generate a set of temporal structural similarity values ​​to calculate the change in the SSIM of the structural similarity values ​​of adjacent frames. , the expression is: Where, represents the structural similarity value between frames t and t+1 in region R, It represents the structural similarity value in region R between frames t+1 and t+2; T represents the number of frames in the infrared image sequence, and the change in the structural similarity value SSIM of adjacent frames is used. The combined indicator inversely proportional to the mean defines the spatial dynamic anomaly index; An interference analysis model construction module trains a machine learning model based on the temperature texture collaborative features and spatial dynamic anomaly features, and outputs a high-temperature non-target object analysis value corresponding to each high-temperature area; The grading module grades and marks the high-temperature area in the infrared image according to the analysis value of the high-temperature non-target object, divides it into a credible target level, an uncertain level and an interference target level, and performs corresponding processing.

2. The multimodal image fusion Tibetan antelope behavior analysis system according to claim 1, characterized in that: Interference analysis model building module, including: The temperature texture synergistic anomaly index and the spatial dynamic anomaly index are converted into a comprehensive feature vector, and the comprehensive feature vector is used as the input of the machine learning model; The machine learning model uses each set of comprehensive feature vectors to predict the high-temperature non-target object analysis value label corresponding to each high-temperature area as the prediction target, and minimizes the sum of the prediction errors of the high-temperature non-target object analysis value labels corresponding to all high-temperature areas as the training target. The machine learning model is trained until the sum of the prediction errors reaches convergence and the model training is stopped. The high-temperature non-target object analysis value corresponding to each high-temperature area is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

3. The multimodal image fusion Tibetan antelope behavior analysis system according to claim 2, characterized in that: The level classification module includes: Comparing the obtained high-temperature non-target object analysis value with a gradient threshold, the gradient threshold including a first threshold and a second threshold, wherein the first threshold is less than the second threshold, and comparing the high-temperature non-target object analysis value with the first threshold and the second threshold respectively; If the high-temperature non-target object analysis value is greater than the second threshold, it is determined to be an interference target level; the area is downgraded in the fused image; If the high-temperature non-target object analysis value is greater than or equal to the first threshold and less than or equal to the second threshold, it is determined to be an uncertain level; the area is marked as an area requiring further evaluation; If the analysis value of the high-temperature non-target object is less than the first threshold, it is judged as a credible target level, and a high fusion weight is assigned to the area, which directly participates in the subsequent Tibetan antelope behavior analysis and processing.

4. A multimodal image fusion-based Tibetan antelope behavior analysis optimization transmission method, used to implement the multimodal image fusion-based Tibetan antelope behavior analysis system described in any one of claims 1-3, characterized in that: include: Collecting multimodal image data including infrared images and visible light images, and preprocessing the multimodal image data; extracting temperature texture collaborative features and spatial dynamic anomaly features from the infrared image; A machine learning model is trained based on the temperature texture collaborative features and the spatial dynamic anomaly features to output the high-temperature non-target object analysis value corresponding to each high-temperature area; According to the high-temperature non-target object analysis value, the high-temperature area in the infrared image is graded and marked, divided into a credible target level, an uncertain level, and an interference target level, and processed accordingly; For high-temperature areas in the uncertainty level, the response strength and weight distribution of the fusion algorithm are combined during the image fusion process to analyze the degree of interference with the target feature expression in the fused image and output the fusion interference risk value; Combined with the fused interference risk value, optimized transmission and intelligent scheduling are performed for high temperature areas of uncertain levels.

5. The multimodal image fusion-based Tibetan antelope behavior analysis and optimization transmission method according to claim 4 is characterized by: in, For high temperature areas in the uncertainty level, during the image fusion process, the intermediate response map information is extracted: the key feature map of the image before fusion ; Feature map of the target area of ​​the fused image ; Extract regions from fusion algorithm The fusion weight value In the area Calculate the degree of change of the key feature map before and after fusion , the expression is: Where, Indicates the number of pixels in the area; the overall degree of distortion and regional fusion weights , define the fusion interference risk value , the expression is: .

6. The Tibetan antelope behavior analysis and optimization transmission method based on multimodal image fusion according to claim 5 is characterized by: Combined with the fused interference risk value, optimized transmission and intelligent scheduling of image data are performed, including: Compare the fused interference risk value of the high temperature area of ​​each uncertainty level with the preset scheduling threshold; If the fused interference risk value is lower than the first scheduling threshold, the data of the corresponding image area is marked as low priority and transmitted later or compressed; If the fusion interference risk value is between the first scheduling threshold and the second scheduling threshold, the region is marked as medium priority and enters the asynchronous fusion cache queue; If the fusion interference risk value is higher than the second scheduling threshold, the area is marked as high priority, immediately transmitted to the central analysis end, and the enhanced fusion mode is enabled for focused processing.

Images