A method for identifying arboreal primates based on thermal infrared and visible light image fusion

Abstract

The application discloses a method for identifying arboreal primates based on thermal infrared and visible light image fusion, and relates to the technical field of target detection, which comprises the following steps: collecting visible light images and thermal infrared images in a target sample plot synchronously; performing image processing on the visible light images and the thermal infrared images; extracting thermal infrared features in the thermal infrared images and biological features in the visible light images; performing a three-layer matching mechanism of coarse matching, static matching and biological constraint based on the extracted thermal infrared features and biological features; performing image fusion on the visible light images and the thermal infrared images; performing model training and optimization; and outputting a species identification result. The method is adaptive to infrared radiation correction enhancement of the complex habitat environment of arboreal primates in high-altitude areas, and can effectively exclude the interference of environmental conditions and improve the identification precision by fusing thermal infrared features and visible light features, performing three-layer matching of cross-modal features and modeling, thereby providing key technical support for physiological monitoring of arboreal primates.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and more specifically, to a method for identifying arboreal primates based on the fusion of thermal infrared and visible light images. Background Technology

[0002] Long-term observation of animal populations is an important scientific research method for maintaining species balance and ecological stability. Existing animal identification methods generally construct end-to-end deep neural networks based on classification or target detection tasks, achieving species identification through sample labeling and model training. For example, a method and system for animal identification based on domain transfer using nighttime infrared images (202311810982.6) acquires visible light and infrared image data, dividing them into training and test sets; establishes a recurrent generative adversarial network (RGAN) model for domain transfer; trains the RGAN model on the training set to obtain a domain transfer network; trains a target detection model on the training set to obtain an image recognition network; inputs infrared image data from the test set into the domain transfer network to obtain transferred visible light image data, and then inputs the transferred visible light image data into the image recognition network. The network ultimately obtains the recognition results; for example, a wildlife species recognition method based on incremental learning (202411671680.X) constructs a detection dataset of known species categories, trains a detection model for known species categories, selects representative samples of each species category, and saves the expression features of each species category; constructs a species discrimination dataset and trains a species discrimination model; manually verifies new species sample data; constructs a species detection fine-tuning dataset, updates the detection model of known species categories based on a feature-based incremental strategy, and simultaneously updates the species discrimination model based on a feature-level sample replay incremental strategy. The recognition system includes a data acquisition module, a transfer module, and a recognition module.

[0003] For arboreal primates in high-altitude areas, differences in habitat and living habits lead to the following problems in species identification: 1) interference from canopy layers such as lichen and swaying branches; 2) the influence of differences in meteorological conditions such as light and seasons; 3) physiological differences such as age and sex affect the accuracy of individual identification, with lower accuracy for juvenile and older individuals; 4) the influence of posture changes; 5) regional differences lead to a decrease in model accuracy; 6) interference from pseudo-heat sources caused by temperature differences in high-altitude environments. Summary of the Invention

[0004] The purpose of this invention is to provide a method for identifying arboreal primates based on the fusion of thermal infrared and visible light images. This method can eliminate the interference of complex environmental conditions and combine the advantages of thermal infrared and visible light imaging to accurately and effectively identify the species and individuals of arboreal primates.

[0005] The embodiments of the present invention are implemented as follows: A method for identifying arboreal primates based on the fusion of thermal infrared and visible light images, comprising: S1. Simultaneously acquire visible light and thermal infrared images at the target sample site; S2. Process the acquired thermal infrared and visible light images; S3. Extract the thermal infrared features from the thermal infrared image and the biological features from the visible light image; S4. Based on the extracted thermal infrared features and biological features, a three-layer matching mechanism of coarse matching, static matching, and biological constraints is performed, and image fusion is then performed; S5. Train and optimize the model, and output the species identification results.

[0006] Furthermore, in other preferred embodiments of the present invention, the acquired thermal infrared image is radiometrically calibrated to convert the grayscale values ​​in the thermal infrared image into radiance values. The formula for radiometric calibration is as follows: , In the formula, This represents the radiance value corresponding to the thermal infrared data wavelength λ. This is the gain quantity. This is the offset. This represents the gain coefficient for the body temperature region. This is the offset.

[0007] Furthermore, in other preferred embodiments of the present invention, pixels in the thermal infrared image... The radiance values ​​were continuously collected and a time series was constructed. ,in, The radiance value of the pixel in the nth frame is given. A static threshold and a dynamic threshold are set. When the change in the radiance value does not exceed the static threshold, median filtering is performed. When the change in the radiance value is not lower than the dynamic threshold, the original value is retained.

[0008] Furthermore, in other preferred embodiments of the present invention, in step S2, the radiance value is corrected according to the following formula: , In the formula, Radiance, These are sensor observations. For path radiation, Atmospheric transmittance. For the hair emissivity of arboreal primates; Furthermore, the altitude was divided into layers every 500 meters, and the temperature, air pressure, and water vapor content of each layer were analyzed using the MODTRAN model. and Make corrections.

[0009] Furthermore, in other preferred embodiments of the present invention, the thermal infrared features include temperature statistical features and thermal texture features, wherein the temperature statistical features include the average temperature within the ROI. Standard deviation Maximum temperature With the lowest temperature temperature difference Thermal texture features include energy, entropy, and contrast extracted based on the gray-level co-occurrence matrix.

[0010] Furthermore, in other preferred embodiments of the present invention, the thermal infrared features further include thermal LBP features, and the method for extracting thermal LBP features is as follows: In the thermal infrared image, the temperature of the center pixel is used as a threshold, and the temperatures of 3×3 neighboring pixels are compared to generate the thermal LBP value of the center pixel; the thermal LBP feature is obtained by statistically analyzing the histogram of the thermal LBP values ​​of all pixels in the thermal infrared image.

[0011] Furthermore, in other preferred embodiments of the present invention, the visible light features include facial morphological features and body posture features. The facial morphological features include the localization and detection of 10-16 facial feature points, and the body posture features include constructing a skeleton model containing 18-25 joints, and optimizing the loss function using an improved OpenPose algorithm. , .

[0012] Furthermore, in other preferred embodiments of the present invention, the visible light features also include color features, texture features, and geometric features of the ROI; the color features include constructing an HSV color gamut quantization histogram for arboreal primates; the texture features employ an improved LBP-TOP operator to extend texture analysis along the time axis, using a triorthogonal plane mode with radii of 3 and 8 neighborhoods to generate a 256-dimensional feature vector; the geometric features include perimeter, area, major axis, minor axis, compactness, and eccentricity.

[0013] Furthermore, in other preferred embodiments of the present invention, the three-layer matching mechanism includes: Coarse matching: Using ORB features and body size constraints, select point pairs whose scale ratio matches the size of arboreal primates; Fine matching: Construct a local biogeometric model and screen matching points based on the biogeometric constraints of the head-to-body ratio and joint symmetry deviation of arboreal primates; Biological semantic constraints: cross-modal feature correlation verification, pose invariance constraints and mismatch elimination enhancement are performed, and point pairs that simultaneously satisfy scale constraints, body temperature-grayscale threshold and biological structure proportion are retained to construct high-precision registration transformation relationship.

[0014] Furthermore, in other preferred embodiments of the present invention, it further includes: S5. Perform multi-scale fusion of visible light images and thermal infrared images to generate a fused image, and then train and optimize the model.

[0015] The beneficial effects of the embodiments of the present invention are: This invention provides a method for identifying arboreal primates based on the fusion of thermal infrared and visible light images. The method includes simultaneously acquiring visible light and thermal infrared images at the target sampling site; processing the thermal infrared and visible light images; extracting thermal infrared features from the thermal infrared images and biological features from the visible light images; performing a three-layer matching mechanism based on the extracted thermal infrared and biological features (coarse matching, static matching, and biological constraints); image fusion; model training and optimization; and outputting species identification results. This method combines the complex ecological environment of high-altitude areas with the processing of thermal infrared and visible light images, and then fuses thermal infrared and visible light features for cross-modal feature three-layer matching. This effectively eliminates interference from environmental conditions, improves identification accuracy, and provides key technical support for the physiological monitoring of arboreal primates. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present invention.

[0017] The following specific examples will provide further explanation. Example

[0018] This embodiment provides a method for identifying arboreal primates based on the fusion of thermal infrared and visible light images, which includes: S1. Simultaneously acquire visible light and thermal infrared images at the target sample site.

[0019] Furthermore, considering the activity characteristics of arboreal primates, a layered vertical detection hardware architecture is set up in the forest to provide full-dimensional coverage of their activity space. Specifically, a four-layer infrared camera three-dimensional monitoring network is deployed, including: Ground level (1.5m): Deploy thermal infrared panoramic cameras (FLIR T840, 180° fisheye lens), with triangular steel brackets (buried 50cm deep), to monitor the area from the ground to the tree trunk within 10m, and accurately locate the nodes where Yunnan snub-nosed monkeys go up and down the tree (positioning accuracy ±5cm).

[0020] Lower layer (5-10m): Dual-lens monitoring unit (4K visible light + 336×256 thermal infrared) covers an area with a horizontal radius of 8m through adjustable clamp bracket (4.5m / 7.5m height), solving the problem of recognizing the behavior of juveniles under the cover of branches and leaves.

[0021] Middle layer (10-15m): Multispectral nodes integrate visible light, thermal infrared and lidar (8m ranging), and achieve 5m vertical height monitoring through a double-arm telescopic bracket, record social interaction details and construct a three-dimensional positioning grid.

[0022] Upper layer (15-20m): A lightweight thermal infrared camera (320g, wind resistance ≥10) is mounted on a biomimetic tree branch support to monitor the radiation area at the top of the canopy, covering the high-altitude migration route of the Yunnan golden monkey.

[0023] Compared to single-height monitoring, this four-layer three-dimensional monitoring network increases the monitoring coverage from 40% to 90% with its layered architecture, achieving for the first time a complete vertical activity chain of "ground-trunk-canopy" monitoring without blind spots.

[0024] Furthermore, the upper-layer camera supports PoE power supply (reducing wiring), the middle-layer lidar (RPLIDAR A2) is resistant to interference from leaf swaying, and the lower-layer dual-lens unit achieves high-precision registration of visible light and thermal infrared images (error < 3 pixels) through a checkerboard calibration board.

[0025] Meanwhile, the biomimetic tree branch support simulates the natural habitat structure. The equipment weight (320g) and installation density (2 upper-level cameras deployed on each dominant tree) have been calculated based on the load-bearing mechanics of trees, which meets the principle of "minimal disturbance" for wildlife protection.

[0026] Furthermore, the acquired thermal infrared images are radiometrically calibrated to eliminate sensor errors and convert grayscale values ​​in the thermal infrared images into radiance values. The method adopts the improved Landsat series data radiometric calibration formula and adds weight to the body temperature sensitive band (such as the 8-14μm thermal infrared band).

[0027] Specifically, the formula for radiation calibration is: , In the formula, This represents the radiance value corresponding to the thermal infrared data wavelength λ. This is the gain quantity. DN The original pixel grayscale value. This is the offset. This represents the gain coefficient for the body temperature region. This is the offset.

[0028] Furthermore, to address the low-frequency temperature fluctuations caused by abiotic heat sources such as rocks and soil in forest environments, a time-series median filter is introduced. By suppressing noise in the time dimension, the high-frequency body temperature signals of arboreal primates are preserved and enhanced. The specific operation is as follows: pixels in thermal infrared images The radiance values ​​were continuously collected (e.g., one frame every 5 minutes, for 4-6 hours, covering the peak activity periods at dawn and dusk), and a time series was constructed. ,in, Set the radiance value of the pixel in the nth frame (unit: W / (m²・sr・μm); set a static threshold and a dynamic threshold. When the change in radiance value does not exceed the static threshold, perform median filtering; when the change in radiance value is not lower than the dynamic threshold, retain the original value.

[0029] Furthermore, based on the temperature change cycle of abiotic heat sources, a window length of W=5 or 7 frames (corresponding to 25-35 minutes) is typically used for median filtering. The temperature of abiotic objects such as rocks changes slowly (e.g., a 2-5°C increase per hour during the day), while the body temperature of arboreal primates is stable, fluctuating by less than 1°C in a short period. Therefore, the window needs to cover a sufficiently long noise change cycle while avoiding the loss of animal dynamic signals. For each frame of imagery, (W-1) / 2 frames are taken forward and backward from the current frame (e.g., when W=5, the current frame and two frames before and after are taken) to calculate the median; if W is even, the average of the two middle values ​​is usually taken, but odd-numbered windows are preferred in actual calculations.

[0030] Setting static and dynamic thresholds can effectively reduce filtering ambiguity in the movement trajectories of arboreal primates, and the change in radiance values ​​can be achieved through... Perform the calculation. The static threshold can be set to 0.5 W / (m²・sr・μm), when... When the static threshold is less than or equal to the stable body temperature region, a medium-range filter is applied to suppress low-frequency noise. The dynamic threshold can be set to 2W / (m²・sr・μm), corresponding to pixel changes caused by animal movement, in which case the original value is retained. To avoid blurring the target during filtering.

[0031] Furthermore, the arboreal primate identification method based on thermal infrared and visible light image fusion provided in this embodiment also includes: S2. Correct and enhance the thermal infrared image.

[0032] Due to the unique differences in water vapor content, aerosols, and topographic factors in high-altitude regions, this embodiment of the invention divides the area into layers every 500 meters of altitude. Based on the temperature, air pressure, and water vapor content (obtained from ground meteorological station data) of each layer, the MODTRAN model is used to analyze... and Make corrections.

[0033] At the same time, the radiance value is corrected for different arboreal primates according to the following formula. , In the formula, Radiance, These are sensor observations. For path radiation, Atmospheric transmittance. The hair emission rate of arboreal primates.

[0034] For different arboreal primates, the hair emissivity ranges from 0.93 to 0.98. For example, for the Yunnan snub-nosed monkey, its hair emissivity is... =0.95.

[0035] After extracting the ROI (Region of Interest), to avoid the interference of background temperature fluctuations on the target features, a quantile normalization method is used for the entire ROI region to map the temperature value range to a fixed percentile range (such as 1%-99%), thus suppressing the influence of abnormal temperature points (such as strong reflection noise).

[0036] In addition, visible light images also require preprocessing. Preprocessing of visible light images includes: Spectral response calibration: A dual-channel response model was established for visible light images from an infrared camera under 850nm near-infrared illumination.

[0037] in, These are the original RGB channel values. Infrared reflectance intensity, and The spectral response coefficients are in the wavelength range of 800-1000nm, obtained through standard whiteboard calibration.

[0038] Dynamic range compression: The Retinex algorithm is used for illumination compensation to construct a local illuminance estimation model.

[0039] in, It is a low-pass filter with a Gaussian kernel size of 31×31, which realizes nonlinear compression in bright light areas (such as snow) and dim light areas (such as tree shade).

[0040] Adaptive Median Filtering: Designing a dynamic window (3×3 to 7×7) based on neighborhood gray-level variance Automatic adjustment: Window size

[0041] in, The noise threshold (empirical value 30) effectively preserves hair edge details.

[0042] Multi-scale guided filtering: Construct a three-layer pyramid (scale factor 0.5) and perform guided filtering in the luminance channel:

[0043] Achieve effective separation of hair texture from the background and enhance coefficient. Set it to 0.01.

[0044] Furthermore, the arboreal primate identification method based on thermal infrared and visible light image fusion provided in this embodiment also includes: S3. Extract thermal infrared features from thermal infrared images and biological features from visible light images.

[0045] Optionally, the thermal infrared features include temperature statistical features and thermal texture features, where the temperature statistical features include the average temperature within the ROI. Standard deviation Maximum temperature With the lowest temperature temperature difference The mean temperature reflects the average level of thermal radiation in the area and can be used to determine the overall thermal state of arboreal primates, such as whether they are within the normal body temperature range. The standard deviation reflects the dispersion of temperature distribution in the area and can reflect the uniformity of temperature across different parts of the body. This is important for analyzing whether Yunnan snub-nosed monkeys have localized overheating or abnormal heat dissipation. The calculation formulas are as follows:

[0046]

[0047] in, It is the number of pixels within the ROI. It is the first Temperature value per pixel.

[0048] The highest temperatures are likely to occur in metabolically active areas of arboreal primates, such as the region near the heart or muscles during movement; the lowest temperatures are likely to occur at the extremities or in areas where heat dissipates more quickly. Temperature differences reflect the thermal gradient between different parts of the body and are of significant reference value for studying the thermoregulation mechanisms and energy metabolism of arboreal primates.

[0049] Thermal texture features include energy, entropy, and contrast extracted based on the gray-level co-occurrence matrix (GLCM). GLCM describes the spatial distribution of pixel pairs with different gray values ​​(corresponding to different temperature values ​​in infrared images). For infrared images, features related to thermal texture, such as energy, entropy, contrast, and correlation, can be extracted by calculating the GLCM. First, an offset is defined. The statistical image contains temperature values. and and distance Frequency of occurrence of pixel pairs Construct the GLCM matrix. Energy reflects the uniformity of thermal texture in the image, and the calculation formula is:

[0050] Entropy measures the complexity of a thermal texture, and the formula is:

[0051] Contrast reflects the degree of difference in temperature within a thermal texture, and the formula is:

[0052] Correlation represents the linear correlation of temperature values ​​in a thermal texture, and the formula is:

[0053] in, , It is a temperature value. and The mean, , It is the standard deviation. These thermal texture features can reflect the patterns and changes in heat distribution on the body surface of arboreal primates, helping to distinguish different behavioral and physiological states.

[0054] Furthermore, thermal infrared features also include thermal LBP features. Traditional LBP is used to describe the texture information of local areas of an image. For infrared images, it can be extended to other applications based on temperature values.

[0055] Specifically, the method for extracting hot LBP features is as follows: In thermal infrared images, the temperature of the center pixel is used as a threshold. The temperatures of 3×3 neighboring pixels are compared; if a neighboring pixel's temperature is higher than the center pixel's, it is marked as 1; otherwise, it is marked as 0. These markers are arranged in a specific direction to form a binary number, which is then converted to a decimal number as the thermal LBP value for that pixel. A histogram of the thermal LBP values ​​of all pixels in the thermal infrared image is plotted to obtain the thermal LBP feature. To enhance the rotation invariance and adaptability to temperature changes, rotation-invariant thermal LBP or improved uniform thermal LBP methods can be used. Thermal LBP features can effectively capture detailed changes in the thermal texture of the arboreal primate's body surface, playing a crucial role in identifying heat-related features such as body posture changes and fur condition.

[0056] Furthermore, visible light features include facial morphological features and body posture features. Facial morphological features include the localization and detection of 10-16 facial feature points. Taking the Yunnan snub-nosed monkey as an example, a cascaded CNN model (based on an improved MTCNN) is used to define 12 key feature points, including: Facial contours: base of left and right ears (2 points), jaw inflection point (2 points) Facial features: outer corner of the eye (2 points), tip of the nose (1 point), cleft lip point (3 points) Coat color demarcation: the apex of the black area on the neck and back (2 points), the boundary point of the white area on the shoulders and back (2 points). Body posture features include constructing a skeleton model with 18-25 joints. Again, using the Yunnan snub-nosed monkey as an example, a skeleton model with 18 joints is constructed. An improved OpenPose algorithm is used, and the loss function is optimized for Yunnan snub-nosed monkey-specific postures (such as climbing and grooming). , .

[0057] Among them, the heatmap loss uses mean squared error, and the affinity field loss uses cosine similarity as a metric.

[0058] Furthermore, visible light features also include the color features, texture features, and geometric features of the ROI.

[0059] Color characteristics include constructing HSV color gamut quantization histograms for arboreal primates. Taking the Yunnan snub-nosed monkey as an example, its HSV color gamut quantization histogram should satisfy the following conditions regarding black areas. , , white area A 3D histogram quantized using 8×8×8 was used.

[0060] Texture features are analyzed using an improved LBP-TOP operator, extending texture analysis along the time axis (each frame is considered a static time point). A triorthogonal plane model with radii of 3 and 8 neighborhoods is employed to generate 256-dimensional feature vectors; five scales are used (…). ), 8-directional filter banks, extracting energy and phase co-occurrence matrices.

[0061] Geometric features include perimeter, area, major axis, minor axis, compactness, and eccentricity. Compactness is defined as area divided by the area of ​​the smallest bounding rectangle, calculated using a contour tracing algorithm (Freeman chaincode).

[0062] Furthermore, this embodiment of the invention employs a multi-attention U-Net model for image segmentation. The encoding path uses ResNet50 as the backbone network, with a spatial attention module added after each residual block.

[0063] Decoding path: A channel attention mechanism is introduced to enhance contextual semantics; the loss function is jointly optimized using the Dice coefficient and cross-entropy.

[0064] Dynamic threshold contour extraction is an improved thresholding method based on the Otsu algorithm, which considers the bimodal characteristics of fur color distribution and constructs a bigaussian mixture model.

[0065] The optimal segmentation threshold is determined by estimating parameters using the EM algorithm. This can effectively improve the accuracy and efficiency of image processing.

[0066] Furthermore, the arboreal primate identification method based on thermal infrared and visible light image fusion provided in this embodiment also includes: S4. Based on the extracted thermal infrared features and biological features, a three-layer matching mechanism of coarse matching, static matching and biological constraints is implemented to output species identification results.

[0067] Furthermore, the three-layer matching mechanism includes: Coarse match: ORB initial matching is performed on feature points in thermal infrared (high-temperature ROI) and visible light (contour edges), using Hamming distance to calculate similarity, and employing the nearest neighbor to second nearest neighbor method (threshold). ) Filter the initial matching point pairs.

[0068] Body size constraints are used to calculate the scale ratio of matching point pairs based on knowledge of the body length of arboreal primates.

[0069] Among them, d is the distance between two points in the image. Taking the Yunnan snub-nosed monkey as an example, according to the prior knowledge of its body length of 80 - 120 cm, the point pairs are retained (background noise points with too large scale differences are removed). For the sizes of other arboreal primates, the range of s can be appropriately adjusted.

[0070] Fine matching: For the SIFT point pairs that pass the rough matching, a local biogeometric model is constructed, and the matching points are screened based on the biogeometric constraints of the head-trunk ratio and joint symmetry deviation of arboreal primates. Taking the Yunnan snub-nosed monkey as an example again, its head-trunk ratio should satisfy 0.3 - 0.5, and the relative position deviation of the limb joint points in the visible light and thermal infrared images needs to be < 10 pixels (corresponding to an actual distance < 10 cm).

[0071] When performing matching, the YOLO detection box is divided into 3 sub-regions (head, trunk, limbs), and the homography matrix is independently calculated for the SIFT point pairs in each sub-region , and the formula is as follows:

[0072] Among them, is the coordinate of the thermal infrared feature point, is the coordinate of the corresponding point in the visible light, and it is solved by the least squares method .

[0073] Biological semantic verification: Set the body temperature thresholds for the bare skin area and the hair-covered area of arboreal primates, and remove the mis-matched points through the constraint of body temperature morphological consistency. For the Yunnan snub-nosed monkey, the threshold for its bare skin area is T > 38 °C and the visible light gray scale V < 150, and the threshold for the hair-covered area is 36 °C < T < 38 °C and V > 180.

[0074] In addition, it can also be constrained by pose invariance. For the same individual of an arboreal primate, in the thermal infrared image and the visible light image, calculate the angle between the line connecting the feature point and the centroid of the region of interest (ROI) in its respective image and the horizontal axis: In the thermal infrared image, assuming the feature point coordinates are ( , ), and the thermal infrared ROI centroid coordinates are ( , ) then the angle is: <>

[0075] Similarly, in the visible light, the angle is:

[0076] When | - Points with an angle of less than 15° are retained.

[0077] Furthermore, dynamic weighted RANSAC can be used to enhance mismatch removal. This includes: Prioritize detection points within the bounding box: In the RANSAC iteration, matching points located within the YOLO detection box are assigned double weights: When determining the inner point, the reprojection error threshold for the inner point of the detection box is relaxed to 2 pixels (1 pixel for the background point), because it belongs to the target area, allowing a certain range of biological morphological deformation.

[0078] The weights of points within the detection box during model voting. Background point This enhances the contribution of target region features to the registration model.

[0079] Multiple rounds of iterative optimization: Round 1: Estimate the initial homography matrix using only SIFT point pairs within the detection box, eliminating background interference; Round 2: Incorporate ORB coarse matching point pairs and use the Round 1 model to filter point pairs that meet biological geometric constraints (e.g., cross-modal position difference of limb joints < 5 pixels). Final model: Point pairs that simultaneously satisfy scale constraints, body temperature-grayscale threshold, and biological structure proportions are retained to construct a high-precision registration transformation relationship.

[0080] Furthermore, the arboreal primate identification method based on thermal infrared and visible light image fusion provided in this embodiment also includes: S5. Perform multi-scale fusion of visible light images and thermal infrared images to generate a fused image.

[0081] An image fusion algorithm based on multi-scale decomposition is adopted to fully utilize the characteristics of three-dimensional thermal infrared images and visible light images to generate fused images containing rich information. The specific technical solution is as follows: Laplacian pyramid decomposition: The registered thermal infrared and visible light images are subjected to Laplacian pyramid decomposition separately, decomposing the images into high-frequency detail information and low-frequency approximation information at different scales. The Laplacian pyramid decomposition process is based on a Gaussian pyramid; first, Gaussian smoothing and downsampling operations are performed on the image to construct a Gaussian pyramid. Assume the original image is... , No. Image of the Gaussian Pyramid It can be achieved by the first Layer Image This is obtained by performing Gaussian convolution and downsampling, i.e.:

[0082] in, This indicates a downsampling operation. This represents the Gaussian convolution operation. After obtaining the Gaussian pyramid, the Laplace pyramid's... Layer Image The difference between two adjacent Gaussian pyramid images can be obtained by calculating:

[0083] in, This indicates an upsampling operation. In this way, the image is decomposed into multiple layers, each containing detailed information at different scales. The low-frequency components retain the general outline and structure of the image, while the high-frequency components contain detailed information such as texture and edges.

[0084] Low-frequency subband fusion: For the low-frequency subband, a weighted average fusion rule is used. Considering the different information richness levels in different regions of thermal infrared and visible light images, the weights are determined based on the image variance. The larger the variance, the richer the information in that region, and the higher the weight. The specific formula is:

[0085] in, For the merged low-frequency sub-band, The low-frequency subband of the thermal infrared image. The low-frequency subband of the visible light image. and These represent the variances of the low-frequency subbands of the thermal infrared and visible light images, respectively. Through this weighted averaging method, the fused low-frequency subband can comprehensively retain the main structural information of both images.

[0086] High-frequency subband fusion: For high-frequency subbands, a fusion rule based on the larger absolute value is adopted. That is, the pixel value with the larger absolute value in the high-frequency subbands of the thermal infrared and visible light images is selected as the pixel value of the fused high-frequency subband. The formula is as follows:

[0087] in, For the fused high-frequency subband, and These are high-frequency subbands from thermal infrared and visible light images, respectively. This fusion method ensures that the fused high-frequency subbands retain more detailed information, such as edges and textures, resulting in a clearer image in terms of detail.

[0088] Image Reconstruction: Finally, the fused 3D image is obtained through inverse Laplacian pyramid transform, while preserving the image's... 3D depth information. The inverse transform process involves reconstructing the fused low-frequency and high-frequency subbands layer by layer. Starting from the top-level low-frequency subband, the original fused image is gradually recovered through upsampling and addition with the corresponding high-frequency subbands. The specific process is as follows:

[0089] in, For the first The image after layer fusion For the first The high-frequency subband after layer fusion. Through layer-by-layer reconstruction, a complete fused image is finally obtained. This image integrates the temperature feature information of the thermal infrared image and the texture detail information of the visible light image, providing a richer and higher-quality image data foundation for subsequent feature extraction and recognition classification.

[0090] Application examples This invention employs a method for identifying arboreal primates based on the fusion of thermal infrared and visible light images, specifically targeting the Yunnan snub-nosed monkey. Habitat data from Baima Snow Mountain and Mangkang were collected, including over 20,000 pairs of thermal infrared-visible light images, labeled with individual ID, age (juvenile / adult), sex, and posture (climbing / sitting / grooming). Statistical results are shown in Tables 1-3.

[0091] Table 1. Species identification results of Yunnan snub-nosed monkey in different environments. No obstruction during the day 8000 98.9% Night / Dense Fog 5000 92.5% strong wind disturbance 3000 88.7% Table 2. Identification results for different Yunnan snub-nosed monkey individuals Larvae 300 96.9% 93.7% Adult 800 98.5% 97.5% Older body 200 97.3% 96.4% Table 3. Results of Yunnan snub-nosed monkey pose recognition for different poses Climbing 350 98.6% Elbow joint angle >120° + sudden increase in thermal texture contrast Grooming 230 98.1% Arm-head distance <15% body length + low local heat drop alert 200 97.9% A sudden 20% increase in aspect ratio plus increased thermal energy in the neck area breast-feeding 150 98.4% Abdominal-larval overlap >60% + temperature difference <2℃ As shown in Table 1, the arboreal primate identification method based on thermal infrared and visible light image fusion provided in this embodiment of the invention achieves an accuracy rate of up to 98.9% under conditions of good, unobstructed vision, and maintains an accuracy rate of 92.5% even under poor visibility conditions such as nighttime or dense fog. However, strong winds significantly interfere with identification, reducing the accuracy rate to 88.7%.

[0092] In addition, as can be seen from Table 2, this identification method has a high recognition rate for individuals. The recognition accuracy for Yunnan snub-nosed monkey juveniles, adults and elderly individuals can all reach more than 96%, and the re-identification accuracy for duplicate individuals can also be maintained above 93%. However, the recognition accuracy is reduced due to the rapid growth of juveniles.

[0093] As can be seen from Table 3, this identification method can achieve relatively accurate identification of different postures of Yunnan snub-nosed monkeys, and can achieve an accuracy of about 98% for some postures with obvious features.

[0094] In summary, this invention provides a method for identifying arboreal primates based on the fusion of thermal infrared and visible light images. The method includes simultaneously acquiring visible light and thermal infrared images at the target site; performing radiometric correction on the thermal infrared images; extracting thermal infrared features from the thermal infrared images and biological features from the visible light images; and performing a three-layer matching mechanism based on the extracted thermal infrared and biological features, consisting of coarse matching, static matching, and biological constraints, to output the species identification result. This method combines the complex ecological environment of high-altitude areas to correct and enhance infrared radiation, and then fuses thermal infrared and visible light features for three-layer cross-modal feature matching. This effectively eliminates interference from environmental conditions, improves identification accuracy, and provides key technical support for the physiological monitoring of arboreal primates.

[0095] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying arboreal primates based on the fusion of thermal infrared and visible light images, characterized in that, It includes: S1. Simultaneously acquire visible light and thermal infrared images at the target sample site; S2. Process the acquired thermal infrared and visible light images; S3. Extract the thermal infrared features from the thermal infrared image and the biological features from the visible light image; S4. Based on the extracted thermal infrared features and biological features, a three-layer matching mechanism of coarse matching, static matching, and biological constraints is performed, and image fusion is then performed; S5. Train and optimize the model, and output the species identification results; The thermal infrared features include temperature statistical features and thermal texture features. The temperature statistical features include the average temperature within the ROI. Standard deviation Maximum temperature With the lowest temperature temperature difference The thermal texture features include energy, entropy, and contrast extracted based on the gray-level co-occurrence matrix. The visible light features also include color features, texture features, and geometric features of the ROI; the color features include constructing an HSV color gamut quantized histogram for the arboreal primate; the texture features employ an improved LBP-TOP operator to extend texture analysis along the time axis, using a triorthogonal plane mode with radii of 3 and 8 neighborhoods to generate a 256-dimensional feature vector; the geometric features include perimeter, area, major axis, minor axis, compactness, and eccentricity; The three-layer matching mechanism includes: Coarse matching: Using ORB features and body size constraints, select point pairs whose scale ratio matches the size of the arboreal primates; Fine matching: Construct a local biogeometric model and screen matching points based on the biogeometric constraints of the head-to-body ratio and joint symmetry deviation of the arboreal primates; Biological semantic constraints: cross-modal feature correlation verification, pose invariance constraints and mismatch elimination enhancement are performed, and point pairs that simultaneously satisfy scale constraints, body temperature-grayscale threshold and biological structure proportion are retained to construct high-precision registration transformation relationship.

2. The method for identifying arboreal primates according to claim 1, characterized in that, The acquired thermal infrared image is radiometrically calibrated by converting the grayscale values ​​in the thermal infrared image into radiance values. The radiometric calibration formula is as follows: ， In the formula, This represents the radiance value corresponding to the thermal infrared data wavelength λ. This is the gain quantity. This is the offset. This represents the gain coefficient for the body temperature region. This is the offset.

3. The method for identifying arboreal primates according to claim 2, characterized in that, For the pixels in the thermal infrared image The radiance values ​​were continuously collected and a time series was constructed. ,in, The radiance value of the pixel in the nth frame is given. A static threshold and a dynamic threshold are set. When the change in the radiance value does not exceed the static threshold, median filtering is performed. When the change in the radiance value is not lower than the dynamic threshold, the original value is retained.

4. The method for identifying arboreal primates according to claim 2, characterized in that, In step S2, the radiance value is corrected according to the following formula. ， In the formula, Radiance, These are sensor observations. For path radiation, Atmospheric transmittance. For the hair emissivity of arboreal primates; Furthermore, the altitude was divided into layers every 500 meters, and the temperature, air pressure, and water vapor content of each layer were analyzed using the MODTRAN model. and Make corrections.

5. The method for identifying arboreal primates according to claim 1, characterized in that, The thermal infrared features also include thermal LBP features, and the method for extracting the thermal LBP features is as follows: In the thermal infrared image, the temperature of the center pixel is used as a threshold, and the temperatures of 3×3 neighboring pixels are compared to generate the thermal LBP value of the center pixel; the thermal LBP feature is obtained by statistically analyzing the histogram of the thermal LBP values ​​of all pixels in the thermal infrared image.

6. The method for identifying arboreal primates according to claim 1, characterized in that, The visible light features include facial morphological features and body posture features. The facial morphological features include the localization and detection of 10-16 facial feature points, and the body posture features include the construction of a skeleton model containing 18-25 joints. The loss function is optimized using an improved OpenPose algorithm. ， 。

Images