Tree-dwelling primate identification method based on thermal infrared and visible light image fusion

Abstract

A tree-dwelling primate identification method based on thermal infrared and visible light image fusion relates to the technical field of target detection, and comprises the following steps: synchronously acquiring a visible light image and a thermal infrared image in a target sample plot; performing image processing on the visible light image and the thermal infrared image; extracting thermal infrared features in the thermal infrared image and biological features in the visible light image; based on the extracted thermal infrared features and biological features, performing a three-layer matching mechanism of rough matching, static matching and biological constraint; carrying out image fusion on the visible light image and the thermal infrared image; carrying out model training and optimization; and outputting a species identification result. According to the method, adaptive infrared radiation correction enhancement is carried out on the complex inhabiting environment of the tree-dwelling primates in the high-altitude area, then thermal infrared features and visible light features are fused to carry out cross-modal feature three-layer matching and modeling, interference of environmental conditions can be effectively eliminated, the recognition precision is improved, and the recognition efficiency is improved. And a key technical support is provided for physiological monitoring of the tree-dwelling primates.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, in particular to a tree-dwelling primate identification method based on thermal infrared and visible light image fusion. BACKGROUND

[0002] Long-term observation of animal populations is an important scientific research method for maintaining species balance and maintaining ecological stability. The existing animal identification methods are generally based on classification tasks or target detection tasks to construct an end-to-end deep neural network, and the species identification is realized through sample labeling and model training. For example, a night infrared image animal identification method and system based on domain migration (202311810982.6) obtains visible light image data and infrared image data, and divides them into a training set and a test set; a cycle generative adversarial network model for domain migration is established; the cycle generative adversarial network model is trained based on the training set, and a domain migration network is obtained; a target detection model is trained based on the training set, and an image recognition network is obtained; the infrared image data in the test set is input into the domain migration network, and the migrated visible light image data is obtained, and then the migrated visible light image data is input into the image recognition network, and finally the recognition result is obtained. For another example, a wild animal species identification method based on incremental learning (202411671680.X) constructs a known category species detection dataset, trains a known category species detection model, selects representative samples of each category species, and saves the expression features of each category species; a species discrimination dataset is constructed, a species discrimination model is trained; new species sample data is manually verified; a species detection fine-tuning dataset is constructed, and the known category species detection model is updated based on the feature playback class incremental strategy, and the species discrimination model is updated based on the feature-level sample playback class incremental strategy. The identification system includes a collection module, a migration module, and an identification module.

[0003] However, for tree-dwelling primates in high-altitude areas, due to differences in habitat environment and living habits, species identification has the following problems: 1) tree crown layer interference such as lichen and branch leaf swing; 2) meteorological condition differences such as light and season; 3) physiological differences such as age and gender affect individual identification accuracy, and the identification accuracy of young and old individuals is low; 4) posture changes affect; 5) regional differences cause model accuracy to decrease; 6) pseudo-thermal source interference caused by temperature difference in high mountain environment. SUMMARY

[0004] The purpose of the present application is to provide a tree-dwelling primate identification method based on thermal infrared and visible light image fusion, which can exclude the interference of complex environmental conditions, and at the same time, fuse the advantages of thermal infrared and visible light imaging, accurately and effectively identify the species and individuals of tree-dwelling primates.

[0005] Embodiments of the present application are implemented as follows: A tree-dwelling primate identification method based on thermal infrared and visible light image fusion, comprising: S1. synchronously collecting visible light images and thermal infrared images in a target sample plot; S2. processing the collected thermal infrared and visible light images; S3. extracting thermal infrared features in the thermal infrared images and biological features in the visible light images; S4. based on the extracted thermal infrared features and biological features, performing a three-layer matching mechanism of coarse matching, static matching and biological constraint, and performing image fusion; S5. performing model training and optimization, and outputting species identification results.

[0006] Further, in other preferred embodiments of the present application, the collected thermal infrared images are radiometrically calibrated, and the gray value in the thermal infrared images is converted into a radiance value, the formula for radiometric calibration is , In the formula, is the radiance value corresponding to the wavelength of the thermal infrared data, is the gain amount, is the offset amount, is the body temperature region gain coefficient, is the offset amount.

[0007] Further, in other preferred embodiments of the present application, the radiance value of a pixel in the thermal infrared image is continuously collected, and a time series is constructed, wherein, is the radiance value of the nth frame of the pixel; a static threshold and a dynamic threshold are set, when the change amount of the radiance value does not exceed the static threshold, median filtering is performed; when the change amount of the radiance value is not less than the dynamic threshold, the original value is retained.

[0008] Further, in other preferred embodiments of the present application, in the S2 step, the radiance value is corrected according to the following formula, , In the formula, is the radiance, is the sensor observation value, is the path radiance, is the atmospheric transmittance, is the hair emissivity of the tree-dwelling primate; and, according to the altitude, one layer is divided every 500 meters, and according to the temperature, air pressure and water vapor content of each layer, the and Make corrections.

[0009] Furthermore, in other preferred embodiments of the present invention, the thermal infrared features include temperature statistical features and thermal texture features, wherein the temperature statistical features include the average temperature within the ROI. Standard deviation Maximum temperature With the lowest temperature temperature difference Thermal texture features include energy, entropy, and contrast extracted based on the gray-level co-occurrence matrix.

[0010] Furthermore, in other preferred embodiments of the present invention, the thermal infrared features further include thermal LBP features, and the method for extracting thermal LBP features is as follows: In the thermal infrared image, the temperature of the center pixel is used as a threshold, and the temperatures of 3×3 neighboring pixels are compared to generate the thermal LBP value of the center pixel; the thermal LBP feature is obtained by statistically analyzing the histogram of the thermal LBP values ​​of all pixels in the thermal infrared image.

[0011] Furthermore, in other preferred embodiments of the present invention, the visible light features include facial morphological features and body posture features. The facial morphological features include the localization and detection of 10-16 facial feature points, and the body posture features include constructing a skeleton model containing 18-25 joints, and optimizing the loss function using an improved OpenPose algorithm. , .

[0012] Furthermore, in other preferred embodiments of the present invention, the visible light features also include color features, texture features, and geometric features of the ROI; the color features include constructing an HSV color gamut quantization histogram for arboreal primates; the texture features employ an improved LBP-TOP operator to extend texture analysis along the time axis, using a triorthogonal plane mode with radii of 3 and 8 neighborhoods to generate a 256-dimensional feature vector; the geometric features include perimeter, area, major axis, minor axis, compactness, and eccentricity.

[0013] Furthermore, in other preferred embodiments of the present invention, the three-layer matching mechanism includes: Coarse matching: Using ORB features and body size constraints, select point pairs whose scale ratio matches the size of arboreal primates; Fine matching: Construct a local biogeometric model and screen matching points based on the biogeometric constraints of the head-to-body ratio and joint symmetry deviation of arboreal primates; Biological semantic constraints: cross-modal feature correlation verification, pose invariance constraints and mismatch elimination enhancement are performed, and point pairs that simultaneously satisfy scale constraints, body temperature-grayscale threshold and biological structure proportion are retained to construct high-precision registration transformation relationship.

[0014] Furthermore, in other preferred embodiments of the present invention, it further includes: S5. Perform multi-scale fusion of visible light images and thermal infrared images to generate a fused image, and then train and optimize the model.

[0015] The beneficial effects of the embodiments of the present invention are: This invention provides a method for identifying arboreal primates based on the fusion of thermal infrared and visible light images. The method includes simultaneously acquiring visible light and thermal infrared images at the target sampling site; processing the thermal infrared and visible light images; extracting thermal infrared features from the thermal infrared images and biological features from the visible light images; performing a three-layer matching mechanism based on the extracted thermal infrared and biological features (coarse matching, static matching, and biological constraints); image fusion; model training and optimization; and outputting species identification results. This method combines the complex ecological environment of high-altitude areas with the processing of thermal infrared and visible light images, and then fuses thermal infrared and visible light features for cross-modal feature three-layer matching. This effectively eliminates interference from environmental conditions, improves identification accuracy, and provides key technical support for the physiological monitoring of arboreal primates. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present invention.

[0017] The following specific examples will provide further explanation. Example

[0018] This embodiment provides a method for identifying arboreal primates based on the fusion of thermal infrared and visible light images, which includes: S1. Simultaneously acquire visible light and thermal infrared images at the target sample site.

[0019] Furthermore, considering the activity characteristics of arboreal primates, a layered vertical detection hardware architecture is set up in the forest to provide full-dimensional coverage of their activity space. Specifically, a four-layer infrared camera three-dimensional monitoring network is deployed, including: Ground level (1.5m): Deploy thermal infrared panoramic cameras (FLIR T840, 180° fisheye lens), with triangular steel brackets (buried 50cm deep), to monitor the area from the ground to the tree trunk within 10m, and accurately locate the nodes where Yunnan snub-nosed monkeys go up and down the tree (positioning accuracy ±5cm).

[0020] Lower layer (5-10m): Dual-lens monitoring unit (4K visible light + 336×256 thermal infrared) covers an area with a horizontal radius of 8m through adjustable clamp bracket (4.5m / 7.5m height), solving the problem of recognizing the behavior of juveniles under the cover of branches and leaves.

[0021] Middle layer (10-15m): Multispectral nodes integrate visible light, thermal infrared and lidar (8m ranging), and achieve 5m vertical height monitoring through a double-arm telescopic bracket, record social interaction details and construct a three-dimensional positioning grid.

[0022] Upper layer (15-20m): A lightweight thermal infrared camera (320g, wind resistance ≥10) is mounted on a biomimetic tree branch support to monitor the radiation area at the top of the canopy, covering the high-altitude migration route of the Yunnan golden monkey.

[0023] Compared to single-height monitoring, this four-layer three-dimensional monitoring network increases the monitoring coverage from 40% to 90% with its layered architecture, achieving for the first time a complete vertical activity chain of "ground-trunk-canopy" monitoring without blind spots.

[0024] Furthermore, the upper-layer camera supports PoE power supply (reducing wiring), the middle-layer lidar (RPLIDAR A2) is resistant to interference from leaf swaying, and the lower-layer dual-lens unit achieves high-precision registration of visible light and thermal infrared images (error < 3 pixels) through a checkerboard calibration board.

[0025] Meanwhile, the biomimetic tree branch support simulates the natural habitat structure. The equipment weight (320g) and installation density (2 upper-level cameras deployed on each dominant tree) have been calculated based on the load-bearing mechanics of trees, which meets the principle of "minimal disturbance" for wildlife protection.

[0026] Furthermore, the acquired thermal infrared images are radiometrically calibrated to eliminate sensor errors and convert grayscale values ​​in the thermal infrared images into radiance values. The method adopts the improved Landsat series data radiometric calibration formula and adds weight to the body temperature sensitive band (such as the 8-14μm thermal infrared band).

[0027] Specifically, the formula for radiation calibration is: , In the formula, This represents the radiance value corresponding to the thermal infrared data wavelength λ. This is the gain quantity. DN The original pixel grayscale value. This is the offset. This represents the gain coefficient for the body temperature region. This is the offset.

[0028] Furthermore, to address the low-frequency temperature fluctuations caused by abiotic heat sources such as rocks and soil in forest environments, a time-series median filter is introduced. By suppressing noise in the time dimension, the high-frequency body temperature signals of arboreal primates are preserved and enhanced. The specific operation is as follows: pixels in thermal infrared images The radiance values ​​were continuously collected (e.g., one frame every 5 minutes, for 4-6 hours, covering the peak activity periods at dawn and dusk), and a time series was constructed. ,in, Set the radiance value of the pixel in the nth frame (unit: W / (m²・sr・μm); set a static threshold and a dynamic threshold. When the change in radiance value does not exceed the static threshold, perform median filtering; when the change in radiance value is not lower than the dynamic threshold, retain the original value.

[0029] Furthermore, based on the temperature change cycle of abiotic heat sources, a window length of W=5 or 7 frames (corresponding to 25-35 minutes) is typically used for median filtering. The temperature of abiotic objects such as rocks changes slowly (e.g., a 2-5°C increase per hour during the day), while the body temperature of arboreal primates is stable, fluctuating by less than 1°C in a short period. Therefore, the window needs to cover a sufficiently long noise change cycle while avoiding the loss of animal dynamic signals. For each frame of imagery, (W-1) / 2 frames are taken forward and backward from the current frame (e.g., when W=5, the current frame and two frames before and after are taken) to calculate the median; if W is even, the average of the two middle values ​​is usually taken, but odd-numbered windows are preferred in actual calculations.

[0030] Setting static and dynamic thresholds can effectively reduce the blurring of the movement trajectories of arboreal primates, and the change in radiance values ​​can be achieved through... Perform the calculation. The static threshold can be set to 0.5 W / (m²・sr・μm), when... When the static threshold is less than or equal to the stable body temperature region, a medium-range filter is applied to suppress low-frequency noise. The dynamic threshold can be set to 2W / (m²・sr・μm), corresponding to pixel changes caused by animal movement, in which case the original value is retained. To avoid blurring the target during filtering.

[0031] Furthermore, the arboreal primate identification method based on thermal infrared and visible light image fusion provided in this embodiment also includes: S2. Correct and enhance the thermal infrared image.

[0032] Due to the unique differences in water vapor content, aerosols, and topographic factors in high-altitude regions, this embodiment of the invention divides the area into layers every 500 meters of altitude. Based on the temperature, air pressure, and water vapor content (obtained from ground meteorological station data) of each layer, the MODTRAN model is used to analyze... and Make corrections.

[0033] At the same time, the radiance value is corrected for different arboreal primates according to the following formula. , In the formula, Radiance, These are sensor observations. For path radiation, Atmospheric transmittance. The hair emission rate of arboreal primates.

[0034] For different arboreal primates, the hair emissivity ranges from 0.93 to 0.98. For example, for the Yunnan snub-nosed monkey, its hair emissivity is... =0.95.

[0035] After extracting the ROI (Region of Interest), to avoid the interference of background temperature fluctuations on the target features, a quantile normalization method is used for the entire ROI region to map the temperature value range to a fixed percentile range (such as 1%-99%), thus suppressing the influence of abnormal temperature points (such as strong reflection noise).

[0036] In addition, visible light images also require preprocessing. Preprocessing of visible light images includes: Spectral response calibration: A dual-channel response model was established for visible light images from an infrared camera under 850nm near-infrared illumination.

[0037] in, These are the original RGB channel values. Infrared reflectance intensity, and The spectral response coefficients are in the wavelength range of 800-1000nm, obtained through standard whiteboard calibration.

[0038] Dynamic range compression: The Retinex algorithm is used for illumination compensation to construct a local illuminance estimation model.

[0039] in, It is a low-pass filter with a Gaussian kernel size of 31×31, which realizes nonlinear compression in bright light areas (such as snow) and dim light areas (such as tree shade).

[0040] Adaptive Median Filtering: Designing a dynamic window (3×3 to 7×7) based on neighborhood gray-level variance Automatic adjustment: Window size

[0041] in, The noise threshold (empirical value 30) effectively preserves hair edge details.

[0042] Multi-scale guided filtering: Construct a three-layer pyramid (scale factor 0.5) and perform guided filtering in the luminance channel:

[0043] Achieve effective separation of hair texture from the background and enhance coefficient. Set it to 0.01.

[0044] Furthermore, the arboreal primate identification method based on thermal infrared and visible light image fusion provided in this embodiment also includes: S3. Extract thermal infrared features from thermal infrared images and biological features from visible light images.

[0045] Optionally, the thermal infrared features include temperature statistical features and thermal texture features, where the temperature statistical features include the average temperature within the ROI. Standard deviation Maximum temperature With the lowest temperature temperature difference The mean temperature reflects the average level of thermal radiation in the area and can be used to determine the overall thermal state of arboreal primates, such as whether they are within the normal body temperature range. The standard deviation reflects the dispersion of temperature distribution in the area and can reflect the uniformity of temperature across different parts of the body. This is important for analyzing whether Yunnan snub-nosed monkeys have localized overheating or abnormal heat dissipation. The calculation formulas are as follows:

[0046]

[0047] in, It is the number of pixels within the ROI. It is the first Temperature value per pixel.

[0048] The highest temperatures are likely to occur in metabolically active areas of arboreal primates, such as the region near the heart or muscles during movement; the lowest temperatures are likely to occur at the extremities or in areas where heat dissipates more quickly. Temperature differences reflect the thermal gradient between different parts of the body and are of significant reference value for studying the thermoregulation mechanisms and energy metabolism of arboreal primates.

[0049] Thermal texture features include energy, entropy, and contrast extracted based on the gray-level co-occurrence matrix (GLCM). GLCM describes the spatial distribution of pixel pairs with different gray values ​​(corresponding to different temperature values ​​in infrared images). For infrared images, features related to thermal texture, such as energy, entropy, contrast, and correlation, can be extracted by calculating the GLCM. First, an offset is defined. The statistical image contains temperature values. and and distance Frequency of occurrence of pixel pairs Construct the GLCM matrix. Energy reflects the uniformity of thermal texture in the image, and the calculation formula is:

[0050] Entropy measures the complexity of a thermal texture, and the formula is:

[0051] Contrast reflects the degree of difference in temperature within a thermal texture, and the formula is:

[0052] Correlation represents the linear correlation of temperature values ​​in a thermal texture, and the formula is:

[0053] in, , It is a temperature value. and The mean, , It is the standard deviation. These thermal texture features can reflect the patterns and changes in heat distribution on the body surface of arboreal primates, helping to distinguish different behavioral and physiological states.

[0054] Furthermore, thermal infrared features also include thermal LBP features. Traditional LBP is used to describe the texture information of local areas of an image. For infrared images, it can be extended to other applications based on temperature values.

[0055] Specifically, the method for extracting hot LBP features is as follows: In thermal infrared images, the temperature of the center pixel is used as a threshold. The temperatures of 3×3 neighboring pixels are compared; if a neighboring pixel's temperature is higher than the center pixel's, it is marked as 1; otherwise, it is marked as 0. These markers are arranged in a specific direction to form a binary number, which is then converted to a decimal number as the thermal LBP value for that pixel. A histogram of the thermal LBP values ​​of all pixels in the thermal infrared image is plotted to obtain the thermal LBP feature. To enhance the rotation invariance and adaptability to temperature changes, rotation-invariant thermal LBP or improved uniform thermal LBP methods can be used. Thermal LBP features can effectively capture detailed changes in the thermal texture of the arboreal primate's body surface, playing a crucial role in identifying heat-related features such as body posture changes and fur condition.

[0056] Furthermore, visible light features include facial morphological features and body posture features. Facial morphological features include the localization and detection of 10-16 facial feature points. Taking the Yunnan snub-nosed monkey as an example, a cascaded CNN model (based on an improved MTCNN) is used to define 12 key feature points, including: Facial contours: base of left and right ears (2 points), jaw inflection point (2 points) Facial features: outer corner of the eye (2 points), tip of the nose (1 point), cleft lip point (3 points) Coat color demarcation: the apex of the black area on the neck and back (2 points), the boundary point of the white area on the shoulders and back (2 points). Body posture features include constructing a skeleton model with 18-25 joints. Again, using the Yunnan snub-nosed monkey as an example, a skeleton model with 18 joints is constructed. An improved OpenPose algorithm is used, and the loss function is optimized for Yunnan snub-nosed monkey-specific postures (such as climbing and grooming). , .

[0057] Among them, the heatmap loss uses mean squared error, and the affinity field loss uses cosine similarity as a metric.

[0058] Furthermore, visible light features also include the color features, texture features, and geometric features of the ROI.

[0059] Color characteristics include constructing HSV color gamut quantization histograms for arboreal primates. Taking the Yunnan snub-nosed monkey as an example, its HSV color gamut quantization histogram should satisfy the following conditions regarding black areas. , , white area A 3D histogram quantized using 8×8×8 was used.

[0060] Texture features are analyzed using an improved LBP-TOP operator, extending texture analysis along the time axis (each frame is considered a static time point). A triorthogonal plane model with radii of 3 and 8 neighborhoods is employed to generate 256-dimensional feature vectors; five scales are used (…). ), 8-directional filter banks, extracting energy and phase co-occurrence matrices.

[0061] Geometric features include perimeter, area, major axis, minor axis, compactness, and eccentricity. Compactness is defined as area divided by the area of ​​the smallest bounding rectangle, calculated using a contour tracing algorithm (Freeman chaincode).

[0062] Furthermore, this embodiment of the invention employs a multi-attention U-Net model for image segmentation. The encoding path uses ResNet50 as the backbone network, with a spatial attention module added after each residual block.

[0063] Decoding path: A channel attention mechanism is introduced to enhance contextual semantics; the loss function is jointly optimized using the Dice coefficient and cross-entropy.

[0064] Dynamic threshold contour extraction is an improved thresholding method based on the Otsu algorithm, which considers the bimodal characteristics of fur color distribution and constructs a bigaussian mixture model.

[0065] The optimal segmentation threshold is determined by estimating parameters using the EM algorithm. This can effectively improve the accuracy and efficiency of image processing.

[0066] Furthermore, the arboreal primate identification method based on thermal infrared and visible light image fusion provided in this embodiment also includes: S4. Based on the extracted thermal infrared features and biological features, a three-layer matching mechanism of coarse matching, static matching and biological constraints is implemented to output species identification results.

[0067] Furthermore, the three-layer matching mechanism includes: Coarse match: ORB initial matching is performed on feature points in thermal infrared (high-temperature ROI) and visible light (contour edges), using Hamming distance to calculate similarity, and employing the nearest neighbor to second nearest neighbor method (threshold). ) Filter the initial matching point pairs.

[0068] Body size constraints are used to calculate the scale ratio of matching point pairs based on knowledge of the body length of arboreal primates.

[0069] Among them, d is the distance between two points in the image. Taking the Yunnan snub-nosed monkey as an example, according to the prior knowledge of its body length of 80 - 120 cm, retain point pairs (eliminating background noise points with too large scale differences). For the sizes of other arboreal primates, the range of s can be appropriately adjusted.

[0070] Fine matching: For the SIFT point pairs that pass the rough matching, construct a local biogeometric model, and screen the matching points based on the biogeometric constraints of the head-trunk ratio and joint symmetry deviation of arboreal primates.Taking the Yunnan snub-nosed monkey as an example again, its head-trunk ratio should satisfy 0.3 - 0.5, and the relative position deviation of the limb joint points in the visible light and thermal infrared images needs to be < 10 pixels (corresponding to an actual distance < 10 cm).

[0071] When performing matching, divide the YOLO detection box into 3 sub-regions (head, trunk, limbs), and independently calculate the homography matrix for the SIFT point pairs in each sub-region , the formula is as follows:

[0072] Among them, are the coordinates of the thermal infrared feature points, are the coordinates of the corresponding points in the visible light, and solve by the least squares method.

[0073] Biological semantic verification: Set the body temperature thresholds for the bare skin area and the hair-covered area of arboreal primates, and eliminate the mis-matched points through the consistency constraint of body temperature morphology. For the Yunnan snub-nosed monkey, the threshold for its bare skin area is T > 38 °C and the visible light gray scale V < 150, and the threshold for the hair-covered area is 36 °C < T < 38 °C and V > 180.

[0074] In addition, it can also be constrained by pose invariance. For the same individual of an arboreal primate, in the thermal infrared image and the visible light image, calculate the angles between the lines connecting the feature points and the centroids of the regions of interest (ROIs) in their respective images and the horizontal axis: In the thermal infrared image, assume that the feature point coordinates are ( , ), and the centroid coordinates of the thermal infrared ROI are ( , [[ID=​​​​​​​​

[0076] When | - Points with an angle of less than 15° are retained.

[0077] Furthermore, dynamic weighted RANSAC can be used to enhance mismatch removal. This includes: Prioritize detection points within the bounding box: In the RANSAC iteration, matching points located within the YOLO detection box are assigned double weights: When determining the inner point, the reprojection error threshold for the inner point of the detection box is relaxed to 2 pixels (1 pixel for the background point), because it belongs to the target area, allowing a certain range of biological morphological deformation.

[0078] The weights of points within the detection box during model voting. Background point This enhances the contribution of target region features to the registration model.

[0079] Multiple rounds of iterative optimization: Round 1: Estimate the initial homography matrix using only SIFT point pairs within the detection box, eliminating background interference; Round 2: Incorporate ORB coarse matching point pairs and use the Round 1 model to filter point pairs that meet biological geometric constraints (e.g., cross-modal position difference of limb joints < 5 pixels). Final model: Point pairs that simultaneously satisfy scale constraints, body temperature-grayscale threshold, and biological structure proportions are retained to construct a high-precision registration transformation relationship.

[0080] Furthermore, the arboreal primate identification method based on thermal infrared and visible light image fusion provided in this embodiment also includes: S5. Perform multi-scale fusion of visible light images and thermal infrared images to generate a fused image.

[0081] An image fusion algorithm based on multi-scale decomposition is adopted to fully utilize the characteristics of three-dimensional thermal infrared images and visible light images to generate fused images containing rich information. The specific technical solution is as follows: Laplacian pyramid decomposition: The registered thermal infrared and visible light images are subjected to Laplacian pyramid decomposition separately, decomposing the images into high-frequency detail information and low-frequency approximation information at different scales. The Laplacian pyramid decomposition process is based on a Gaussian pyramid; first, Gaussian smoothing and downsampling operations are performed on the image to construct a Gaussian pyramid. Assume the original image is... , No. Image of the Gaussian Pyramid It can be achieved by the first Layer Image This is obtained by performing Gaussian convolution and downsampling, i.e.:

[0082] in, This indicates a downsampling operation. This represents the Gaussian convolution operation. After obtaining the Gaussian pyramid, the Laplace pyramid's... Layer Image The difference between two adjacent Gaussian pyramid images can be obtained by calculating:

[0083] in, This indicates an upsampling operation. In this way, the image is decomposed into multiple layers, each containing detailed information at different scales. The low-frequency components retain the general outline and structure of the image, while the high-frequency components contain detailed information such as texture and edges.

[0084] Low-frequency subband fusion: For the low-frequency subband, a weighted average fusion rule is used. Considering the different information richness levels in different regions of thermal infrared and visible light images, the weights are determined based on the image variance. The larger the variance, the richer the information in that region, and the higher the weight. The specific formula is:

[0085] in, For the merged low-frequency subband, For the low-frequency subband of thermal infrared images, The low-frequency subband of the visible light image. and These represent the variances of the low-frequency subbands of the thermal infrared and visible light images, respectively. Through this weighted averaging method, the fused low-frequency subband can comprehensively retain the main structural information of both images.

[0086] High-frequency subband fusion: For high-frequency subbands, a fusion rule based on the larger absolute value is adopted. That is, the pixel value with the larger absolute value in the high-frequency subbands of the thermal infrared and visible light images is selected as the pixel value of the fused high-frequency subband. The formula is as follows:

[0087] in, For the fused high-frequency subband, and These are high-frequency subbands from thermal infrared and visible light images, respectively. This fusion method ensures that the fused high-frequency subbands retain more detailed information, such as edges and textures, resulting in a clearer image in terms of detail.

[0088] Image Reconstruction: Finally, the fused 3D image is obtained through inverse Laplacian pyramid transform, while preserving the image's... 3D depth information. The inverse transform process involves reconstructing the fused low-frequency and high-frequency subbands layer by layer. Starting from the top-level low-frequency subband, the original fused image is gradually recovered through upsampling and addition with the corresponding high-frequency subbands. The specific process is as follows:

[0089] in, For the first The image after layer fusion For the first The high-frequency subband after layer fusion. Through layer-by-layer reconstruction, a complete fused image is finally obtained. This image integrates the temperature feature information of the thermal infrared image and the texture detail information of the visible light image, providing a richer and higher-quality image data foundation for subsequent feature extraction and recognition classification.

[0090] Application examples This invention employs a method for identifying arboreal primates based on the fusion of thermal infrared and visible light images, specifically targeting the Yunnan snub-nosed monkey. Habitat data from Baima Snow Mountain and Mangkang were collected, including over 20,000 pairs of thermal infrared-visible light images, labeled with individual ID, age (juvenile / adult), sex, and posture (climbing / sitting / grooming). Statistical results are shown in Tables 1-3.

[0091] Table 1. Species identification results of Yunnan snub-nosed monkey in different environments. Sample size Accuracy Daytime unobstructed 8000 98.9% Night / Heavy fog 5000 92.5% Strong wind disturbance 3000 88.7% Table 2. Identification results for different Yunnan snub-nosed monkey individuals Sample size Accuracy Re-identification accuracy Juvenile 300 96.9% 93.7% Adult 800 98.5% 97.5% Old 200 97.3% 96.4% Table 3. Results of Yunnan snub-nosed monkey pose recognition for different poses Sample size Accuracy Key features Climbing 350 98.6% Elbow joint angle > 120° + sudden increase in thermal texture contrast Grooming 230 98.1% Wrist-head distance < 15% of body length + local entropy reduction Alertness 200 97.9% Sudden increase in aspect ratio 20% + increased thermal energy in neck Nursing 150 98.4% Abdomen - Larval overlap > 60% + Temperature difference < 2°C As shown in Table 1, the arboreal primate identification method based on thermal infrared and visible light image fusion provided in this embodiment of the invention achieves an accuracy rate of up to 98.9% under conditions of good, unobstructed vision, and maintains an accuracy rate of 92.5% even under poor visibility conditions such as nighttime or dense fog. However, strong winds significantly interfere with identification, reducing the accuracy rate to 88.7%.

[0092] In addition, as can be seen from Table 2, this identification method has a high recognition rate for individuals. The recognition accuracy for Yunnan snub-nosed monkey juveniles, adults and elderly individuals can all reach more than 96%, and the re-identification accuracy for duplicate individuals can also be maintained above 93%. However, the recognition accuracy is reduced due to the rapid growth of juveniles.

[0093] As can be seen from Table 3, this identification method can achieve relatively accurate identification of different postures of Yunnan snub-nosed monkeys, and can achieve an accuracy of about 98% for some postures with obvious features.

[0094] In summary, this invention provides a method for identifying arboreal primates based on the fusion of thermal infrared and visible light images. The method includes simultaneously acquiring visible light and thermal infrared images at the target site; performing radiometric correction on the thermal infrared images; extracting thermal infrared features from the thermal infrared images and biological features from the visible light images; and performing a three-layer matching mechanism based on the extracted thermal infrared and biological features, consisting of coarse matching, static matching, and biological constraints, to output the species identification result. This method combines the complex ecological environment of high-altitude areas to correct and enhance infrared radiation, and then fuses thermal infrared and visible light features for three-layer cross-modal feature matching. This effectively eliminates interference from environmental conditions, improves identification accuracy, and provides key technical support for the physiological monitoring of arboreal primates.

[0095] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying arboreal primates based on the fusion of thermal infrared and visible light images, characterized in that, It includes: S1. Simultaneously acquire visible light and thermal infrared images at the target sample site; S2. Process the acquired thermal infrared and visible light images; S3. Extract the thermal infrared features from the thermal infrared image and the biological features from the visible light image; S4. Based on the extracted thermal infrared features and biological features, a three-layer matching mechanism of coarse matching, static matching, and biological constraints is performed, and image fusion is then performed; S5. Train and optimize the model, and output the species identification results.

2. The method for identifying arboreal primates according to claim 1, characterized in that, The acquired thermal infrared image is radiometrically calibrated by converting the grayscale values ​​in the thermal infrared image into radiance values. The radiometric calibration formula is as follows: ， In the formula, This represents the radiance value corresponding to the thermal infrared data wavelength λ. This is the gain quantity. This is the offset. This represents the gain coefficient for the body temperature region. This is the offset.

3. The method for identifying arboreal primates according to claim 2, characterized in that, For the pixels in the thermal infrared image The radiance values ​​were continuously collected and a time series was constructed. ,in, The radiance value of the pixel in the nth frame is given. A static threshold and a dynamic threshold are set. When the change in the radiance value does not exceed the static threshold, median filtering is performed. When the change in the radiance value is not lower than the dynamic threshold, the original value is retained.

4. The method for identifying arboreal primates according to claim 2, characterized in that, In step S2, the radiance value is corrected according to the following formula. ， In the formula, Radiance, These are sensor observations. For path radiation, Atmospheric transmittance, The hair emissivity of arboreal primates; Furthermore, the altitude was divided into layers every 500 meters, and the temperature, air pressure, and water vapor content of each layer were analyzed using the MODTRAN model. and Make corrections.

5. The method for identifying arboreal primates according to claim 1, characterized in that, The thermal infrared features include temperature statistical features and thermal texture features. The temperature statistical features include the average temperature within the ROI. Standard deviation Maximum temperature With the lowest temperature temperature difference The thermal texture features include energy, entropy, and contrast extracted based on the gray-level co-occurrence matrix.

6. The method for identifying arboreal primates according to claim 5, characterized in that, The thermal infrared features also include thermal LBP features, and the method for extracting the thermal LBP features is as follows: In the thermal infrared image, the temperature of the center pixel is used as a threshold, and the temperatures of 3×3 neighboring pixels are compared to generate the thermal LBP value of the center pixel; the thermal LBP feature is obtained by statistically analyzing the histogram of the thermal LBP values ​​of all pixels in the thermal infrared image.

7. The method for identifying arboreal primates according to claim 1, characterized in that, The visible light features include facial morphological features and body posture features. The facial morphological features include the localization and detection of 10-16 facial feature points, and the body posture features include the construction of a skeleton model containing 18-25 joints. The loss function is optimized using an improved OpenPose algorithm. ， 。 8. The method for identifying arboreal primates according to claim 1, characterized in that, The visible light features also include color features, texture features, and geometric features of the ROI; the color features include establishing an HSV color gamut quantization histogram for the arboreal primate; the texture features employ an improved LBP-TOP operator to extend texture analysis along the time axis, using a triorthogonal plane mode with radii of 3 and 8 neighborhoods to generate a 256-dimensional feature vector; the geometric features include perimeter, area, major axis, minor axis, compactness, and eccentricity.

9. The method for identifying arboreal primates according to claim 1, characterized in that, The three-layer matching mechanism includes: Coarse matching: Using ORB features and body size constraints, select point pairs whose scale ratio matches the size of the arboreal primates; Fine matching: Construct a local biogeometric model and screen matching points based on the biogeometric constraints of the head-to-body ratio and joint symmetry deviation of the arboreal primates; Biological semantic constraints: cross-modal feature correlation verification, pose invariance constraints and mismatch elimination enhancement are performed, and point pairs that simultaneously satisfy scale constraints, body temperature-grayscale threshold and biological structure proportion are retained to construct high-precision registration transformation relationship.

Images