A wild animal target detection method based on edge computing deployment

Abstract

This invention provides a wildlife target detection method based on edge computing deployment, belonging to the field of wildlife target detection technology. This invention achieves edge-end model compression by performing hybrid precision quantization and knowledge distillation on wildlife images. The compressed model is deployed to a multimodal temporal attention fusion detection model to achieve visible light and thermal infrared dual-stream fusion detection. Adaptive inference is achieved by dynamically adjusting inference precision and memory allocation based on resource adaptation evaluation values. Early termination mechanisms and sparse convolutions reduce computational power consumption. A hierarchical wake-up architecture compresses cold start latency. Furthermore, a cross-domain feature alignment algorithm based on Riemannian manifold geodesic interpolation, combined with elastic weight solidification and a test-time adaptive mechanism, continuously suppresses cross-seasonal visual domain drift at the edge. This solves the technical problem of continuous performance degradation of artificial intelligence detection models caused by cross-seasonal visual domain drift in wildlife monitoring scenarios using edge computing devices.

Description

Technical Field

[0001] This invention belongs to the field of wildlife target detection technology, and more specifically, relates to a wildlife target detection method based on edge computing deployment. Background Technology

[0002] Wildlife target detection is a crucial technique in ecological monitoring. Existing methods typically employ target detection frameworks based on deep convolutional neural networks, combined with visible light cameras or thermal infrared sensors for single-modal image analysis, and then deploy the model on cloud servers or local workstations for inference. In practical field deployment scenarios, researchers have explored migrating lightweight detection models to edge computing devices to reduce data transmission latency and adapt to monitoring needs in areas without network coverage. Related applications include forest patrols, endangered species tracking, and population density estimation.

[0003] However, the aforementioned methods face significant domain drift issues when deployed over long periods across seasons. In the field, climatic factors such as leaf color changes in spring, summer, autumn, and winter, snow cover, and rain and fog can cause a persistent shift in the covariance structure of image feature distributions. Traditional domain adaptation methods generally align the covariance matrix based on Euclidean linear interpolation, ignoring the inherent Riemannian manifold geometry of the covariance matrix as a symmetric positive definite matrix. The linear interpolation results are prone to falling outside the positive definite cone, causing distortion in the feature distribution after alignment. This fails to effectively eliminate the statistical structure differences introduced by cross-seasonal visual shifts, leading to a continuous degradation in model detection accuracy over time.

[0004] In other words, existing technologies suffer from the technical problem of continuous performance degradation of detection models caused by cross-seasonal visual domain drift in wildlife monitoring scenarios. Summary of the Invention

[0005] In view of this, the present invention provides a wildlife target detection method based on edge computing deployment, which can solve the technical problem in the prior art where the performance of detection models of edge computing devices continuously degrades due to cross-seasonal visual domain drift in wildlife monitoring scenarios.

[0006] This invention is implemented as follows: This invention provides a wildlife target detection method based on edge computing deployment, comprising the following steps:

[0007] The collected wildlife images are subjected to mixed precision quantization processing. The low-frequency feature layer of the backbone network is quantized with INT8, while the detection head and classification branch retain FP16. Quantization-aware training and knowledge distillation are performed to complete the compression of the edge model.

[0008] The compressed model is deployed to a multimodal temporal attention fusion detection model. The input is a visible light image stream and a thermal infrared image stream. The feature extraction and fusion are completed through a cross-modal cross attention module and temporal difference convolution, and the target detection result is output.

[0009] Calculate the resource adaptation evaluation value of the current frame, and dynamically adjust the memory allocation ratio and inference accuracy parameters of the multimodal temporal attention fusion detection model according to the interval to which the resource adaptation evaluation value belongs, so as to achieve adaptive inference;

[0010] An early termination mechanism is triggered for static background frames, and results for easily identifiable targets are output in advance in the middle layer of the backbone network. Sparse convolution is used for fine calculation of the foreground region to reduce computing power consumption.

[0011] The architecture is designed with a tiered wake-up mechanism. After PIR triggering, the ultra-low-power coprocessor is woken up first to run the ultra-lightweight detection model for coarse screening. Once the confidence level of the coarse screening exceeds the confidence threshold, the main neural network processing unit is woken up, and the core model weights are preloaded into the static random access memory reserved area.

[0012] A cross-domain feature alignment algorithm based on Riemannian manifold geodesic interpolation is adopted. The feature distributions of the source and target domains are represented by symmetric positive definite matrices. The midpoint of the geodesic is calculated as the alignment reference point. Cross-seasonal domain drift correction is completed through logarithmic and exponential mapping. The batch normalization layer parameters of the detection head are fine-tuned at the edge by combining a continuous learning framework and an elastic weight solidification mechanism. The batch normalization statistics are updated online during the inference stage using a test-time adaptive mechanism.

[0013] In the hybrid precision quantization process, quantization-aware training refers to simulating quantization errors during the training phase and introducing pseudo-quantization nodes in the forward propagation to make the weight distribution converge in a direction that is friendly to low-bit representation, thus compensating for the precision loss caused by FP32 to INT8 conversion.

[0014] The knowledge distillation process supervises the training of the student model by using the soft-label output of the high-precision FP32 teacher model. This makes the relative probability distribution between classes of the student model approximate that of the teacher model, thus compensating for the decrease in fine-grained species classification accuracy caused by the accumulation of quantization errors.

[0015] The backbone network of the multimodal temporal attention fusion detection model adopts the improved ConvNeXt-Tiny, inserting a cross-modal cross-attention module at the end of each stage, using the channel mean and variance of the thermal imaging feature map as the query vector, and dynamically modulating the response weights of each channel of the visible light feature map.

[0016] The temporal differential convolution encodes motion trajectory information using frame-by-frame differential tensors of five consecutive feature maps, and performs iterative updates of the hidden state within the network through lightweight convolutional long short-term memory units, forming a spatiotemporal feature stream with memory capabilities.

[0017] The resource adaptation evaluation value is calculated by taking the current edge device's video memory usage, CPU usage, and inference queue backlog as inputs, and calculating the weighted normalized resource adaptation evaluation value. The weight coefficients are determined by least squares regression analysis after running stress tests on the target edge hardware platform.

[0018] The resource adaptation evaluation value range is divided into high load range, medium load range and low load range. The high load range triggers the video memory compression strategy, the medium load range adopts the standard inference configuration, and the low load range enables the full precision extended configuration.

[0019] The early termination mechanism adds a lightweight classification head to the middle layer of the backbone network. The confidence threshold for triggering early termination is determined by progressively enumerating on the validation set and selecting the value that minimizes the computational load while ensuring that the accuracy loss does not exceed a preset proportion of the average accuracy of the validation set.

[0020] In the hierarchical wake-up architecture, the weight scale of the ultra-lightweight detection model running on the ultra-low power coprocessor is less than the weight scale threshold, and the inference latency is less than the latency threshold; after the core detection model weights are preloaded into the static random access memory reserved area, the cold start latency is compressed to within the cold start latency threshold.

[0021] In the cross-domain feature alignment algorithm based on Riemannian manifold geodesic interpolation, the midpoint of the geodesic is obtained by applying a logarithmic mapping to two symmetric positive definite matrices, taking the arithmetic mean, and then mapping back to the manifold through an exponential mapping. The aligned features are used to drive the fine-tuning of the parameters of the first batch of normalized layers in the detection, and the number of fine-tuned parameters is less than the parameter ratio threshold of the total number of model parameters.

[0022] The elastic weight solidification mechanism calculates the diagonal approximation of the Fisher information matrix of the old task parameters as the parameter importance weights. When fine-tuning the new domain, it applies a secondary penalty term to the parameters with high importance to constrain their offset magnitude and prevent catastrophic forgetting.

[0023] The test-time adaptive mechanism does not use labeled samples during the inference phase, but only utilizes the feature distribution information of the current input batch. It updates the mean and variance statistics of the batch normalization layer online by minimizing the prediction entropy of the model output.

[0024] The computational complexity of the cross-domain feature alignment algorithm is... , For the feature vector dimension, The specific value is determined by binary search experiments based on the available memory at the edge before deployment, with the constraint that the time for a single alignment calculation does not exceed a preset proportion of the inference frame interval.

[0025] The coarse screening confidence threshold is determined by using receiver operating characteristic (ROC) curve analysis on the validation set of the ultralight detection model with the constraint that the false negative rate does not exceed the false negative rate threshold.

[0026] The weight size threshold is 1. The delay threshold is 50. The cold start delay threshold is 150. The threshold for fine-tuning parameters is 0.1%; the early termination accuracy loss ratio is 2% of the average accuracy of the validation set; the early termination confidence threshold is determined by enumeration in steps of 0.05 within the range of 0.5 to 0.9; the batch size for online batch normalization updates is 4 to 16 frames; the elastic weight solidification penalty coefficient is... to Determined by logarithmic uniform sampling search within the range; feature vector dimension The value range is 128 to 512; the false negative rate threshold is 5%.

[0027] This invention employs a cross-domain feature alignment algorithm based on Riemannian manifold geodesic interpolation. It represents the feature distributions of the source and target domains as symmetric positive definite matrices, calculates the midpoint of the geodesic on the Riemannian manifold as the alignment reference point, and completes cross-seasonal domain drift correction through logarithmic and exponential mapping. This solves the fundamental defect of feature distribution distortion caused by the destruction of positive definiteness constraints due to Euclidean linear interpolation of the covariance matrix.

[0028] Since the logarithmic mapping locally linearizes the curved Riemannian manifold into a tangent space, and the exponential mapping precisely maps the alignment result back to the manifold, the two constitute an inverse coordinate transformation pair. This ensures that the entire alignment process always takes place within a symmetric positive definite matrix space, eliminating cross-seasonal covariance structure differences in a geometrically optimal sense and guaranteeing the statistical validity of the aligned feature distribution. Furthermore, by combining the elastic weight solidification mechanism and the test-time adaptive mechanism, the performance drift and degradation of the edge model caused by seasonal changes are continuously suppressed without relying on paired labeled samples.

[0029] In summary, this invention solves the technical problem mentioned in the background art of continuous performance degradation of detection models caused by cross-seasonal visual domain drift in edge computing devices in wildlife monitoring scenarios. Attached Figure Description

[0030] Figure 1 This is a flowchart of the method of the present invention.

[0031] Figure 2 This diagram shows the hardware architecture of the edge node and the connection relationship between the dual-modal sensor acquisition link.

[0032] Figure 3 A comparison chart showing the bit width allocation of parameters in each layer and the retention of model accuracy before and after mixed precision quantization.

[0033] Figure 4 This is a graph showing the relationship between accuracy retention rate and computational compression ratio as a function of threshold during the early retirement confidence threshold enumeration process.

[0034] Figure 5 A visualization comparison of the dimensionality reduction of the feature distributions of the source and target domains before and after geodesic interpolation alignment of the Riemannian manifold.

[0035] Figure 6 This diagram illustrates the changes in timing status and confidence output at each stage of a PIR-triggered event in a graded wake-up architecture. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below.

[0037] like Figure 1 The diagram shown is a flowchart of a wildlife target detection method based on edge computing deployment provided by the present invention. This method includes the following steps:

[0038] S01. Perform mixed precision quantization on the collected wildlife images. Use INT8 quantization for the low-frequency feature layer of the backbone network, retain FP16 for the detection head and classification branch, and perform quantization-aware training and knowledge distillation to complete edge model compression.

[0039] S02. Deploy the compressed model to the multimodal temporal attention fusion detection model. Input visible light image stream and thermal infrared image stream. Complete feature extraction and fusion through cross-modal cross attention module and temporal difference convolution, and output target detection results.

[0040] S03. Calculate the resource adaptation evaluation value of the current frame, and dynamically adjust the memory allocation ratio and inference accuracy parameters of the multimodal temporal attention fusion detection model according to the interval to which the resource adaptation evaluation value belongs, so as to achieve adaptive inference.

[0041] S04. Trigger an early termination mechanism for static background frames, output results for easily judged targets in advance in the middle layer of the backbone network, and use sparse convolution for fine calculation of the foreground region to reduce computing power consumption.

[0042] S05. Design a hierarchical wake-up architecture. After PIR triggering, the ultra-low power coprocessor is woken up first to run the ultra-lightweight detection model for coarse screening. After the confidence level of the coarse screening exceeds the confidence threshold, the main neural network processing unit is woken up. The core model weights are preloaded into the static random access memory reserved area.

[0043] S06. A cross-domain feature alignment algorithm based on Riemannian manifold geodesic interpolation is adopted. The feature distributions of the source and target domains are represented by symmetric positive definite matrices. The midpoint of the geodesic is calculated as the alignment reference point. Cross-seasonal domain drift correction is completed through logarithmic mapping and exponential mapping. The batch normalization layer parameters of the detection head are fine-tuned at the edge by combining a continuous learning framework and an elastic weight solidification mechanism. The batch normalization statistics are updated online during the inference stage using a test-time adaptive mechanism.

[0044] Hybrid precision quantization refers to the technique of using different numerical precisions for parameter representation and inference at different layers of the neural network. The low-frequency feature layers of the backbone network are less sensitive to precision; using INT8 can significantly reduce storage and computation. However, the activation value distribution of the detector head and classification branches exhibits a long-tail effect, so FP16 is retained to prevent a non-linear decrease in classification accuracy. Quantization-aware training simulates quantization errors during the training phase, introducing pseudo-quantization nodes into the model during forward propagation. This causes the weight distribution to converge towards a lower-bit representation-friendly direction, thus compensating for the precision loss caused by converting FP32 to INT8. Knowledge distillation refers to the process of supervising the training of the student model using the soft-label output of the high-precision FP32 teacher model. This makes the inter-class relative probability distribution of the student model approximate that of the teacher model, further compensating for the fine-grained species classification accuracy decrease caused by the accumulation of quantization errors. The original decrease was 8–15 percentage points; this loss can be effectively controlled after combined processing with hybrid precision quantization and knowledge distillation.

[0045] The specific structure of the multimodal temporal attention fusion detection model is as follows: The multimodal temporal attention fusion detection model uses visible light image stream and thermal infrared image stream as dual input branches. The backbone network adopts an improved ConvNeXt-Tiny, and a cross-modal cross-attention module is inserted at the end of each stage. The channel mean and variance of the thermal imaging feature map are used as the query vector to dynamically modulate the response weights of each channel of the visible light feature map, thereby achieving gating enhancement of texture features by physical temperature information. In the temporal dimension, learnable temporal differential convolution is introduced. Motion trajectory information is encoded by frame-by-frame differential tensors of five consecutive frames of feature maps, and the hidden state is iteratively updated within the network through lightweight convolutional long short-term memory units to form a spatiotemporal feature stream with memory capabilities. The detection head adopts a dual-output structure with decoupled anchor box prediction path and center offset prediction path. Global context modeling adopts an attention mechanism based on sparse Transformer. The dynamic receptive field is implemented using deformable convolution version 3. The quantization strategy is a hybrid precision scheme of backbone network INT8 and detection head FP16. When an occlusion frame occurs... The model automatically triggers internal jump paths to directly reuse the hidden state output of the previous frame, reducing redundant computation. Regarding resource allocation, the model divides memory allocation into three levels based on resource adaptation evaluation values: when the resource adaptation evaluation value is in the high-load range, the query computation of the cross-modal attention module is switched to a single-channel statistical approximation, the number of attention heads in the sparse Transformer is compressed to 50% of the original design, the hidden state dimension of the convolutional long short-term memory unit is reduced to 64, and the feature map cache for each stage of the backbone network uses INT8 static values. The system optimizes storage, keeping peak VRAM usage below 70% of the device's total VRAM. When the resource adaptation evaluation value is in the medium load range, the cross-modal attention module restores dual-channel statistics, the number of attention heads is restored to 75%, the hidden state dimension of the convolutional long short-term memory unit is increased to 128, and the upper limit of VRAM allocation is relaxed to 85%. When the resource adaptation evaluation value is in the low load range, the model runs at full precision and full head count, with an upper limit of VRAM allocation of 95%. At this time, the number of sampling points in the third version of deformable convolution is expanded from the default 4 to 8 to enhance the dynamic receptive field coverage.

[0046] The steps for establishing the training dataset for the multimodal temporal attention fusion detection model specifically include: collecting visible light images and synchronous thermal infrared images of wild animals covering all four seasons and various climatic conditions such as sunshine, rain, snow, and fog, with an image resolution range of 640×480. Up to 1920×1080 The images were manually annotated, including species category bounding boxes and instance masks. Data augmentation was performed on the annotated data, including random horizontal flipping, multi-scale cropping, and random perturbation of brightness and color temperature (the color temperature perturbation range was based on the sensor's aging color temperature drift of 500–1000). The dataset was divided into training, validation, and test sets in an 8:1:1 ratio. Background frame sequences without animal targets were collected for negative sample training of the early termination mechanism.

[0047] The training steps of the multimodal temporal attention fusion detection model specifically include: using the AdamW optimizer, with an initial learning rate set to... The weight decay coefficient is set to The training consists of 200 rounds. The first 10 rounds use a linear warm-up strategy, followed by cosine annealing. The loss function is a weighted sum of the classification cross-entropy loss, bounding box regression loss, and center offset loss of the detection head. The parameters of the cross-modal cross-attention module and the temporal differential convolution are trained at a low learning rate in the first 50 rounds. The preheating training is conducted, and after 50 rounds, it is jointly trained with the backbone network using the same learning rate. The quantization perception training phase starts at the 150th round, and after inserting pseudo-quantization nodes, it continues to train for another 50 rounds. The knowledge distillation teacher model adopts the full-precision version of FP32. The distillation temperature coefficient is determined by selecting the optimal value based on the average precision of the validation set after training 30 rounds on each of the three candidate values ​​of 4, 6, and 8 on the validation set.

[0048] The multimodal temporal attention fusion detection model brings the following technical benefits to the overall solution: The cross-modal cross-attention module injects thermal infrared temperature distribution information as a dynamic gating signal into the visible light feature channel, enabling the model to recover the target contour from the temperature gradient even at night or in low light conditions, compensating for the degradation of texture features under extreme lighting; Temporal differential convolution explicitly encodes the motion difference information between consecutive frames into feature tensors, enabling the model to predict fast-moving targets, while the hidden state memory mechanism of the convolutional long short-term memory unit allows the model to accumulate contextual evidence of target appearance across frames, reducing the probability of missed detection caused by occlusion or brief frame drops; The internal jump path mechanism allows occluded frames to avoid repeating full inference, directly reusing the hidden state to reduce redundant computation, thus improving overall inference efficiency; The resource allocation linkage mechanism enables the model to actively shrink the computation graph size when hardware resources fluctuate at the edge, avoiding inference interruptions caused by memory overflow and ensuring long-term deployment stability.

[0049] The resource adaptation evaluation function is defined as follows: Using the current edge device's memory usage, CPU usage, and inference queue backlog of frames as input, a weighted normalized resource adaptation evaluation value is calculated, expressed by the following formula: ;in This represents the video memory usage ratio. Processor utilization rate Queue backlog ratio , , The weighting coefficients and The values ​​of these three parameters were determined using least squares regression analysis after running 100 stress tests under different load scenarios on the target edge hardware platform; when When this is identified as a high-load zone, a memory compression strategy is triggered; when When the load is determined to be in the medium load range, the standard inference configuration is used; when When the load is low, the full-precision extended configuration is enabled.

[0050] The early termination mechanism involves adding a lightweight classification head to the intermediate layer of the backbone network. For easily identifiable targets with concentrated confidence distributions, the detection result is output and subsequent calculations are terminated before feature extraction is complete. Only targets with dispersed confidence are propagated to the complete network, thereby reducing the average computational cost per frame. The confidence threshold for triggering early termination is determined by progressively enumerating the range of 0.5 to 0.9 on the validation set with a step size of 0.05, selecting the value that minimizes computational cost while ensuring that the accuracy loss does not exceed 2% of the average accuracy of the validation set.

[0051] Sparse convolution refers to performing convolution kernel operations only on spatial locations with non-zero activation values, skipping zero-value calculations in the background region, thereby concentrating the computation on the foreground target region. It is particularly effective for images with sparsely distributed targets in outdoor scenes. The average computational power consumption compression ratio is estimated by statistically analyzing the proportion of foreground pixels on typical wildlife image sets. The proportion of foreground pixels is usually in the range of 5% to 25%.

[0052] In the tiered wake-up architecture, the weight scale of the ultra-lightweight detection model running on the ultra-low-power coprocessor is less than 1. Inference delay less than 50 The cold start latency of the core detection model weights is compressed to 150 after preloading them into the static random access memory reserved area. Within, the 150 The threshold was determined by statistically analyzing the time it took for fast-moving animals such as birds to cross the camera's field of view (with a sample size of no less than 200 valid crossing events, and the 5th percentile of the crossing time distribution), ensuring that the first frame inference is completed within the target's effective dwell window. The coarse-screen confidence threshold was determined by using receiver operating characteristic (ROC) curve analysis on the validation set of the lightweight detection model, with a false negative rate not exceeding 5%, to determine the optimal operating point.

[0053] The specific implementation of the cross-domain feature alignment algorithm based on Riemannian manifold geodesic interpolation is as follows: Extract the feature vector sets of the source and target domains respectively, calculate the sample covariance matrix of the feature vectors of each domain, where the covariance matrix is ​​a symmetric positive definite matrix belonging to a point on the symmetric positive definite matrix Riemannian manifold; on the symmetric positive definite matrix Riemannian manifold, the geodesic distance between two points is given by a logarithmic Euclidean metric, and the midpoint of the geodesic is obtained by applying a logarithmic mapping to the two symmetric positive definite matrices, taking the arithmetic mean, and then mapping back to the manifold via an exponential mapping; project the features of the source and target domains onto the tangent space at the midpoint of the geodesic through logarithmic mapping, perform standard Euclidean subtraction alignment within the tangent space, and then map the aligned features back to the symmetric positive definite matrix manifold via an exponential mapping; the aligned features are used to drive the fine-tuning of the parameters of the first batch of normalization layers in the multimodal temporal attention fusion detection model, with the number of fine-tuned parameters being less than 0.1% of the total number of model parameters. The computational complexity of the algorithm is... , For the feature vector dimension, The value ranges from 128 to 512. The specific value is determined by binary search experiments before deployment based on the available memory at the edge. The constraint is that the time for a single alignment calculation does not exceed 10% of the inference frame interval.

[0054] The cross-domain feature alignment algorithm based on Riemannian manifold geodesic interpolation brings the following technical benefits: Traditional domain adaptation methods in Euclidean space treat the covariance matrix as an ordinary vector for linear interpolation, ignoring the positive definiteness constraint of the covariance matrix. The interpolation result is prone to falling outside the positive definite cone, leading to distortion of the feature distribution. Geodesic interpolation on Riemannian manifolds is always performed within the symmetric positive definite matrix space, and the aligned feature distribution maintains statistical validity. Logarithmic mapping locally linearizes the curved manifold into the tangent space, and exponential mapping accurately maps the tangent space result back to the manifold. The two constitute an inverse coordinate transformation pair, which eliminates the covariance structure difference caused by cross-seasonal visual distribution shifts (leaf color changes, snow cover, etc.) in a geometrically optimal sense. The characteristic of not requiring paired samples makes it directly applicable in unsupervised deployment scenarios in the field, effectively suppressing the continuous drift and degradation of model performance.

[0055] The implementation of the continuous learning framework combined with the elastic weight solidification mechanism is as follows: Elastic weight solidification uses the diagonal approximation of the Fisher information matrix of the old task parameters as the parameter importance weights. During fine-tuning in the new domain, a quadratic penalty term is applied to parameters with high importance to constrain their offset magnitude and prevent catastrophic forgetting. The diagonal approximation of the Fisher information matrix is ​​estimated by the mean squared gradient after performing one forward propagation on the old task validation set. The elastic weight solidification penalty coefficient is obtained by sampling logarithmically uniformly on the continuous learning benchmark validation set. to Search for 5 candidate values ​​within the range, and select the value with the highest accuracy retention rate of the old task.

[0056] The test-time adaptive mechanism refers to the use of no labeled samples during the inference phase, but only the feature distribution information of the current input batch. By minimizing the prediction entropy of the model output, the mean and variance statistics of the batch normalization layer are updated online, so that the batch normalization parameters of the model are closer to the current domain distribution, thus achieving unsupervised online adaptation. The batch size for each update is 4 to 16 frames. The specific value is determined by compromise after testing the memory usage and adaptive convergence speed under different batch sizes on the target edge hardware.

[0057] Optionally, the present invention also provides a method for implementing a wildlife target detection system based on edge computing deployment via a computer, wherein the computer is provided with a readable storage medium storing program instructions, which can execute the above-described method when the computer is run.

[0058] The specific implementation of step S01 is as follows: First, the collected wildlife images are subjected to hierarchical precision allocation. The low-frequency feature layers of the backbone network are less sensitive to quantization errors, so INT8 fixed-point quantization is used for their weights and activation values ​​to significantly reduce storage usage and multiplication / addition operations. The activation value distribution of the detection head and classification branch exhibits a long-tail effect, so FP16 half-precision floating-point representation is retained to prevent a non-linear decrease in classification accuracy. After precision allocation, pseudo-quantization nodes are inserted into each layer of the backbone network during the training phase to simulate the rounding error introduced by INT8 quantization, so that the weight distribution converges in a direction friendly to low-bit representation during forward propagation. This process is called quantization-aware training. Based on quantization-aware training, the soft label probability distribution output by the teacher model with full-precision FP32 parameters is used as a supervision signal to train the quantized student model, so that the inter-class relative probability distribution of the student model approximates that of the teacher model. Knowledge distillation is used to compensate for the decrease in fine-grained species classification accuracy caused by the accumulation of quantization errors. The distillation temperature coefficient is selected as the optimal value based on the average precision of the validation set after training 30 rounds on each of the three candidate values ​​of 4, 6, and 8. After completing the above steps, a compression model suitable for edge deployment is obtained.

[0059] The specific implementation of step S02 is as follows: The compressed model obtained in step S01 is deployed to the multimodal temporal attention fusion detection model. This model uses visible light image streams and thermal infrared image streams as dual input branches. The backbone network adopts an improved ConvNeXt-Tiny architecture. A cross-modal cross-attention module is inserted at the end of each stage. A query vector is constructed using the channel mean and variance of the thermal imaging feature map. The response weights of each channel of the visible light feature map are dynamically modulated, and physical temperature gradient information is injected into the texture feature channel in a gated manner. A learnable temporal differential convolution is introduced in the temporal dimension. The frame-by-frame differential tensors of five consecutive frames of feature maps are used as motion trajectory encoding inputs. Lightweight convolutional long short-term memory units are used to iteratively update the hidden states within the network, forming a spatiotemporal feature stream that accumulates contextual evidence across frames. When an occluded frame appears, the model automatically triggers an internal jump path, directly reusing the hidden state output of the previous frame, avoiding repeated full inference. The detection head adopts a dual-output structure that decouples the anchor frame prediction path and the center offset prediction path. The global context modeling is based on the sparse Transformer attention mechanism. The dynamic receptive field is implemented using deformable convolution version 3. The quantization strategy is a hybrid precision scheme of backbone network INT8 and detection head FP16.

[0060] The specific implementation of step S03 is as follows: using the current edge device's video memory usage, CPU usage, and inference queue backlog of frames as input, and applying the weighted normalization formula... Computational resource adaptation assessment value Weighting coefficient , , satisfy The three factors were determined using least squares regression analysis after running 100 stress tests on the target hardware platform. When the load is determined to be high, the query calculation of the cross-modal attention module is switched to single-channel statistical approximation, the number of sparse Transformer attention heads is compressed to 50%, the hidden state dimension of the convolutional long short-term memory unit is reduced to 64, and the peak memory usage is controlled within 70% of the total memory. When the load is determined to be in the medium load range, the number of attention heads recovers to 75%, the hidden state dimension increases to 128, and the memory limit is relaxed to 85%; When the time is determined to be a low load range, the model runs at full precision and full head count. The number of sampling points for deformable convolution third edition has been expanded from the default 4 to 8, and the maximum memory limit is 95%.

[0061] The specific implementation of step S04 is as follows: A lightweight classification head is added to the intermediate layer of the backbone network of the multimodal temporal attention fusion detection model. For each input frame, it is first determined whether it is a static background frame. If the difference magnitude between the current frame and the previous frame is lower than the background judgment threshold, an early termination mechanism is triggered. For easily judged targets with concentrated confidence distribution in the intermediate layer, the detection result is directly output and subsequent feature propagation is terminated; for difficult-to-judge targets with dispersed confidence, propagation continues to the complete network. Sparse convolution is used for the foreground region, and convolution kernel operation is performed only on spatial locations with non-zero activation values, skipping redundant calculations at zero-value background locations, and concentrating computing power on the sparsely distributed foreground region of the target. The confidence threshold for triggering early termination is determined by gradually enumerating in the range of 0.5 to 0.9 with a step size of 0.05 on the validation set, and selecting the value that minimizes the computational cost while ensuring that the accuracy loss does not exceed 2% of the average accuracy of the validation set.

[0062] The specific implementation of step S05 is as follows: Deploy a hierarchical wake-up architecture. After the PIR passive infrared sensor detects a motion signal, it prioritizes waking up the ultra-low-power coprocessor, which has a weight scale of less than 1. Inference delay less than 50 The ultra-lightweight detection model performs coarse screening on the current frame. The coarse screening confidence threshold is determined by using receiver operating characteristic (ROC) curve analysis on the validation set, with a false negative rate not exceeding 5%. If the coarse screening confidence exceeds this threshold, the main neural network processing unit is activated to run the core detection model. The weights of the core detection model are preloaded into the static random access memory (SRAM) reserve area at system startup, and the cold start latency is compressed to 150. Within 150 The threshold is determined by statistically analyzing at least 200 valid crossing events of fast-moving animals such as birds across the camera's field of view, and taking the 5th percentile of the crossing time distribution to ensure that the first frame inference is completed within the target's effective dwell window.

[0063] The specific implementation of step S06 is as follows: Extract the feature vector sets of the source domain and the target domain respectively, calculate the sample covariance matrix, and treat it as a point on the symmetric positive definite matrix Riemannian manifold. Apply a logarithmic mapping to the covariance matrices of the source domain and the target domain respectively, project the two matrices to the tangent space, take the arithmetic mean, and then map them back to the manifold through an exponential mapping to obtain the midpoint of the geodesic as the alignment reference point; project the features of the source domain and the target domain to the tangent space through a logarithmic mapping, perform Euclidean subtraction alignment in the tangent space, and then map the aligned features back to the manifold through an exponential mapping. The aligned features are used to drive the fine-tuning of the parameters of the first batch of normalization layers in the detection. The number of fine-tuned parameters is less than 0.1% of the total number of model parameters, and the computational complexity of the algorithm is O(n log n). Feature vector dimension The value ranges from 128 to 512, with the specific value determined through binary search experiments. The constraint is that the alignment time for a single iteration does not exceed 10% of the inference frame interval. The elastic weight solidification mechanism performs forward propagation on the old task validation set, estimating the diagonal approximation of the Fisher information matrix using the mean squared gradient. During fine-tuning in the new domain, a secondary penalty constraint is applied to important parameters, with the penalty coefficient being... to The algorithm searches for 5 candidate values ​​using logarithmic uniform sampling within the range before determining the final value. During testing, the adaptive mechanism utilizes the feature distribution of frames 4-16 of the current batch during the inference phase to minimize the prediction entropy and update the batch normalized statistics online, thus achieving unsupervised online adaptation.

[0064] It should be noted that the key technologies of this invention include: a cross-domain feature alignment technology based on Riemannian manifold geodesic interpolation, which replaces traditional Euclidean linear interpolation with geometric alignment along a symmetric positive definite matrix manifold geodesic, fundamentally ensuring the positive definiteness of the covariance matrix after alignment and eliminating feature distribution distortion caused by linear interpolation falling outside the positive definite cone; a cross-modal cross-attention and temporal differential convolution fusion technology, which injects the thermal infrared temperature gradient into the visible light feature channel in a gated manner, while explicitly encoding the motion difference between consecutive frames as spatiotemporal features, enabling the model to recover the target contour from temperature information even under low light conditions; and an adaptive inference technology driven by resource adaptation evaluation, which adjusts the computation graph size in conjunction with weighted normalized evaluation values, actively shrinking the model complexity when edge hardware resources fluctuate, and avoiding inference interruption caused by memory overflow. The synergistic effect of these three technologies enables the model to simultaneously possess cross-seasonal robustness, multimodal perception capability, and long-term stable deployment capability at the edge, a comprehensive effect that no single technology can achieve independently.

[0065] It should be noted that with long-term deployment of wildlife monitoring equipment at the edge of the ecosystem, the feature covariance structure continuously shifts due to visual factors such as leaf color, snow cover, rain, and fog in different seasons. Without retraining, the model's detection accuracy will cumulatively degrade over time. The reason for this technical problem is that the covariance matrix is ​​a symmetric positive definite matrix, its geometry described by a Riemannian manifold. Linear interpolation in Euclidean space cannot maintain positive definiteness constraints. When the alignment result falls outside the positive definite cone, the feature statistics lose their physical meaning, and the model's response to new domain samples exhibits systematic bias. This bias gradually amplifies with seasonal changes and cannot be automatically corrected. Common solutions to this problem include periodically re-collecting labeled data and retraining the model, or using a batch normalized statistical replacement method in Euclidean space for simple domain adaptation. However, periodic retraining requires significant manual annotation costs and suffers from deployment window interruptions. Euclidean space batch normalization replacement ignores the Riemannian geometry of the covariance matrix, limiting its ability to correct overall shifts in multi-layer feature distributions and failing to fundamentally eliminate cross-seasonal covariance structure differences. Consequently, model performance continues to drift over long-term deployments. This invention effectively solves this technical problem by using Riemannian manifold geodesic interpolation to confine the alignment operation within a symmetric positive definite matrix manifold. The inverse relationship between logarithmic and exponential mappings ensures the geometric optimality of the alignment path, enabling statistically legitimate elimination of cross-seasonal covariance structure differences. Simultaneously, an elastic weight solidification mechanism prevents catastrophic forgetting of learned knowledge from previous seasons during fine-tuning, and an adaptive mechanism corrects temporal offsets in real-time during inference. These three elements together form a fully automatic cross-seasonal domain adaptation closed loop that requires no paired labeled samples, fundamentally cutting off the propagation path of performance degradation caused by domain drift.

[0066] Specifically, the principle of this invention is:

[0067] This invention addresses the technical problem of continuous performance degradation in detection models caused by cross-seasonal visual domain drift. The fundamental reason lies in its alignment algorithm designed based on the geometric essence of the covariance matrix. The covariance matrix is ​​a symmetric positive definite matrix, and its entirety constitutes a curved Riemannian manifold, i.e., a symmetric positive definite matrix Riemannian manifold, rather than a flat Euclidean space. Traditional methods treat the covariance matrix as an ordinary vector for linear interpolation, essentially performing operations in an incorrect geometric space. The interpolation path does not follow the geodesic of the manifold, inevitably producing invalid results that deviate from the positive definite cone. The resulting distortion in feature distribution is the direct cause of model accuracy degradation.

[0068] This invention maps the covariance matrices of the source and target domains from a curved Riemannian manifold to a tangent space at the midpoint of the geodesic through a logarithmic mapping. The tangent space is a locally flat Euclidean approximation, and performing a standard Euclidean subtraction alignment operation within the tangent space is mathematically valid and precise. Subsequently, an exponential mapping maps the alignment result back to the manifold, ensuring that the output remains a valid symmetric positive definite matrix. The logarithmic and exponential mappings are inverses of each other, constituting a precise bidirectional coordinate transformation between the manifold and the tangent space. This mathematical structure guarantees that the alignment process proceeds along the geodesic, representing a geometrically optimal path in the Riemannian metric sense, fundamentally avoiding the distortion problem of Euclidean linear interpolation.

[0069] Building upon this foundation, the elastic weight solidification mechanism estimates the importance of each parameter to the old task through a diagonal approximation of the Fisher information matrix. During fine-tuning in the new domain, it applies a secondary penalty constraint to important parameters to prevent catastrophic forgetting, allowing the model to retain existing detection capabilities while continuously adapting to new seasonal feature distributions. The test-time adaptive mechanism utilizes the feature distribution of the current batch input during the inference phase, updating the batch normalized statistics online by minimizing the prediction entropy. This achieves unsupervised online adaptation, further mitigating the instantaneous domain shift during inference. The synergistic effect of these three mechanisms enables the model to continuously resist cross-seasonal visual distribution changes even under field deployment conditions with unlabeled paired samples. This mechanism fundamentally cuts off the propagation path of performance degradation caused by domain drift, demonstrating logical consistency and technical feasibility.

[0070] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0071] The specific implementation of step S01 is as follows: When performing mixed-precision quantization processing on the acquired wildlife images, the low-frequency feature layer of the backbone network uses INT8 quantization, while the detection head and classification branch retain FP16 precision, balancing computational efficiency and classification accuracy. During the quantization-aware training phase, pseudo-quantization nodes are inserted into the weights and activation values ​​during forward propagation to simulate INT8 fixed-point quantization errors, causing the weight distribution to converge towards a lower-bit representation-friendly direction, thereby compensating for the precision loss introduced by the FP32 to INT8 conversion. In the knowledge distillation phase, the soft-label output of the FP32 full-precision teacher model supervises the INT8 student model. The distillation loss function is expressed as follows:

[0072] ;

[0073] In the formula, For distillation loss, The distillation temperature coefficient was determined by training on the validation set for 30 rounds each with candidate values ​​4, 6, and 8, and then using the average precision of the validation set. This represents the batch sample size. The total number of species categories, For the teacher model to the first The first sample The class soft label probability is output as a logit vector by the teacher model. Calculated using temperature scaling Softmax, i.e. , For the teacher model to the first The first sample The class outputs the logit value. For the student model to the first The first sample The soft label probability of the class is output by the logit value from the student model. Calculated using Softmax scaling at the same temperature. and All are dimensionless probability values. It is a dimensionless scalar. The original decrease in accuracy for fine-grained species classification was 8–15 percentage points, but this loss can be effectively controlled after the above combined treatment.

[0074] The specific implementation of step S02 is as follows: the compressed model is deployed to a multimodal temporal attention fusion detection model, with visible light image streams and thermal infrared image streams as inputs. The backbone network adopts an improved convolutional neural network structure, with a cross-modal cross-attention module inserted at the end of each stage. A query vector is constructed using the channel mean and variance of the thermal imaging feature map, and gating enhancement is applied to each channel of the visible light feature map. The response weight calculation formula of the cross-modal cross-attention module is expressed as follows:

[0075] ;

[0076] In the formula, For the first The dimensionless gating weight of the channel, It is the Sigmoid activation function. To query the projection matrix, The first thermal infrared feature map Channel mean with standard deviation The query vector is constructed by concatenation. The key projection matrix, The visible light feature map The feature vector of the channel, The dimension of the key vector is denominator. This is used to scale the dot product result, making the overall fraction a dimensionless scalar. A learnable temporal difference convolution is introduced to the temporal dimension, encoding the frame-by-frame difference tensors of the feature maps across five consecutive frames. The difference tensors are defined as follows:

[0077] ;

[0078] In the formula, For the first Frame and the Difference between frame feature maps and These are the feature map tensors for the current frame and the previous frame, respectively. All three have the same dimensions and are feature activation value tensors. The convolutional long short-term memory unit iteratively updates the hidden state. When an occluded frame appears, the model automatically triggers an internal jump path to directly reuse the hidden state output of the previous frame to reduce redundant computation. The detection head adopts a decoupled dual-output structure to predict the anchor box and center offset separately. Global context modeling uses an attention mechanism based on a sparse transformer, and the dynamic receptive field is implemented using deformable convolution version 3.

[0079] The specific implementation of step S03 is as follows: using the current edge device's video memory usage, processor usage, and inference queue backlog of frames as input, the resource adaptation evaluation value is calculated, and the formula is expressed as follows:

[0080] ;

[0081] In the formula, For dimensionless resource adaptation assessment values, This represents the current video memory usage. This refers to the total video memory of the device; both are in the same unit. This is a dimensionless video memory usage ratio. This represents the current processor load. Both refer to the total processor load capacity and are in the same unit. This is a dimensionless processor utilization rate. The number of frames currently backlogged in the inference queue. This represents the maximum capacity of the queue; both units are the same. This is the dimensionless queue backlog ratio. , , The weight coefficients are dimensionless and satisfy the following conditions: The three factors were determined using least squares regression analysis after running 100 sets of stress tests under different load scenarios on the target edge hardware platform. When the system is identified as being in a high-load zone, the cross-modal attention module is switched to a single-channel statistical approximation, the number of attention heads is reduced to 50% of the original design, the hidden state dimension of the convolutional long short-term memory unit is reduced to 64, and the peak memory usage is controlled within 70% of the total memory usage of the device. When the load is determined to be in the medium load range, the number of attention heads recovers to 75%, the hidden state dimension increases to 128, and the memory limit is relaxed to 85%; When the time is determined to be a low load range, the model runs at full precision and full head count, with a memory limit of 95%, and the number of sampling points for deformable convolution third edition is expanded from the default 4 to 8.

[0082] The specific implementation of step S04 is as follows: An early termination mechanism is triggered for static background frames. A lightweight classification head is added to the intermediate layer of the backbone network. Results are output early for easily identifiable targets within the confidence distribution set, terminating subsequent calculations. The confidence threshold for triggering early termination is determined by progressively enumerating on the validation set in the range of 0.5 to 0.9 with a step size of 0.05. The constraint is that the accuracy loss does not exceed 2% of the average accuracy of the validation set. Sparse convolution is used for fine calculation in the foreground region, performing convolution kernel operations only on spatial locations with non-zero activation values, skipping zero-value calculations in the background region. The proportion of foreground pixels is typically 5% to 25%, effectively compressing computational power consumption.

[0083] The specific implementation of step S05 is to design a hierarchical wake-up architecture, in which the ultra-low power coprocessor is woken up first after being triggered by the passive infrared sensor, and the weight scale of the coprocessor is less than 1. Inference delay less than 50 A lightweight detection model is used for coarse screening. The confidence threshold for coarse screening is determined by using receiver operating characteristic (ROC) curve analysis on the validation set with a false negative rate not exceeding 5%. Once the coarse screening confidence exceeds the confidence threshold, the main neural network processing unit is activated, the core model weights are preloaded into the static random access memory (SRAM) reserve area, and the cold start latency is compressed to 150. Within this threshold, the threshold is determined by statistically analyzing at least 200 effective crossing events of fast-moving animals such as birds crossing the camera's field of view, and taking the 5th percentile of the crossing time distribution to ensure that the first frame inference is completed within the target's effective dwell window.

[0084] The specific implementation of step S06 is to extract the source domain feature vector set by using a cross-domain feature alignment algorithm based on Riemannian manifold geodesic interpolation. With the set of feature vectors of the target domain The covariance matrix of samples from each domain is calculated separately, and the formula is expressed as follows:

[0085] ;

[0086] ;

[0087] In the formula, and These are the feature covariance matrices of the source domain and the target domain, respectively. A symmetric positive definite matrix is ​​a point on a symmetric positive definite matrix Riemannian manifold. and These represent the number of samples in the source and target domains, respectively. Let be the sample mean vector of the source domain feature vectors. Let be the sample mean vector of the feature vectors of the target domain. For the source domain 1 eigenvector For the target domain 1 eigenvector for dimensional outer product matrix, The dimension of the feature vector is 1, and both sides of the equals sign are 1. A 3D matrix. The midpoint of a geodesic is calculated using logarithmic and exponential mappings, as expressed by the following formula:

[0088] ;

[0089] In the formula, Let be the midpoint of the geodesic of the covariance matrices of the source and target domains on the Riemannian manifold, and let be 3D symmetric positive definite matrix For The Riemann logarithmic mapping with base point will Mapped to The tangent space at point A is computed in the form of: ,in This is the matrix logarithm function, obtained by performing eigenvalue decomposition on the matrix, taking the natural logarithm of each eigenvalue, and then reconstructing the matrix. For the corresponding Riemann index mapping, mapping the tangent space vector back to the symmetric positive definite matrix manifold, the two form a mutually inverse coordinate transformation pair, and both sides of the equation are... A symmetric positive definite matrix. Aligned features are used to drive fine-tuning of the normalization layer parameters in the first batch of detection. The number of fine-tuned parameters is less than 0.1% of the total number of model parameters, and the overall algorithm computational complexity is O(n log n). To prevent catastrophic forgetting by incorporating a flexible weight solidification mechanism, the formula for the flexible weight solidification penalty term loss is as follows:

[0090] ;

[0091] In the formula, For the dimensionless elastic weight fixed penalty term loss, To solidify the penalty coefficient for dimensionless elastic weights, logarithmic uniform sampling is performed on the continuous learning benchmark validation set. to After searching for 5 candidate values ​​within the range, the value with the highest accuracy retention rate from the old task is selected as the final value. For model parameter indexing, For the first The diagonal approximation of the Fisher information matrix corresponding to each parameter is a dimensionless scalar. This is the first stage of the current fine-tuning process. Parameter values, After completing the training for the old task, the first Reference values ​​for each parameter. The parameter is the relative offset, and is a dimensionless scalar. As a numerically stable term, it typically takes the value of ,and Having the same dimensions is used to prevent the denominator from being zero. The diagonal approximation of the Fisher information matrix. The calculation formula is expressed as follows:

[0092] ;

[0093] In the formula, This represents the total number of samples in the old task validation set. For the model in parameters Next to the One verification sample The output probability, For the first The input features of a validation sample, For the corresponding tags, For the log-likelihood with respect to the th Partial derivatives of each parameter, As a dimensionless scalar, both sides of the equation have the same dimensions. During testing, the adaptive mechanism does not use labeled samples in the inference phase. It updates the mean and variance statistics of the batch normalized layer online by minimizing the prediction entropy of the model output. The prediction entropy formula is expressed as follows:

[0094] ;

[0095] In the formula, This represents the dimensionless prediction entropy output by the model for the current batch. For the model to the first Minimize the output probability of the class This approach brings the mean and variance statistics of the batch normalization layer closer to the current domain distribution, achieving unsupervised online adaptation. The batch size is updated in 4 to 16 frames each time. The specific value is determined by compromise after testing the memory usage and adaptive convergence speed under different batch sizes on the target edge hardware. This effectively suppresses the continuous drift and degradation of model performance caused by cross-seasonal visual distribution shift.

[0096] To better understand and implement this invention, the following is a specific application scenario of the invention, Example 2: To verify the effect of the invention, technicians built a test environment and deployed edge computing nodes in a mountainous forest area, integrating visible light cameras and uncooled thermal infrared sensors to carry out cross-seasonal wildlife target detection experiments. The test period covered four complete seasons: spring, summer, autumn, and winter. The target species included sika deer, wild boar, wolverine, and various forest birds.

[0097] In terms of hardware, the edge node is equipped with a main neural network processing unit and an ultra-low-power coprocessor, and the static random access memory reserved area has a capacity of 512. The system's total video memory is 8 The visible light sensor has a resolution of 1920×1080. The thermal infrared sensor has a resolution of 640×480. Both sensors have a frame rate of 25 frames / s, and the PIR passive infrared sensor has a coverage radius of 12m. For example... Figure 2 As shown, the connection between the edge node hardware architecture and the dual-modal sensor acquisition link is illustrated.

[0098] For dataset creation, technicians collected 48,000 pairs of visible light and synchronous thermal infrared images under various climatic conditions, including all four seasons, sunny, rainy, snowy, and foggy weather. The images were manually labeled with species category bounding boxes and instance masks, and divided into training, validation, and test sets in an 8:1:1 ratio. Additionally, 5,000 background frame sequences without animal targets were collected for negative sample training of the early termination mechanism. The color temperature perturbation range was set to 500–1000 based on the color temperature drift caused by sensor aging. The sample distribution for each season is shown in Table 1.

[0099] Table 1. Sample distribution in four seasons

[0100]

[0101] For model training, the AdamW optimizer was used, with an initial learning rate of [missing information]. The weight decay coefficient is The training consists of 200 rounds, with the first 10 rounds using linear warm-up followed by cosine annealing scheduling. The cross-modal attention module and temporal difference convolution are trained using a learning rate of [missing information - likely a specific learning rate] in the first 50 rounds. The system undergoes 50 rounds of warm-up training, followed by joint training with the backbone network using the same learning rate. Quantization perception training begins at round 150, and after inserting pseudo-quantization nodes, it continues training for another 50 rounds. The knowledge distillation temperature coefficient is trained for 30 rounds with three candidate values ​​(4, 6, and 8), and the optimal value is selected based on the average precision of the validation set. Ultimately, 6 is chosen as the distillation temperature coefficient.

[0102] After mixed-precision quantization, the low-frequency feature layers of the backbone network are stored in INT8 format, while the detection head and classification branch retain FP16. The total number of model parameters is approximately 8.7M. After INT8 quantization, the storage size of the backbone network weights is reduced by about 75% compared to FP32, and the memory usage for edge inference is significantly compressed. Figure 3 As shown in the figure, a comparison diagram of the bit width allocation of each layer parameter and the model accuracy retention before and after mixed precision quantization is presented.

[0103] In terms of calculating resource compatibility assessment values, weighting coefficients were determined by running 100 sets of different load stress tests on the target hardware platform and using least squares regression analysis. , , In a typical test scenario, when video recording is running in the background simultaneously, the video memory usage is as follows: Processor utilization Queue backlog ratio Calculated Falling into the medium load range The model automatically switched to the standard inference configuration, with the number of attention heads restored to 75%, the hidden state dimension of the convolutional short-term memory units set to 128, and the maximum memory allocation set to 85%. The inference process was uninterrupted. The model configuration parameters for each load range are shown in Table 2.

[0104] Table 2. Model Configuration Parameters for Level 3 Load Ranges

[0105]

[0106] Regarding the early termination mechanism, the technicians progressively enumerated the early termination confidence threshold on the validation set with a step size of 0.05, ranging from 0.5 to 0.9. The final threshold was determined to be 0.75. Under the constraint that the accuracy loss does not exceed 2% of the average accuracy of the validation set, the early termination trigger rate of the backbone network's intermediate layers in static background frames reached 63%, and the statistical average proportion of foreground pixels in sparse convolutions in the foreground region was 14%, demonstrating a significant compression effect on the average computational power consumption per frame. Figure 4 As shown in the figure, the relationship between the accuracy retention rate and the computing power compression ratio as a function of the threshold is displayed during the early retirement confidence threshold enumeration process.

[0107] Regarding the tiered wake-up architecture, after PIR triggering, the ultra-low-power coprocessor starts an extremely lightweight detection model with a weight scale of 0.87. The measured average inference latency on the coprocessor is 38. The confidence threshold for coarse screening was determined to be 0.42 through receiver operating characteristic (ROC) curve analysis, with a false negative rate constraint of no more than 5%. The core detection model weights were pre-loaded into the static random access memory (SRAM) reserve area, and the measured cold start latency was 132. 150 Threshold requirements ensure that inference in the first frame can be completed within the effective dwell window when fast-moving animals such as birds cross the camera's field of view.

[0108] Regarding cross-domain feature alignment, researchers used autumn-collected data as the source domain and winter-collected data as the target domain to verify the cross-seasonal domain adaptation effect. Feature vector dimension Through binary search experiments, the value was determined to be 256 within the range of 128–512. The single alignment calculation time was 7.3% of the inference frame interval, satisfying the constraint of not exceeding 10%. The elastic weight solidification penalty coefficient is... to After uniformly sampling 5 candidate values ​​within the range, select the one with the highest accuracy retention rate from the old task. During testing, the batch size for the adaptive mechanism was set to 8 frames. After updating the batch normalization statistics online, the model's response stability for target detection in winter snow scenes was significantly improved. The detection performance for each season before and after cross-domain feature alignment is shown in Table 3.

[0109] Table 3. Comparison of mean accuracy for each season before and after cross-domain feature alignment.

[0110]

[0111] like Figure 5 As shown, the distribution changes of the source and target domain features in the dimensionality-reduced visualization space before and after Riemannian manifold geodesic interpolation alignment are illustrated. It can be observed that the overlap of the feature distributions of the two domains is significantly improved after alignment, and the inter-domain differences are effectively compressed. Figure 6 The diagram shows the changes in timing states and confidence outputs at each stage of a complete PIR trigger event for the hierarchical wake-up architecture.

[0112] The advancements of this invention compared to traditional methods are reflected in the following aspects: Traditional Euclidean domain adaptation methods treat the covariance matrix as an ordinary vector for linear interpolation, which cannot guarantee the positive definiteness of the interpolation results, leading to the cumulative amplification of feature distribution distortion with the seasons. This invention, through Riemannian manifold geodesic interpolation, confines the alignment within a symmetric positive definite matrix space, ensuring the statistical validity of the aligned features from a geometrical perspective and eliminating the systematic bias introduced by Euclidean interpolation. Traditional single-modal detection suffers from a high false negative rate at night or under low light conditions due to texture feature degradation. This invention, through a cross-modal cross-attention module, dynamically injects the thermal infrared temperature gradient into the visible light feature channel in a gated manner, compensating for the loss of texture feature perception under extreme lighting conditions. Traditional edge-end models lack an active adaptation mechanism when hardware resources fluctuate, and are prone to inference interruption due to memory overflow. This invention, through resource adaptation evaluation value linkage adjustment of the computation graph size, actively shrinks the model complexity when resources are scarce, ensuring long-term deployment stability—a mechanism not possessed by traditional static models.

[0113] It should be noted that the variables involved in this invention are explained in detail in Table 4.

[0114] Table 4. Variable Explanation Table

[0115]

[0116] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting wildlife targets based on edge computing deployment, characterized in that, Includes the following steps: The collected wildlife images are subjected to mixed precision quantization processing. The low-frequency feature layer of the backbone network is quantized with INT8, while the detection head and classification branch retain FP16. Quantization-aware training and knowledge distillation are performed to complete the compression of the edge model. The compressed model is deployed to a multimodal temporal attention fusion detection model. The input is a visible light image stream and a thermal infrared image stream. The feature extraction and fusion are completed through a cross-modal cross attention module and temporal difference convolution, and the target detection result is output. Calculate the resource adaptation evaluation value of the current frame, and dynamically adjust the memory allocation ratio and inference accuracy parameters of the multimodal temporal attention fusion detection model according to the interval to which the resource adaptation evaluation value belongs, so as to achieve adaptive inference; An early termination mechanism is triggered for static background frames, and results for easily identifiable targets are output in advance in the middle layer of the backbone network. Sparse convolution is used for fine calculation of the foreground region to reduce computing power consumption. The architecture is designed with a tiered wake-up mechanism. After PIR triggering, the ultra-low-power coprocessor is woken up first to run the ultra-lightweight detection model for coarse screening. Once the confidence level of the coarse screening exceeds the confidence threshold, the main neural network processing unit is woken up, and the core model weights are preloaded into the static random access memory reserved area. A cross-domain feature alignment algorithm based on Riemannian manifold geodesic interpolation is adopted. The feature distributions of the source and target domains are represented by symmetric positive definite matrices. The midpoint of the geodesic is calculated as the alignment reference point. Cross-seasonal domain drift correction is completed through logarithmic and exponential mapping. The batch normalization layer parameters of the detection head are fine-tuned at the edge by combining a continuous learning framework and an elastic weight solidification mechanism. The batch normalization statistics are updated online during the inference stage using a test-time adaptive mechanism.

2. The wildlife target detection method based on edge computing deployment according to claim 1, characterized in that, In the hybrid precision quantization process, quantization-aware training refers to simulating quantization errors during the training phase and introducing pseudo-quantization nodes during forward propagation to cause the weight distribution to converge in a direction that is friendly to low-bit representation, thus compensating for the precision loss caused by converting FP32 to INT8.

3. The wildlife target detection method based on edge computing deployment according to claim 2, characterized in that, The knowledge distillation process supervises the training of the student model by using the soft-label output of the high-precision FP32 teacher model. This makes the inter-class relative probability distribution of the student model approximate that of the teacher model, thus compensating for the decrease in fine-grained species classification accuracy caused by the accumulation of quantization errors.

4. The wildlife target detection method based on edge computing deployment according to claim 3, characterized in that, The backbone network of the multimodal temporal attention fusion detection model adopts the improved ConvNeXt-Tiny, inserting a cross-modal cross-attention module at the end of each stage. The channel mean and variance of the thermal imaging feature map are used as the query vector to dynamically modulate the response weights of each channel of the visible light feature map.

5. The wildlife target detection method based on edge computing deployment according to claim 4, characterized in that, The temporal differential convolution encodes motion trajectory information using frame-by-frame differential tensors of five consecutive feature maps, and performs iterative updates of the hidden state within the network through lightweight convolutional long short-term memory units, forming a spatiotemporal feature stream with memory capabilities.

6. The wildlife target detection method based on edge computing deployment according to claim 5, characterized in that, The resource adaptation evaluation value is calculated by taking the current edge device's video memory utilization rate, CPU utilization rate, and inference queue backlog of frames as inputs, and calculating the weighted normalized resource adaptation evaluation value. The weight coefficients are determined by least squares regression analysis after running stress tests on the target edge hardware platform.

7. The wildlife target detection method based on edge computing deployment according to claim 6, characterized in that, The resource adaptation evaluation value range is divided into high load range, medium load range and low load range. The high load range triggers the memory compression strategy, the medium load range adopts the standard inference configuration, and the low load range enables the full precision extended configuration.

8. The wildlife target detection method based on edge computing deployment according to claim 7, characterized in that, The early termination mechanism adds a lightweight classification head to the middle layer of the backbone network. The confidence threshold for triggering early termination is determined by progressively enumerating on the validation set and selecting the value that minimizes the computational load while ensuring that the accuracy loss does not exceed a preset proportion of the average accuracy of the validation set.

9. The wildlife target detection method based on edge computing deployment according to claim 8, characterized in that, In the hierarchical wake-up architecture, the weight scale of the ultra-lightweight detection model running on the ultra-low power coprocessor is less than the weight scale threshold, and the inference latency is less than the latency threshold; after the core detection model weights are preloaded into the static random access memory reserved area, the cold start latency is compressed to within the cold start latency threshold.

10. The wildlife target detection method based on edge computing deployment according to claim 9, characterized in that, In the cross-domain feature alignment algorithm based on Riemannian manifold geodesic interpolation, the midpoint of the geodesic is obtained by applying a logarithmic mapping to two symmetric positive definite matrices, taking the arithmetic mean, and then mapping back to the manifold through an exponential mapping. The aligned features are used to drive the fine-tuning of the parameters of the first batch of normalized layers in the detection, and the number of fine-tuned parameters is less than the parameter ratio threshold of the total number of model parameters.

Images