A Data Detection Method and System for Giant Pandas Based on Learnable Motion Saliency Modulation

Abstract

This application discloses a method and system for giant panda data detection based on learnable motion saliency modulation, belonging to the fields of endangered animal protection and intelligent infrared image processing technology. It can be applied to the rapid screening of long-term continuous monitoring data from fixed-position infrared cameras. The method constructs a multi-channel temporal difference input for continuous infrared image frames and generates a motion saliency weight map. The weights are embedded as conditional control signals into the convolution calculation process of a lightweight feature backbone network, enabling differentiated feature extraction between motion-saliency regions and static background regions. At the detection output, suppressive modulation is applied to the confidence score to reduce false alarms in static backgrounds. Simultaneously, perturbation frames are identified, and weight updates are maintained or limited to suppress false activations across the entire image and confidence score jitter across consecutive frames. The technical solution of this application solves the problem of balancing efficiency and stability caused by target sparsity, high background ratio, and pseudo-motion interference.

Description

Technical Field

[0001] This invention relates to the fields of endangered animal protection and intelligent infrared image processing technology, and in particular to a method and system for detecting giant panda data based on learnable motion saliency modulation. Background Technology

[0002] Infrared camera technology, with its advantages of all-weather operation, non-invasiveness, and long-term continuous monitoring, has become a common technique in wildlife surveys and endangered species conservation. It has been applied in areas such as giant panda habitat monitoring, activity pattern analysis, and population dynamics assessment. By deploying infrared cameras in fixed areas over a long period, a large amount of infrared image or video data can be continuously acquired, providing a data foundation for subsequent target detection, individual statistics, behavioral analysis, and conservation decisions.

[0003] However, the long-term operation of infrared cameras generates massive amounts of continuous image data, the vast majority of which only contain static backgrounds such as vegetation, terrain changes, and lighting or temperature disturbances, while the proportion of valid images actually containing giant pandas is extremely low. Currently, the screening and processing of giant panda infrared image data still mainly relies on manual frame-by-frame review or semi-automatic assisted annotation methods. This is not only time-consuming, labor-intensive, and costly, but also easily influenced by subjective human experience, making it difficult to meet the efficiency and consistency requirements of large-scale, long-term monitoring tasks. To improve the processing efficiency of infrared camera image data, existing research and products have gradually introduced infrared image recognition or target detection schemes based on deep learning. These schemes typically use convolutional neural networks or target detection networks as their core to automatically identify target categories or locations in infrared images, thus alleviating the workload of manual frame-by-frame screening to some extent.

[0004] For example, Chinese invention patent CN111046964B discloses a method for recognizing people and vehicles from infrared thermal images based on convolutional neural networks. It primarily uses DenseNet as the basis for its recognition network and introduces channel weight learning within the network layers to weight features, thereby enhancing the single-frame category recognition performance of infrared thermal images. This scheme mainly focuses on single-frame recognition, emphasizing the improvement of feature representation and classification accuracy.

[0005] For example, Chinese invention patent CN111382683A discloses a target detection method based on feature fusion of a color camera and an infrared thermal imager. This method extracts color texture features and thermal infrared temperature features separately through a dual-channel network, and then fuses them within the network layer to perform target prediction, thereby improving detection performance in environments with low visibility or complex backgrounds. This approach emphasizes multi-sensor joint observation and feature fusion to enhance detection capabilities.

[0006] However, the existing infrared identification and detection schemes mentioned above still have significant shortcomings under the actual conditions of long-term continuous monitoring of giant pandas using infrared cameras.

[0007] First, existing solutions mostly rely on single-frame recognition or fixed structure detection. The networks typically use fixed computational paths, performing convolution operations of approximately equal intensity on the entire image. This makes it difficult to differentiate regional computing power allocation for data distributions in continuous infrared monitoring where static backgrounds account for a long period and effective target frames account for a low percentage. As a result, many image regions containing only static vegetation, terrain, and background thermal radiation still consume similar computational resources as the target regions. For example, in long-term monitoring from a fixed camera position, most frames only show slow changes in background thermal radiation or static terrain textures. The time window for the actual appearance of the giant panda target is short and discontinuous. If deep feature extraction is still performed on the entire image using a fixed path, problems such as accumulated processing latency, increased power consumption, or difficulty in real-time filtering of batch data can easily occur when deployed on edge devices or low-power platforms.

[0008] Secondly, false motion phenomena caused by imaging links and environmental conditions are common in continuous infrared monitoring. Existing methods often lack the ability to calculably distinguish between global synchronous disturbances and local real motion, easily misjudging global disturbances as motion, resulting in false activations, increased false alarms, and unstable output across the entire image. For example, when the ambient temperature changes slowly from night to dawn, the overall rise or fall of background thermal radiation will cause the difference between adjacent frames to increase synchronously over a large area. Changes in camera automatic gain and bias adjustments, as well as thermal noise levels, may also cause synchronous drift in the brightness or temperature distribution of the entire image. In these situations, if the judgment is based solely on frame difference energy or simple motion cues, almost the entire image may be considered to be in motion, triggering a large number of invalid candidates, increasing false detections, and causing unnecessary fluctuations in the detection results during disturbances.

[0009] Therefore, there is still a lack of target detection methods that can be applied to long-term continuous monitoring scenarios using giant panda infrared cameras, enabling motion information to participate in feature calculation at the network structure level, and structurally distinguishing and stably controlling global synchronous pseudo-motion and local real motion, so as to simultaneously take into account batch screening efficiency, weak motion target detection capability, and long-term operational stability. Summary of the Invention

[0010] To address the challenges of sparse targets, high static background ratios, and the tendency for weakly moving targets to be ignored by general detection networks in continuous infrared monitoring scenarios, this invention proposes a panda data detection method and system based on learnable motion saliency modulation. Unlike existing techniques that treat motion information as an independent selection condition or posterior constraint, this invention constructs a lightweight temporal motion saliency encoder to encode the motion intensity and temporal variation patterns in continuous infrared images into continuously modulated motion saliency weights. These weights are then directly embedded as conditional control signals into the convolutional computation process of the feature backbone network, enabling motion information to participate in the modulation of spatial computation intensity and the allocation of computational power during the feature extraction stage.

[0011] Through the above structural design, the network can implement differentiated computational strategies for different spatial locations based on the motion saliency distribution during the feature extraction stage. This allows regions with significant motion to receive higher computational attention, while convolutional computations in low-motion or static background regions are actively suppressed. This creates an adaptive computational power allocation mechanism at the network structure level for continuous infrared monitoring scenarios. This mechanism effectively reduces computational redundancy caused by a large number of invalid background regions in continuous monitoring scenarios without significantly increasing the number of model parameters, and improves the perception and recognition capabilities for weakly moving giant panda targets.

[0012] The technical solution of this invention is:

[0013] On the one hand, a method for detecting giant panda data based on learnable motion saliency modulation is provided, which includes the following steps:

[0014] S1: Obtain a continuous infrared image frame sequence. For each current frame, extract multiple historical frames at different time intervals within a preset time window before its timestamp. Perform pixel-by-pixel difference operations on the current frame and each historical frame respectively, and stitch the difference results together in the channel dimension to obtain a multi-channel time difference input that represents motion changes at different time scales.

[0015] S2: Input the multi-channel time difference input to the motion saliency encoder, output the motion saliency response map, and perform normalization or mapping processing on the motion saliency response map to generate the motion saliency weight map.

[0016] S3: After scale alignment, the motion saliency weight map is used as a conditional control signal and input to the saliency modulation convolution blocks of each level of feature extraction layer of the lightweight feature backbone network. This allows the motion saliency weight map to participate in the spatial modulation of the convolution calculation process, so as to obtain the motion saliency modulated feature map.

[0017] S4: Construct detection features based on feature mapping and input them into the target detection head. Output the target category probability and bounding box regression parameters. After weighting and modulating the detection confidence corresponding to the target category probability using the motion saliency weight map, combine it with non-maximum suppression to generate and output the giant panda target detection result.

[0018] The time intervals are set to be non-equidistant, allowing the temporal difference input tensor to simultaneously represent rapid motion changes at short timescales and slow displacement changes at long timescales. The motion saliency encoder includes a lightweight feature extraction unit and a saliency mapping unit. The saliency mapping unit maps the extracted motion features to a single-channel pre-response map and applies a non-linear activation function to the pre-response map to obtain a single-channel motion saliency response map. Generating the motion saliency weight map involves normalizing the motion saliency response map so that the motion saliency weight map participates in the convolution calculation and modulation of the saliency modulation convolution block with continuous values.

[0019] The panda data detection method based on learnable motion saliency modulation also includes a pseudo-motion suppression step: calculating the global difference intensity index and spatial consistency index of the difference map between the current frame and one or more historical frames. When the global difference intensity index exceeds a first threshold and the spatial consistency index meets the global synchronization change characteristics, the corresponding frame is identified as a globally synchronized perturbation frame. During the globally synchronized perturbation frame, a control strategy is adopted for the motion saliency weight map to maintain the weights of the previous time step or limit the weight update amplitude, in order to suppress false activation of the entire map caused by global synchronization perturbation. When the above conditions are not met simultaneously, the corresponding frame is identified as a non-globally synchronized perturbation frame, and the motion saliency weight map is updated based on the difference results between the current frame and historical frames.

[0020] The saliency-modulated convolutional block comprises a motion-aware convolutional branch and a semantic-preserving convolutional branch. The convolutional output of the motion-aware convolutional branch is modulated element-wise by the motion saliency weight map. The semantic-preserving convolutional branch is used to preserve the basic semantic features. The outputs of the motion-aware convolutional branch and the semantic-preserving convolutional branch are fused to obtain the output features of the saliency-modulated convolutional block.

[0021] The motion saliency weight map in the saliency modulation convolution block is used to differentially modulate the convolution computation intensity at different spatial locations, so that the motion saliency region receives higher computational attention, while the convolution response of the low motion or static background region is suppressed.

[0022] The lightweight feature backbone network further includes a thermal gradient enhancement modulation mechanism: gradient features are extracted from the input features in the spatial dimension, and the gradient features are modulated using a motion saliency weight map to enhance the target edge and contour response in motion saliency regions and suppress ineffective gradient activation in low motion regions.

[0023] The confidence level weighting includes suppressive modulation of the confidence level of the classification prediction based on the motion saliency weight map.

[0024] On the other hand, a system for detecting giant panda data based on learnable motion saliency modulation is provided, including: a temporal difference construction module, a motion saliency encoding module, a lightweight feature backbone network module, and a detection and output module.

[0025] The temporal difference construction module is used to acquire a continuous infrared image frame sequence. For each current frame, multiple historical frames are extracted within a preset time window before its timestamp at different time intervals. Pixel-by-pixel difference operations are performed on the current frame and each historical frame respectively, and the difference results are stitched together in the channel dimension to obtain a multi-channel temporal difference input that represents motion changes at different time scales.

[0026] The motion saliency encoding module is used to input multi-channel time difference inputs into the motion saliency encoder, output a motion saliency response map, and perform normalization or mapping processing on the motion saliency response map to generate a motion saliency weight map.

[0027] The lightweight feature backbone network module is used to scale-align the motion saliency weight map and input it as a conditional control signal to the saliency modulation convolution blocks of each level of feature extraction layer of the lightweight feature backbone network, so that the motion saliency weight map participates in the spatial modulation of the convolution calculation process to obtain the motion saliency modulated feature map.

[0028] The detection and output module is used to construct detection features based on feature mapping and input them into the target detection head. It outputs the target category probability and bounding box regression parameters. After weighting and modulating the detection confidence corresponding to the target category probability using the motion saliency weight map, it combines non-maximum suppression to generate and output the giant panda target detection result.

[0029] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0030] 1. By constructing multi-scale temporal difference inputs and generating motion saliency weights in long-term continuous monitoring scenarios using infrared cameras, the network can invest higher computational intensity in motion-saliency regions and suppress redundant computation in static background regions during the feature extraction stage. This enables rapid batch data screening and low-overhead operation at the edge. Compared with existing technologies that use fixed computation paths to perform equal-intensity convolution on the entire image, which wastes computational power, this significantly reduces average processing latency and energy consumption and improves deployability in monitoring tasks where giant pandas appear infrequently and have a very low percentage of effective frames.

[0031] 2. By computably distinguishing between global synchronous disturbances and local real motion, and by applying stable constraints to the activation of motion-driven regions when disturbances dominate, false activations and invalid candidate expansion throughout the image are avoided during pseudo-motion. This achieves stabilization of detection output and suppression of false alarms during disturbance periods. Compared with existing technologies that rely solely on frame difference energy or simple motion cues, which lead to almost the entire image being judged as motion due to temperature drift from night to dawn and automatic gain adjustment, this technology reduces false detections and output jitter and minimizes unnecessary computational power consumption in continuous infrared monitoring.

[0032] 3. By integrating motion saliency information into the feature extraction and detection confidence generation process, and by suppressing the consistency of detection responses in low-motion regions, the effective responses of weakly moving targets are preserved while suppressing false responses caused by background disturbances. This achieves a balance between detection rate and stability. Compared with existing technologies that leave motion information to pre-processing screening or post-processing rule constraints and are independent of the network's internal computing mechanism, this approach improves detection consistency and reduces fluctuations in results between adjacent frames in common scenarios such as pandas moving slowly or stopping briefly, weak boundary contrast, and slight swaying of background vegetation. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A flowchart of a giant panda data detection method based on learnable motion saliency modulation provided in this application embodiment;

[0035] Figure 2 A schematic diagram of the structure of the giant panda data detection system based on learnable motion saliency modulation provided in this application embodiment;

[0036] Figure 3 This is a schematic diagram of the giant panda data detection method and system based on learnable motion saliency modulation provided in an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of the saliency modulation feature processing flow. Detailed Implementation

[0038] The following provides explanations of some terms used in this application. It should be noted that these explanations are for the convenience of those skilled in the art and do not constitute a limitation on the scope of protection claimed in this application.

[0039] The embodiments of this application involve at least one, including one or more; where "multiple" means two or more. Furthermore, it should be understood that in the description of this specification, terms such as "first," "second," and "third" are used only for descriptive purposes and should not be construed as indicating relative importance or order. For example, "first device" and "second device" do not represent the degree of importance of the two or their order, but are merely for descriptive distinction. In the embodiments of this application, "and / or" merely describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0040] The directional terms mentioned in the embodiments of this application, such as "up", "down", "left", "right", "inner", and "outer", are only for reference to the directions in the accompanying drawings. Therefore, the directional terms used are for better and clearer explanation and understanding of the embodiments of this application, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0041] References to "one embodiment," "in some examples," or "some embodiments" as described in the embodiments of this application mean that one or more embodiments of this specification include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in some examples," "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0042] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0043] Embodiment 1 of the present invention: Figure 1 The diagram shown is a flowchart of the giant panda data detection method based on learnable motion saliency modulation provided in this application embodiment. Figure 3 , Figure 3 This is a schematic diagram of a method and system flow for giant panda data detection based on learnable motion saliency modulation, provided in an embodiment of the present invention. The method includes the following steps:

[0044] S1: Acquire the infrared image frame sequence obtained by the infrared camera in continuous acquisition. In this embodiment, the infrared images are processed at a uniform resolution, with an image height of H pixels and a width of W pixels. A single frame image is represented as... In this embodiment of the invention, H=480 and W=640 are uniformly set, that is, the image resolution is 480×640. t is the frame timestamp. Images can cover both daytime color and nighttime grayscale scenes. Grayscale normalization is performed on each frame, mapping pixel values ​​to [0, 1]. The images are then divided into training, validation, and test sets in chronological order, with a ratio of 7:2:1, totaling approximately 15,000 frames. During the training phase, images are randomly horizontally flipped, randomly cropped, and randomly adjusted for brightness and contrast to improve adaptability to different infrared scenes.

[0045] To simultaneously characterize both short-term rapid motion and long-term slow displacement of a target in a continuous infrared monitoring scenario, K different time intervals are set within a preset time window. (or equivalent frame interval) The input is constructed using a non-uniformly spaced multi-scale differential method. Assuming the infrared camera frame rate is f (frames / second), the time interval between adjacent frames is 1 / f. This embodiment preferably uses a frame interval-time interval correspondence method: k=1, ...,K. Where, The frame interval (unit: frame) corresponding to the k-th scale satisfies < <…< ,and The maximum available frame interval does not exceed the length of the time window. To cover typical motion patterns with low computational overhead, this embodiment of the invention sets K=3 and sets the frame interval to... , , The corresponding time interval is , .in, Corresponding to the shortest time baseline, it is used to sensitively capture the rapid local motion of the target. Provides a medium temporal baseline to enhance the visibility of weakly moving targets in the difference map and suppress single-frame noise. Providing a longer time baseline makes slow displacements more readily apparent under the differential accumulation effect, thus achieving unified coverage of fast, medium, and slow motion changes. Using a non-equidistant setting can expand the time baseline range with a smaller number of channels, avoiding insufficient characterization of long-term slow displacements or computational channel redundancy caused by using equidistant multi-scale methods. Furthermore, The value of can be determined statistically based on historical monitoring data: calculate the difference intensity distribution under different r values ​​for both the target segment and the segment without the target segment, and select the value that improves the separability of the two distributions while satisfying the calculation constraints. As a multi-scale set. It can also guarantee... Based on the interval between adjacent frames, subsequent scales are extended in an approximately doubling relationship to cover a longer time baseline and maintain inter-scale complementarity.

[0046] Perform pixel-wise differencing on the current frame and each historical frame, and then concatenate them along the channel dimension to obtain a multi-channel temporal difference input. :

[0047] ;

[0048] in, This represents pixel-by-pixel absolute value operation. This indicates a channel-level concatenation operation. When K=3, It is a 3-channel input used to provide a compact temporal variation representation for subsequent motion saliency encoding.

[0049] S2: The multi-channel temporal difference input is fed into the motion saliency encoder. The motion saliency encoder includes a lightweight feature extraction unit and a saliency mapping unit. The saliency mapping unit maps the extracted motion features to a motion saliency response map and obtains a single-channel motion saliency response map through nonlinear activation. In this embodiment, the motion saliency encoder... This is a lightweight convolutional network used to map multi-channel temporal difference inputs. Single-channel motion saliency response plot The encoder employs a convolutional structure that preserves spatial resolution, with each convolutional layer having a stride of 1 and using padding of the same size to ensure that the output matches the input in spatial dimensions.

[0050] The first lightweight convolutional unit uses A depthwise separable convolutional structure performs channel-wise spatial convolution on the input tensor, and its output is represented as:

[0051]

[0052] The second lightweight convolutional unit uses a dilation rate of 2. A depthwise separable convolutional structure performs cross-scale fusion on the output features of the first lightweight convolutional unit, and its output is represented as:

[0053]

[0054] The saliency mapping unit uses 1 Pointwise convolution is used to compress the channels of the fused motion features, and a single-channel motion saliency response map is generated using the Sigmoid activation function. The calculation process is as follows:

[0055]

[0056] in, This represents a 3×3 depthwise separable convolution. This represents a 3×3 depthwise separable convolution with a dilation rate d=2. This represents a 1×1 pointwise convolution. This represents the Sigmoid activation function, which is used to compress the input to the range of 0 to 1. This represents a one-to-one pointwise convolution operation, which means linearly combining different channels at the same spatial location without changing the spatial resolution, thereby achieving information fusion and channel number transformation between channels. To normalize features by channel during training and introduce learnable scale and bias parameters, the feature distribution is made more stable, thereby improving the network training convergence speed and reducing the risk of gradient instability. It is a non-linear activation function.

[0057] After outputting the motion saliency response map, perform continuous mapping / normalization on the motion saliency response map to generate a motion saliency weight map. This value is set to 0 or 1 and used as a modulation factor in subsequent convolution calculations. This embodiment employs local neighborhood normalization:

[0058] ;

[0059] At the same time, Crop to Where q is the pixel index within the neighborhood, and p is the pixel position. It is a local neighborhood centered at p (e.g., an m×m window). ε is the number of neighboring pixels. The side length m of the neighborhood window is an odd number to ensure center alignment. It can be selected from the validation set within {5, 7, 9, 11} to minimize confidence jitter and the false alarm rate; in this embodiment, m=7. ε is a constant to prevent division by zero (in this embodiment, ε is ε). = ).

[0060] S3: After scale alignment, the motion saliency weight map is used as a conditional control signal input to the lightweight feature backbone network. The lightweight feature backbone network consists of multiple sequentially connected feature extraction layers. Each layer contains at least one saliency modulation convolutional block, with the input being the feature map of the current layer. The output is the feature map for the next layer. The backbone network performs scale alignment on the motion saliency weight map at each feature level and embeds the aligned motion saliency weights as conditional computation control signals into each level of convolution operation. This enables the backbone network to perform feature extraction with non-uniform computational intensity in the spatial dimension, thereby achieving adaptive allocation of computing power between motion-saliency regions and static background regions. The saliency modulation convolutional block includes a motion-aware convolutional branch and a semantic-preserving convolutional branch. The convolutional output of the motion-aware convolutional branch is modulated element-wise by the motion saliency weight map. The semantic-preserving convolutional branch is used to preserve basic semantic features. The outputs of the motion-aware convolutional branch and the semantic-preserving convolutional branch are fused to obtain the output features of the saliency modulation convolutional block.

[0061] For different levels of feature resolution, After scaling to the same resolution as the features of that layer using bilinear interpolation or nearest neighbor methods, the motion saliency weight map obtained in step S2 is used in the calculation. Perform scale alignment to obtain the feature mapping with the input of this layer. Weighted graph with consistent spatial dimensions . The weight map is at the same resolution as the features of the first layer. If it is a single-channel weight, it is copied / broadcast along the channel dimension to be consistent with the feature dimension.

[0062] In this embodiment of the invention, the motion-aware convolution branch adopts... Depthwise separable convolution performs spatial convolution on the input features and modulates the convolution result element-wise using motion saliency weights. The calculation process is as follows: .

[0063] Where l is the feature level index, l=1, 2, ..., L. L can be selected based on the trade-off between computing power budget and detection accuracy. In mobile or edge scenarios, it is usually selected between 3 and 6. In this embodiment, in order to meet the balance between real-time performance and accuracy, L=5 is chosen.

[0064] Semantic-preserving convolution branches adopt Pointwise convolution maps input features to channels, and its calculation process is as follows: .

[0065] The outputs of the two branches are fused and a nonlinear activation function is applied to obtain the output features of the saliency modulation convolutional block. (Level 1):

[0066] ;

[0067] in, This is element-wise multiplication. For the output of the motion-aware convolution branch, Preserve semantics in the output of convolutional branches. For non-linear activation functions, SiLU is used in this embodiment. Input feature mapping. The number of channels is semantically preserved branches The number of output channels is set to This is done so that its output is consistent with the output of the motion perception branch in the channel dimension, thereby enabling element-wise summation and fusion. For depthwise convolution, taking the depth multiplier as 1 does not change the number of channels; only spatial convolution is performed. If the first convolutional unit needs to map the input temporal difference tensor channels from K channels to 16 channels, and the second convolutional unit further maps the feature channels to 32 channels, then a corresponding configuration is needed after each depthwise convolution. Complete the channel mapping. For a single-channel weight map with the same spatial dimensions as the l-th layer, broadcast the weights along the channel dimension to each channel feature, then multiply them element-wise to achieve saliency modulation.

[0068] The above structure enhances the convolutional response in motion-salient regions, suppresses the response in static background regions, and further strengthens edge and contour information in motion-salient regions, reducing invalid edge activation introduced by large static backgrounds.

[0069] S4: Channel integration is performed on the output feature map of the last stage of the backbone network to obtain the detection features. In this embodiment of the invention, the last stage feature map obtained in step S3 is denoted as... . It contains multi-scale spatial semantic information modulated by motion saliency. Feature alignment and channel integration are performed to construct a detection feature representation for object detection. The calculation process is expressed as follows: .

[0070] in, Indicates adoption The convolution operation of the convolution kernel is used to linearly map the feature channels while maintaining the spatial resolution, so as to meet the input dimension requirements of the detection head.

[0071] The detection features are input into the target detection prediction head, and the prediction is based on the detection feature representation. A target detection prediction head is constructed to jointly predict the target's class probability and bounding box parameters. The target detection prediction head uses a shared feature input structure and outputs classification prediction results and location regression results respectively. Its calculation process is as follows: ; .

[0072] in, This represents a probability map of the predicted target category. This represents the set of regression parameters for the target bounding box, which includes the coordinates of the target's center position and the target's width and height information. It is a classification prediction head. It is the bounding box regression prediction head.

[0073] Motion saliency-guided detection confidence modulation. To further reduce the false detection probability in static background regions, the motion saliency weight map is... The process of calculating the confidence level of the detection results is introduced to modulate the prediction response in the classification prediction stage. The modulation method is expressed as follows: .

[0074] in, P(x,y) represents the target detection confidence at the modulated position (x,y), and W represents the target class prediction probability at the position (x,y). t (x, y) represents the motion saliency weight at position (x, y). This confidence modulation is not an independent post-processing step, but a joint modulation strategy consistent with the motion saliency modulation mechanism in the aforementioned feature extraction stage. This strategy further enhances the consistency of computational power allocation and response suppression at the detection result level. Through this method, the detection network maintains a high response in motion-saliency regions and suppresses false detection results in low-motion regions, thereby improving overall detection stability.

[0075] Based on the modulated target detection confidence level Regression results with corresponding bounding boxes The final target detection result set is generated. After non-maximum suppression processing is applied to the detection results, the detection bounding box and corresponding category information of the giant panda target in the infrared image are output.

[0076] To obtain the parameters of the motion saliency encoder and detection network, this embodiment adopts the following training process: The motion saliency encoder is trained using the temporal difference input constructed in step S1 as the model input. This embodiment uses three temporal scales to construct the input, therefore the number of channels for the temporal difference input is 3. The motion saliency encoder consists of a first convolutional unit, a second convolutional unit, and a saliency mapping unit connected in sequence, used to output a single-channel motion saliency response map. The first convolutional unit uses a 3x3 depthwise separable convolutional structure with 3 input channels and 16 output channels. A batch normalization layer is set after the first convolutional unit to normalize the feature distribution. A SiLU activation function is set after the batch normalization layer to introduce non-linear expressive power. The second convolutional unit uses a 3x3 depthwise separable convolutional structure with an expansion ratio of 2, with 16 input channels and 32 output channels. A batch normalization layer is set after the second convolutional unit, and a SiLU activation function is set after the batch normalization layer. The saliency mapping unit employs a one-to-one pointwise convolution structure with 32 input channels and 1 output channel. A sigmoid activation function is applied after the saliency mapping unit to map the output to the range of 0 to 1, thereby obtaining a single-channel motion saliency response map.

[0077] The training supervision signal uses a motion saliency ground truth map generated by labeling motion regions, where the motion saliency ground truth map represents the labeling intensity or binary mask of the motion region at pixel locations. The training loss function uses mean squared error loss to measure the consistency between the predicted response map and the ground truth map. This means that for each pixel location in the entire image, the difference between the predicted saliency response value and the ground truth labeling value is calculated point-by-point, the difference is squared, and then averaged over the entire image. This allows for the simultaneous suppression of local overactivation and local underactivation during training, resulting in a more concentrated output response map in motion regions and lower activation in static background regions. The optimizer uses the Adam optimization algorithm with an initial learning rate of 0.001. The batch size is 16. The training epochs are 50. The learning rate, batch size, and training epochs can be obtained through validation set parameter tuning: under the premise of ensuring stable training convergence, the parameter combination that makes the validation set loss decrease more smoothly and the boundaries of the saliency response map clearer is selected as the setting in this embodiment.

[0078] The lightweight feature backbone network and target detection head are trained as follows: After training the motion saliency encoder, the motion saliency encoder parameters are fixed or loaded, and used to generate a motion saliency weight map for the training samples. This weight map is then used as a conditional control signal to participate in the training of the subsequent detection network. The lightweight feature backbone network consists of multiple sequentially connected feature extraction layers, each containing at least one saliency modulation convolutional block. The number of output channels for each feature extraction layer can be set to 16, 32, 64, 128, and 256, respectively, and the stride of each feature extraction layer is set to 1 to maintain spatial resolution, thus adapting to the fine-grained target localization requirements in continuous infrared monitoring scenarios. The target detection head includes a classification prediction head and a bounding box regression prediction head, used to output the target category prediction results and bounding box regression parameters, respectively.

[0079] During training, the last-level feature map obtained in step S3 is recorded as the backbone output feature, and channel integration is performed on the backbone output feature to construct the detection feature, which meets the input dimension requirements of the detection head. Based on the detection feature, the classification prediction head outputs a target category probability map, and the bounding box regression prediction head outputs a set of bounding box regression parameters. The target category probability map is used to characterize the predicted probability of each spatial location belonging to the target category, and the regression parameter set is used to characterize the position and size parameters of the target bounding box (including the target center position and regression values ​​such as width and height). To ensure that the detection results are consistent with the motion saliency modulation strategy and to further reduce false detections in static background regions, this embodiment introduces a saliency consistency constraint in the training objective, enabling the network to maintain a higher response in motion saliency regions and suppress invalid responses in low-motion or static background regions. Specifically, the objective function for training the detection network is constructed using a multi-objective joint constraint approach, including at least: a classification loss, used to constrain the target category prediction result to be consistent with the labeled category; and a regression loss, used to constrain the position and size of the predicted bounding box to be consistent with the labeled bounding box. The saliency consistency constraint loss is used to constrain the network's classification response to be more concentrated and higher in salient motion regions and lower in low-motion regions, thereby reducing false alarms and improving the stability of continuous frame output. The above loss terms are combined with preset weight coefficients during training to form a joint training objective. These weight coefficients can be obtained by parameter tuning based on the validation set: using the validation set false alarm rate, detection accuracy metrics (e.g., mAP), and the fluctuation range of continuous frame confidence as comprehensive evaluation criteria, the weight combination that reduces the false alarm rate and provides more stable output is prioritized, provided that the detection accuracy does not significantly decrease. As a feasible example, the weights of the classification loss and regression loss can be set to the same order of magnitude, while the weight of the saliency consistency constraint loss can be set to a smaller order of magnitude to avoid excessive suppression of the real target response. The optimizer uses the Adam optimization algorithm with an initial learning rate of 0.001. The batch size is 16. The training epochs are 200. The learning rate, batch size, and training epochs mentioned above can be obtained through validation set parameter tuning: under the premise of ensuring stable training convergence, the parameter combination that keeps the validation set detection accuracy stable, reduces the false alarm rate, and reduces confidence jitter is selected as the setting of this embodiment.

[0080] To verify the performance of this embodiment, it was compared with YOLOv8n, Faster R-CNN, and MobileNet-SSD. The experimental results are shown in Table 1. Here, mAP@0.5(%) represents the average accuracy calculated under an intersection-over-union (IoU) threshold of 0.5, expressed as a percentage, used to measure the accuracy of target detection; FPS represents the number of frames processed per second (FramesPerSecond), used to measure inference speed and real-time performance. A higher FPS value indicates more frames detected per unit time and better real-time performance. While maintaining detection accuracy comparable to YOLOv8n, the method of this invention increases the number of model parameters by no more than 6M. However, through the joint design of saliency modulation and a lightweight backbone network, it can significantly improve the ability to recognize low-motion targets and effectively suppress false responses in static background regions. Compared with Faster R-CNN, this invention improves inference speed by approximately 3 times and reduces model size by nearly half while achieving slightly lower accuracy. Compared with MobileNet-SSD, the method of this invention improves mAP by approximately 6% in infrared weak-motion target detection while maintaining low computational overhead. Experimental results show that the combined application of motion saliency pre-filtering and lightweight feature backbone network can achieve a balance between high accuracy, high efficiency and low computational overhead in infrared giant panda target detection tasks, which has significant engineering application value and enables real-time and reliable monitoring and identification.

[0081] Table 1 compares the prediction method used with the other three methods.

[0082]

[0083] Therefore, this embodiment maintains high detection accuracy under conditions of low parameter quantity and high inference speed. Furthermore, since the motion saliency weights are embedded in the feature extraction calculation modulation, false alarms and confidence fluctuations in static backgrounds can be reduced, improving the stability of long-term continuous operation and its engineering practical value.

[0084] Embodiment 2 of the present invention: Based on Embodiment 1, in order to suppress pseudo-motion caused by thermal drift, automatic gain or automatic exposure in the full-image synchronous brightness disturbance during continuous infrared monitoring, a pseudo-motion suppression step is added after generating the motion saliency weight map. The remaining steps are still executed according to S1, S3 and S4 of Embodiment 1, and the detection results are output.

[0085] In this embodiment, the shortest time baseline is used. The difference is used as the criterion for perturbation discrimination, and pixel-level difference is defined as follows: .

[0086] Calculate the global difference strength index: ;in, This indicates the average change across the entire graph.

[0087] Calculate the spatial consistency index (percentage of synchronous changes across the entire map): ;in, δ is the indicator function. δ is the background noise threshold.

[0088] When satisfied > and When the current frame is within the global synchronization disturbance range, it is determined that the current frame is in the threshold range. , δ is a set of background frames of giant pandas without targets selected from historical monitoring data (which can be obtained by filtering out target-free frames from labeled data or by manual sampling), and the difference between the background frames is statistically analyzed. δ is the mean of the background difference ΔI plus three standard deviations. Take background 95th percentile. Take background The 95th percentile. When a frame is identified as a perturbation frame, a control strategy is implemented for the weighted graph update to suppress false activations of the entire graph caused by full-graph synchronization perturbations.

[0089] This embodiment provides two optional strategies: (1) the preservation strategy, which directly sets the weight of the current frame to the weight of the previous frame, i.e. (2) Limiting strategy: first calculate the change in weight. To suppress the weights of the previous frame and prevent them from continuing to affect the current frame, the weights are then cropped to obtain the cropped weight change. This limits it to [ Between ], and update the current weights using the changes after amplitude clipping to obtain the weights of the current frame, that is + .in, This represents the upper limit of the weight change range, which can be statistically analyzed in normal, undisturbed sequences. The 95th percentile was obtained.

[0090] After completing the above pseudo-motor inhibition, The updated version continues to execute S3 and S4 according to Example 1, thereby outputting the giant panda target detection results.

[0091] Embodiment 3 of the present invention: Based on Embodiment 1, the generation method of the motion saliency weight map is changed from local neighborhood normalization to mapping processing. The remaining steps are still executed according to S1, S3 and S4 of Embodiment 1, and the detection results are output.

[0092] In this embodiment, the motion saliency response map is still obtained by the motion saliency encoder. Then, a global statistical mapping is performed to generate a weight graph. :

[0093] ;

[0094] ;

[0095] = ;

[0096] in, This is the global mean of the current frame response graph. This represents the global standard deviation of the current frame response graph. ε is a positive constant to prevent division by zero. Sigmoid For the Sigmoid mapping function, make The value falls within [0, 1].

[0097] After the mapping process is completed, In S3 of Example 1, saliency modulation features are extracted, and in S4, weighted modulation and non-maximum suppression are applied to the candidate box confidence, and the detection results are output.

[0098] In other alternative implementations, a linear mapping based on the background frame quantile can also be used to generate the data. For example, the low and high quantiles of R can be statistically analyzed using the background set, then linearly mapped and clipped to [0, 1]. This alternative implementation also falls under the category of mapping processing.

[0099] Embodiment 4 of the present invention: Figure 4 As shown, Figure 4 This is a schematic diagram of the saliency modulation feature processing flow. In the l-th level feature extraction layer, based on the input feature map l, the motion saliency weight map obtained in step S2 is introduced. ,right After scale alignment, a weight map with the same feature resolution as that layer is obtained. Within the saliency-modulated convolutional block, the input features... They respectively enter the motion-aware convolution branch, the semantic-preserving convolution branch, and the gradient enhancement branch: among which, the motion-aware convolution branch... Performing spatial convolution yields the motion response, and then... Element-wise modulation of the motion response enhances the response intensity in motion-salient regions. Semantic-preserving convolutional branches are used. Perform channel mapping to preserve underlying semantic information. Gradient enhancement branch pairs. Extract spatial gradient magnitude features, and from Element-wise modulation of the gradient features enhances the edge and contour responses in motion-prone regions while suppressing ineffective gradient responses in low-motion regions. Subsequently, the outputs of the three branches are synthesized according to a fusion rule to obtain the output features of this layer. The final feature representation is generated through a nonlinear activation function, thereby achieving motion saliency-guided feature enhancement and background suppression. The specific calculation process is as follows:

[0100] Based on Example 1, in order to enhance the target edge and contour response in the salient motion region and suppress the activation of invalid edges in the low motion region, a thermal gradient enhancement modulation mechanism is introduced into the salient modulation convolution block. The remaining steps are still performed as in Example 1, and the detection results are output.

[0101] In this embodiment, S1 and S2 can be generated according to Embodiment 1. , and In the saliency modulation convolutional block of S3, in addition to the motion-aware branch and semantic-preserving branch of Example 1, a gradient enhancement branch is added, specifically as follows:

[0102] For the l-th level input features Calculate gradient magnitude features in the spatial dimension This embodiment uses the Sobel operator as an example, first calculating the gradient in the horizontal direction. gradient in the vertical direction Then take the amplitude:

[0103] ;

[0104] in, It can be calculated independently per channel and kept in line with Same spatial resolution. For numerical stability, it can be... Perform linear normalization to [0, 1].

[0105] Aligned weight map Modulation is applied to the gradient features to obtain the enhancement term:

[0106] ;

[0107] The final output features are: ;in, and The definition is consistent with that of Example 1. α is the gradient enhancement coefficient, used to control the enhancement intensity. The validation set can be selected to reduce the false alarm rate without significantly reducing detection accuracy. In this embodiment, the validation set can be selected. =0.1, and can be limited to the empirical range [0.05, 0.2]. It is a non-linear activation function.

[0108] After completing the above enhancements, construct the detection features according to S4 of Example 1 and perform detection head prediction, confidence-weighted modulation and non-maximum suppression to output the final detection result.

[0109] Embodiment 5 of the present invention: Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of a giant panda data detection method system based on learnable motion saliency modulation provided in an embodiment of this application. The system, employing a giant panda data detection method based on learnable motion saliency modulation, includes: a temporal difference construction module, a motion saliency encoding module, a lightweight feature backbone network module, and a detection and output module.

[0110] The temporal difference construction module is used to acquire a continuous infrared image frame sequence. For each current frame, multiple historical frames are extracted within a preset time window before its timestamp at different time intervals. Pixel-by-pixel difference operations are performed on the current frame and each historical frame respectively, and the difference results are stitched together in the channel dimension to obtain a multi-channel temporal difference input that represents motion changes at different time scales.

[0111] The motion saliency encoding module is used to input multi-channel time difference inputs into the motion saliency encoder, output a motion saliency response map, and perform normalization or mapping processing on the motion saliency response map to generate a motion saliency weight map.

[0112] The lightweight feature backbone network module is used to scale-align the motion saliency weight map and input it as a conditional control signal to the saliency modulation convolution blocks of each level of feature extraction layer of the lightweight feature backbone network, so that the motion saliency weight map participates in the spatial modulation of the convolution calculation process to obtain the motion saliency modulated feature map.

[0113] The detection and output module is used to construct detection features based on feature mapping and input them into the target detection head. It outputs the target category probability and bounding box regression parameters. After weighting and modulating the detection confidence corresponding to the target category probability using the motion saliency weight map, it combines non-maximum suppression to generate and output the giant panda target detection result.

[0114] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope and intent of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and variations.

Claims

1. A method for detecting giant panda data based on learnable motion saliency modulation, characterized in that, Includes the following steps: S1: Obtain a continuous infrared image frame sequence. For each current frame, extract multiple historical frames at different time intervals within a preset time window before its timestamp. Perform pixel-by-pixel difference operation on the current frame and each historical frame respectively, and stitch the difference results in the channel dimension to obtain a multi-channel time difference input that represents motion changes at different time scales. S2: Input the multi-channel time difference input to the motion saliency encoder, output the motion saliency response map, and perform normalization or mapping processing on the motion saliency response map to generate a motion saliency weight map; S3: After scale alignment, the motion saliency weight map is used as a conditional control signal and input to the saliency modulation convolution blocks of each level of feature extraction layer of the lightweight feature backbone network, so that the motion saliency weight map participates in the spatial modulation of the convolution calculation process to obtain the motion saliency modulated feature map. S4: Construct detection features based on the feature mapping and input them into the target detection head, output the target category probability and bounding box regression parameters, and use the motion saliency weight map to weight and modulate the detection confidence corresponding to the target category probability, and then combine non-maximum suppression to generate and output the giant panda target detection result; The motion saliency encoder includes a lightweight feature extraction unit and a saliency mapping unit. The saliency mapping unit is used to map the extracted motion features into a single-channel pre-response map and apply a nonlinear activation function to the pre-response map to obtain a single-channel motion saliency response map.

2. The giant panda data detection method based on learnable motion saliency modulation as described in claim 1, characterized in that: The different time intervals are set to be non-equidistant, so that the time difference input tensor can simultaneously represent rapid motion changes at short time scales and slow displacement changes at long time scales.

3. The giant panda data detection method based on learnable motion saliency modulation as described in claim 1, characterized in that: Generating the motion saliency weight map includes performing normalization processing on the motion saliency response map, so that the motion saliency weight map participates in the convolution calculation modulation of the saliency modulation convolution block with continuous values.

4. The giant panda data detection method based on learnable motion saliency modulation as described in claim 1, characterized in that, It also includes a pseudo-motor inhibition step: Calculate the global difference intensity index and spatial consistency index of the difference map between the current frame and one or more historical frames; When the global differential intensity index exceeds the first threshold and the spatial consistency index meets the global synchronization change characteristics, the corresponding frame is determined to be a global synchronization disturbance frame. During the global synchronization perturbation frame, a control strategy is adopted for the motion saliency weight map to maintain the weights of the previous time step or limit the weight update magnitude, so as to suppress the full map false activation caused by the global synchronization perturbation. If the above conditions are not met simultaneously, the corresponding frame is determined to be a non-global synchronization disturbance frame, and the motion saliency weight map is updated based on the difference results between the current frame and the historical frames.

5. The giant panda data detection method based on learnable motion saliency modulation as described in claim 1, characterized in that: The saliency modulation convolutional block includes motion-aware convolutional branches and semantic-preserving convolutional branches; The convolution output of the motion-aware convolution branch is modulated element-wise by the motion saliency weight map; The semantic-preserving convolutional branch is used to preserve basic semantic features; The outputs of the motion-aware convolutional branch and the semantic-preserving convolutional branch are fused to obtain the output features of the saliency modulation convolutional block.

6. The giant panda data detection method based on learnable motion saliency modulation as described in claim 1, characterized in that: The motion saliency weight map in the saliency modulation convolution block is used to differentially modulate the convolution calculation intensity at different spatial locations, so that the motion saliency region receives higher computational attention, while the convolution response of the low motion or static background region is suppressed.

7. The giant panda data detection method based on learnable motion saliency modulation as described in claim 1, characterized in that: The lightweight feature backbone network further includes a thermal gradient enhancement modulation mechanism: Gradient features are extracted from the input features in the spatial dimension, and the gradient features are modulated using the motion saliency weight map to enhance the target edge and contour response in the motion saliency region and suppress invalid gradient activation in the low motion region.

8. The giant panda data detection method based on learnable motion saliency modulation as described in claim 1, characterized in that: The weighted modulation of the confidence score includes suppressive modulation of the confidence score of the classification prediction based on the motion saliency weight map.

9. A giant panda data detection system based on learnable motion saliency modulation, employing the giant panda data detection method based on learnable motion saliency modulation as described in any one of claims 1-8, characterized in that, include: The module consists of a temporal difference construction module, a motion saliency encoding module, a lightweight feature backbone network module, and a detection and output module. The time difference construction module is used to acquire a continuous infrared image frame sequence. For each current frame, multiple historical frames are extracted at different time intervals within a preset time window before its timestamp. Pixel-by-pixel difference operations are performed on the current frame and each historical frame respectively, and the difference results are stitched together in the channel dimension to obtain a multi-channel time difference input that represents motion changes at different time scales. The motion saliency encoding module is used to input the multi-channel time difference input to the motion saliency encoder, output a motion saliency response map, and perform normalization or mapping processing on the motion saliency response map to generate a motion saliency weight map. The lightweight feature backbone network module is used to scale-align the motion saliency weight map and input it as a conditional control signal to the saliency modulation convolution blocks of each level of feature extraction layer of the lightweight feature backbone network, so that the motion saliency weight map participates in the spatial modulation of the convolution calculation process to obtain the motion saliency modulated feature map. The detection and output module is used to construct detection features based on the feature mapping and input them into the target detection head, output the target category probability and bounding box regression parameters, and use the motion saliency weight map to weight modulate the detection confidence corresponding to the target category probability, and then combine it with non-maximum suppression to generate and output the giant panda target detection result.

Images