Lightweight multi-target intelligent identification and tracking method for infrared scene

Abstract

The invention provides a lightweight multi-target intelligent identification and tracking method for an infrared scene. The method comprises a detection module and a tracking module. In the detection module, a lightweight feature extraction network is adopted to perform multi-scale feature extraction on the infrared image, feature expression with higher discrimination is constructed through feature selection and a cross-level fusion mechanism, and finally, the category and position information of a target is output through a detection head; in the tracking module, deep measurement learning is performed on the apparent features of the target based on the pre-trained lightweight network, and the distinguishing capability of the target features is enhanced by optimizing the measurement distance of the feature space, so that the robustness and precision of the tracking algorithm in the infrared dynamic scene are improved. According to the method, through network structure optimization and fine configuration of computing resources, the model parameter quantity and the computing complexity are remarkably reduced, the reasoning efficiency is improved, and meanwhile, the tracking performance of multiple targets in complex environments such as dynamic shielding and scale change is considered.

Description

Technical Field

[0001] This invention relates to a lightweight multi-target intelligent recognition and tracking method for infrared scenes, belonging to the field of infrared guided imaging target detection and tracking. Background Technology

[0002] Infrared and visible light detection differ significantly in their imaging principles and data characteristics. Infrared images rely on the thermal radiation characteristics of the target and lack the texture details and color information found in visible light images, but they are better at capturing target contours and motion features. Visible light images, on the other hand, contain rich color and texture information but are sensitive to lighting conditions. Therefore, infrared detection networks typically tend to use shallow structures to reduce computational complexity and inference latency, adapting to the simple features and high noise levels of infrared images while meeting real-time requirements. Visible light detection networks, on the other hand, often employ deep structures to extract multi-scale, high-order semantic features to address the complexity and diversity of visible light images. The shallow design of infrared detection is mainly due to its data characteristics, application scenarios such as low light, nighttime, camouflage scenarios, and resource constraints. Although shallow networks have limitations in feature representation capabilities, through lightweight optimization and algorithm improvements, they can still effectively balance detection accuracy and computational efficiency, becoming the mainstream choice for infrared target recognition.

[0003] ReID networks are deep learning models for target appearance feature learning and matching. Their core objective is to extract discriminative features such as color, texture, and pose, and achieve accurate target identification and association in complex environments across cameras, scenes, or timeframes. These networks typically use convolutional neural networks for multi-scale feature extraction and combine metric learning methods to optimize the feature space, ensuring that features of similar targets are as close as possible in the embedding space, while features of dissimilar targets are as far apart as possible, thereby improving feature discrimination and robustness. In multi-target tracking tasks, ReID networks achieve trajectory association across frames or cameras through appearance feature matching, effectively addressing challenges such as occlusion and viewpoint changes, and are widely used in video surveillance. It should be noted that ReID technology is primarily designed for pedestrian tracking, such as in surveillance scenarios, where its research and application are most mature. Its adaptability to other targets, such as vehicles and animals, is relatively limited. Therefore, in practical applications, targeted design and optimization based on the specific characteristics of the target are necessary.

[0004] To date, many papers have conducted research on infrared intelligent target detection and tracking. Reference 1: Wu X, Hong D, Chanussot J. UIU-Net: U-Net in U-Net for infrared small object detection[J]. IEEE Transactions on Image Processing, 2022, 32: 364-376. Existing infrared small target detection methods heavily rely on classification backbone networks. As network depth increases, this easily leads to the loss of small targets and limited feature discrimination capabilities. UIU-Net maintains the learned depth feature resolution by using single-layer dilated convolutions in shallow networks and multiple layers of dilated convolutions with different dilation rates in deep networks. Simultaneously, it utilizes interactive cross-attention modules to encode low-level details and high-level semantic features, replacing the skip connections in the original UNet framework to retain more contextual information. Reference 2: Guo C, Ren K, Chen Q. YOLO-SGF: Lightweight network for object detection in complex infrared images based on improved YOLOv8[J]. Infrared Physics & Technology, 2024, 142: 105539. The feature extraction layer is constructed using the lightweight architecture ShuffleNetV2, and lightweight convolution GSConv is used instead of ordinary convolution in the deep network to reduce the computational cost of deep features. Reference 3: Bewley A, Ge Z, Ott L, et al. Simple online and realtime tracking[C] / / 2016 IEEE international conference on image processing (ICIP). IEEE, 2016: 3464-3468. DeepSort, based on the Sort algorithm's prediction of the detection box's state in the next frame using Kalman filtering, uses a simple CNN network to extract the appearance features of the detected object. After detection and tracking in each frame, the object's appearance features are extracted and saved.Reference 4: Zhou, Kaiyang, et al. "Omni-scale feature learning for person re-identification." Proceedings of the IEEE / CVF international conference on computer vision. 2019. A lightweight network is designed for the characteristics of the ReID task. A unified aggregation gating mechanism is used to achieve dynamic multi-scale feature fusion based on the channel weights dependent on the input. To effectively learn spatial and channel correlations and avoid overfitting, a combination of point convolutions and depthwise convolutions is used simultaneously when constructing the network modules.

[0005] The shortcomings of the existing technologies are as follows: Existing ReID-based tracking methods exhibit excellent performance in tracking pedestrians, but their detection and tracking effectiveness for non-pedestrian targets drops significantly. The fundamental reason is that the model design relies excessively on the unique appearance features of pedestrians, resulting in insufficient generalization ability and difficulty adapting to diverse target shapes and motion patterns in cross-modal scenarios. Furthermore, existing detection networks have a large number of parameters and high computational complexity, leading to a dual challenge in infrared image detection: on the one hand, the inherent low resolution, noise interference, and feature sparsity of infrared images make it difficult for deep networks to effectively extract discriminative features; on the other hand, the high computational cost limits the deployment capability of the model in embedded devices or real-time scenarios, making it difficult to meet the real-time requirements of practical applications. These shortcomings severely restrict the applicability and engineering implementation of existing methods in complex dynamic scenarios. Summary of the Invention

[0006] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a lightweight multi-target intelligent recognition and tracking method for infrared scenes. The method improves the accuracy of infrared multi-target detection and tracking by using a lightweight target detection model and a lightweight ReID model.

[0007] The technical solution of this invention is: Firstly, a lightweight multi-target intelligent recognition and tracking method for infrared scenes is provided, including: The system comprises a detection module and a tracking module. The detection module consists of a lightweight feature extraction network, a feature selection and cross-layer fusion mechanism layer, and a detection head, connected in sequence. The tracking module consists of a pre-trained lightweight feature extraction ReID network. The infrared image is input into the detection module, where a lightweight feature extraction network extracts multi-scale features from the infrared image and outputs multi-scale features. The feature selection and cross-level fusion mechanism layer processes the multi-scale features and outputs the fused feature representation. The fused feature representation is then input into the detection head for calculation and outputs the target image patch. The target image patch is input into the pre-trained lightweight feature extraction ReID network of the tracking module to generate the target appearance feature vector; The cosine similarity is calculated using the target appearance feature vector during the cascaded matching process, and the minimum cosine distance value is selected as the appearance cost index; at the same time, the target motion state is predicted by the Kalman filter to generate the motion cost index. The appearance cost index and motion cost index are weighted and fused to obtain the final matching cost function. The target tracking state is updated and optimized based on the final matching cost function to achieve dynamic tracking and association of the target trajectory.

[0008] Preferably, the lightweight feature extraction network adopts the YOLOv8 detection framework and combines a lightweight full-scale convolution operator on the backbone of the YOLOv8 framework. The lightweight feature extraction network includes: ordinary convolution, full-scale convolution, and a full-scale module, specifically: A single ordinary convolutional layer connects to a combination of four consecutive full-scale convolutions and full-scale modules. The output of the fourth full-scale convolution and full-scale module is connected to a fast spatial pyramid pooling layer (SPPF). The first group of full-scale convolutions and full-scale module combinations outputs the first multi-scale feature F2, the second group of full-scale convolutions and full-scale module combinations outputs the second multi-scale feature F3, the third group of full-scale convolutions and full-scale module combinations outputs the third multi-scale feature F4, and the fast spatial pyramid pooling layer SPPF outputs the fourth multi-scale feature F5. The full-scale module consists of: a 1×1 convolutional layer connected in sequence, multiple full-scale convolutional branches, a feature splicing layer, and a final 1×1 convolutional layer.

[0009] Preferably, the feature selection and cross-level fusion mechanism layer uses BiFPN combined with full-scale convolution operators to process the input multi-scale features, specifically as follows: The first multi-scale feature F2 is directly output as the first fusion feature P2; The first multi-scale feature F2 is concatenated with the second multi-scale feature F3 through convolution to output the first intermediate feature M1. After downsampling, M1 is concatenated with the third multi-scale feature F4 to output the second intermediate feature M2. The second multi-scale feature F3 and M1 are concatenated to obtain the second fused feature P3; The third multi-scale feature F4 and M2 are concatenated to obtain the third fusion feature P4; The fourth fusion feature P5 is obtained by concatenating M2 downsampling with the fourth multi-scale feature F5. The obtained P2, P3, P4, and P5 values ​​are output to the detection head.

[0010] Preferably, the pre-trained lightweight feature extraction ReID network includes, in sequence: a size preprocessing module, a stemming extraction module, two basic processing modules, three full-scale processing modules, and a post-stemming processing module; specifically: The target image patch output by the detection module is resized by the size preprocessing module to fit the input requirements of the subsequent processing modules. The resized image is then processed by the simple extraction module to extract features and generate a basic feature map. The basic feature map is then processed by two basic processing modules to obtain a locally enhanced multi-scale feature representation. The locally enhanced multi-scale features are then fused by three full-scale processing modules to obtain cross-scale semantic information fusion features. The cross-scale semantic information fusion features are then input into the post-processing module to complete the target category classification output.

[0011] Preferably, the simplified extraction module includes: two consecutive convolutional layers and one max pooling layer.

[0012] Preferably, the basic module includes two consecutive convolutional layers and one downsampling layer.

[0013] Preferably, the full-scale processing module includes: a 1×1 convolutional layer, multiple full-scale convolutional branches, a feature concatenation layer and a final 1×1 convolutional layer connected in sequence. By processing the input features in parallel through multiple scale branches and fusing the output, it achieves efficient representation of target features and optimization of computational resources.

[0014] Preferably, the post-extraction module includes: an adaptive average pooling layer, a first linear transformation layer, a one-dimensional batch normalization layer, a ReLU activation function layer, and a second linear output layer connected in sequence, which achieves efficient compression of input features and generation of classification features through multi-level feature mapping and nonlinear activation.

[0015] Secondly, a terminal device is provided, comprising: Memory, used to store at least one instruction executed by a processor; A processor is used to execute instructions stored in memory to implement the methods described above.

[0016] Thirdly, a computer-readable storage medium is provided that stores computer instructions that, when executed on a computer, cause the computer to perform the methods described above.

[0017] Compared with the prior art, the present invention has the following advantages: (1) This invention features lightweight design and multimodal adaptability. By combining the YOLOv8 framework with a lightweight full-scale convolution operator structure, it significantly reduces the number of model parameters and computational complexity by utilizing modules such as depthwise separable convolution, thus meeting the real-time requirements of embedded devices in infrared scenarios. At the same time, it supports efficient feature extraction and appearance representation of multiple types of targets such as vehicles and pedestrians, breaking through the high dependence of traditional deep networks on hardware resources, and extending to multi-target tracking applications in complex dynamic scenarios.

[0018] (2) This invention has a multi-scale feature fusion mechanism and detection robustness. By using the BiFPN cross-level feature selection and fusion strategy, combined with the lightweight full-scale module to optimize feature expression, it effectively solves the problems of target outline blurring and noise interference in infrared images, improves the detection accuracy of small targets and complex backgrounds, and performs better than traditional shallow networks, especially in low-resolution and weak-texture scenes. Attached Figure Description

[0019] Figure 1 This is an overall flowchart of a lightweight multi-target intelligent recognition and tracking method for infrared scenes according to an embodiment of the present invention; Figure 2 This is a flowchart of the detection module of the lightweight multi-target intelligent recognition and tracking method for infrared scenes according to an embodiment of the present invention; Figure 3 This is a diagram of the lightweight feature extraction network structure of the detection module in the lightweight multi-target intelligent recognition and tracking method for infrared scenes according to an embodiment of the present invention. Figure 4 This is a layer structure diagram of the feature selection and cross-level fusion mechanism of the detection module in the lightweight multi-target intelligent recognition and tracking method for infrared scenes according to an embodiment of the present invention; Figure 5 This is a structural diagram of the tracking module of the lightweight multi-target intelligent recognition and tracking method for infrared scenes according to an embodiment of the present invention; Figure 6 This is a structural diagram of the lightweight ReID network model for the tracking module of the lightweight multi-target intelligent recognition and tracking method for infrared scenes according to an embodiment of the present invention. Figure 7 This is a structural diagram of the lightweight feature extraction module and the simplified extraction module of the ReID network in the infrared scene lightweight intelligent recognition and tracking method according to an embodiment of the present invention. Figure 8 This is a diagram of the lightweight feature extraction ReID network basic module of the tracking module in the lightweight multi-target intelligent recognition and tracking method for infrared scenes according to an embodiment of the present invention. Figure 9 This is a lightweight multi-target intelligent recognition and tracking method for infrared scenes according to an embodiment of the present invention, featuring a lightweight feature extraction ReID network full-scale convolutional graph for the tracking module. Figure 10 This is a full-scale module structure diagram of the lightweight feature extraction ReID network for the tracking module of the lightweight multi-target intelligent recognition and tracking method in infrared scenes according to an embodiment of the present invention. Figure 11 This is a diagram of the lightweight multi-target intelligent recognition and tracking method for infrared scenes according to an embodiment of the present invention, which includes a lightweight feature extraction module, a ReID network, and a post-extraction module. Detailed Implementation

[0020] To address the problems of low detection accuracy, high model computational complexity, poor generalization ability, and insufficient adaptability to multiple target categories in existing infrared target detection and tracking technologies, this invention proposes a lightweight multi-target intelligent recognition and tracking method for infrared scenes. By constructing a lightweight detection network and a lightweight ReID network, it achieves efficient recognition and accurate tracking of multiple target categories in infrared images.

[0021] The specific implementation method includes the following steps: A lightweight multi-target intelligent recognition and tracking method for infrared scenes includes a detection module and a tracking module.

[0022] In the detection module, a lightweight feature extraction network performs multi-scale feature extraction on the infrared image, outputting multi-scale features. The output multi-scale features are then input into a feature selection and cross-layer fusion mechanism layer, which outputs the fused feature representation. This fused feature representation is then input into the detection head, which outputs the target prediction bounding box, target category, and location information.

[0023] In the tracking module, the target image patch, cropped from the target prediction bounding box output in the detection phase, is input into a pre-trained lightweight feature extraction network to generate a target appearance feature vector. This target appearance feature vector is used to calculate cosine similarity during the cascaded matching process, and the minimum cosine distance value is selected as the appearance cost index. Simultaneously, a Kalman filter is used to predict the target motion state, generating a Mahalanobis distance metric as the motion cost index. The appearance cost index and the motion cost index are weighted and fused to obtain the final matching cost function. Based on this final matching cost function, the target tracking state is updated and optimized, thereby achieving dynamic tracking and association of the target trajectory.

[0024] The detection module specifically performs multi-scale feature extraction on infrared images through a lightweight feature extraction network. The extracted multi-scale features are input into the feature selection and cross-level fusion mechanism layer, which outputs the fused feature representation. The fused feature representation is then input into the detection head to output the target's category and location information.

[0025] The specific process of multi-scale feature extraction from infrared images using a lightweight feature extraction network is as follows: the infrared image is input into the lightweight feature extraction network, and the network model outputs features at different scales extracted by different network layers of different depths.

[0026] The lightweight feature extraction network is specifically as follows: The lightweight feature extraction network employs the YOLOv8 detection framework and incorporates a lightweight full-scale convolution operator on the YOLOv8 framework backbone. Infrared images are input into this network to extract multi-scale features.

[0027] The multi-scale feature input feature selection and cross-level fusion mechanism layer is specifically as follows: BiFPN combined with full-scale convolution operators is used to select multi-scale features from the input, and the selected multi-scale features are then fused and output.

[0028] The fused feature representation is input into the detection head to output the target's category and location information. Specifically, F2 is convolved with F3 to output intermediate feature M1, and M1 is downsampled and convolved with F4 to output intermediate feature M2. P2 is directly output by F2, P3 is convolved with F3 and M1, P4 is convolved with F4 and M2, and P5 is downsampled with M2 and convolved with F5. The features P2, P3, P4, and P5 are input into the detection head for calculation to obtain the target prediction result, i.e., the target's category and location.

[0029] The tracking module specifically employs an improved SORT tracking framework, inputting the target image patch cropped from the target prediction box output in the detection phase into a pre-trained lightweight feature extraction network to generate target appearance features. The improvement to the SORT tracking framework specifically involves improving the appearance feature extraction ReID network used in the SORT framework, while keeping the Kalman filter tracking unchanged.

[0030] The multi-scale features output are input into the feature selection and cross-level fusion mechanism layer output, resulting in a fused feature representation.

[0031] Input an infrared image, perform multi-scale feature extraction through a lightweight feature extraction network, and output multi-scale feature maps of different levels; The multi-scale features output are input into the feature selection and cross-level fusion mechanism layer. BiFPN combined with full-scale convolution operators is used to select and fuse the multi-scale features, and the fused feature representation is output. The fused feature representation is input into the detection head, and the target prediction box, target category, and location information are output. A lightweight feature extraction network is used to extract local feature information at different levels from the input infrared image and output the extracted local features. The lightweight feature extraction network includes: ordinary convolution, full-scale convolution and full-scale module. Specifically, the lightweight feature extraction network first goes through one convolution layer and then through four consecutive full-scale convolutions and full-scale modules. The full-scale module consists of a 1×1 convolution layer, multiple full-scale convolution branches (each containing 1 to 4 full-scale convolution operations), a feature concatenation layer and the final 1×1 convolution layer.

[0032] In the tracking module, the input image is cropped into image patches based on the target prediction bounding boxes output in the detection phase. These patches are then input into a pre-trained lightweight feature extraction network to generate a target appearance feature vector. This target appearance feature vector is used to calculate cosine similarity during the cascaded matching process, and the minimum cosine distance value is selected as the appearance cost index. Simultaneously, the target motion state is predicted using a Kalman filter, generating a Mahalanobis distance metric as the motion cost index. The appearance cost index and the motion cost index are weighted and fused to obtain the final matching cost function. Based on the final matching cost function, the target tracking state is updated and optimized, thereby achieving dynamic tracking and association of the target trajectory.

[0033] The lightweight feature extraction ReID network extracts features from the target image patch after cropping the target prediction box output from the detection stage, and outputs the generated target representation feature vector. The lightweight feature extraction ReID network includes a feature extraction model suitable for multiple categories of targets such as vehicles and pedestrians. Its core structure integrates a lightweight full-scale convolution operator. By optimizing the network parameter scale and computational complexity, it achieves efficient feature extraction while ensuring the feature representation capability.

[0034] The specific pre-trained lightweight feature extraction ReID network, to adapt to more targets beyond just people with fixed aspect ratios, first scales the input image patch to 128x128 before feeding it into the network. The network first enters two 3x3 convolutional kernels, followed by max-pooling downsampling, and then two basic modules, each containing two convolutional layers. Both convolutions employ Conv-BN-ReLU. Next, it passes through three full-scale modules, consisting of convolutions and depthwise separable convolutions. Finally, it outputs a class classification through adaptive average downsampling and a fully connected layer.

[0035] The lightweight feature extraction network is specifically constructed based on the YOLOv8 detection framework, and a lightweight convolution operator is introduced into its backbone network to reduce the number of network parameters and computational complexity while maintaining feature extraction capability. The lightweight feature extraction network outputs multi-scale feature maps of different depths after the infrared image is input, which are used for subsequent feature selection and fusion. The feature selection and cross-level fusion mechanism layer specifically employs a BiFPN structure, combining full-scale convolution operators for feature selection and cross-level feature fusion to enhance feature representation capabilities. Specifically, F2 outputs intermediate feature M1 by convolution and concatenating it with F3; M1 is downsampled and concatenated with F4 to output intermediate feature M2. P2 is directly output by F2, P3 is output by concatenating F3 and M1, P4 is output by concatenating F4 and M2, and P5 is output by downsampling M2 and concatenating it with F5.

[0036] The detection head specifically uses multi-scale features P2, P3, P4, and P5 as inputs to the detection head and finally outputs the target prediction box, category, and location information. The tracking module employs an improved SORT tracking framework, which crops the target prediction box output from the detection phase into image patches, inputs them into a pre-trained lightweight ReID network, and generates a target appearance feature vector. The lightweight ReID network is specifically a feature extraction model applicable to multiple categories of targets such as vehicles and pedestrians. Its core structure integrates a full-scale lightweight operator. By optimizing the network parameter scale and computational complexity, it achieves efficient feature extraction while ensuring the feature appearance representation capability. In the improved SORT framework, the appearance feature extraction part is replaced by a lightweight ReID network, while the Kalman filter tracking part remains unchanged to ensure the stability and real-time performance of the tracking process. The target appearance feature vector is used to calculate cosine similarity during the cascade matching process, and the minimum cosine distance value is selected as the appearance cost index. Meanwhile, the target motion state is predicted by using a Kalman filter, and a Mahalanobis distance metric is generated as a motion cost indicator. The appearance cost index and motion cost index are weighted and fused to obtain the final matching cost function; The target tracking state is updated and optimized based on the final matching cost function, thereby realizing dynamic tracking and association of the target trajectory.

[0037] This invention significantly improves the accuracy and real-time performance of multi-target detection and tracking in infrared scenes by constructing a lightweight feature extraction network and a lightweight ReID network. Through feature selection and cross-level fusion mechanisms, it enhances the model's ability to recognize complex backgrounds and small targets. Through the improved SORT tracking framework, it improves the stability and robustness of target association, effectively solving the problems of poor adaptability to multiple target categories, high computational complexity, and low tracking accuracy in existing technologies, and has good prospects for engineering applications.

[0038] Example: This embodiment is implemented based on the technical solution of the present invention, and provides detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiment.

[0039] The overall structure of a lightweight multi-target intelligent recognition and tracking method for infrared scenes is as follows: Figure 1 As shown, the input image passes through the detection module and outputs the detection result. The detection result is then input into the tracking module and outputs the tracking result.

[0040] Detection module such as Figure 2 As shown, the input image is processed by a lightweight feature extraction network to extract multi-scale features, outputting multi-scale features. The multi-scale features are input to the feature selection and cross-layer fusion mechanism layer, which outputs the fused feature representation. The fused feature representation is input to the detection head, which outputs the target prediction bounding box, target category, and location information. The output target prediction bounding box is combined with the input image to output a target slice image patch.

[0041] Lightweight feature extraction backbone network for detection module, such as Figure 3 As shown, the input image goes through one convolutional layer and then through a series of full-scale convolutions and full-scale modules, with the corresponding features F2, F3, F4, and F5 output in the full-scale module respectively.

[0042] The detection module features lightweight feature extraction, network feature selection, and cross-layer fusion mechanism, as shown in the following layers. Figure 4 As shown, the input features F2, F3, F4, and F5 are convolved, downsampled, and concatenated to output P2, P3, P4, and P5. Specifically, F2 is convolved with F3 to output intermediate feature M1, and M1 is downsampled and concatenated with F4 to output intermediate feature M2. P2 is directly output from F2, P3 is concatenated with F3 and M1, P4 is concatenated with F4 and M2, and P5 is downsampled from M2 and concatenated with F5.

[0043] Tracking module such as Figure 5 As shown, the input target image patch outputs the target appearance feature vector through a pre-trained lightweight ReID network.

[0044] The lightweight ReID network model structure diagram of the tracking module is shown below. Figure 6 As shown, the input image is first resized to fit the input requirements of the subsequent processing modules. Then, the resized image is processed by the simple extraction module to extract features and generate a basic feature map. The basic feature map is then input into two basic processing modules to enhance the expression of local features. Subsequently, the multi-scale features are fused through three full-scale processing modules to obtain cross-scale semantic information fusion features. Finally, the fused features are input into the post-processing module to complete the classification output of the target category.

[0045] Lightweight feature extraction module for tracking; ReID network stem extraction module, such as... Figure 7 As shown, the simple extraction module consists of two consecutive convolutional layers and one max pooling layer.

[0046] Lightweight feature extraction tracking module ReID network basic modules such as Figure 8 As shown, it consists of two consecutive convolutional layers and one downsampling layer.

[0047] Lightweight feature extraction in tracking modules; full-scale convolution in ReID networks, such as... Figure 9 As shown, it consists of sequentially connected convolutional layers, depthwise separable convolutional layers, batch normalization layers, and ReLU activation functions.

[0048] Lightweight feature extraction tracking module, ReID network full-scale module, such as Figure 10 As shown, it includes a 1×1 convolutional layer, multiple full-scale convolutional branches (each containing 1 to 4 layers of full-scale convolutional operations), a feature concatenation layer, and a final 1×1 convolutional layer connected in sequence. By processing the input features in parallel through multiple scale branches and fusing the output, it achieves efficient representation of target features and optimization of computational resources.

[0049] Lightweight feature extraction module for tracking; ReID network post-processing module, such as... Figure 11 As shown, it includes an adaptive average pooling layer, a first linear transformation layer, a one-dimensional batch normalization layer, a ReLU activation function layer, and a second linear output layer connected in sequence. Through multi-level feature mapping and nonlinear activation, it achieves efficient compression of input features and generation of classification features.

[0050] Existing technologies for infrared multi-target detection and tracking have significant shortcomings. Infrared target detection datasets are difficult to obtain and have high annotation costs. When existing visible light detection models are directly transferred to infrared scenes, the difference in feature distribution leads to decreased detection accuracy, resulting in high false negative and false positive rates, making it difficult to meet the real-time and robustness requirements of complex dynamic scenes. To address these technical problems, this invention provides a lightweight intelligent multi-target recognition and tracking method for infrared scenes, comprising: performing hierarchical feature extraction on the input infrared image through a multi-scale feature extraction network, and generating highly discriminative feature representations by combining feature selection and cross-level fusion mechanisms; inputting the fused features into a detection head to output target prediction boxes, category, and location information; in the tracking module, cropping image patches from the detection boxes and inputting them into a pre-trained lightweight ReID network to extract appearance features, calculating the matching cost through cosine similarity and Mahalanobis distance, and combining Kalman filtering to achieve dynamic association of target trajectories. Specifically, the lightweight feature extraction network adopts the YOLOv8 framework and integrates full-scale convolution operators. Multi-scale features are fused across layers through the BiFPN structure, and the detection head adopts a multi-scale prediction mechanism. The ReID network is based on a lightweight full-scale modular architecture and optimizes feature expression through depthwise separable convolution, effectively improving the appearance representation ability and cross-scene generalization performance of multi-class targets such as vehicles and pedestrians.

[0051] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

[0052] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A lightweight multi-target intelligent recognition and tracking method for infrared scenes, characterized in that... include: The system comprises a detection module and a tracking module. The detection module consists of a lightweight feature extraction network, a feature selection and cross-layer fusion mechanism layer, and a detection head, connected in sequence. The tracking module consists of a pre-trained lightweight feature extraction ReID network. The infrared image is input into the detection module, where a lightweight feature extraction network extracts multi-scale features from the infrared image and outputs multi-scale features. The feature selection and cross-level fusion mechanism layer processes the multi-scale features and outputs the fused feature representation. The fused feature representation is then input into the detection head for calculation and outputs the target image patch. The target image patch is input into the pre-trained lightweight feature extraction ReID network of the tracking module to generate the target appearance feature vector; The cosine similarity is calculated using the target appearance feature vector during the cascaded matching process, and the minimum cosine distance value is selected as the appearance cost index; at the same time, the target motion state is predicted by the Kalman filter to generate the motion cost index. The appearance cost index and motion cost index are weighted and fused to obtain the final matching cost function. The target tracking state is updated and optimized based on the final matching cost function to achieve dynamic tracking and association of the target trajectory.

2. The lightweight multi-target intelligent recognition and tracking method for infrared scenes according to claim 1, characterized in that: The lightweight feature extraction network adopts the YOLOv8 detection framework and combines a lightweight full-scale convolution operator on the backbone of the YOLOv8 framework. Lightweight feature extraction networks include: ordinary convolution, full-scale convolution, and full-scale modules, specifically: A single ordinary convolutional layer connects to a combination of four consecutive full-scale convolutions and full-scale modules. The output of the fourth full-scale convolution and full-scale module is connected to a fast spatial pyramid pooling layer (SPPF). The first group of full-scale convolutions and full-scale module combinations outputs the first multi-scale feature F2, the second group of full-scale convolutions and full-scale module combinations outputs the second multi-scale feature F3, the third group of full-scale convolutions and full-scale module combinations outputs the third multi-scale feature F4, and the fast spatial pyramid pooling layer SPPF outputs the fourth multi-scale feature F5. The full-scale module consists of: a 1×1 convolutional layer connected in sequence, multiple full-scale convolutional branches, a feature splicing layer, and a final 1×1 convolutional layer.

3. The lightweight multi-target intelligent recognition and tracking method for infrared scenes according to claim 2, characterized in that: The feature selection and cross-layer fusion mechanism layer uses BiFPN combined with full-scale convolution operators to process the input multi-scale features, specifically: The first multi-scale feature F2 is directly output as the first fusion feature P2; The first multi-scale feature F2 is concatenated with the second multi-scale feature F3 through convolution to output the first intermediate feature M1. After downsampling, M1 is concatenated with the third multi-scale feature F4 to output the second intermediate feature M2. The second multi-scale feature F3 and M1 are concatenated to obtain the second fused feature P3; The third multi-scale feature F4 and M2 are concatenated to obtain the third fusion feature P4; The fourth fusion feature P5 is obtained by concatenating M2 downsampling with the fourth multi-scale feature F5. The obtained P2, P3, P4, and P5 values ​​are output to the detection head.

4. The lightweight multi-target intelligent recognition and tracking method for infrared scenes according to claim 1, characterized in that: The pre-trained lightweight feature extraction ReID network includes, in sequence: a size preprocessing module, a stemming extraction module, two basic processing modules, three full-scale processing modules, and a post-stemming module; specifically: The target image patch output by the detection module is resized by the size preprocessing module to fit the input requirements of the subsequent processing modules. The resized image is then processed by the simple extraction module to extract features and generate a basic feature map. The basic feature map is then processed by two basic processing modules to obtain a locally enhanced multi-scale feature representation. The locally enhanced multi-scale features are then fused by three full-scale processing modules to obtain cross-scale semantic information fusion features. The cross-scale semantic information fusion features are then input into the post-processing module to complete the target category classification output.

5. A lightweight multi-target intelligent recognition and tracking method for infrared scenes according to claim 4, characterized in that: The simplified extraction module consists of two consecutive convolutional layers and one max pooling layer.

6. A lightweight multi-target intelligent recognition and tracking method for infrared scenes according to claim 4, characterized in that: The basic module consists of two consecutive convolutional layers and one downsampling layer.

7. A lightweight multi-target intelligent recognition and tracking method for infrared scenes according to claim 4, characterized in that: The full-scale processing module includes a 1×1 convolutional layer, multiple full-scale convolutional branches, a feature concatenation layer, and a final 1×1 convolutional layer connected in sequence. By processing input features in parallel through multiple scale branches and fusing the output, it achieves efficient representation of target features and optimization of computational resources.

8. A lightweight multi-target intelligent recognition and tracking method for infrared scenes according to claim 4, characterized in that: The post-extraction module includes: an adaptive average pooling layer, a first linear transformation layer, a one-dimensional batch normalization layer, a ReLU activation function layer, and a second linear output layer connected in sequence. It achieves efficient compression of input features and generation of classification features through multi-level feature mapping and nonlinear activation.

9. A terminal device, characterized in that, include: Memory, used to store at least one instruction executed by a processor; A processor for executing instructions stored in memory to implement the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-8.

Images