Infrared image target detection method and system based on deep learning
Through deep learning technology, including CNN feature extraction, time dimension analysis of recurrent neural networks and Kalman filter tracking, the problem of feature similarity false alarms and overlapping heat source separation in infrared image object detection is solved, and the system's recognition accuracy and adaptation are improved to ensure the reliability of home security monitoring.
Patent Information
- Application Number
- CN202410784050.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-18
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-06-18
AI Technical Summary
The existing infrared image object detection technology can easily lead to false alarms when processing different target characteristics are similar, making it difficult to separate overlapping heat sources, and insufficient processing of unlabeled targets, resulting in insufficient response speed and degree of intelligence of the system in complex scenarios.
Using a deep learning-based method, CNN is used for feature extraction and standardization processing, combined with the time-dimensional characteristics of the recurrent neural network to analyze the target, dynamically adjust the identification threshold and the improved non-maximum suppression algorithm, using Kalman filter and Hungarian algorithm Improve the tracking stability of the target and introduce self-learning methods to optimize model parameters.
It significantly improves the accuracy and robustness of infrared image object detection and recognition, reduces false alarms and missed alarms, improves the system's adaptability and intelligence capabilities in dynamic environments, and provides more efficient and reliable home security monitoring guarantees.
Smart Images

Figure CN118644723B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of security monitoring, and more specifically, to an infrared image target detection method and system based on deep learning. Background Art
[0002] Infrared image target detection is a method that uses infrared imaging technology and image processing algorithms to identify and track target objects in a scene. Infrared imaging technology generates images by detecting the thermal radiation on the surface of an object. It can work in a completely dark environment, so it has a wide range of applications in security monitoring, military reconnaissance, and industrial detection. In target detection, the algorithm preprocesses the infrared image, such as noise filtering, grayscale adjustment, and image enhancement, and then uses target detection algorithms (such as YOLO, SSD, etc.) to identify the target location in the image, classify it, and track it.
[0003] The existing infrared image target detection technology still has some shortcomings in practical applications. First, when the features of different targets are similar, it is easy to cause false alarms, affecting the accuracy of the system. Secondly, for overlapping heat sources, it is difficult for existing technologies to effectively separate and identify them, resulting in missed and false alarms in target detection. In addition, for targets without labels, the existing system lacks intelligent processing methods and cannot make reasonable judgments based on dynamic scenes, resulting in insufficient response speed and intelligence of the system when processing complex scenes. These shortcomings limit the application effect of infrared image target detection technology in complex scenes.
[0004] In order to solve the above problems, a technical solution is now provided. Summary of the invention
[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an infrared image target detection method and system based on deep learning, which uses CNN to extract features, obtain the time dimension features of the target, and process overlapping and changing targets; by dynamically adjusting the recognition threshold and the improved non-maximum suppression algorithm, it ensures that the heat source is independently identified and processed. Combined with the Kalman filter and the Hungarian algorithm, the association and tracking stability of the target between consecutive frames are improved. The self-learning method is introduced to optimize the model parameters and enhance the adaptability and robustness of the system in a dynamic environment. By detecting unlabeled targets and specific category labels, the intelligence and responsiveness of the system are improved. The accuracy and robustness of infrared image target detection and recognition are improved, false alarms and missed alarms are reduced, and more efficient and reliable home security monitoring guarantees are provided to solve the problems raised in the above background technology.
[0006] To achieve the above object, the present invention provides the following technical solutions:
[0007] S1, using an infrared camera to collect infrared images in the environment, using a target detection algorithm to detect targets in the preprocessed images, and locating and marking the positions of each target;
[0008] S2, identifies the detected targets. When it is determined that the features of different targets are similar, a recurrent neural network is used to analyze the continuous infrared image sequence, dynamically adjust the threshold through the information of the time dimension, separate the different heat sources, and identify the separated heat sources separately, and attach labels to all identified targets;
[0009] S3, continuously tracking the identified target, recording the target's motion trajectory, using the recorded target's motion trajectory and recognition results as input data for self-learning, and optimizing the parameters of the target detection and recognition algorithm;
[0010] S4, at the latest time point, analyzes whether the identified targets in the scene are labeled. If there are targets without labels and the labels of all identified targets in the scene do not fall within the specified conditions, an early warning reminder is issued.
[0011] In a preferred embodiment, S2 includes the following contents:
[0012] S1.1, calibrate the infrared camera using a standard blackbody radiation source;
[0013] S1.2, collect infrared images at different time periods and angles to capture target objects under various environmental conditions;
[0014] S1.3, applying median filtering and Gaussian filtering to remove image noise, and using histogram equalization to enhance image contrast;
[0015] S1.4, using YOLO and FasterR-CNN as the base models, fine-tune them respectively, and fuse their prediction results:
[0016] S1.4-1, the fusion process combines the bounding box sets and category sets of the two models through the weighted average method;
[0017] S1.4-2, using an adaptive module based on a recurrent neural network, with a continuous infrared image sequence as input and an adjustment coefficient of the model parameters as output, to dynamically adjust the model;
[0018] S1.4-3, during the training process, multi-scale transformation is performed on each original image to generate a multi-scale image, and the image size is adjusted by scaling operation;
[0019] S1.4-4, use the generative adversarial network to expand the training data set and generate additional infrared image data. The generative adversarial network includes a generator and a discriminator. The generator inputs a noise vector and outputs a generated image.
[0020] In a preferred embodiment, S1.5, the preprocessed infrared image is input, and the fused YOLO and FasterR-CNN model is used to perform target detection, and the bounding box and category label of each target in the image are output, wherein the bounding box describes the position and size of the target, and the category label identifies the type of the target:
[0021] S1.5-1, combining the image sequence of the current frame and several frames before and after, using a convolutional neural network to extract temporal features, and fusing the temporal features with the image features of the current frame;
[0022] S1.5-2, using an adaptive module based on a recurrent neural network to adjust the detection threshold of the current frame according to the detection results of the previous and next frames;
[0023] S1.5-3, process the detected multiple bounding boxes and use the non-maximum suppression algorithm to remove redundant and overlapping bounding boxes to ensure that each target object is marked only once;
[0024] S1.5-4, mark the detected target location on the image, draw the bounding box and category label, and overlay the bounding box and label on the image;
[0025] S1.5-5, save the detection results to a file or database, including the bounding box coordinates, category label, detection confidence and timestamp of each target.
[0026] In a preferred embodiment, S2 includes the following contents:
[0027] S2.1, use convolutional neural network to extract features of each detected target and generate feature vector;
[0028] S2.1-1, standardize the feature vector of each target;
[0029] S2.1-2, calculate the cosine similarity of the two target feature vectors after normalization;
[0030] S2.1-3, calculate the average similarity and standard deviation between feature vectors, and determine the dynamic threshold by the sum of the two;
[0031] S2.1-4, use the dynamic threshold to determine whether the target features are similar. If the cosine similarity of the target feature vector is higher than the dynamic threshold, the target features are determined to be similar and marked as the target to be analyzed. Otherwise, the features are considered to be dissimilar.
[0032] In a preferred embodiment, S2.2, for the target to be analyzed, a continuous infrared image sequence is analyzed using a recurrent neural network to extract the characteristics of the target in the time dimension:
[0033] S2.2-1, the feature vectors of the target to be analyzed in different time frames form a time series;
[0034] S2.2-2, the state update function of the recurrent neural network combines the input feature vector of the current time step and the hidden state of the previous time step to gradually extract the dynamic change information in the time series;
[0035] S2.2-3, the entire time series is processed by a recurrent neural network, and the hidden state of the last time step is used as the feature representation of the time dimension, reflecting the change pattern of the target to be analyzed in the entire time series;
[0036] S2.3, dynamically adjust the target recognition threshold according to the time dimension characteristics to adapt it to different targets and scenarios. Through the threshold adjustment function, a new recognition threshold is calculated according to the time dimension characteristics:
[0037] S2.3-1, the feature aggregation function combines the time dimension features with the current frame features to generate a comprehensive feature representation;
[0038] S2.3-2, dynamically generating a recognition threshold through a threshold calculation function based on the comprehensive feature representation;
[0039] S2.3-3, combining temporal and spatial features, a dynamically adjusted recognition threshold is obtained through feature aggregation and threshold calculation function.
[0040] In a preferred embodiment, S2.4, the target to be analyzed is separated by dynamically adjusting the recognition threshold, and the heat source is separated by using a non-maximum suppression algorithm to ensure that each heat source is identified separately:
[0041] S2.4-1, use cosine similarity to calculate the similarity between the comprehensive feature representations of each target to be analyzed, and record it as the similarity of the target to be analyzed;
[0042] S2.4-2, combining the similarity of the target to be analyzed and the dynamically adjusted recognition threshold to perform non-maximum suppression, retaining targets whose similarity is less than the dynamically adjusted recognition threshold, and removing redundant bounding boxes;
[0043] S2.4-3, individually identifying and re-extracting features of the retained targets to be analyzed, and repeating steps S2.1 to S2.4 until each target to be analyzed is identified;
[0044] S2.5, marking the identified target as an identified target, and labeling the identified target.
[0045] In a preferred embodiment, S3 includes the following contents:
[0046] S3.1, set the initial state vector and covariance matrix for each identified target, use the Kalman filter to predict the next position of the identified target, and use the Hungarian algorithm to associate the identified targets detected in the current frame with the identified targets in the previous frame;
[0047] S3.2, according to the motion model of the identified target, the next state of the target is predicted using the state transfer matrix and the process noise covariance matrix, and the predicted state vector and covariance matrix are corrected in combination with the control vector and the control matrix;
[0048] S3.3, use the YOLO and FasterR-CNN models fused in step S1.4 to detect the identified target, obtain the detection result, use the observation matrix to map the detection result to the state space, and update the state vector and covariance matrix of the identified target by combining the observation data through the Kalman gain;
[0049] S3.4, record the motion trajectory of each identified target and store the state vector of each time frame in a trajectory set;
[0050] S3.5, using the recorded trajectory and recognition results, calculates the gradient according to the trajectory and recognition results, and uses the gradient descent algorithm to optimize the parameters of the identified target detection and recognition model for self-learning.
[0051] In a preferred embodiment, S4 includes the following contents:
[0052] S4.1, check whether each identified object present in the scene at the latest time point has a label attached;
[0053] S4.2, for the identified targets without labels, check whether the labels of all identified targets in the scene contain a specific category;
[0054] S4.3, if there are unlabeled targets in the scene and there are no targets of a specific category among all identified targets, a remote warning alert is generated.
[0055] The infrared image target detection system based on deep learning includes: infrared image acquisition module, target detection module, target recognition module, target tracking module, self-learning optimization module and label analysis module;
[0056] The infrared image acquisition module uses an infrared camera to collect infrared images in the environment, performs preliminary preprocessing on the images to ensure image quality, and passes the preprocessed infrared images to the target detection module;
[0057] The target detection module uses the target detection algorithm to detect the targets in the preprocessed infrared image, locate and mark the positions of each target, and pass the marked target information to the target recognition module;
[0058] The target recognition module recognizes the targets transmitted by the target detection module. When it is determined that the features of different targets are similar, it uses a recurrent neural network to analyze the continuous infrared image sequence, dynamically adjusts the threshold through the information of the time dimension, separates different heat sources, and identifies the separated heat sources separately. It attaches labels to all identified targets and transmits the recognition and label information to the target tracking module.
[0059] The target tracking module continuously tracks the identified target transmitted by the target recognition module, records the target's motion trajectory, and transmits the tracked target's motion trajectory and recognition results to the label analysis module and the self-learning optimization module;
[0060] The self-learning optimization module uses the trajectory records and recognition results transmitted by the target tracking module to perform self-learning, optimize the parameters of the target detection and recognition algorithms through the self-learning process, and transmit the latest parameter adjustments back to the target detection module and the target recognition module;
[0061] The label analysis module analyzes whether the identified targets in the scene are labeled at the latest time point. If there are targets without labels and the labels of all identified targets in the scene do not fall within the specified conditions, an early warning reminder is issued.
[0062] The technical effects and advantages of the infrared image target detection method and system based on deep learning of the present invention are as follows:
[0063] 1. The present invention uses CNN to extract features and standardize them, making feature similarity determination more accurate; combines RNN to analyze the time dimension features of the target, captures the motion trajectory, and processes overlapping and changing targets; and ensures that the heat source is independently identified and processed by dynamically adjusting the recognition threshold and the improved non-maximum suppression algorithm. This helps to significantly improve the accuracy and robustness of infrared image target detection and recognition in home security monitoring, reduce false positives and false negatives, improve detection accuracy and system adaptability, and provide more efficient and reliable home security monitoring protection.
[0064] 2. The present invention significantly improves the accuracy and continuity of target tracking and reduces the situation of target loss through target tracking and position prediction. The combination of Kalman filter and Hungarian algorithm ensures the effective association of targets between consecutive frames and improves the stability of tracking. At the same time, the introduction of self-learning method enables the system to continuously obtain feedback from the tracking and recognition process, and enhances the adaptability and robustness of the system in a dynamic environment by optimizing model parameters. Self-learning not only improves the accuracy of target detection and recognition, but also enables the system to have the ability of self-improvement, can continuously adapt to environmental changes and the diversity of target characteristics, and ensure long-term reliability and performance stability.
[0065] 3. By detecting unlabeled targets and specific category labels in the scene, the intelligence and responsiveness of the system are improved. The system can timely detect and warn unlabeled potential intrusion targets without the supervision of identified targets, reduce false alarms, and enhance the reliability and effectiveness of home security monitoring. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 It is a flowchart of the infrared image target detection method based on deep learning of the present invention;
[0067] Figure 2 It is a structural schematic diagram of the infrared image target detection system based on deep learning of the present invention. DETAILED DESCRIPTION
[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0069] Example 1
[0070] Figure 1 The infrared image target detection method based on deep learning of the present invention is given:
[0071] S1, using an infrared camera to collect infrared images in the environment, using a target detection algorithm to detect targets in the preprocessed image, and locating and marking the positions of each target;
[0072] S2, identifies the detected targets. When it is determined that the features of different targets are similar, a recurrent neural network is used to analyze the continuous infrared image sequence, dynamically adjust the threshold through the information of the time dimension, separate the different heat sources, and identify the separated heat sources separately, and attach labels to all identified targets;
[0073] S3, continuously tracking the identified target, recording the target's motion trajectory, using the recorded target's motion trajectory and recognition results as input data for self-learning, and optimizing the parameters of the target detection and recognition algorithm;
[0074] S4, analyzing whether the identified targets existing in the scene are labeled at the latest time point. If no label is attached and the labels of all identified targets in the same scene do not fall within the specified conditions, an early warning reminder is issued.
[0075] In home security monitoring scenarios, especially at night or in low light conditions, traditional visible light cameras often have difficulty capturing clear images, leading to increased safety hazards. Infrared image cameras are able to work in completely dark environments and generate images by detecting the thermal radiation of objects. However, infrared images are affected by problems such as ambient light interference, noise, and insufficient contrast, which may lead to reduced accuracy in target detection. In order to solve these problems and ensure accurate detection of target objects under various environmental conditions, it is necessary to perform detailed image acquisition and preprocessing steps. Step S1 improves the quality of infrared images and the accuracy of detection through a series of innovative technical processing, providing more reliable protection for home security.
[0076] Step S1 includes the following contents:
[0077] S1.1, first, calibrate the infrared camera using a standard black body radiation source to ensure the accuracy of temperature measurement. Adjust the ambient light conditions and use a sunshade or filter to reduce the interference of ambient light on the infrared image to ensure that the camera can accurately capture the thermal radiation of the target object.
[0078] S1.2, collect infrared images at different time periods and angles to capture target objects under various environmental conditions. By collecting images at different time periods of the day (such as morning, noon, and evening) and from multiple angles, the different states and positions of the target objects can be fully covered, improving the reliability and accuracy of subsequent detection.
[0079] S1.3, pre-process the collected infrared images to improve image quality and detection accuracy. First, apply median filtering and Gaussian filtering to remove image noise and reduce the impact of high-frequency noise. Then, use histogram equalization or adaptive histogram equalization to enhance image contrast and ensure that the target objects in the image are clearer. Finally, normalize the image pixel values to the [0,1] interval to meet the input requirements of the deep learning model and ensure that the model can correctly process the image data.
[0080] S1.4, the prediction results of two pre-trained models, YOLO and FasterR-CNN, are combined through the multi-model fusion method to improve the robustness of detection. Adopt an adaptive fine-tuning strategy and introduce an adaptive module based on a recurrent neural network (RNN) to adjust the model parameters in real time according to the changes in the image sequence, thereby enhancing the adaptability of the model in different environments. Multi-scale input images are used during training to improve the model's ability to detect targets of different sizes. More training samples are generated through data augmentation technology and generative adversarial networks (GANs), the training data set is expanded, and the generalization ability of the model is further improved. Through these innovative processing steps, efficient target detection of infrared images is achieved.
[0081] S1.4-1, select YOLO and FasterR-CNN as the basic models, fine-tune them respectively, and fuse their prediction results;
[0082] Let the output of YOLO be the bounding box set B YOLO and category set C YOLO , the output of FasterR-CNN is the bounding box set B R-CNN and category set C R-CNN The bounding box indicates the location of the object, and the category indicates the type of the object.
[0083] Fusion formula: B fusion =αB YOLO +(1-α)B R-CNN ; C fusion =βC YOLO +(1-β)C R-CNN ;
[0084] Among them, α and β are fusion weights, which are between 0 and 1 respectively. The weights are used to control the relative importance of the outputs of the two models.
[0085] S1.4-2, using an adaptive module based on a recurrent neural network (RNN) to dynamically adjust the adaptive model. The input of the adaptive module is a continuous infrared image sequence, and the output is the adjustment coefficient of the model parameters. Specifically, the input of the RNN module is the image sequence X at the tth moment t , the output is the adjustment coefficient Δθ. The model parameter adjustment formula is: θ t+1 =θ t +Δθ
[0086] Among them, θ t is the model parameter at the tth moment; Δθ is the adjustment coefficient of the adaptive model output.
[0087] Through this process, model parameters can be adaptively adjusted according to environmental changes, enhancing the adaptability of the model in different environments.
[0088] S1.4-3, during the training process, use images of different scales to improve the model's ability to detect objects of different sizes. Specifically, perform multi-scale transformation on each original image to generate a multi-scale image. Let the original image be Ι and the multi-scale image be Ι s , the transformation formula is: s =Resize(Ι,s);
[0089] Among them, s is the scaling ratio (such as 0.5, 1.0, 1.5, etc.); Resize(Ι,s) represents a scaling operation, and the function resizes the original image Ι according to the scaling ratio s.
[0090] Through multi-scale training methods, the model can learn the characteristics of targets at different scales, thereby improving the accuracy and robustness of detection.
[0091] S1.4-4, Generate diverse training samples through data augmentation technology to expand the training data set. Data augmentation includes operations such as rotation, flipping, and translation, and the generative adversarial network is used to generate additional infrared image data. Specifically, GAN includes a generator and a discriminator. The input of the generator is a noise vector, and the output is a generated infrared image. The input of the discriminator is a real image or a generated image, and the output is the probability of being real or fake.
[0092] By using multi-model fusion and adaptive fine-tuning strategies, the generalization ability and detection accuracy of the model are improved, providing a solid foundation for subsequent target recognition and tracking.
[0093] S1.5, input the preprocessed infrared image, use the fused YOLO and FasterR-CNN model to perform target detection, output the bounding box and category label of each target in the image, where the bounding box describes the location and size of the target, and the category label identifies the type of target. Finally, the detection results are saved to a file or database for subsequent target recognition and tracking.
[0094] S1.5-1, input the preprocessed infrared image and use the deep learning model for target detection. The model will output the bounding box and category label of each target in the image. The bounding box describes the location and size of the target, while the category label identifies the type of target.
[0095] S1.5-2, combine the image sequence of the current frame and several frames before and after, and use the convolutional neural network (CNN) to extract the time series features. The time series features can capture the dynamic changes of the target in consecutive frames and improve the stability of detection. The extracted time series features are fused with the image features of the current frame to enhance the robustness of detection.
[0096] S1.5-3, dynamically adjust the confidence threshold of target detection according to the timing characteristics. Use an adaptive module based on a recurrent neural network (RNN) to adjust the detection threshold of the current frame according to the detection results of the previous and next frames to adapt to changes in different scenes and environments. The dynamically adjusted threshold makes target detection more flexible and adaptable.
[0097] According to the time series characteristics, the confidence threshold τ of target detection is dynamically adjusted, and the adaptive threshold is predicted using RNN: τ = RNN(Τ);
[0098] Filter the model output using the adjusted threshold: τ ={Β i conf(Β i )>τ}.
[0099] Among them, τ Represents the set of bounding boxes after confidence filtering; Β i Represents the i-th bounding box detected, containing the location and size information of the target; conf(Β i ) represents the bounding box Β i , that is, the confidence score of the model that the bounding box contains the target; τ represents the confidence threshold, and only bounding boxes with confidence scores higher than the threshold are retained.
[0100] S1.5-4, process the detected multiple bounding boxes and remove redundant and overlapping bounding boxes. Use the non-maximum suppression (NMS) algorithm to calculate the overlap of each pair of bounding boxes, and only retain the bounding box with the highest confidence, and remove the remaining bounding boxes with high overlap. Ensure that each target object is marked only once to improve the accuracy of the detection results.
[0101] Calculate bounding box Β i and Β j The intersection over union (IoU) of
[0102] Use the IoU threshold θ′ to remove redundant bounding boxes: B NMS ={Β i IoU(Β i ,Β j )<θ′}.
[0103] B NMS : The bounding box set after non-maximum suppression (NMS) processing. The bounding box set contains all bounding boxes filtered by NMS, ensuring that each target object is marked only once.
[0104] S1.5-5, mark the detected target location on the image, draw the bounding box and category label. Use visualization tools to overlay the bounding box and label on the image to clearly show the location and type of the target. The marked image is convenient for further analysis and processing.
[0105] S1.5-6, save the detection results to a file or database for subsequent target identification and tracking. The saved content includes the bounding box coordinates, category label, detection confidence, and timestamp of each target. The saved results are used for further target identification, trajectory tracking, and anomaly detection.
[0106] Through detailed image acquisition and preprocessing steps, combined with multi-model fusion and adaptive fine-tuning strategies, the accuracy and robustness of infrared image target detection are significantly improved. Images in different time periods and environmental conditions are effectively processed, false positives and false negatives are reduced, and the system's adaptability and detection capabilities in complex environments are enhanced.
[0107] In modern home security monitoring systems, infrared cameras are widely used for all-weather monitoring. Infrared technology can clearly capture heat source images at night or in low light conditions, greatly enhancing security protection capabilities. However, with the increase in the number of targets in infrared images and the complexity of scenes, the system needs to identify and analyze these targets more accurately.
[0108] In a typical home surveillance scenario, the camera may detect multiple heat sources, such as family members, pets, and even moving objects outside the window (such as thermal images reflected by tree branches or vehicles). Traditional single-frame detection methods often have difficulty distinguishing these targets, especially when they have similar thermal radiation characteristics, which can easily lead to false positives and false negatives. For example, the heat source generated by a pet at home when moving may be similar to the heat source characteristics of some intruders, and there are also family members holding pets, etc. How to effectively distinguish these targets has become an important issue in ensuring family safety.
[0109] In addition, the position and state of the target in the scene may change over time, such as family members walking around the room and pets wandering around the room. These dynamic changes increase the difficulty of target detection and recognition. In order to ensure that the system can accurately identify and track each target, in-depth feature extraction and time series analysis of the detected targets are required to provide higher detection accuracy and robustness.
[0110] The implementation of step S2 provides a refined processing method for these actual usage scenarios. Through feature extraction, time dimension analysis and dynamic threshold adjustment, it ensures that the system can accurately identify and separate multiple targets in a complex and changing environment, thereby improving the intelligence and reliability of the monitoring system.
[0111] S2 includes the following:
[0112] S2.1, use convolutional neural network (CNN) to extract features from each detected target and generate feature vectors. Calculate the cosine similarity between the standardized feature vectors, and dynamically adjust the similarity judgment threshold through the average similarity and standard deviation of the feature vector set, and finally use the dynamic threshold to determine whether the target features are similar.
[0113] S2.1-1, the feature vector of each target is standardized to eliminate the influence of different feature dimensions. By subtracting the mean of the feature vector and dividing it by its standard deviation, the standardized feature vector has the characteristics of mean 0 and standard deviation 1, so that it is more stable and comparable in subsequent calculations.
[0114] S2.1-2, calculate the cosine similarity of the two target feature vectors after normalization, and measure their similarity by calculating the angle between the two vectors. The closer the cosine similarity value is to 1, the more similar the directions of the two vectors are, and the more similar the features are.
[0115] S2.1-3, calculate the average similarity and standard deviation between feature vectors, and determine the dynamic threshold by the sum of the two. The dynamic threshold can adapt to the changes of different feature sets and improve the accuracy of judgment.
[0116] S2.1-4, use the dynamic threshold to determine whether the target features are similar. If the cosine similarity of the target feature vector is higher than the dynamic threshold, the target features are determined to be similar and marked as the target to be analyzed. Otherwise, the features are considered to be dissimilar.
[0117] The purpose of feature vector standardization is to eliminate the influence of different feature dimensions, making the feature vector more accurate when calculating similarity. Cosine similarity is used to measure the similarity of two feature vectors, reflecting the angle relationship between target features in high-dimensional space. Adaptive threshold adjustment dynamically determines the threshold for similarity judgment based on the distribution of feature vectors, so that the algorithm can adapt to changes in different feature sets. Finally, through the feature similarity judgment function, the target features are judged to be similar based on the dynamically adjusted threshold, ensuring sufficient flexibility and accuracy in feature similarity judgment.
[0118] S2.2, for the target to be analyzed, a recurrent neural network (RNN) is used to analyze the continuous infrared image sequence and extract the target's features in the time dimension. RNN is good at processing sequence data. By memorizing the information of the previous and next moments, it can capture the dynamic changes of the target in the time dimension. The input is the feature vector of the target in different time frames, and the RNN output is the feature representation of the time dimension. These time dimension features can provide information such as the target's motion trajectory and change pattern, which helps to more accurately identify and separate in subsequent steps.
[0119] S2.2-1, the feature vectors of the target to be analyzed in different time frames form a time series to facilitate subsequent RNN processing.
[0120] S2.2-2, the state update function of the RNN combines the input feature vector of the current time step and the hidden state of the previous time step to gradually extract the dynamic change information in the time series;
[0121] S2.2-3, the entire time series is processed by RNN, and the hidden state of the last time step is used as the feature representation of the time dimension, reflecting the change pattern of the target to be analyzed in the entire time series;
[0122] RNN processes continuous time series through memory and update mechanisms, extracts dynamic change information in the time dimension, and finally outputs a comprehensive time feature representation.
[0123] S2.3, dynamically adjust the target recognition threshold according to the time dimension characteristics to adapt it to different targets and scenarios. Through the threshold adjustment function, a new recognition threshold is calculated according to the time dimension characteristics. This dynamic adjustment process enables the recognition system to flexibly respond to changes in target characteristics, such as different heat source intensities and morphological changes. The threshold adjustment function can be implemented through a regression model or a deep learning model to ensure that the adjusted threshold can effectively distinguish different targets.
[0124] S2.3-1, the feature aggregation function combines the time dimension features with the current frame features to generate a comprehensive feature representation, providing richer target information.
[0125] S2.3-2, based on the comprehensive feature representation, the recognition threshold is dynamically generated through the threshold calculation function, so that the system can adapt to the changes of the target characteristics.
[0126] S2.3-3, the feature aggregation and threshold calculation function combines temporal and spatial features to dynamically adjust the recognition threshold to flexibly adapt to changes in different scenarios and target features.
[0127] S2.4, use the dynamically adjusted recognition threshold to separate the targets to be analyzed, and use the improved non-maximum suppression (NMS) algorithm to separate the heat sources to ensure that each heat source is identified separately. The improved NMS algorithm combines the time dimension features and dynamic thresholds to screen and separate the targets to be analyzed and remove redundant bounding boxes. After each independent heat source is separated, feature extraction and recognition are performed again to ensure that the specific category and attributes of each heat source can be accurately identified. The final recognition results will be marked and recorded for subsequent target tracking and further analysis.
[0128] S2.4-1, use cosine similarity to calculate the similarity between the comprehensive feature representations of each target to be analyzed, marked as the target similarity to be analyzed, and quantify their similarity.
[0129] S2.4-2, combining the similarity of the target to be analyzed and the dynamically adjusted recognition threshold to perform non-maximum suppression, retaining the target whose similarity to the target to be analyzed is less than the dynamically adjusted recognition threshold, and removing redundant bounding boxes;
[0130] S2.4-3, individually identify and re-extract features for the retained targets to be analyzed, and continue to execute steps S2.1 to S2.4 until each target to be analyzed is identified, ensuring accurate identification of the category and attributes of each heat source.
[0131] S2.5, marking the identified target as an identified target, and labeling the identified target, for example, an adult, a child, a pet cat, a pet dog, etc. This label can be manually marked by an administrator based on the identified target.
[0132] The purpose of labeling identified targets is to improve the system's ability to distinguish different targets, facilitate subsequent target tracking and behavior analysis, and enhance the system's response flexibility in specific scenarios. For example, no warning will be issued when there are adults in the scene, but a remote warning reminder will be triggered when only unlabeled targets invade, ensuring safety and user experience.
[0133] In the home security monitoring scenario, the traditional infrared image target detection system is easily disturbed by factors such as ambient light and noise, resulting in a decrease in detection accuracy, especially when multiple target features are similar, targets overlap or targets change over time, it is more likely to have false positives and false negatives. Through the detailed processing of step S2, the system can effectively solve these problems and significantly improve the accuracy and robustness of target detection and recognition. First, the detected targets are feature extracted and standardized by convolutional neural network (CNN) to ensure the consistency of different feature dimensions, and then the target feature similarity is determined by cosine similarity calculation and dynamic threshold. The dynamic threshold can be automatically adjusted according to the change of feature set, so that the system has higher adaptability and accuracy when facing different targets. Secondly, for the target to be analyzed, the continuous infrared image sequence is analyzed by recurrent neural network (RNN) to extract the features of the target in the time dimension, so as to capture the target's motion trajectory and change pattern, which is particularly important for dealing with target overlap and change over time. Through feature aggregation and dynamic threshold adjustment function, the system can flexibly adjust the recognition threshold in combination with time and space features, making target recognition more accurate and reliable. Finally, the improved non-maximum suppression (NMS) algorithm is combined to screen and separate the targets to be analyzed, ensuring that each heat source is identified separately, removing redundant bounding boxes, and further improving the accuracy of the detection results. These beneficial effects not only improve the detection accuracy and robustness of the system, but also enhance the adaptability of the system in complex monitoring environments, and can better cope with various complex situations in home security monitoring, providing more efficient and reliable security protection.
[0134] The present invention uses CNN to extract features and standardize them, making feature similarity determination more accurate; combines RNN to analyze the time dimension features of the target, captures the motion trajectory, and processes overlapping and changing targets; and ensures that the heat source is independently identified and processed by dynamically adjusting the recognition threshold and the improved non-maximum suppression algorithm. This helps to significantly improve the accuracy and robustness of infrared image target detection and recognition in home security monitoring, reduce false positives and false negatives, improve detection accuracy and system adaptability, and provide more efficient and reliable home security monitoring protection.
[0135] After the infrared camera detects and identifies the target, it needs to continuously track these targets to ensure that the dynamic position and status of each target is mastered in real time. For example, the monitoring system detects people or pets moving around the room at home. By continuously tracking these targets, it can promptly issue an alarm and take countermeasures when abnormal situations (such as illegal intrusion or emergency) occur. However, as the number of targets increases and the complexity of the scene increases, how to accurately predict the target position and effectively associate the targets between frames becomes the main challenge facing the system. To achieve this goal, the system needs to use advanced tracking and prediction algorithms to ensure the continuity and accuracy of target tracking.
[0136] S3 includes the following:
[0137] S3.1, use Kalman filter to predict the next position of the identified target, and use Hungarian algorithm to solve the association problem of the identified target between frames. The state vector of Kalman filter includes the position and velocity of the identified target, and the initial state vector and covariance matrix are set for each identified target;
[0138] The state vector of the Kalman filter includes the position and velocity of the identified targets, and the Hungarian algorithm is used to associate the detected identified targets in each frame with the identified targets in the previous frame. The initialization process sets the initial state and covariance matrix for each identified target.
[0139] S3.2, based on the motion model of the identified target, use the state transfer matrix and process noise covariance matrix to predict the next state of the target, and modify the prediction by combining the control vector and control matrix. The predicted state vector and covariance matrix provide the basis for subsequent updates.
[0140] S3.3, use the YOLO and FasterR-CNN models fused in step S1.4 to detect the identified target, obtain the detection results, use the observation matrix to map the detection results to the state space, and update the state vector and covariance matrix of the identified target through the Kalman gain combined with the observation data.
[0141] S3.4, recording the motion trajectory of each identified target, and storing the state vector of each time frame in a trajectory set. The trajectory set includes the state vector of each identified target in continuous time frames, providing motion information of the identified target.
[0142] S3.5, using the recorded trajectories and recognition results, self-learning is performed through a deep learning algorithm. A loss function is defined, the gradient is calculated based on the trajectory and recognition results, and the parameters of the identified target detection and recognition model are optimized using a gradient descent algorithm. The model parameters are updated based on the learning rate and the gradient of the loss function.
[0143] Assume the loss function is L, and the parameter update formula is: θ″=θ″-η▽θ ″ L;
[0144] Among them, θ″ is the model parameter, η is the learning rate, and L is the loss function defined according to the trajectory and recognition results.
[0145] The self-learning mechanism enables the system to improve itself based on the actual target tracking and recognition results. By recording the movement trajectory and recognition results of each target, the system can continuously obtain feedback, update and optimize the parameters of the deep learning model. Self-learning not only improves the accuracy of target detection and recognition, but also enables the system to cope with environmental changes and the diversity of target features. In long-term operation, the system can automatically adapt to new target types and changing environmental conditions, maintaining a high level of detection and tracking performance. In addition, the self-learning mechanism can reduce the need for manual intervention, improve the system's automation and intelligence level, and make the home security monitoring system more reliable and efficient.
[0146] The present invention significantly improves the accuracy and continuity of target tracking and reduces the loss of targets through target tracking and position prediction. The combination of Kalman filter and Hungarian algorithm ensures the effective association of targets between consecutive frames and improves the stability of tracking. At the same time, the introduction of self-learning method enables the system to continuously obtain feedback from the tracking and recognition process, and enhances the adaptability and robustness of the system in a dynamic environment by optimizing model parameters. Self-learning not only improves the accuracy of target detection and recognition, but also enables the system to have the ability of self-improvement, can continuously adapt to environmental changes and the diversity of target characteristics, and ensure long-term reliability and performance stability.
[0147] In a home security monitoring system, when an infrared camera detects multiple targets, some targets may not be labeled in time. The "adult" label in the identified targets can monitor these unlabeled targets to ensure safety. However, if there is no "adult" label in all identified targets, the system needs to detect and issue an early warning to prevent potential security threats. By executing step S4, the system can intelligently distinguish and process unlabeled targets to ensure the safety of the home environment.
[0148] S4 includes the following:
[0149] S4.1, check whether each identified target existing in the scene at the latest time point has a label. Define a label state vector, where a value of 1 indicates that the target has a label, and a value of 0 indicates that the target has no label.
[0150] S4.2, for the identified targets without labels, check whether the labels of all identified targets in the scene contain a specific category, such as "adult". Use the label matching function to determine whether each label belongs to the category, and check the label state vector to determine whether there is an object with the label "adult".
[0151] S4.3, if there is an unlabeled target in the scene and there is no target of a specific category among all identified targets, a remote early warning alert is generated, indicating that there is an unlabeled target intrusion.
[0152] In the home security monitoring system, when the infrared camera detects multiple targets, the system defines a label state vector (a value of 1 indicates that the target is labeled, and a value of 0 indicates that the target is unlabeled) by checking whether each identified target present in the scene at the latest time point is labeled. For identified targets that are not labeled, the system checks whether the labels of all identified targets in the scene contain a specific category (such as "adult"), uses a label matching function to determine whether each label belongs to the category, and checks the label state vector to determine whether there is a target with the "adult" label. If there are unlabeled targets in the scene and there are no targets of a specific category among all identified targets, the system generates a remote early warning reminder, indicating that there is an unlabeled target intrusion. In this way, the system significantly improves its intelligence and responsiveness, timely detects and warns unlabeled potential intrusion targets without supervision of identified targets, reduces false alarms, enhances the reliability and effectiveness of home security monitoring, and ensures the safety of the home environment.
[0153] By detecting unlabeled targets and specific category labels in the scene, the system's intelligence and responsiveness are improved. The system can timely detect and warn unlabeled potential intrusion targets without the supervision of identified targets, reduce false alarms, and enhance the reliability and effectiveness of home security monitoring.
[0154] Example 2
[0155] Figure 2 The present invention provides an infrared image target detection system based on deep learning, including: an infrared image acquisition module, a target detection module, a target recognition module, a target tracking module, a self-learning optimization module and a label analysis module;
[0156] The infrared image acquisition module uses an infrared camera to collect infrared images in the environment, performs preliminary preprocessing on the images to ensure image quality, and passes the preprocessed infrared images to the target detection module;
[0157] The target detection module uses the target detection algorithm to detect the targets in the preprocessed infrared image, locate and mark the positions of each target, and pass the marked target information to the target recognition module;
[0158] The target recognition module recognizes the targets transmitted by the target detection module. When it is determined that the features of different targets are similar, it uses a recurrent neural network to analyze the continuous infrared image sequence, dynamically adjusts the threshold through the information of the time dimension, separates different heat sources, and identifies the separated heat sources separately. It attaches labels to all identified targets and transmits the recognition and label information to the target tracking module.
[0159] The target tracking module continuously tracks the identified target transmitted by the target recognition module, records the target's motion trajectory, and transmits the tracked target's motion trajectory and recognition results to the label analysis module and the self-learning optimization module;
[0160] The self-learning optimization module uses the trajectory records and recognition results transmitted by the target tracking module to perform self-learning, optimize the parameters of the target detection and recognition algorithms through the self-learning process, and transmit the latest parameter adjustments back to the target detection module and the target recognition module;
[0161] The label analysis module analyzes whether the identified targets in the scene are labeled at the latest time point. If there are targets without labels and the labels of all identified targets in the scene do not fall within the specified conditions, an early warning reminder is issued.
[0162] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.
[0163] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.
[0164] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0165] In the several embodiments provided in the present application, it should be understood that the disclosed systems and devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0166] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0167] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0168] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0169] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0170] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. Infrared image target detection method based on deep learning, characterized by: S1, using an infrared camera to collect infrared images in the environment, using a target detection algorithm to detect targets in the preprocessed images, and locating and marking the positions of each target; S2, identifies the detected targets. When it is determined that the features of different targets are similar, a recurrent neural network is used to analyze the continuous infrared image sequence, dynamically adjust the threshold through the information of the time dimension, separate the different heat sources, and identify the separated heat sources separately, and attach labels to all identified targets; S3, continuously tracking the identified target, recording the target's motion trajectory, using the recorded target's motion trajectory and recognition results as input data for self-learning, and optimizing the parameters of the target detection and recognition algorithm; S4, analyzing whether the identified targets in the scene are labeled at the latest time point. If there are targets without labels and the labels of all identified targets in the scene do not fall within the specified conditions, an early warning reminder is issued; S2 includes the following: S2.1, use convolutional neural network to extract features of each detected target and generate feature vector: S2.1-1, standardize the feature vector of each target; S2.1-2, calculate the cosine similarity of the two target feature vectors after normalization; S2.1-3, calculate the average similarity and standard deviation between feature vectors, and determine the dynamic threshold by the sum of the two; S2.1-4, using a dynamic threshold to determine whether the target features are similar. If the cosine similarity of the target feature vector is higher than the dynamic threshold, the target features are determined to be similar and marked as the target to be analyzed. Otherwise, the features are considered to be dissimilar. S2.2, for the target to be analyzed, a recurrent neural network is used to analyze the continuous infrared image sequence and extract the characteristics of the target in the time dimension: S2.2-1, the feature vectors of the target to be analyzed in different time frames form a time series; S2.2-2, the state update function of the recurrent neural network combines the input feature vector of the current time step and the hidden state of the previous time step to gradually extract the dynamic change information in the time series; S2.2-3, the entire time series is processed by a recurrent neural network, and the hidden state of the last time step is used as the feature representation of the time dimension, reflecting the change pattern of the target to be analyzed in the entire time series; S2.3, dynamically adjust the target recognition threshold according to the time dimension characteristics to adapt it to different targets and scenarios. Through the threshold adjustment function, a new recognition threshold is calculated according to the time dimension characteristics: S2.3-1, the feature aggregation function combines the time dimension features with the current frame features to generate a comprehensive feature representation; S2.3-2, dynamically generating a recognition threshold through a threshold calculation function based on the comprehensive feature representation; S2.3-3, combining temporal and spatial features, obtaining a dynamically adjusted recognition threshold through feature aggregation and threshold calculation function; S2.4, using dynamically adjusted recognition thresholds to separate the targets to be analyzed, using non-maximum suppression algorithms to separate heat sources, ensuring that each heat source is identified individually: S2.4-1, use cosine similarity to calculate the similarity between the comprehensive feature representations of each target to be analyzed, and record it as the similarity of the target to be analyzed; S2.4-2, combining the similarity of the target to be analyzed and the dynamically adjusted recognition threshold to perform non-maximum suppression, retaining targets whose similarity is less than the dynamically adjusted recognition threshold, and removing redundant bounding boxes; S2.4-3, individually identifying and re-extracting features of the retained targets to be analyzed, and repeating steps S2.1 to S2.4 until each target to be analyzed is identified; S2.5, marking the identified target as an identified target, and labeling the identified target.
2. The infrared image target detection method based on deep learning according to claim 1, characterized in that: S1 includes the following: S1.1, calibrate the infrared camera using a standard blackbody radiation source; S1.2, collect infrared images at different time periods and angles to capture target objects under various environmental conditions; S1.3, applying median filtering and Gaussian filtering to remove image noise, and using histogram equalization to enhance image contrast; S1.4, using YOLO and FasterR-CNN as the base models, fine-tune them respectively, and fuse their prediction results: S1.4-1, the fusion process combines the bounding box sets and category sets of the two models through the weighted average method; S1.4-2, using an adaptive module based on a recurrent neural network, with a continuous infrared image sequence as input and an adjustment coefficient of the model parameters as output, to dynamically adjust the model; S1.4-3, during the training process, multi-scale transformation is performed on each original image to generate a multi-scale image, and the image size is adjusted by scaling operation; S1.4-4, use the generative adversarial network to expand the training data set and generate additional infrared image data. The generative adversarial network includes a generator and a discriminator. The generator inputs a noise vector and outputs a generated image.
3. The infrared image target detection method based on deep learning according to claim 2 is characterized in that: S1.5, input the preprocessed infrared image, use the fused YOLO and FasterR-CNN model for target detection, and output the bounding box and category label of each target in the image, where the bounding box describes the location and size of the target, and the category label identifies the type of target: S1.5-1, combining the image sequence of the current frame and several frames before and after, using a convolutional neural network to extract temporal features, and fusing the temporal features with the image features of the current frame; S1.5-2, using an adaptive module based on a recurrent neural network to adjust the detection threshold of the current frame according to the detection results of the previous and next frames; S1.5-3, process the detected multiple bounding boxes and use the non-maximum suppression algorithm to remove redundant and overlapping bounding boxes to ensure that each target object is marked only once; S1.5-4, mark the detected target location on the image, draw the bounding box and category label, and overlay the bounding box and label on the image; S1.5-5, save the detection results to a file or database, including the bounding box coordinates, category label, detection confidence and timestamp of each target.
4. The infrared image target detection method based on deep learning according to claim 1, characterized in that: S3 includes the following: S3.1, set the initial state vector and covariance matrix for each identified target, use the Kalman filter to predict the next position of the identified target, and use the Hungarian algorithm to associate the identified targets detected in the current frame with the identified targets in the previous frame; S3.2, according to the motion model of the identified target, the next state of the target is predicted using the state transfer matrix and the process noise covariance matrix, and the predicted state vector and covariance matrix are corrected in combination with the control vector and the control matrix; S3.3, use the YOLO and FasterR-CNN models fused in step S1.4 to detect the identified target, obtain the detection result, use the observation matrix to map the detection result to the state space, and update the state vector and covariance matrix of the identified target by combining the observation data through the Kalman gain; S3.4, record the motion trajectory of each identified target and store the state vector of each time frame in a trajectory set; S3.5, using the recorded trajectory and recognition results, calculates the gradient according to the trajectory and recognition results, and uses the gradient descent algorithm to optimize the parameters of the identified target detection and recognition model for self-learning.
5. The infrared image target detection method based on deep learning according to claim 4 is characterized in that: S4 includes the following: S4.1, check whether each identified object present in the scene at the latest time point has a label attached; S4.2, for the identified targets without labels, check whether the labels of all identified targets in the scene contain a specific category; S4.3, if there are unlabeled targets in the scene and there are no targets of a specific category among all identified targets, a remote warning alert is generated.
6. An infrared image target detection system based on deep learning, used to implement the infrared image target detection method according to any one of claims 1 to 5, comprising: Infrared image acquisition module, target detection module, target recognition module, target tracking module, self-learning optimization module and label analysis module; The infrared image acquisition module uses an infrared camera to collect infrared images in the environment, performs preliminary preprocessing on the images to ensure image quality, and passes the preprocessed infrared images to the target detection module; The target detection module uses the target detection algorithm to detect the targets in the preprocessed infrared image, locate and mark the positions of each target, and pass the marked target information to the target recognition module; The target recognition module recognizes the targets transmitted by the target detection module. When it is determined that the features of different targets are similar, it uses a recurrent neural network to analyze the continuous infrared image sequence, dynamically adjusts the threshold through the information of the time dimension, separates different heat sources, and identifies the separated heat sources separately. It attaches labels to all identified targets and transmits the recognition and label information to the target tracking module. The target tracking module continuously tracks the identified target transmitted by the target recognition module, records the target's motion trajectory, and transmits the tracked target's motion trajectory and recognition results to the label analysis module and the self-learning optimization module; The self-learning optimization module uses the trajectory records and recognition results transmitted by the target tracking module to perform self-learning, optimize the parameters of the target detection and recognition algorithms through the self-learning process, and transmit the latest parameter adjustments back to the target detection module and the target recognition module; The label analysis module analyzes whether the identified targets in the scene are labeled at the latest time point. If there are targets without labels and the labels of all identified targets in the scene do not fall within the specified conditions, an early warning reminder is issued.
