Target intrusion identification method and system based on infrared thermal imaging

Abstract

The invention discloses a target intrusion identification method and system based on infrared thermal imaging, and aims to solve the problems of poor robustness and missed detection and false alarm in a complex environment in a traditional target identification method based on RGB images. According to the method, an RGB image and an infrared image are collected at the same time through an infrared thermal imaging camera, an RGB change detection model and a target detection model are used for carrying out change detection and target recognition on the RGB image, and if a target is not detected or the change amplitude does not reach a threshold value, an infrared change detection model is used for detecting the infrared image subjected to normalization processing. The system comprises an image acquisition device, a data processing device, a data storage device and a communication transmission device, can reliably recognize an abnormal intrusion target under the conditions of low illumination, severe weather and the like, and improves the applicability and robustness of the monitoring system.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to a target intrusion recognition method and system based on infrared thermal imaging. Background Art

[0002] At key locations such as railway transport intersections and substations, high-definition cameras are usually installed for real-time monitoring to promptly detect abnormal intrusions, such as people, animals, and vehicles. Currently, intelligent recognition methods based on machine vision are gradually being applied in this scenario. There are two main types of traditional target detection methods:

[0003] The first type of method is the foreground target extraction technology based on traditional image processing. For example, by differentiating the real-time monitoring image from the preset background, the moving target can be identified. This type of method is simple and direct, with low computational cost, but its robustness is poor and it is easily affected by changes in the background environment, such as light, shadow, etc., which can lead to false detection and missed detection, affecting the accuracy of recognition.

[0004] The second type of method is target detection technology based on deep learning. For example, target detection algorithms such as YOLO are used to directly identify targets. These algorithms are trained based on high-definition RGB images and can achieve high recognition accuracy when there are sufficient samples. However, due to the wide variety of targets and their inexhaustibility, targets that are not included in the training data may not be recognized. In addition, RGB images are greatly affected by imaging conditions, especially at night or in bad weather (such as rainy and foggy weather), the quality of RGB images is reduced, resulting in reduced recognition performance, and prone to missed reports and false alarms.

[0005] In order to improve the robustness and adaptability of abnormal target recognition, it is necessary to design a target intrusion recognition method that can adapt to complex environments and improve recognition results. Infrared thermal imaging technology can clearly capture the thermal radiation characteristics of the target under low light conditions, which complements the RGB image, thereby achieving more stable and reliable target detection. Therefore, the target intrusion recognition method based on the combination of infrared thermal imaging and RGB images has gradually become an effective way to solve the above problems. Summary of the invention

[0006] The present invention relates to a target intrusion recognition method and system based on infrared thermal imaging, which is particularly suitable for abnormal target monitoring in important places such as railway transportation intersections and substations. Existing monitoring technologies mainly rely on high-definition cameras and RGB image processing, but under conditions such as nighttime and rainy and foggy weather, these technologies are easily affected by imaging quality, resulting in reduced recognition accuracy. At the same time, target recognition methods based on traditional image processing have poor robustness and are easily affected by background disturbances, such as light changes and shadows, resulting in a high false detection rate. In addition, target detection methods based on deep learning perform well when there are sufficient training samples, but it is difficult to handle abnormal targets that are not within the training range.

[0007] In view of the above problems, the present invention proposes a method for identifying target intrusions by combining infrared thermal imaging technology with RGB images. By simultaneously acquiring high-definition RGB images and infrared images, an RGB change detection model, a target detection model, and an infrared change detection model are constructed to monitor the target in all directions. This method uses the temperature information of infrared images to make up for the shortcomings of RGB images in low-light environments, thereby enhancing the robustness and applicability of the recognition system.

[0008] The core of the present invention is:

[0009] The convolutional neural network (CNN) is used to perform change detection on RGB images and infrared images respectively, and the changes of targets in the images are detected by feature map difference.

[0010] Target recognition is performed based on RGB images. If the RGB change detection model detects an anomaly and the target detection model confirms the existence of the target, an alarm message is output and the relevant image is saved;

[0011] When no abnormality is detected in the RGB image, the infrared image is detected through the infrared change detection model. If there is an abnormality in the infrared image, an alarm message is generated and the abnormal image is saved.

[0012] The method and system have the following advantages:

[0013] Combining infrared thermal imaging with RGB images enables the system to maintain a high target recognition accuracy in low light or complex weather conditions;

[0014] The deep learning model improves the robustness of change detection and reduces false alarms caused by environmental interference;

[0015] The combination of change detection and target detection effectively improves the applicability of the system, especially for non-exhaustive target categories, with strong generalization ability;

[0016] The system has the ability to continuously learn and can continuously update the training set and improve model performance by manually labeling unidentified targets.

[0017] The present invention can be widely used in real-time monitoring and abnormal target detection of important places, and is particularly suitable for scenes with complex environments and large changes in light, such as railway crossings, substations, airports and other public safety places. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A schematic diagram of the work flow of the target intrusion identification method of the present invention;

[0020] Figure 2 This is a schematic diagram of an example of a high-definition RGB image according to an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of an example of an infrared image according to an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of an example of a binary change graph according to an embodiment of the present invention;

[0023] Figure 5 The figure is a schematic diagram of an example of target detection result according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0025] It should be noted that the references to "one embodiment", "an embodiment", "an exemplary embodiment", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0026] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0027] Example 1

[0028] The present invention proposes a target intrusion recognition method, device and system based on infrared thermal imaging. Figure 1 As shown, the invention uses an infrared thermal imaging camera to provide high-definition RGB images and infrared images at the same time. First, RGB images are collected to build a change detection model and a target detection model, which are used to perform change detection and target detection on the real-time scene respectively; at the same time, infrared images are collected to train a new change detection model for change detection on infrared images. During on-site reasoning and analysis, the image is firstly detected for change through the RGB change detection model. If a change is detected and the change amplitude exceeds the set threshold, the target detection continues, that is, the specific intrusion target category and position are identified through the target detection model. If the target detection model identifies the target, an alarm record is formed and the intrusion target is output, otherwise an alarm record is formed and the abnormal image is saved. If the RGB change detection model does not detect the change or the change amplitude does not reach the set threshold, the infrared change detection model is used to perform change detection on the normalized infrared image. If a change is detected and the change amplitude exceeds the set threshold, an alarm record is formed and the abnormal image is saved.

[0029] 1. Image acquisition

[0030] The present invention uses an infrared thermal imaging camera to collect images. The camera provides real-time high-definition RGB images and infrared images, such as Figure 2 and Figure 3 As shown in the figure, the RGB change detection model and target detection model are trained based on high-definition RGB images, and the infrared change detection model is trained based on infrared images.

[0031] Before modeling the infrared image, the infrared image needs to be normalized. Since the infrared image returns the temperature value of each pixel, the image is normalized considering the temperature difference between the background and different targets, and the following transformation operations are performed:

[0032]

[0033] 2. Change Detection Model

[0034] In the present invention, change detection models are constructed for RGB images and infrared images respectively. The traditional image difference detection method has poor robustness and is easily affected by on-site disturbances such as shadows and light changes, which produces many pseudo changes and causes many false alarms. Therefore, in the present invention, a convolutional neural network is used to learn the deep features of the image, and change detection is performed by feature map difference. Preferably, the present invention is implemented through FDCNN: first, the algorithm performs deep feature learning through VGG-16; on this basis, a feature difference CNN is constructed to produce a change amplitude map; finally, a binarization operation is performed through a threshold to generate a binary change map, such as Figure 4 The area of ​​each closed region in the binary change map is counted. If it exceeds the set threshold, the region is considered to be a change target, otherwise it is discarded.

[0035] It should be noted that no matter which deep learning-based change detection algorithm is used to implement change detection within the region, it does not affect the essence of this solution.

[0036] 3. Object Detection Model

[0037] In the present invention, a target detection model is constructed for RGB images. First, images of known targets appearing in the region are collected and annotated to construct a training set. Then a target detection model is selected for modeling. Currently, commonly used target detection algorithms include R-CNN, FastR-CNN, FasterR-CNN, SSD, YOLO, RetinaNet, etc. Preferably, the present invention uses YOLOv7 as a specific implementation method for target detection. Figure 5 Schematic diagram of target detection.

[0038] 4. Reasoning and recognition process

[0039] In the present invention, change detection is first performed on the RGB image to determine whether a changed target appears. If the changed target reaches the set threshold, it is considered that there is an intruder in the picture, and then the target detection model is used to identify and determine which type of target it belongs to. If the target detection model cannot detect it, the image is saved and collected, and added to the training set after manual annotation for further learning. If no change is detected in the RGB image, change detection based on the infrared image is performed. If the infrared change detection model recognizes that there is a change, the image is saved and collected, and then manually annotated and added to the training set for further learning, otherwise it is considered that there is no intrusion.

[0040] 5. The equipment

[0041] A target intrusion recognition device based on infrared thermal imaging includes real-time video acquisition equipment, communication transmission equipment, data processing equipment, data storage equipment, etc.

[0042] The image acquisition device is an infrared thermal imaging camera that can provide at least 1080P high-definition RGB images and infrared thermal imaging.

[0043] Communication transmission equipment includes equipment required for wireless transmission methods such as 5G and industrial wifi, as well as equipment required for wired transmission methods such as USB3.0 and Ethernet.

[0044] The edge processing device uses an AI edge computing server, which has a certain computing power and memory to complete the deep learning model described in the present invention.

[0045] The data storage device is an SD card or a hard disk with a large storage capacity, which is used to store images identified as intrusions.

[0046] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0047] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A target intrusion recognition method based on infrared thermal imaging, characterized in that: The following steps are involved: (1) Using an infrared thermal imaging camera to simultaneously capture RGB images and infrared images; (2) Build an RGB change detection model and a target detection model based on RGB images, and build an infrared change detection model based on infrared images, where: The RGB change detection model is used to detect whether there is a change target in the RGB image; The target detection model is used to identify the target category and position in the RGB image; The infrared change detection model is used to detect changes in infrared images; (3) The RGB change detection model is used to detect changes in the collected RGB image. If a change is detected and the magnitude of the change exceeds a preset threshold, the target is identified through the target detection model: If the target detection model identifies the target, an alarm record is generated and the intrusion target information is output; If the target detection model fails to identify the target, an alarm record is generated and the abnormal RGB image is saved; (4) If the RGB change detection model does not detect a change, or the magnitude of the change does not reach the preset threshold, the infrared image is normalized and detected by the infrared change detection model. If a change is detected and the magnitude of the change exceeds the set threshold, an alarm record is generated and the abnormal infrared image is saved.

2. The target intrusion identification method and system based on infrared thermal imaging according to claim 1 is characterized in that: The RGB change detection model and the infrared change detection model perform deep feature learning through a convolutional neural network (CNN) and detect changes through feature map differences.

3. The target intrusion identification method and system based on infrared thermal imaging according to claim 1 is characterized in that: The convolutional neural network performs deep feature extraction based on the VGG-16 model, generates a change amplitude map, and generates a binary change map by setting a threshold.

4. A method and system for identifying target intrusion based on infrared thermal imaging according to any one of claims 1 to 3, characterized in that: The target detection model performs target detection based on the YOLOv7 algorithm.

5. A method and system for identifying target intrusion based on infrared thermal imaging according to any one of claims 1 to 4, characterized in that: The normalization process includes normalizing the temperature value in the infrared image to reduce the influence of the temperature difference between the background and the target on the detection result.

6. A target intrusion recognition system based on infrared thermal imaging, the system comprising: (1) Image acquisition equipment, used to acquire RGB images and infrared images; (2) a data processing device for performing change detection and target recognition on an image based on an RGB change detection model, a target detection model, and an infrared change detection model; (3) a data storage device for storing the identified intrusion target information and abnormal images; (4) Communication transmission equipment, used to transmit the collected images and detection results to the data processing equipment.

7. A target intrusion recognition system based on infrared thermal imaging as claimed in claim 6, wherein: The image acquisition device is an infrared thermal imaging camera that can at least provide 1080P high-definition images.

8. The target intrusion recognition system based on infrared thermal imaging according to claim 6 is characterized in that: The data processing device is an edge computing server capable of performing deep learning model reasoning.

9. The target intrusion identification method and system based on infrared thermal imaging according to claim 6, characterized in that: The communication transmission device includes a wireless communication device or a wired communication device.

10. The target intrusion identification method and system based on infrared thermal imaging according to claim 6, characterized in that: The data storage device is an SD card or a hard disk, which is used to store image data.

Images