A method and system for monitoring abnormal security scenarios using infrared thermal imaging

Abstract

This invention relates to the field of image recognition technology, specifically to a method and system for monitoring abnormal security scenarios using infrared thermal imaging. The method includes: acquiring images of the monitoring area using an infrared thermal imager to establish a video frame sequence; constructing a Gaussian mixture model for each pixel; performing spatial analysis on the video frame sequence to establish foreground patches, and determining the artificial environmental spatial interference of each pixel in the current frame through the foreground patches; performing temporal analysis on the video frame sequence in conjunction with the artificial environmental spatial interference to obtain the artificial environmental spatiotemporal interference of each pixel in the current frame; using the Gaussian mixture model to evaluate the artificial environmental spatiotemporal interference, determining and marking whether each pixel in the current frame is a candidate state; setting a preset decision threshold, counting the number of consecutive frames with candidate state markings, and comparing the number of consecutive frames with the decision threshold to distinguish between normal and abnormal environmental changes. This effectively distinguishes between benign environmental changes and real threats, preventing the generation of a large number of invalid alarms.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to an infrared thermal imaging method and system for monitoring abnormal security scenarios. Background Technology

[0002] To achieve proactive security early warning and effectively protect the safety of personnel, property, and areas, it is of great significance to conduct abnormal security scenario monitoring. Among various monitoring technologies, infrared thermal imaging technology is particularly suitable for such tasks due to its unique advantages. This technology has powerful all-weather monitoring capabilities, does not rely on visible light, and can clearly image in harsh environments such as darkness, rain, snow, and fog, enabling true 24-hour uninterrupted monitoring. Moreover, infrared thermal imaging is a passive receiving technology, which does not emit any light during operation, making it highly concealed and difficult for monitored targets to detect, thus providing excellent confidentiality.

[0003] In dynamic background environments, the core objective of abnormal security scene monitoring based on infrared thermal imaging technology is to accurately separate foreground targets from the background. Gaussian mixture models are typically used for background modeling, constructing a probability model composed of multiple Gaussian distributions for each pixel in the video sequence. Multimodal modeling effectively describes the various brightness states that pixels may exhibit at different times. Through continuous learning, the Gaussian mixture model can adapt to slow dynamic changes in the background, such as slow changes in lighting. When processing new video frames, the pixel value is compared with all Gaussian distributions in the background model to check the matching degree. If the pixel value matches the background model, it is determined to be a background pixel; conversely, if it does not match any background distribution, it is considered a new, rare pattern and is determined to be a foreground target.

[0004] In existing technologies, when non-threatening but persistent changes to the environment occur, such as newly parked vehicles, newly piled materials, or temporary construction barriers, these objects are initially identified as new foreground objects by the background model. Their thermal radiation characteristics differ significantly from the original background, leading to false alarms. Furthermore, the static nature of these new objects aligns with the static anomaly detection rules after motion monitoring, continuously triggering system alarms. This false alarm phase persists throughout the background model's learning cycle until the thermal signal characteristics of the new object are accepted as part of the background. However, before this absorption process is complete, existing conventional monitoring algorithms cannot effectively distinguish between non-threatening, benign environmental changes and threatening, real-world security scenarios, resulting in a large number of invalid alarms. Summary of the Invention

[0005] To address the technical problem that existing monitoring algorithms cannot effectively distinguish between non-threatening benign environmental changes and threatening real-world security scenarios, leading to invalid alarms, the present invention aims to provide an infrared thermal imaging method for monitoring abnormal security scenarios. The specific technical solution adopted is as follows:

[0006] An infrared thermal imager was used to capture images of the monitored area to establish a video frame sequence, and a Gaussian mixture model was constructed for each pixel.

[0007] Spatial analysis is performed on the video frame sequence to establish foreground patches, and the spatial interference of the artificial environment at each pixel in the current frame is determined through the foreground patches.

[0008] By combining the spatial interference of the artificial environment with the temporal analysis of the video frame sequence, the spatiotemporal interference of the artificial environment at each pixel in the current frame can be obtained.

[0009] Gaussian mixture model is used to evaluate the spatiotemporal interference of artificial environment, and to determine and mark whether each pixel in the current frame is a candidate state.

[0010] A preset decision threshold is set, the number of consecutive frames with candidate state markers is counted, and the number of consecutive frames is compared with the decision threshold to distinguish between normal and abnormal environmental changes.

[0011] Preferably, an infrared thermal imager is used to acquire images of the monitoring area to establish a video frame sequence, and a Gaussian mixture model is constructed for each pixel, including:

[0012] Infrared thermal imagers are deployed in the monitoring area to collect radiation information and generate infrared thermal images, which are then integrated to form a video frame sequence of the monitoring area.

[0013] For each pixel, a mixture model containing multiple Gaussian distributions is constructed to create a Gaussian mixture model.

[0014] Preferably, spatial analysis is performed on the video frame sequence to establish foreground patches, and the spatial interference of the artificial environment at each pixel in the current frame is determined through the foreground patches, including:

[0015] Foreground pixels are selected from each frame of the video frame sequence and clustered to form foreground patches;

[0016] Construct the initial spatial disturbance of the artificial environment for each pixel in the current frame based on the foreground patches;

[0017] Based on the statistics and analysis of foreground patches and edge pixels, and combined with the preliminary spatial interference of the artificial environment, the further spatial interference of the artificial environment of each pixel in the current frame is constructed.

[0018] The thermal radiation uniformity of the foreground patch is obtained, and the spatial interference of the artificial environment is further determined by combining the spatial interference of the artificial environment.

[0019] Preferably, the method involves statistically analyzing edge pixels based on foreground patches, including:

[0020] Starting from any edge pixel in the foreground patch, construct a closed edge by moving along the edge pixels, and convert the closed edge into an edge chain code by numbering.

[0021] The curvature corresponding to each number is determined based on the edge chain code, and the number of edge pixels in the foreground patch is counted.

[0022] The edge regularity of each pixel in the current frame is constructed by combining the curvature and the number of edge pixels.

[0023] Preferably, obtaining the thermal radiation uniformity of the foreground patch specifically involves:

[0024] The surface temperature of each pixel in the foreground patch is obtained from the infrared thermal imager, and the standard deviation of the surface temperature is obtained based on the surface temperatures of all pixels in the corresponding foreground patch.

[0025] Preferably, temporal analysis of the video frame sequence is performed in conjunction with the spatial interference of the artificial environment to obtain the spatiotemporal interference of the artificial environment at each pixel in the current frame, including:

[0026] Based on the current frame, several historical frames are extracted from the video frame sequence. Combining the current frame and historical frames, the mutability of the position of each pixel in the current frame is analyzed through artificial environmental spatial interference.

[0027] Foreground patch matching is performed on the current frame and historical frames, and the spatiotemporal interference of the artificial environment for each pixel in the current frame is constructed by combining the variability of the position.

[0028] Preferably, foreground patch matching is performed on the current frame and historical frames, specifically as follows:

[0029] The similarity of the foreground patches corresponding to any pixel location in the current frame and historical frames is evaluated, and the foreground patch matching results between the current frame and historical frames are obtained by combining the spatial interference of the artificial environment.

[0030] Preferably, a Gaussian mixture model is used to evaluate the spatiotemporal interference of the artificial environment, and to determine and mark whether each pixel in the current frame is a candidate state, specifically:

[0031] A preset evaluation threshold is used to compare the spatiotemporal interference of the artificial environment with the evaluation threshold. When the spatiotemporal interference of the artificial environment is greater than or equal to the evaluation threshold, the pixel corresponding to the spatiotemporal interference of the artificial environment is marked as a candidate state.

[0032] Preferably, a preset decision threshold is used to count the number of consecutive frames with candidate state markers, and the number of consecutive frames is compared with the decision threshold to distinguish between normal and abnormal environmental changes, including:

[0033] Acquire new video frames within the monitoring area, and use a Gaussian mixture model to verify whether the brightness values ​​of the new video frames match those of the pixels with candidate state markers in the current frame. If they do not match, start counting again; if they match, add them to the frame count, and continue until they do not match, and then count the number of consecutive frames.

[0034] By comparing the number of consecutive frames with the decision threshold, if the number of consecutive frames exceeds the decision threshold, it indicates a normal environmental change; if the number of consecutive frames does not exceed the decision threshold, it indicates an abnormal environmental change.

[0035] To address the aforementioned problems, the present invention also provides: an infrared thermal imaging-based abnormal security scenario monitoring system, the system comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus, and the processor calls logical instructions in the memory to execute any of the aforementioned infrared thermal imaging-based abnormal security scenario monitoring methods.

[0036] The present invention has the following beneficial effects:

[0037] 1. A Gaussian mixture model is constructed for each pixel in the video frame sequence formed by the infrared thermal imager capturing the monitoring area, i.e., dynamic background modeling. Then, spatial analysis is performed on the foreground patches established in the current frame of the video frame sequence to evaluate the changes of each pixel and construct the spatial interference of the artificial environment. Next, temporal analysis is performed to determine the spatiotemporal interference of the artificial environment by combining the spatial interference of the artificial environment. The Gaussian mixture model is used for evaluation to determine whether the pixel is marked as a candidate state. The candidate state is treated as a temporary state, and the number of frames for the temporary state is counted and compared with the decision threshold to analyze whether it is a short-term or long-term environmental interference, distinguishing between benign environmental changes and real abnormal security scenarios. This enables the evaluation of environmental changes during the operation of the Gaussian mixture model, preventing a large number of invalid alarms in the monitoring area, avoiding the continuous periodicity of false alarms, improving the accuracy of security monitoring, and thus providing a more accurate and reliable abnormal target identification capability for security work.

[0038] 2. The infrared thermal imaging abnormal security scene monitoring system provided by this invention has the same beneficial effects as the infrared thermal imaging abnormal security scene monitoring method provided by this invention, and will not be described in detail here. Attached Figure Description

[0039] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating the steps of an abnormal security scenario monitoring method using infrared thermal imaging, as provided in one embodiment of the present invention. Detailed Implementation

[0041] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an infrared thermal imaging-based abnormal security scenario monitoring method and system proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0043] The following description, in conjunction with the accompanying drawings, details the specific scheme of an infrared thermal imaging method and system for monitoring abnormal security scenarios provided by this invention.

[0044] In existing technologies, during dynamic background modeling, changes in the artificial environment become new objects in the background model, continuously triggering alarms until they are absorbed into the background model. During this stage, abnormal security scene monitoring is easily interfered with, making it impossible to distinguish between benign and malicious environmental changes. Therefore, an infrared imager is used to collect video frame sequences of the monitoring area, analyze the changes in each pixel to analyze the interference in the artificial environment, assess the spatiotemporal interference of the artificial environment, and mark the candidate states of the corresponding pixels. By monitoring whether the number of consecutive frames of the candidate states exceeds the decision threshold, normal or abnormal environmental changes can be distinguished. An infrared thermal imaging abnormal security scene monitoring system is proposed. When operating, it requires an infrared thermal imaging abnormal security scene monitoring method. Therefore, whether the system and program data are integrated or different hardware is configured to produce functions with similar effects to those achieved by this invention, they all fall within the protection scope of this invention.

[0045] Please see Figure 1 The diagram illustrates a flowchart of the steps in an infrared thermal imaging method for monitoring abnormal security scenarios according to a first embodiment of the present invention. The method includes:

[0046] Step S1: Use an infrared thermal imager to collect images of the monitoring area to establish a video frame sequence, and construct a Gaussian mixture model for each pixel;

[0047] Step S2: Perform spatial analysis on the video frame sequence, establish foreground patches, and determine the spatial interference of the artificial environment for each pixel in the current frame through the foreground patches;

[0048] Step S3: Perform temporal analysis on the video frame sequence based on the spatial interference of the artificial environment to obtain the spatiotemporal interference of the artificial environment for each pixel in the current frame;

[0049] Step S4: Use a Gaussian mixture model to evaluate the spatiotemporal interference of the artificial environment, and determine and mark whether each pixel in the current frame is a candidate state;

[0050] Step S5: Set a decision threshold, count the number of consecutive frames with candidate state markers, compare the number of consecutive frames with the decision threshold, and distinguish between normal and abnormal environmental changes.

[0051] To better illustrate this, identifying and assessing potential security risks and abnormal behaviors in the monitored area, and adaptively triggering security alarms, can effectively improve the initiative and accuracy of security precautions, promptly detect potential security hazards, reduce the incidence of security accidents, and protect the safety of people and property and normal order within the monitored area. However, existing technologies cannot effectively distinguish between benign environmental changes and genuine abnormal security scenarios, resulting in a large number of invalid alarms. In the monitored area, genuine abnormal security scenarios refer to changes in the scene such as unauthorized intrusions by personnel or small animals, which are sudden and potentially dangerous and may directly or indirectly affect the personal safety, property safety, or operation of critical facilities within the area. These are identified as genuine abnormal security scenarios. Benign environmental changes refer to a series of artificial environmental changes that are non-threatening but persistent, such as newly parked vehicles, newly piled materials, and temporary construction barriers. These changes alter the original scene layout and visual characteristics, but are essentially legal and compliant human actions that do not pose a real threat to security targets.

[0052] In existing technologies, dynamic background modeling based on Gaussian mixture models (GMMs) struggles to effectively distinguish between real security threats and benign changes in the artificial environment. Objects that are not inherently security anomalies, but whose thermal radiation characteristics differ significantly from the original background upon initial appearance, may be identified as new objects in the foreground by the GMM, leading to continuous false alarms until the model, after a learning period, gradually absorbs the thermal signal characteristics of these new objects as part of the background. While existing methods can achieve correct differentiation over time, the resulting false alarms waste resources and hinder appropriate staff intervention. Therefore, this paper proposes an infrared thermal imaging-based method for monitoring abnormal security scenarios to effectively distinguish between normal and abnormal environmental changes, improve the accuracy of abnormal security scenario identification, and avoid invalid alarms caused by misjudgments.

[0053] Further, step S1 includes:

[0054] Step S11: Deploy infrared thermal imagers in the monitoring area, collect radiation information of the monitoring area and generate infrared thermal images, and integrate them to form a video frame sequence of the monitoring area.

[0055] An infrared thermal imager is a detection device capable of detecting the temperature distribution on the surface of an object and converting it into a visual image. Based on the principle of infrared radiation, it forms a thermal image reflecting the temperature differences on the object's surface. The installation position and angle of the infrared thermal imager are rationally planned according to the specific scene of the monitoring area to capture images of the entire monitoring area. The infrared thermal imager continuously collects radiation information from the monitoring area, converts it into electrical signals, and after processing by an infrared imager, generates a thermal image with temperature gradient color coding. Different colors represent different temperature values, generating an infrared thermal image reflecting the temperature distribution on the surface of objects within the monitoring area. These images are then integrated to form a video frame sequence corresponding to the monitoring area, providing a reliable data foundation for subsequent image analysis, temperature assessment, and decision-making.

[0056] Step S12: Build a mixture model containing multiple Gaussian distributions for each pixel, and construct a Gaussian mixture model.

[0057] As an optional implementation, it is usually set to include 3-5 Gaussian distributions, which can be adjusted according to the actual situation.

[0058] To better illustrate, Gaussian mixture modeling for dynamic background modeling is a statistical method that learns the pixel intensity distribution characteristics of the background region in a scene to model the dynamic background, effectively distinguishing pixel value fluctuations caused by environmental changes from the movement of foreground objects. Specifically, for each pixel, the Gaussian mixture model is used to learn its pixel value distribution parameters in different time windows. When a new video frame is input, the matching degree between the current pixel and each Gaussian component in the Gaussian mixture model is calculated to determine whether the pixel belongs to the background or the foreground, thus distinguishing environmental changes in the video frames generated in the monitored area.

[0059] Specifically, after acquiring consecutive video frames, i.e., obtaining a video frame sequence, background modeling initialization is performed. This involves constructing a mixture model containing multiple Gaussian distributions for each pixel in the infrared thermal image of each frame in the video frame sequence. The Gaussian distributions cover different static or slowly changing temperature variations that may occur in the background area, thereby describing the various brightness states that may occur at the corresponding pixel location.

[0060] Understandably, in the monitored area, real security threats such as unauthorized intruders or small animals typically have relatively small physical dimensions, appearing as small areas on infrared thermal images. Conversely, benign environmental changes caused by artificial elements, such as newly parked vehicles, newly stockpiled materials, and temporary construction barriers, which are non-threatening but persistent alterations to the scene, typically have large physical dimensions and thus appear as larger areas on infrared thermal images. Therefore, the spatial interference of the artificial environment can be initially assessed by observing the area representation on infrared thermal images. Here, spatial interference of the artificial environment refers to the degree and scope of change caused by human activities or objects within the monitored area to the original spatial structure or state of the scene.

[0061] Further, step S2 includes:

[0062] Step S21: Filter foreground pixels based on each frame of the video frame sequence and cluster them to form foreground patches.

[0063] Specifically, a Gaussian mixture model is used to filter foreground pixels in each frame of a video frame sequence, i.e., a preset threshold. For each pixel in each frame, a probability calculation is performed to determine whether the probability of the pixel belonging to the background is lower than the threshold. If it is lower than the threshold, it is determined to be a foreground pixel. In practical applications, the Gaussian mixture model does not judge a single pixel in isolation, but clusters all foreground pixels in each frame to form independent foreground patches, so as to effectively reduce noise interference and avoid isolated pixels being misjudged as foreground. At the same time, using foreground patches for analysis is more in line with the continuous characteristics of object motion in video scenes.

[0064] Step S22: Construct the preliminary spatial interference of the artificial environment for each pixel in the current frame based on the foreground patches.

[0065] As explained, in this embodiment, based on the first [frame] in the current frame... Analyze the pixels to determine the first pixel. The initial spatial interference of an artificial environment for each pixel is calculated using the following formula:

[0066]

[0067] in, Indicates the first in the current frame Preliminary spatial interference of artificial environment at the pixel level; This represents the maximum and minimum value normalization function; Indicates the first in the current frame The number of pixels in the foreground patch corresponding to each pixel.

[0068] It can be explained that the maximum and minimum value normalization function This indicates the number of pixels compared to other foreground patches. Standardization processing is performed, that is, normalization processing is used to unify the number of pixels in foreground patches of different sizes to a uniform value. Within the interval, it can objectively reflect the number of frames in the current frame. The area of ​​the foreground patch where each pixel is located represents the size of the foreground patch. When the area of ​​the foreground patch is large, that is, a large area patch corresponds to the initial spatial interference of the artificial environment. The larger the value, the greater the likelihood that the current frame represents a benign environmental change that is non-threatening but will persistently alter the scene; conversely, the smaller the value, the greater the likelihood of initial spatial interference from the artificial environment. The smaller the value, the smaller the area of ​​the foreground patch feedback, which may indicate a real threat; similarly, the preliminary spatial interference of the artificial environment at each pixel in the current frame can be determined.

[0069] Understandably, given the large area of ​​foreground patches in the current frame, their outlines are composed of complex biological structures due to real security threats such as intruders or animals. In thermal imaging, these patches typically appear as irregular, zigzag-edged shapes. Furthermore, such targets are usually in motion, and once their posture changes, their outlines become dynamically distorted, reducing the regularity of their edges. Conversely, man-made objects such as vehicles, cargo boxes, and large equipment are industrially manufactured, and their shapes are usually composed of standard rectangular or square geometric shapes. In infrared thermal images, these objects appear as straight edges and sharp corners, exhibiting high edge regularity. In other words, large patches with regular edges highly conform to the characteristics of typical man-made objects.

[0070] Step S23: Based on the statistics and analysis of edge pixels of the foreground patch, and combined with the preliminary spatial interference of the artificial environment, construct the further spatial interference of the artificial environment for each pixel in the current frame.

[0071] To clarify, edge pixels refer to the pixels at the boundaries between different objects in the foreground patch, which are used to provide key information about the image's outline and shape.

[0072] Further, in step S23, edge pixels are statistically analyzed based on foreground patches, including:

[0073] Step S231: Starting from any edge pixel in the foreground patch, construct a closed edge by moving along the edge pixels, and convert the closed edge into an edge chain code using numbering.

[0074] Specifically, an edge pixel is randomly selected from the foreground patch and used as the edge starting point. Starting from the edge starting point, the movement proceeds along the boundary, and the direction of movement at each step is recorded using digital codes to form a closed-loop code. This code is then converted into a series of numbers to obtain the edge chain code. In other words, the edge pixels are numbered to establish the edge chain code. The digital codes used are 0-7 from Freeman chain code, which represents the eight possible adjacent directions (horizontal, vertical, and 45° diagonal) with numbers from 0 to 7 to record the direction of the pixel at each step. That is, the chain code is used to describe the contour of the object. The edge of the target is approximated by a series of short line segments with directions to completely describe the contour shape existing in the entire foreground patch.

[0075] Step S232: Determine the curvature corresponding to each number based on the edge chain code, and count the number of edge pixels in the foreground patch.

[0076] It is explained that each number in the edge chain code corresponds to a pixel movement in a specific direction. The curvature of the pixel is determined by a geometric method using the number and the corresponding pixel position. That is, the curvature is determined by calculating the second derivative of the curve to describe the degree of bending of the pixel, thus realizing a quantitative description of the bending characteristics of the pixel corresponding to the number. The number of all edge pixels in each foreground patch in the current frame is counted, that is, the number of edge pixels used to construct the edge chain code.

[0077] Step S233: Combine curvature and the number of edge pixels to construct the edge regularity of each pixel in the current frame.

[0078] Understandably, the curvature and the number of edge pixels are used to comprehensively reflect the edge regularity of objects in the foreground patch. For regular shapes, the edge chain code direction of the straight edge is constant, so the directional difference between any two adjacent numbers on the edge chain code is zero, meaning the curvature change is minimal. Even if a sudden change in direction occurs at the corner of the contour, the change is concentrated on a limited number of pixels, and then it enters a new stable phase. For example, a typical regular shape like a square, its corner is turned 90° by only two pixels, and then immediately returns to the stable direction of the next edge. Conversely, for irregular shapes, such as irregular natural contours like human figures or animals, the edge chain code usually alternates rapidly in eight basic directions, resulting in a tortuous and varied edge direction. That is, the direction code in the edge chain code switches frequently and randomly, indicating that the difference between adjacent numbers is sometimes large and sometimes small, and the curvature fluctuates violently and irregularly throughout the edge.

[0079] To clarify, the current frame will still be the first... The description is based on each pixel, combining edge regularity with preliminary spatial interference from the artificial environment to construct the first pixel in the current frame. The artificial environment further reduces spatial interference for each pixel, and the corresponding calculation formula is:

[0080]

[0081] in, Indicates the first in the current frame The artificial environment of each pixel further exacerbates spatial interference; Indicates the first in the current frame The number of edge pixels of the foreground patch corresponding to each pixel; Represented by natural constant An exponential function with base 0; , These represent the first and second digits in the current frame, respectively. The foreground patch corresponding to the nth pixel is on the edge chain code. The number and the first Curvature of each number; This represents the absolute value operation; Indicates the first in the current frame Preliminary spatial interference of artificial environment at the pixel level.

[0082] It can be explained that, Indicates the first in the current frame The edge regularity of the foreground patch corresponding to each pixel is fed back by the curvature and the number of edge pixels, which together reflect the contour changes of the corresponding object in the foreground patch. When the curvature fluctuates less and the number of edge pixels is certain or small, the edge regularity is stronger, indicating that the edge changes in the foreground patch are more regular. Conversely, when the curvature fluctuates more and the number of edge pixels is larger, it indicates that the edge changes in the foreground patch are more irregular. Similarly, the artificial environment spatial interference of each pixel in the current frame is further determined.

[0083] Understandably, natural and artificial objects exhibit fundamental differences in their thermal radiation characteristics in infrared thermal images. Real security threats, such as humans or animals, typically display distinct physiological characteristics in their surface temperature distribution. For instance, the core and torso of a natural object tend to have higher temperatures while the extremities are lower. Combined with the shielding and insulation effects of clothing and hair, this results in unevenly distributed patches of varying temperatures on the infrared thermal image. Conversely, artificial objects like vehicle roofs, metal cargo boxes, and plastic tarpaulins, composed of homogeneous materials and with larger volumes, exhibit high heat capacity and uniform heat exchange under similar environmental conditions to natural objects, typically presenting continuous thermal regions with minimal temperature differences. Therefore, analyzing the temperature characteristics of foreground patches and combining the aforementioned area and edge regularity analyses ultimately determines the spatial interference of the artificial environment.

[0084] Step S24: Obtain the thermal radiation uniformity of the foreground patch, and combine it with the artificial environment spatial interference to determine the artificial environment spatial interference of each pixel in the current frame.

[0085] To clarify, thermal radiation uniformity refers to the degree of consistency in the distribution of thermal radiation intensity among individual pixels in a foreground patch. It is usually expressed in terms of temperature to reflect the surface temperature stability and differences in the corresponding foreground patch.

[0086] Further, in step S24, the thermal radiation uniformity of the foreground patch is obtained, specifically as follows:

[0087] The surface temperature of each pixel in the foreground patch is obtained from the infrared thermal imager, and the standard deviation of the surface temperature is obtained based on the surface temperatures of all pixels in the corresponding foreground patch.

[0088] Specifically, in the infrared thermal image corresponding to the current frame generated by the infrared thermal imager, the grayscale value or color of each pixel corresponds to the radiation intensity of that pixel. Based on the temperature calibration algorithm built into the infrared thermal imager, the radiation intensity is converted into the surface temperature of each pixel in that frame, i.e., the temperature of the object surface in the foreground patch. Then, the standard deviation of the surface temperature of all pixels in the corresponding foreground patch is calculated, denoted as . .

[0089] Based on the apparent temperature standard deviation and combined with further spatial interference from the artificial environment, the current frame is determined as the [missing information]. The formula for calculating the spatial interference of artificial environment per pixel is:

[0090]

[0091] in, Indicates the first in the current frame The spatial interference of artificial environment at each pixel; Represented by natural constant An exponential function with base 0; Indicates the first in the current frame The standard deviation of the rendering temperature of all pixels in the foreground patch corresponding to each pixel; Indicates the first in the current frame The artificial environment of each pixel further exacerbates spatial interference.

[0092] It can be explained that in the current frame, given the large area and strong regularity of the foreground patch, the thermal radiation intensity distribution within the foreground patch is uniform, indicating a consistent temperature standard deviation. When the value is small, it indicates that the current monitoring area has a high degree of artificial environmental interference; similarly, the artificial environmental spatial interference of each pixel in the current frame is determined.

[0093] Understandably, in real life, the essential function of areas such as parking lots, loading and unloading areas, and roads is to allow the legal movement and temporary parking of vehicles and goods. This functional attribute will inevitably manifest as a specific spatiotemporal pattern in infrared thermal images. That is, the pixels in such areas will be repeatedly and regularly covered by foreground patches with large areas, strong regularity, and uniform distribution of thermal radiation intensity, resulting in a high frequency of "artificial environmental interference" patterns. This situation is a manifestation of the normal function of such areas and not an abnormal environmental change. Conversely, the core security requirement for walls, entrances and exits, and confidential areas is "prohibition of unauthorized stay and alteration." Under normal conditions, such areas maintain background stability. Therefore, based on the constructed video frame sequence, a comprehensive analysis of the spatial interference of the artificial environment in historical frames and the current frame is conducted to assess the changes in pixel positions and determine the spatiotemporal interference of the artificial environment. The spatiotemporal interference of the artificial environment refers to the degree of interference with a specific temporal distribution pattern caused by legal or expected human activities within the monitoring area, in order to quantitatively analyze the spatiotemporal characteristics within the monitoring area.

[0094] Furthermore, step S3 includes:

[0095] Step S31: Extract several historical frames from the video frame sequence based on the current frame, combine the current frame and historical frames, and analyze the mutability of the position of each pixel in the current frame through artificial environmental spatial interference analysis.

[0096] Specifically, in actual operation, based on the video frame sequence, several historical frames are selected from the current frame to construct a historical time period; the method for determining the artificial environmental spatial interference of each pixel in the current frame based on step S2 is similarly used to determine the artificial environmental spatial interference of each pixel in the historical frames; the artificial environmental spatial interference of each pixel position in the current frame within the historical time period is statistically analyzed, and the permissible changeability of each pixel is inferred in reverse, that is, the mutability of the position of each pixel in the current frame is constructed, and the corresponding calculation formula is:

[0097]

[0098] in, Indicates the first in the current frame The ability to change the position of each pixel; This indicates the number of frames corresponding to a historical time period; Indicates the first in the current frame The pixel in the historical time period The spatial interference of the artificial environment of the frame.

[0099] It can be explained that, This indicates that the current frame count includes the current frame; when the pixel position is changeable. A higher value indicates frequent artificial environmental spatial interference throughout the historical period; similarly, the changeability of the position of each pixel in the current frame can be determined.

[0100] Understandably, if a pixel in the current frame has already been marked as "highly mutable" due to frequent artificial environmental spatial interference in the past, it means that the current monitoring area may be a frequently changing area such as a parking lot or loading and unloading area. At this time, if the pixel in the current frame shows artificial interference characteristics similar to those in historical frames in the historical time period, it forms evidence of "spatiotemporal consistency" and confirms that the current change in the current frame is benign.

[0101] To further explain, when the foreground patches in the current frame are similar to those in historical frames—that is, when the foreground patches in the current frame show vehicles and cargo boxes of similar size, regular shape, and uniform heat distribution common in that area—it indicates that the changes in the current frame are not only within the permitted area, but the pattern of change also matches the normal operating history of that monitored area. This means that the spatial permissions of the current frame and the historical frame are correct, and the temporal characteristics match. For example, vehicles of similar size have re-entered the parking lot, and standard-sized cargo stacks have reappeared in the loading and unloading area. Therefore, by verifying the similarity between the artificial environmental interference in the current frame and that in the historical frame, the "possibly benign" inference obtained through spatial analysis is upgraded to a "very high probability of benign" deterministic judgment obtained through temporal analysis. This allows for a more accurate classification of the corresponding changes as high spatiotemporal interference, effectively suppressing false alarms.

[0102] Step S32: Perform foreground patch matching on the current frame and historical frames, and construct the artificial environmental spatiotemporal interference of each pixel in the current frame by combining the changeability of position.

[0103] Further, in step S32, foreground patch matching is performed on the current frame and historical frames, specifically as follows:

[0104] The similarity of the foreground patches corresponding to any pixel location in the current frame and historical frames is evaluated, and the foreground patch matching results between the current frame and historical frames are obtained by combining the spatial interference of the artificial environment.

[0105] Specifically, around the first in the current frame To explain each pixel, first, confirm the similarity between the foreground patch corresponding to the current frame and the foreground patch corresponding to the previous frame, denoted as . Similarity is typically obtained using color features, texture features, and shape features. Then, based on the ratio of the artificial environmental spatial interference of the historical frame to the sum of the artificial environmental spatial interference of all historical frames, the proportion of interference of the current historical frame in all historical frames is determined to measure the consistency of change between the current frame and historical frames.

[0106] Construct the first position in the current frame by considering the mutability of the position. The formula for calculating the spatiotemporal interference of artificial environment per pixel is:

[0107]

[0108] in, Indicates the first in the current frame The spatiotemporal interference of artificial environment at each pixel level; This indicates the number of frames corresponding to a historical time period; Indicates the first in the current frame The pixel in the historical time period The spatial interference of the artificial environment of the frame; Indicates the first in the current frame The similarity between the foreground patch corresponding to a pixel in the current frame and the foreground patch corresponding to a pixel in a previous frame; Indicates the first in the current frame The changeability of the position of each pixel.

[0109] It can be explained that, and These represent excluding and including the current frame, respectively. Used to provide feedback on the first of all historical frames The extent of artificial environmental spatial interference at each pixel level is used to measure the performance of foreground patches under historical interference. It is used to provide feedback on the matching status between the current frame and historical frames, thereby increasing the deterministic assessment of benign artificial environment changes; similarly, it determines the spatiotemporal interference of the artificial environment at each pixel in the current frame.

[0110] Understandably, since existing technologies cannot distinguish between real security threats and benign changes in the artificial environment, a state machine mechanism is introduced to combine spatial and temporal analysis to obtain the spatiotemporal interference of the artificial environment at each pixel. Based on the spatiotemporal interference of the artificial environment, the state of the corresponding pixel is evaluated, which facilitates subsequent corresponding operations for different states.

[0111] Artificial environmental spatial interference and artificial environmental spatiotemporal interference differ fundamentally in their assessment dimensions and functions, yet both are indispensable. Artificial environmental spatial interference focuses on the instantaneous assessment of morphological and thermodynamic characteristics. This involves analyzing the size, edge regularity, and thermal radiation uniformity of foreground patches based on single-frame image analysis to determine whether the target possesses typical appearance features of man-made objects such as vehicles or cargo boxes, thus distinguishing it from biological targets with irregular edges and complex thermal distributions. In contrast, artificial environmental spatiotemporal interference emphasizes the verification of scene semantics and historical patterns. This involves combining video frame sequences with the historical mutability of the location and the visual similarity between objects to determine whether the currently appearing man-made object conforms to the historical operational patterns of the area, thereby confirming whether the change is a benign, anticipated alteration.

[0112] This application constructs a complementary dual-filtering mechanism using these two indicators to address the false alarm problem in background modeling. Relying solely on spatial interference, the system cannot determine whether a man-made object is present in an unauthorized area or disguised as an intruder; relying solely on spatiotemporal interference makes it difficult to distinguish between long-term lingering intruders and legally parked vehicles. Therefore, it is necessary to first use spatial interference of the artificial environment as an initial screening, eliminating most biological threats based on the physical differences between man-made objects and living organisms in infrared imaging. Then, spatiotemporal interference of the artificial environment is used as a verification to confirm whether the man-made object appears in an area where changes are permissible and conforms to historical patterns. Only by combining spatial appearance judgment with temporal pattern verification can non-threatening, benign, and persistent environmental changes be accurately identified, effectively reducing the false alarm rate.

[0113] Furthermore, in step S4, specifically:

[0114] A preset evaluation threshold is used to compare the spatiotemporal interference of the artificial environment with the evaluation threshold. When the spatiotemporal interference of the artificial environment is greater than or equal to the evaluation threshold, the pixel corresponding to the spatiotemporal interference of the artificial environment is marked as a candidate state.

[0115] As an optional implementation, the evaluation threshold is set to 0.7, which can be adjusted according to the actual situation.

[0116] Specifically, the calculation formula for comparing the spatiotemporal interference of the artificial environment and the evaluation threshold is as follows:

[0117]

[0118] in, Indicates the first in the current frame Is each pixel a candidate state? Indicates the first Each pixel is marked as a candidate state. Indicates the first The pixel is not a candidate state; Indicates the first in the current frame The spatiotemporal interference of artificial environment at the pixel level.

[0119] It should be noted that, to distinguish between real threats and benign environmental changes, when using artificial environmental spatiotemporal disturbances to label pixels, they are not directly classified as foreground anomalies, but are temporarily labeled as "candidate states." In practical applications, when moving objects appear in the monitoring area, the possibility of foreground anomalies includes real threats such as intruders or animals, and benign environmental changes such as pedestrians or vehicles passing by. If these are directly classified as foreground anomalies, it will lead to a large number of false alarms, reducing practicality and reliability. Therefore, the "candidate state" label is used for screening to make subsequent evaluation steps. That is, when the artificial environmental spatiotemporal disturbance of a pixel is greater than or equal to the evaluation threshold, the pixel is marked as 1, indicating that it is a candidate state; conversely, when the artificial environmental spatiotemporal disturbance of a pixel is less than the evaluation threshold, the pixel is marked as 0, indicating that it is not a candidate state. This clarifies whether the pixels corresponding to the candidate states point to real threats or benign environmental changes, avoiding false alarms caused by direct judgment, ensuring timely identification and response to potential real threats, and using "candidate states" to help effectively distinguish between real threats and benign changes in a variable environment, thereby improving the accuracy of the Gaussian mixture model.

[0120] Understandably, benign environmental changes are persistent, while real threats are often transient. For example, if a parked vehicle appears in a monitoring area and remains stable for a certain number of consecutive frames, the Gaussian mixture model will determine it as a benign environmental change. On the other hand, birds or unauthorized intruders usually only last for a few frames before disappearing quickly, thus being determined as transient threats. Therefore, when the number of consecutive frames of a candidate state exceeds a pre-defined threshold, the Gaussian mixture model can adaptively determine that the change in the monitoring area is not a transient foreground target, but a persistent and stable environmental change, thereby distinguishing between benign and malicious environmental changes.

[0121] Furthermore, step S5 includes:

[0122] Step S51: Acquire new video frames within the monitoring area. Use a Gaussian mixture model to verify whether the brightness values ​​of the new video frames match those of the pixels with candidate state markers in the current frame. If they do not match, start counting again. If they match, add them to the frame count. Continue until they do not match, and then count the number of consecutive frames.

[0123] Preferably, when counting frames, a continuous frame counter is introduced to record video frames within the monitoring area that meet the decision-making conditions in real time, and a decision threshold is set. In this embodiment, the decision threshold is set to 20 frames, but it can be adjusted according to the actual situation. This is used to distinguish between "temporary interference" caused by real threats and "persistent changes" caused by benign environmental changes.

[0124] Specifically, based on the monitoring area, new video frames are continuously input into the Gaussian mixture model by acquiring new video frames through an infrared imager. The brightness value of the pixels in the newly input video frame is checked to see if it matches the pixels with candidate state markers in the current frame. A continuous frame count counter is then started. If they match, that is, the difference between the brightness values ​​of the two pixels is within a preset range, the frame is counted and the continuous frame count counter is incremented by 1. If they do not match, that is, the difference between the brightness values ​​of the two pixels exceeds the preset range, the continuous frame count counter is interrupted and reset. A new round of continuous frame count is started when a match occurs again, so as to accurately assess subsequent environmental changes through the accumulation of continuous frame counts.

[0125] Step S52: Compare the number of continuous frames with the decision threshold. When the number of continuous frames exceeds the decision threshold, it indicates a normal environmental change; when the number of continuous frames does not exceed the decision threshold, it indicates an abnormal environmental change.

[0126] Specifically, in this embodiment, if the number of continuous frames counted in the current monitoring area exceeds the decision threshold, it indicates that the environmental change is normal and benign, i.e., a persistent and stable environmental change. At this time, the Gaussian mixture model is updated to make corresponding model changes for the current background changes, and the weight of the candidate state distribution presented in the current frame is significantly increased, and it is formally incorporated into the Gaussian mixture model, so that subsequent video frames generated for the monitoring area will not trigger alarms for similar brightness values ​​of that pixel. Conversely, if the number of continuous frames does not exceed the decision threshold, i.e., the matching is interrupted before the continuous frame counter reaches the decision threshold, it indicates that the current situation is only a brief interference. In this case, the situation is immediately judged as a real foreground abnormal target, and a security alarm is generated accordingly. At the same time, the candidate state distribution corresponding to the new video frame input is quickly forgotten or replaced by the Gaussian mixture model, ensuring the reliability and practicality of the security system built based on the Gaussian mixture model in complex dynamic scenarios.

[0127] Understandably, a Gaussian mixture model is constructed for each pixel in the video frame sequence formed by the infrared thermal imager capturing the monitoring area, i.e., dynamic background modeling. Then, spatial analysis is performed on the foreground patches established in the current frame of the video frame sequence to evaluate the changes in each pixel and construct the spatial interference of the artificial environment. Next, temporal analysis is performed to determine the spatiotemporal interference of the artificial environment by combining the spatial interference of the artificial environment. The Gaussian mixture model is used for evaluation to determine whether a pixel is marked as a candidate state. The candidate state is treated as a temporary state, and the number of frames for the temporary state is counted and compared with the decision threshold to analyze whether it is a short-term or long-term environmental interference, distinguishing between benign environmental changes and real abnormal security scenarios. This allows for the evaluation of environmental changes during the operation of the Gaussian mixture model, preventing a large number of invalid alarms from being generated in the monitoring area, avoiding the continuous periodicity of false alarms, improving the accuracy of security monitoring, and thus providing a more accurate and reliable abnormal target identification capability for security work.

[0128] The second embodiment of the present invention provides an infrared thermal imaging abnormal security scene monitoring system. The system includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus. The processor calls logical instructions in the memory to execute an infrared thermal imaging abnormal security scene monitoring method according to any embodiment of the present invention. This system has the same beneficial effects as the aforementioned infrared thermal imaging abnormal security scene monitoring method, and will not be described in detail here.

[0129] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0130] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for monitoring abnormal security scenarios using infrared thermal imaging, characterized in that, The method includes: An infrared thermal imager was used to capture images of the monitored area to establish a video frame sequence, and a Gaussian mixture model was constructed for each pixel. Spatial analysis is performed on the video frame sequence to establish foreground patches, and the spatial interference of the artificial environment at each pixel in the current frame is determined through the foreground patches. By combining the spatial interference of the artificial environment with the temporal analysis of the video frame sequence, the spatiotemporal interference of the artificial environment at each pixel in the current frame can be obtained. Gaussian mixture model is used to evaluate the spatiotemporal interference of artificial environment, and to determine and mark whether each pixel in the current frame is a candidate state. A preset decision threshold is set, the number of consecutive frames with candidate state markers is counted, and the number of consecutive frames is compared with the decision threshold to distinguish between normal and abnormal environmental changes. The step of performing spatial analysis on the video frame sequence, establishing foreground patches, and determining the artificial environmental spatial interference of each pixel in the current frame through the foreground patches includes: Foreground pixels are selected from each frame of the video frame sequence and clustered to form foreground patches; Construct the initial spatial disturbance of the artificial environment for each pixel in the current frame based on the foreground patches; Based on the statistics and analysis of foreground patches and edge pixels, and combined with the preliminary spatial interference of the artificial environment, the further spatial interference of the artificial environment of each pixel in the current frame is constructed. The thermal radiation uniformity of the foreground patch is obtained, and the spatial interference of the artificial environment is further determined by combining the spatial interference of the artificial environment to determine the spatial interference of the artificial environment for each pixel in the current frame. The step of combining artificial environmental spatial interference with the temporal analysis of the video frame sequence to obtain the artificial environmental spatiotemporal interference of each pixel in the current frame includes: Based on the current frame, several historical frames are extracted from the video frame sequence. Combining the current frame and historical frames, the mutability of the position of each pixel in the current frame is analyzed through artificial environmental spatial interference. Foreground patch matching is performed on the current frame and historical frames, and the spatiotemporal interference of the artificial environment for each pixel in the current frame is constructed by combining the variability of the position.

2. The method for monitoring abnormal security scenarios using infrared thermal imaging according to claim 1, characterized in that, An infrared thermal imager was used to capture images of the monitored area to create a video frame sequence. A Gaussian mixture model was constructed for each pixel, including: Infrared thermal imagers are deployed in the monitoring area to collect radiation information and generate infrared thermal images, which are then integrated to form a video frame sequence of the monitoring area. For each pixel, a mixture model containing multiple Gaussian distributions is constructed to create a Gaussian mixture model.

3. The method for monitoring abnormal security scenarios using infrared thermal imaging according to claim 1, characterized in that, Based on the statistics and analysis of edge pixels in the foreground patches, including: Starting from any edge pixel in the foreground patch, construct a closed edge by moving along the edge pixels, and convert the closed edge into an edge chain code by numbering. The curvature corresponding to each number is determined based on the edge chain code, and the number of edge pixels in the foreground patch is counted. The edge regularity of each pixel in the current frame is constructed by combining the curvature and the number of edge pixels.

4. The method for monitoring abnormal security scenarios using infrared thermal imaging according to claim 1, characterized in that, To obtain the thermal radiation uniformity of the foreground patch, specifically: The surface temperature of each pixel in the foreground patch is obtained from the infrared thermal imager, and the standard deviation of the surface temperature is obtained based on the surface temperatures of all pixels in the corresponding foreground patch.

5. The method for monitoring abnormal security scenarios using infrared thermal imaging according to claim 1, characterized in that, Foreground patch matching is performed between the current frame and historical frames, specifically as follows: The similarity of the foreground patches corresponding to any pixel location in the current frame and historical frames is evaluated, and the foreground patch matching results between the current frame and historical frames are obtained by combining the spatial interference of the artificial environment.

6. The method for monitoring abnormal security scenarios using infrared thermal imaging according to claim 1, characterized in that, The spatiotemporal interference of the artificial environment is evaluated using a Gaussian mixture model. Specifically, each pixel in the current frame is determined and labeled as a candidate state. A preset evaluation threshold is used to compare the spatiotemporal interference of the artificial environment with the evaluation threshold. When the spatiotemporal interference of the artificial environment is greater than or equal to the evaluation threshold, the pixel corresponding to the spatiotemporal interference of the artificial environment is marked as a candidate state.

7. The method for monitoring abnormal security scenarios using infrared thermal imaging according to claim 1, characterized in that, A preset decision threshold is set, and the number of consecutive frames with candidate state markers is counted. The number of consecutive frames is compared with the decision threshold to distinguish between normal and abnormal environmental changes, including: Acquire new video frames within the monitoring area, and use a Gaussian mixture model to verify whether the brightness values ​​of the new video frames match those of the pixels with candidate state markers in the current frame. If they do not match, start counting again; if they match, add them to the frame count, and continue until they do not match, and then count the number of consecutive frames. By comparing the number of consecutive frames with the decision threshold, if the number of consecutive frames exceeds the decision threshold, it indicates a normal environmental change; if the number of consecutive frames does not exceed the decision threshold, it indicates an abnormal environmental change.

8. An infrared thermal imaging-based abnormal security scenario monitoring system, characterized in that, The system includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus. The processor calls logical instructions in the memory to execute the abnormal security scenario monitoring method of infrared thermal imaging as described in any one of claims 1 to 7.

Images