Analysis, learning, and update system for multi-site AI thermal imaging camera-linked security models

Abstract

The present invention relates to an analysis learning and update system for a multi-site artificial intelligence thermal imaging camera-linked security model and a control method thereof. Specifically, it relates to a system and method that ensures the reliability of data fusion by verifying the physical spatiotemporal consistency of heterogeneous sensor data acquired through a real-time camera and a thermal imaging camera, minimizes false alarms through a precise risk calculation algorithm based on object tracking status and thermodynamic change trends, and compensates and controls alarm output based on available channels in emergency situations such as communication failure.

Description

Technology Field

[0001] The present invention relates to an analysis learning and update system for a multi-site artificial intelligence thermal imaging camera-linked security model and a control method thereof. Specifically, it relates to a system and method that ensures the reliability of data fusion by verifying the physical spatiotemporal consistency of heterogeneous sensor data acquired through a real-time camera and a thermal imaging camera, minimizes false alarms through a precise risk calculation algorithm based on object tracking status and thermodynamic change trends, and compensates and controls alarm output based on available channels in emergency situations such as communication failure. Background Technology

[0002] Recently, surveillance systems combining CCTV and fire detectors are being widely adopted for safety management in industrial sites and public places. Initially, these systems were limited to simply recording video or detecting smoke particles, but recently, there has been an increasing number of attempts to detect danger in real time by integrating deep learning-based object recognition technology with thermal imaging cameras.

[0003] However, conventional technologies have the following limitations.

[0004] When fusing data from a visible light camera and a thermal light camera, a method of simply overlapping the two images based on coordinates is used. In this case, if a time error occurs between the two sensors due to network delay or frame drops, a 'false mapping' phenomenon occurs where a heat source is displayed at the location of a fast-moving object's afterimage, causing a false alarm.

[0005] When abnormal behavior (falling, fighting, etc.) is detected, the judgment is based solely on changes in joint points or movement speed. Consequently, there is a problem where false alarms are generated by mistaking tracking error situations—such as frequent ID switching caused by multiple people crossing paths or objects temporarily obscuring objects—for violent movements.

[0006] When a fire is detected, it simply determines whether a set threshold temperature (e.g., 80 degrees) has been reached. This causes the golden time for initial fires to be missed or leads to continuous false alarms for fixed heat sources such as stoves. Furthermore, it only considers the physical size of the fire and fails to reflect the actual distance from people (potential for casualties), which limits its ability to determine response priorities.

[0007] Despite the fact that reporting to an external control server becomes impossible if the network is disconnected in the event of a dangerous situation, the on-site alarm device operates only at the same level as usual, resulting in a problem where the ability to propagate danger in the isolated site is significantly reduced.

[0008] Therefore, there is a growing need for an intelligent sensing system capable of verifying the physical integrity of sensor data, distinguishing between noise (tracking errors) and valid signals, and actively responding to communication availability. The problem to be solved

[0009] The present invention has been devised to solve the problems of the aforementioned prior art, and the specific problems that the present invention aims to solve are as follows.

[0010] The first task is to prevent false alarms caused by mismapping by introducing a spatiotemporal consistency verification logic based on the camera's frame rate and object movement speed when fusing real image and thermal image data, and blocking fusion when the physical synchronization error exceeds an allowable range.

[0011] The second task aims to suppress misjudgments caused by tracking errors and select only valid risk situations by defining the switching frequency of object identifiers (IDs) as tracking noise during abnormal behavior detection and applying a negative feedback logic that attenuates it inversely proportional to the operation intensity score.

[0012] The third task aims to preemptively detect signs of thermal runaway by monitoring the acceleration of temperature, which is the second derivative of the temperature, rather than simply whether the temperature is reached when a fire is detected, and to calculate a risk level focused on human protection by assigning weights to proximity to the human body.

[0013] The fourth task aims to prevent flickering of the alarm device by using hysteresis control based on the mode of the risk level calculation when controlling response according to the risk level, and to secure the survivability and responsiveness of the system through compensation control logic that maximizes the field alarm output according to the law of conservation of total information when external communication is interrupted.

[0014] The problems of the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0016] According to an embodiment of the present invention for solving the above problem

[0017] A composite imaging unit that simultaneously acquires visual and thermal information of a surveillance target area;

[0018] An image data processing module that performs spatiotemporal alignment verification on real image data and thermal image data received from the above-mentioned composite shooting unit to determine the validity of data fusion, and detects abnormal behavior by analyzing the amount of change in an object's joint points and tracking status from the verified data;

[0019] A thermal data analysis module that detects signs of fire by analyzing thermal data received from the above-mentioned composite imaging unit;

[0020] A risk grade calculation module that calculates a risk grade by synthesizing the analysis results of the above image data processing module and thermal data analysis module; and

[0021] An alarm output unit that transmits a physical alarm signal according to the above-calculated risk level; is included,

[0022] The above image data processing module is,

[0023] It is characterized by being configured to block false alarms caused by physical errors of the sensor and errors of the tracking algorithm and to select only valid dangerous situations by providing a spatiotemporal consistency judgment logic that analyzes physical synchronization errors between real image frames and thermal image frames in conjunction with movement speed, and a tracking stability-based severity determination logic that corrects the motion intensity score by considering the identifier (ID) switching frequency occurring during the object tracking process as noise.

[0024] The spatiotemporal consistency determination logic performed by the above image data processing module is,

[0025] A step of automatically calculating and setting a physical frame period value, obtained by dividing 1 second (1,000ms) by the frame rate based on the hardware frame rate (FPS) setting value of the composite shooting unit, as an allowable maximum time error threshold;

[0026] A step of calculating a time error (Δ) which is the difference in timestamps between a received real image frame and a thermal image frame, and blocking data fusion by considering that a physical positional discrepancy between real image coordinates and thermal image coordinates has occurred due to the movement speed of an object if the time error exceeds the threshold; and

[0027] It is characterized by including a fail-safe step that accumulates and counts the number of discontinuous frame sequence numbers assigned during network transmission, and if the accumulated number of frame drops exceeds a preset threshold, determines that the real-time nature of the data has been lost and forcibly sets the fusion reliability to 0.

[0028] The above spatiotemporal consistency determination logic is,

[0029] Even if the above time error is within the above threshold, dynamic threshold adjustment is performed to reduce the threshold inversely proportionally as the movement speed of the object calculated by the image data processing module increases, wherein

[0030] To prevent unnecessary fusion blocking caused by minute sensor jitter, dead-band filtering is further performed, in which variations within a preset minimum time error range are considered noise and ignored.

[0031] When the above data fusion is blocked, it is characterized by automatically switching to a standalone mode in which coordinate information of thermal image data with unguaranteed positional consistency is discarded and analysis is performed using only the object recognition results of the visible light camera.

[0032] The tracking stability-based severity determination logic performed by the above image data processing module is,

[0033] A step of calculating motion intensity through the sum of joint point movement vectors of an object in a video, and simultaneously calculating a switching count which is the number of times the unique identifier (ID) of the object changes within a preset window time; and

[0034] The method includes the step of defining the switching count as a noise component that impairs tracking reliability and the operation intensity as an effective signal component, and performing a negative feedback operation that inversely attenuates the operation intensity score as the switching count increases according to the signal-to-noise ratio (SNR) improvement principle of communication theory.

[0035] This is characterized by lowering the risk score to suppress false alarms when the tracking status is unstable even if violent motion is detected, and reflecting the motion intensity as is to confirm it as a dangerous situation only when the tracking status is stable.

[0036] The above tracking stability-based severity determination logic is,

[0037] If the above attenuated motion intensity score exceeds the risk threshold, the duration for which the abnormal behavior is maintained is measured and reflected in the risk score,

[0038] To prevent the risk score from increasing linearly indefinitely over time, a logarithmic accumulation function is applied so that the rate of increase in the score gradually decreases after the initial occurrence period and converges to a maximum value, and

[0039] It is characterized by further including a repeat pattern suppression filter that, when a certain periodicity is confirmed in the detected motion, considers it as a repetitive work behavior and stops the application of weights based on the accumulation of duration, and

[0040] The above image data processing module is,

[0041] In the case where an object is obscured by an obstacle and then reappears, causing the identifier to change, an exception handling logic is further performed to determine whether it is the same object by comparing the predicted position using a Kalman filter with the position of the reappearing object, and if it is determined to be the same object, the corresponding identifier switching count is excluded from the penalty factor of the aforementioned negative feedback operation, thereby

[0042] It can be characterized by distinguishing between physical disconnection situations and algorithmic error situations in object tracking to correct the operation intensity score. Effects of the invention

[0044] According to the present invention as described above, the following effects can be expected.

[0045] By using a spatiotemporal consistency judgment logic based on physical laws (the relationship between speed and time), the system's data reliability is dramatically improved by fundamentally blocking the forced fusion of time-distorted data even if network delays or frame drops occur.

[0046] By applying the signal-to-noise ratio (SNR) improvement principles of communication theory to image analysis, it automatically filters out noise data generated when object tracking is unstable in congested environments, thereby reducing administrator fatigue from false alerts and increasing detection accuracy.

[0047] By detecting rapid-spreading fires even at low temperatures through thermodynamic acceleration detection, it secures the golden time; furthermore, by classifying cases where a fire is close to the human body as the highest risk, even if the scale is small, it maximizes the effectiveness of preventing actual casualties.

[0048] It reduces on-site confusion by preventing unnecessary flashing of alarm devices through a dual safety control loop, and provides a fail-safe function that automatically amplifies on-site alarms to elicit assistance from the surroundings even in the worst-case disaster scenario where communication networks are cut off.

[0049] The effects according to the present invention are not limited to those exemplified above, and a wider variety of effects are included within the present invention. Brief explanation of the drawing

[0051] Figure 1 illustrates the overall system operation flowchart according to the present invention. FIG. 2 illustrates a spatial alignment-based reliability judgment flowchart according to the present invention. FIG. 3 illustrates a flowchart of a severity determination based on tracking stability according to the present invention. Figure 4 illustrates a flowchart of the dynamic calculation of fire risk according to the present invention. Figure 5 illustrates an available channel-based compensation control flowchart according to the present invention. Specific details for implementing the invention

[0052] Hereinafter, various embodiments are described in more detail with reference to the attached drawings. The embodiments described in this specification may be modified in various ways. Specific embodiments may be depicted in the drawings and described in detail in the detailed description. However, specific embodiments disclosed in the attached drawings are intended only to facilitate understanding of various embodiments. Accordingly, the technical concept is not limited by specific embodiments disclosed in the attached drawings, and it should be understood that it includes all equivalents or substitutions that fall within the spirit and scope of the invention.

[0053] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but these components are not limited by the aforementioned terms. The aforementioned terms are used solely for the purpose of distinguishing one component from another.

[0054] The functions related to artificial intelligence according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0055] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a basic artificial intelligence model is trained using a number of training data by a learning algorithm, thereby creating predefined rules of operation or artificial intelligence models configured to perform desired characteristics (or objectives). Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0056] An artificial intelligence model can be composed of multiple neural network layers. Each of the multiple neural network layers has multiple nodes and weight values, and performs neural network operations through calculations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, multiple weights can be updated so that the loss value or cost value obtained by the artificial intelligence model during the learning process is reduced or minimized. Additionally, to minimize the loss value or cost value, multiple weights can be updated in a direction that minimizes the gradient associated with the loss value or cost value. Artificial neural networks may include deep neural networks (DNNs), such as Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), or Deep Q-Networks, but are not limited to the examples mentioned above.

[0057] A network is a network that serves as a transmission path for web pages; it may be a closed network such as a LAN (Local Area Network) or WAN (Wide Area Network), but it is desirable for it to be an open network such as the Internet. The Internet refers to a global open computer network structure that provides the TCP / IP protocol and various services existing at its upper layers, namely HTTP (HyperText Transfer Protocol), Telnet, FTP (File Transfer Protocol), DNS (Domain Name System), SMTP (Simple Mail Transfer Protocol), SNMP (Simple Network Management Protocol), NFS (Network File Service), and NIS (Network Information Service).

[0058] Terminals can be implemented in various forms. For example, the terminals described in this specification may include mobile terminals such as smartphones, tablet PCs, PDAs, portable multimedia players, and MP3 players, as well as fixed terminals such as smart TVs and desktop computers.

[0059] In this specification, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. When a component is described as being “connected” or “connected” to another component, it should be understood that it may be directly connected to or connected to that other component, or that there may be other components in between. On the other hand, when a component is described as being “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

[0060] Meanwhile, a "module" or "part" for a component as used in this specification performs at least one function or operation. Furthermore, a "module" or "part" may perform a function or operation by hardware, software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts," excluding a "module" or "part" that must be performed on specific hardware or on at least one processor, may be integrated into at least one module. A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0061] In addition, power, power transmission, and control therefor for the following assembly configurations and embodiments, including "by control," follow conventional technology including terminals, applications, hardware control modules, etc., so they are omitted to avoid redundancy.

[0062] In addition, the operation embodiments and configurations described in a general manner without being explained in detail below follow the prior art and are omitted in order to focus on describing the purpose of the present invention and the resulting effects.

[0063] Furthermore, in describing the present invention, if it is determined that a detailed description of related known functions or configurations may unnecessarily obscure the essence of the invention, such detailed description is abbreviated or omitted.

[0064] An analysis learning and update system for a multi-site artificial intelligence thermal imaging camera-linked security model according to one embodiment of the present invention is composed of a plurality of organically combined hardware and software units to collect physical environment information of a monitored area, analyze it using an artificial intelligence algorithm to detect fire and abnormal behavior at an early stage, and perform an optimal response appropriate to the situation.

[0065] Specifically, the system according to the present invention may include a composite shooting unit (110) for acquiring visual and thermal data, an image data processing module (120) for analyzing acquired data to determine abnormal behavior of an object, a thermal data analysis module (130) for analyzing temperature changes to detect signs of fire, a risk grade calculation module (140) for quantifying the severity of a situation by synthesizing analyzed information, a response control module (150) for determining an optimal response plan according to the grade, an alarm output unit (160) for transmitting a physical warning signal, and a learning server unit (170) for accumulating data and enhancing the model. Each of the above components is interconnected via a wired or wireless network or a system bus to exchange data in real time, and each functional feature described below may be implemented by one or more processors and memories.

[0066] According to one embodiment, data exchange between the components is performed in units of “frame records” and “event records.” The frame record includes a frame time (timestamp), a frame sequence number, a sensor status flag (normal / dropped / synchronization mismatch / needs correction), a resolution and a frame rate, and thermal image scale information (minimum / maximum temperature or scale factor). Additionally, the event record includes a risk level, an image-based risk score, a thermal-based risk score, a network availability index, alarm output parameters (blinking pattern identifier, volume level, duration, number of repetitions), and the result of an external notification call (success / failure / number of retries / delay time), and the records are utilized for relearning and threshold correction by the learning server unit (170) described below.

[0067] Hereinafter, the detailed functions of each of the above components, the data processing method, and the core control process of the present invention implemented therethrough will be described in detail with reference to the drawings.

[0068] The composite shooting unit (110) operates as a hardware device in which a visible light camera that captures the visible light region and a thermal camera that detects the infrared region are physically combined to simultaneously acquire visual and thermal information of the surveillance target area. Beyond simple image acquisition, the composite shooting unit (110) performs the role of a synchronization source that generates basic timestamps and sequence data for verifying spatiotemporal alignment. To minimize physical errors between heterogeneous sensors, the composite shooting unit (110) performs a spatiotemporal alignment-based synchronization signal generation function and applies hardware triggering that aligns the exposure start times of visible and thermal frames in microseconds based on an internal clock. Additionally, sequential sequence numbers are assigned to each frame header and transmitted to detect frame drops during network transmission. At this time, the physically acceptable time error threshold follows a logic that is automatically calculated inversely proportional to the camera's frame rate setting value. The composite shooting unit (110) obtains timestamp data, which is the system time at the moment the shutter of each sensor opens at the sensor interface level, sequence data, which is the unique serial number of the transmitted packet, and camera internal temperature and illuminance sensor values ​​for monitoring the operating environment. As an example, when the composite shooting unit (110) is set to 30 frames per second, the physical synchronization threshold is calculated to be approximately 33 milliseconds, and if the measured time error is 40 milliseconds, it is determined to be a risk of afterimage occurrence and fusion is blocked, and if it is 15 milliseconds, it is determined to be within the normal range and data is transmitted.

[0069] The image data processing module (120) is a software configuration that recognizes objects and behaviors by analyzing real image data received from the composite shooting unit (110), and operates a filtering engine that verifies the validity of abnormal behavior by applying a signal-to-noise ratio improvement principle that regards tracking errors as noise. The image data processing module (120) performs a severity determination logic based on tracking stability, calculates a signal which is the amount of change in the object's joint points, and simultaneously monitors noise which is whether the object's unique identifier changes frequently. If the number of identifier switching increases, it is considered a tracking failure, and negative feedback control is performed to inversely deduct or attenuate the motion intensity score. Additionally, as abnormal behavior persists, the score is accumulated in the form of a logarithmic function to perform an operation that secures initial responsiveness while preventing infinite divergence. The above image data processing module (120) calculates in real time, through an image analysis engine, a joint movement vector which is the sum of the inter-frame movement pixel distances of 17 joint points, a switching count which is the number of times an object identifier has changed within a recent 1-second window, and a duration which is the elapsed time since a specific action flag was activated. As an example, even if a motion intensity of 100 points is input due to violent motion, if identifier switching occurs 2 times per second, a damping factor of 0.5 is applied to lower the final score to 50 points and ignore it, and if there is no switching, the score is maintained at 100 points and confirmed as dangerous.

[0070] The thermal data analysis module (130) is a software configuration that detects signs of fire based on thermal imaging data. It goes beyond simple temperature measurement to dynamically calculate the fire risk by performing a complex calculation of the temperature rise acceleration, which is a thermodynamic change, and the proximity of the human body, which is a safety engineering indicator. The thermal data analysis module (130) performs thermal runaway prediction and human life priority weighting logic, and monitors acceleration by differentiating the temperature rise rate per unit time in real time. Even if the current temperature is low, if the acceleration exceeds a threshold, it determines this as a precursor to thermal runaway and activates a predictive trigger to immediately raise the risk level. Additionally, it calculates the Euclidean distance between the high-temperature area and the human object, and performs a calculation that assigns a higher weight to the possibility of human casualties than to the scale of the fire as the distance increases. The above thermal data analysis module (130) obtains the temperature rise acceleration, which is the rate of change of the difference between the current temperature and the previous temperature divided by the time interval from the raw data of the thermal image sensor; the human body mapping distance, which is the pixel distance between the coordinates of the center point of the high-temperature area and the center point of the human body bounding box in the real image; and the high-temperature area spread, which is the rate of increase per hour of the number of pixels above the reference temperature. As an example, in an environment where there is no change in temperature in an initial temperature of 40 degrees, the risk level is calculated as zero, but if acceleration occurs by rapidly rising by 5 degrees per second, the risk level is calculated as ninety points to preemptively warn of thermal runaway.

[0071] The above-described risk level calculation module (140) is a software configuration that quantifies the risk level of the current situation into grades 1 through 3 by synthesizing the results of the image and thermal data analysis and the communication status. The above-described risk level calculation module (140) performs voting-based hysteresis logic to prevent flickering of the alarm device. Within a voting window of 30 frames set considering the human cognitive reaction speed of approximately 1 second, the mode of the calculated grades is determined as the final grade. Through this, it suppresses instantaneous data spikes or vibrations at the grade boundary line, thereby providing a stable judgment result. The above-described risk level calculation module (140) acquires a grade history buffer, which is a frame-by-frame risk level list calculated during the last 30 frames, and a communication status flag indicating whether an external application program interface call was successful and the response time. As an example, in an unstable situation where the risk level fluctuates between grades 2 and 3, a grade change occurs 5 times per second during real-time judgment, causing confusion at the site; however, it is confirmed and applied that if the 30-frame window of the present invention is applied, no grade change occurs, allowing for stable alarm maintenance.

[0072] According to one embodiment, the risk grade calculation module (140) includes a grade mapping table that calculates risk grades (1 to 3) by inputting an image-based risk score, a heat-based risk score, and a communication status flag. For example, if the heat-based risk score is above a first threshold and the communication status is normal, the risk grade is set to 3; if only the image-based risk score is above a second threshold, the risk grade is set to 2; and if both scores are below the threshold, the risk grade is set to 1. Additionally, the risk grade calculation module (140) counts the “number of grade changes” and the “minimum number of maintained frames” to suppress flicker, and applies a hysteresis rule that delays downgrade when the number of grade changes is above a third threshold.

[0073] The above response control module (150) is a software configuration that controls physical devices and communication interfaces according to a determined risk level, and in particular performs a compensation control function that maximizes the availability of the system. The above response control module (150) performs compensation control logic based on available channels, and in accordance with the law of conservation of total information, if the external communication channel is disconnected or the response time exceeds a threshold and availability converges to zero, it automatically switches to an independent amplification mode that amplifies the output of the field alarm, which is an internal channel, to the hardware limit. In addition, if the alarm is prolonged, it performs control that periodically modulates the frequency pattern to prevent sensory adaptation. The above response control module (150) obtains a network availability index calculated based on the number of call failures and response delay time, and alarm duration timer data, which is the cumulative time the alarm of the current level has been maintained. As an example, in a situation where communication is disconnected simultaneously with the occurrence of a fire, when the communication status is normal, it outputs an alarm of 70 decibels according to the standard manual, but when communication is disconnected, it performs compensation control that amplifies to a maximum of 100 decibels to induce a proxy report by a bystander.

[0074] The above alarm output unit (160) is a hardware device that converts the control signal of the corresponding control module (150) into physical signals, namely light and sound, and is equipped with a high-brightness warning light for visual warning and a speaker for auditory warning. The above alarm output unit (160) does not simply turn on and off, but precisely modulates the output according to the frequency, duty cycle, and tone pattern parameters included in the control signal. In particular, when receiving an independent reinforcement mode, it performs output within the maximum output range allowed by the hardware protection circuit, and performs an environment-adaptive clipping function that limits the maximum output at night in conjunction with ambient light and noise sensor values. The above alarm output unit (160) acquires light sensor and noise sensor data, as well as feedback data on whether the light-emitting diode module is disconnected and the speaker impedance status. As an example, in daytime and noisy environments, the warning light flashes at maximum brightness and the speaker volume is maximum, and in nighttime and quiet environments, the warning light brightness is dimmed to 50 percent and the speaker volume is limited to 80 percent to prevent complaints and glare.

[0075] The above-mentioned learning server unit (170) is a central server that accumulates data generated by the system and updates the artificial intelligence model, and selectively learns noise filtered from the modules and confirmed risk data rather than indiscriminate data collection. The above-mentioned learning server unit (170) performs edge-case-based retraining logic, and, for example, continuously improves the precision of the model by classifying data that is judged as ignored due to identifier switching in the image processing module as training data for improving the tracking model, and classifying and storing data that is identified as a fixed heat source and processed as an exception in the thermal data analysis module (130) as a false alarm prevention dataset. The above-mentioned learning server unit (170) acquires filtering logs, which are original data that is judged as noise or error and blocked in each module, and confirmed events, which are video data before and after an incident that has finally propagated into a dangerous situation. As an example, it deploys a model that improves object tracking maintenance performance during multiple person crossings by retraining videos where identifier switching was frequent, or performs an update to fine-tune the dead zone threshold by learning data where instantaneous high-temperature acceleration, such as welding sparks, occurred but was not a fire.

[0076] According to one embodiment, the learning server unit (170) includes the steps of: configuring a dataset at a set period (e.g., once a week or based on the accumulated amount of events); separating samples determined to be false alarms or missed detections to generate retraining candidates; checking whether verification indicators (false alarm rate, average detection time, number of grade changes) satisfy criteria for the retrained model or threshold set; and distributing an update only if criteria are satisfied, and rolling back to a previous version if criteria are not satisfied. Additionally, the update is performed in the form of correcting at least one of a “weight,” a “sensitivity constant,” a “dead zone threshold,” and a “threshold of a grade mapping table.”

[0077] A system according to one embodiment of the present invention performs a four-step core logic process to verify the physical consistency of sensor data and actively control the variability of the situation through the organic combination of the components described above.

[0078] First, to prevent mismapping caused by physical viewpoint mismatch when fusion of heterogeneous sensor data, the integrity of the input data is verified by performing a 'spatial-temporal consistency-based reliability judgment logic' that dynamically calculates an allowable error based on the camera's frame rate and object movement speed.

[0079] Next, false alarms are suppressed through a 'tracking stability-based severity determination logic' that defines the identifier switching phenomenon occurring during the object tracking process as noise and automatically attenuates detection sensitivity when tracking is unstable by applying the signal-to-noise ratio improvement principle of communication theory.

[0080] Next, by monitoring the acceleration of temperature, which is the second derivative of the temperature rather than a simple temperature threshold, it preemptively detects signs of thermal runaway and realizes a life-saving early warning through a 'dynamic fire risk calculation logic' that assigns weights to proximity to the human body.

[0081] Finally, the system's survivability and responsiveness are maximized through hysteresis control that suppresses frequent fluctuations in risk levels and 'available channel-based compensation control logic' that maximizes field alarm output according to the law of conservation of total information in the event of external communication failure.

[0082] The specific operating principles and logical processing procedures of the four core processes mentioned above are explained in detail below.

[0084] The image data processing module (120) performs a spatiotemporal alignment-based fusion reliability judgment process to prevent malfunctions caused by incorrect mapping by verifying physical errors in advance when fusion data is fused between heterogeneous sensors, namely a visible light camera and a thermal light camera. The image data processing module (120) aims not merely to overlap the images of the two sensors, but to verify whether the two fused data points to the same physical time and space. To this end, it defines and operates three key judgment criteria. The first of the judgment criteria is synchronization time error, which refers to the difference in the physical time when the visible light frame and the thermal light frame were captured, and is judged to be the main cause of positional errors of objects moving at high speed. The second is data transmission continuity, which refers to the number of accumulated frame drops during the process of transmitting data from the sensor to the processor, and is used as a measure to determine whether the real-time nature of the data is lost. The third is spatial mapping deviation, which is determined by calculating the Euclidean distance difference between the center point of the visible light object and the center point of the thermal light source on a calibrated coordinate system.

[0085] The image data processing module (120) does not evaluate the judgment criteria individually, but performs an operation to calculate the overall reliability according to an interrelated logical flow. Specifically, taking into account that there is no problem with fusion if the object is stationary even if a time error exists, but that the faster the object moves, the more minute the time difference is amplified into a large spatial error, the image data processing module (120) applies a logic that imposes a penalty proportional to the value obtained by multiplying the time error by the object's movement speed or frame rate. In addition, the image data processing module (120) determines that if frame drops occur continuously exceeding a specific threshold, the currently received data is highly likely to be a past afterimage, and in this case, regardless of other conditions, it activates a safety device that forcibly sets the fusion reliability to zero and immediately stops the fusion process.

[0086] The above image data processing module (120) performs the operation of acquiring data for each judgment criterion at the hardware level. Specifically, it extracts a timestamp by recording the exposure start time of each frame in microseconds at the field programmable gate array or camera driver level. In addition, it performs the operation of counting the omission of expected frame sequence numbers in the receiving buffer to aggregate the number of frame drops in real time. Furthermore, it acquires spatial data by calculating the bounding box center coordinates of an object extracted through a real image analysis algorithm and the center coordinates of a high-temperature area extracted through thermal image analysis in pixel units.

[0087] According to one embodiment, in order to exclude the influence of pixel size that varies depending on the installation environment, the spatial data is converted into a distance ratio (e.g., center distance / object diagonal) normalized based on at least one of the diagonal length or width and height of the object bounding area, or converted into a real-world distance (e.g., in centimeters) by a transformation matrix calculated through calibration at installation. In this case, the calibration includes the step of calculating a correspondence between a visible light coordinate system and a thermal image coordinate system using a reference pattern or the position of a fixed reference point, and the step of storing the calculated correspondence in memory.

[0088] The above image data processing module (120) utilizes the reliability determination result calculated through the logic as a control signal to determine the operating mode of the system. In the normal mode, where the reliability meets the threshold, the real image and thermal image data are mapped pixel by pixel to perform precise analysis. On the other hand, if the reliability is below the threshold, the coordinate information of the thermal image data is determined to be unreliable and discarded, and the system switches to a standalone mode that analyzes using only the object recognition result of the real image camera, thereby blocking false alarms that mistake an unrelated heat source for a person. In addition, if frame drops persist, a warning mode is executed to send a notification to the administrator requesting a sensor inspection.

[0089] The reliability judgment of the above image data processing module (120) proceeds sequentially according to a time-series procedure. In the first step, when real image and thermal image frames reach the buffer, the difference in timestamps of each frame is calculated, and if the error is within 33 milliseconds corresponding to an interval of about 1 frame, it is determined to pass the first step. In the second step, the number of frame drops accumulated over the last 1 second is checked, and if the accumulated drops are 5 or more, it is considered that the communication status is unstable, and the reliability grade is lowered. In the third step, if the preceding steps are passed, the coordinate distance between the person object detected in the real image and the heat source detected in the thermal image is calculated, and if this distance is located within 20 percent of the object size, it is determined to be a successful match. In the final step, a fusion approval signal is generated only if all of the above steps are satisfied, and a fusion rejection signal is generated if even one is not satisfied.

[0090] The above image data processing module (120) applies a deadband to suppress unnecessary reliability fluctuations by excluding errors caused by minute jitter from the error value, considering them as system noise, in order to ensure the reliability of the logic and handle exceptions. In addition, if a sensor loss situation occurs where the signal from one sensor is not received for more than 1 second, control is performed to immediately switch to a single sensor mode and, upon recovery, return to a fusion mode after a certain stabilization period.

[0091] The above image data processing module (120) performs operations that are differentiated from prior art for various situation scenarios. Specifically, unlike the existing method that generates a false fire alarm by overwriting past high-temperature data at the current location when a sensor delay occurs where thermal image data arrives late, it operates in a standalone mode that detects the time error exceeding limit and excludes the thermal image data to block false alarms. In addition, in the event of a communication overload situation where many frame drops occur, instead of causing analysis errors by forcibly fusing disconnected data, it maintains analysis reliability by detecting the accumulation of drops, stopping the fusion, and waiting for stabilization. Furthermore, to prevent matching failure caused by a heat source being displayed at the afterimage location when an object moves quickly, it verifies accurate mapping by dynamically reducing the error tolerance range by reflecting the movement speed.

[0092] The above image data processing module (120) operates based on simulation results for continuous frame data processing. For example, if the time error exceeds 30 milliseconds and is 45 milliseconds, it is determined that the time error threshold has been exceeded and fusion is rejected. Since absolute pixel standards such as “pixels” in spatial error determination can vary depending on the installation distance and resolution, in one embodiment, it is configured to determine that matching is impossible if the distance ratio normalized to center distance / object size (e.g., bounding diagonal) exceeds the first standard. In addition, if the time error is normal but the cumulative number of drops exceeds 5 and reaches 6, it is determined that the continuity standard is not satisfied and fusion is rejected, and the rejection state is maintained until the drop state is resolved. Even if the spatial mapping distance is measured excessively, such as 150 pixels, and the distance error is large, it is determined that matching is impossible and fusion is rejected, and an approval decision is made only when all standard values ​​return to within the normal range.

[0093] The above image data processing module (120) operates according to a priority-assigned judgment rule table. Data continuity is checked with the highest priority, and if the accumulated drop exceeds a threshold, unconditional blocking is performed because the freshness of the data cannot be guaranteed. Time synchronization is checked with the next highest priority, and if the time error exceeds the frame period, it is determined that physical mapping is impossible due to a mismatch in viewpoint, and fusion is rejected. Next, spatial consistency is checked, and if the mapping distance exceeds the allowed radius, it is considered not to be the same object and processed as a mapping failure, and fusion is approved only when all of the above conditions are passed.

[0094] The judgment criteria adopted by the above-mentioned image data processing module (120) are based on physical laws and technical necessity. Linking the allowable time error range with the movement speed of an object is based on the fundamental law of physics that speed is distance in minutes of time, and using frame drops as a measure of system load is based on network theory that data packet loss causes delay, and this is a deterministic algorithm that operates mechanically based on measured physical data.

[0095] The above-mentioned image data processing module (120) adopts this spatiotemporal consistency judgment logic, thereby providing the effect of fundamentally blocking ghost heat sources or false fire alarms that occur when data with distorted time is forcibly fused. In addition, it achieves the technical effect of minimizing monitoring gaps by enabling partial operation that selectively excludes only low-reliability data without stopping the entire system even if a single sensor is temporarily unstable.

[0096] The above image data processing module (120) defines and uses key terms and reference values ​​for clear operation. The number of synchronized valid frames is defined as the number of frames with a time error of about 33 milliseconds or less among the total received frames, and the number of accumulated frame drops is calculated by aggregating the number of discontinuous sequence numbers received from the sensor interface in a sliding window method in 1-second increments. The threshold value used at this time is not a fixed value, but has an inverse relationship with the frame rate setting value of the installed camera and is automatically calculated and set during system initialization.

[0097] In the present invention, the time error threshold, which serves as the criterion for determining the number of synchronized valid frames, is automatically calculated based on the physical frame rate of the camera sensor. Specifically, the threshold is based on the logic that it must match the physical period obtained by dividing 1 second (1,000 ms) by the number of frames per second of the camera.

[0098] [Logical Formula 1]

[0099] Synchronization threshold 1,000 milliseconds ÷ camera frame rate

[0100] That is, for a 30fps camera, the frame interval of approximately 33ms becomes the physically acceptable maximum error range, and reliability is designed to have an inverse relationship with the time error.

[0101] [Logical Formula 2]

[0102] Fusion reliability ∝ 1 ÷ (1 + (current time error - synchronization threshold))

[0103] The table below shows simulation results illustrating how the physically acceptable synchronization threshold is derived based on changes in frame rate (FPS). (Unit: ms)

[0104]

[0105] This is the result of comparing the malfunction rate when the above-derived threshold (33ms based on 30fps) is applied and when it is not applied (simple fusion).

[0106]

[0108] The aforementioned abnormal behavior and fire indication judgment module performs an abnormal behavior severity determination process with applied tracking stability weights to distinguish tracking errors or transient noise, which inevitably occur during the video analysis process, from actual dangerous situations. The module does not judge a situation as dangerous simply because the movement is large; rather, it aims to verify whether the movement originated from a reliable object and persisted for a valid period of time. To this end, it defines and operates four core judgment criteria. The first criterion, motion intensity, refers to the sum of the inter-frame displacement distances of multiple joint points constituting an object and is defined as a physical signal indicating the intensity of the situation. The second criterion, tracking stability, is an indicator representing whether the object's unique identifier is maintained or frequently changes within the same coordinate area, and is utilized as a measure to judge the noise level of the analysis data. The third criterion, event persistence, refers to the time during which the detected abnormal behavior is maintained and is used as a standard to distinguish between instantaneous movements and actual accidents. The fourth criterion, AI model confidence, refers to the probability value of the behavior class primarily classified by the deep learning model.

[0109] The above-mentioned abnormal behavior and fire sign judgment module performs operations that process the above criteria in a logical order based on the signal-to-noise ratio improvement principle. Specifically, taking into account that data cannot be trusted if the noise in tracking stability is high, even if the signal called operation intensity is very strong, it applies negative feedback logic that subtracts or attenuates the operation intensity score inversely proportional to the number of unique identifier switching cycles. This provides the effect of suppressing false alarms in situations where tracking jumps due to multiple people crossing or being obscured. Furthermore, to prevent the risk score from increasing indefinitely linearly over time and potentially leading to minor actions being misjudged as dangerous situations if they persist for a long time, the above-mentioned abnormal behavior and fire sign judgment module applies logarithmic accumulation logic in which the score increases rapidly during an initial period, but thereafter the rate of increase slows down to converge to a maximum value.

[0110] The above-mentioned abnormal behavior and fire indication judgment module performs the operation of acquiring data for each judgment criterion in real time from the video analysis engine. It extracts the coordinates of 17 joint points through a pose estimation algorithm and calculates the amount of joint change by calculating the sum of pixel distances moved relative to the previous frame. In addition, it obtains switching count data by counting the number of times the object's unique identifier changed within a specific region of interest during the last 1 second. Furthermore, it obtains duration data by measuring the elapsed time in milliseconds from the moment the abnormal behavior flag was switched to true until the present.

[0111] The above-mentioned abnormal behavior and fire sign judgment module utilizes the final severity score calculated through the logic to determine the situation response stage. If the score is below the bottom 20 percent, it primarily corresponds to a situation of unstable tracking; therefore, an ignore stage is applied, treating it as noise and filtering it out without recording it in the logs. If the score is in the middle range, an interest stage is applied, highlighting the corresponding object on the administrator monitoring screen without sounding an alarm. Conversely, if the score exceeds the threshold and tracking stability is ensured, an alarm stage is executed, immediately activating the on-site warning light and saving the event to the server.

[0112] The determination process of the aforementioned abnormal behavior and fire sign judgment module proceeds step-by-step according to specific scenarios. For example, in the case where a person collapses in a crowded passageway, the first step detects a rapid increase in the change of joint coordinates as the object moves quickly toward the floor, thereby calculating a high motion intensity. The second step confirms that the unique identifier is maintained because there are no other pedestrians around the collapsed object, and since the switching count is 0, the original score is maintained without attenuation. The third step amplifies the score by applying double the weight as the deep learning model classifies the motion as a collapse. The fourth step confirms that the collapsed state is maintained for more than 3 seconds and applies duration saturation logic to calculate the score to reach the maximum value of 99 points. Finally, since the severity exceeds the threshold of 80 points, it is confirmed as a dangerous situation, and a command to issue an alarm is transmitted.

[0113] To ensure the reliability of the logic and handle exceptions, the above-mentioned abnormal behavior and fire sign judgment module applies an exception rule in which, if an object is obscured by a pillar or another person within 0.5 seconds and then reappears, causing its identifier to change, it is re-identified as the same object by comparing it with the Kalman filter predicted location, and the switching in this case is excluded from the penalty. Additionally, to prevent the score from increasing when vigorous but regular behaviors, such as a cleaner sweeping, persist for a long time, it executes a repeating pattern suppression logic that forcibly lowers the duration weight when periodicity of the action is detected.

[0114] The aforementioned abnormal behavior and fire indication judgment module produces advanced operational results compared to prior art for various situational scenarios. Unlike existing technology that issues an alarm by mistaking coordinate changes for a violent fight based solely on the amount of coordinate fluctuation in situations where identifiers constantly change and coordinates fluctuate due to tracking errors, this module significantly deducts points upon detecting multiple identifier switching to classify the incident as noise and ignore it. Furthermore, unlike existing technology that issues an immediate alarm in the case of a simple fall where the user gets up within one second, this module terminates the situation at the "concern" stage as the short duration causes the score to drop before reaching a threshold. Conversely, in the case of an actual fainting incident where the user collapses and is unable to get up, it assigns the highest grade, Grade 3, and issues an alarm by reflecting the intensity of the action, AI confidence, and accumulated duration.

[0115] The above-mentioned abnormal behavior and fire indication judgment module determines actions based on a generalized decision matrix. If the identifier jumps due to an unstable tracking status, the data is deemed unreliable and ignored, even if the action intensity is high. If the tracking status is stable and the action intensity is high, but the duration is short (less than 2 seconds), it is classified as a "concern" stage due to the possibility of a temporary impact. However, if the tracking status is stable, the action intensity is high, the duration is long (5 seconds or more), and the AI ​​judgment is "falling over," it is confirmed as a valid accident and classified as a "danger" stage.

[0116] The judgment criteria of the aforementioned abnormal behavior and fire indication detection module are based on objective data rather than the operator's subjectivity. The number of unique identifier switches is a clear value counted as an integer by the image processing engine and serves as an objective indicator of tracking performance. Furthermore, the principle of improving the signal-to-noise ratio—which requires reducing the influence of unreliable data such as noise—is based on fundamental principles of signal processing and control engineering, proving that this design is driven by technical necessity.

[0117] By adopting these components, the above-mentioned abnormal behavior and fire sign judgment module provides the effect of reducing false alarm fatigue by reducing malfunctions caused by wind-blown objects or insects passing in front of the camera—which are the biggest problems in the field—through tracking stability checks. In addition, it achieves the technical effect of quantitatively distinguishing the urgency of accidents, such as simple falls and fainting, by reflecting not only whether an accident has occurred but also its duration.

[0118] The aforementioned abnormal behavior and fire indication judgment module defines and uses key terms and reference values ​​for clear operation. The traceability index is defined as the reciprocal of the number of unique identifier switches that occurred in the last 1 second, and has a maximum value when the switching is zero. Saturation time refers to the inflection point time at which the accumulation of risk scores over time transitions from linear growth to logarithmic growth, and this is automatically set by learning the average traffic patterns of the target area.

[0119] The severity calculation logic of the present invention does not involve the application of arbitrary weights; instead, it incorporates the concept of 'Signal-to-Noise Ratio (SNR)' from communication theory into image analysis and is realized through the following operational logic.

[0120] [Logical Formula 3]

[0121] Final Risk Score = Operation Intensity (Signal) × (1 ÷ (1 + Weight × Number of Unique Identifier Switches (Noise)))

[0122] In the above equation, by applying a 'attenuation filter' algorithm that increases the denominator as the number of identifier switching (noise) increases so that the total score converges to 0, malfunctions caused by noise are mathematically blocked.

[0123] This is data measuring how reliability changes according to the number of ID switches when the same operation intensity (on a 100-point scale) is detected. (Basis for setting the threshold to 80 points)

[0124]

[0125] This is the result of a comparison of false alarm suppression performance that occurs when multiple people cross paths in a crowded environment.

[0126]

[0128] The above-described thermal data analysis module (130) is equipped with a dynamic fire risk calculation logic that dynamically generates a fire risk index by comprehensively considering not only whether a simple temperature threshold is exceeded, but also the acceleration of temperature change and proximity to the human body, which are thermodynamic change trends. The above-described thermal data analysis module (130) aims to detect thermal runaway characteristics, where the temperature rises rapidly in the early stages of a fire, and to evaluate the possibility of human casualties with the highest priority. To this end, it defines and operates three core judgment criteria. The first criterion, acceleration of temperature rise, refers to the amount of change in the rate of temperature change per unit time, and is utilized as a thermodynamic indicator to preemptively predict the fire spread speed and explosion risk rather than a simple high-temperature state. The second criterion, proximity to the human body, refers to the physical distance between the high-temperature area and the human object detected in the actual fire, and is defined as a safety engineering measure to determine the possibility of direct human casualties regardless of the scale of the fire. The third criterion, high-temperature area diffusion, refers to the rate of increase in the area of ​​high-temperature pixel clusters over time and is used as a criterion to determine the physical growth rate of the fire.

[0129] The thermal data analysis module (130) performs calculations to determine the risk level by introducing safety engineering concepts such as the rate of fire spread and the possibility of human casualties based on the above criteria. Specifically, taking into account the thermal runaway characteristic where the temperature rises exponentially rather than linearly in the initial stages of a fire, the thermal data analysis module (130) performs a logic to monitor the temperature rise rate per unit time by differentiating it in real time. If an acceleration section in which the rise rate itself increases is detected, a predictive trigger is activated to immediately raise the risk level to the highest level, even if the current temperature is below a preset threshold. Additionally, even if the physical size of the high-temperature area is small, if the heat source is identified as a person or the mapping distance to a person converges to zero, it is considered a human ignition or burn accident rather than a simple fire. Accordingly, a command is transmitted to determine the response stage by applying a human-first calculation rule that assigns a higher weighting factor to human proximity than to the fire scale.

[0130] The thermal data analysis module (130) performs the operation of acquiring data for each judgment criterion from the thermal imaging sensor and the image analysis module. It collects temperature data for each pixel of the thermal imaging sensor on a frame-by-frame basis and calculates the temperature rise rate in degrees per second by dividing the temperature difference between the previous frame and the current frame by the time interval. For example, if the temperature at a specific point rises from 50 degrees to 60 degrees in 1 second, the rise rate is calculated as 10 degrees per second. In addition, human body proximity data is acquired by calculating the Euclidean distance between the bounding box coordinates of a person object acquired as a result of real-image analysis and the center coordinates of the high-temperature area within the thermal image on a pixel-by-pixel basis. Furthermore, the number of pixels that are above the reference temperature is counted to obtain area data of the high-temperature area in real time.

[0131] The above thermal data analysis module (130) dynamically controls the alarm level of the system based on the determination result. If the acceleration of temperature rise exceeds a threshold, a preliminary alarm is issued immediately even if the current temperature is low, and a preemptive measure is taken to secure the golden time. In situations where human proximity is high, an emergency evacuation notification is immediately transmitted even if the scale of the fire is small, and a signal requesting rescue is sent to the fire department linkage system as a top priority. On the other hand, if the temperature is high but the rate of rise is low and there are no people, it is determined to be simple equipment overheating and operates in a maintenance mode that only sends an inspection notification to the facility manager.

[0132] The processing procedure of the thermal data analysis module (130) proceeds step by step according to a specific scenario. Taking the scenario in which a fire starts in a trash can as an example, the first step detects that the temperature of a specific pixel has risen rapidly from 40 degrees to 45 degrees and calculates that the rate of rise is 5 degrees per second. The second step confirms that the rate of rise was 2 degrees per second in the previous 1 second but has now increased to 5 degrees per second, and determines that this is a precursor to thermal runaway with an accelerating rise. The third step confirms that a human object exists within 50 pixels around the high-temperature area and applies the maximum human injury weighting. The final step confirms that the current maximum temperature is only 50 degrees, which does not reach the general fire standard of 80 degrees, but determines the risk level as an emergency situation based on the accelerating rise and human proximity criteria, and automatically transmits a command to issue an alarm.

[0133] The above thermal data analysis module (130) includes dead zone logic to ensure the reliability of the logic and prevent malfunctions, so that minute temperature fluctuations caused by noise in the sensor itself are not mistaken for a fire. Specifically, minute fluctuations with a temperature rise rate of less than 1 degree per second are considered environmental factors rather than fire precursors and are excluded from the risk calculation. In addition, if there is a fixed heating element that maintains a high temperature normally, such as a heater or monitor, exception processing is performed to block unnecessary alarms by masking the area or applying a relaxed rise rate standard.

[0134] The above thermal data analysis module (130) provides advanced operational results compared to prior art. In the case of prior art, which sounds an alarm only when the temperature reaches a threshold of 80 degrees, there is a problem in that the fire is detected only after it has spread because the initial firefighting time is missed. However, the present invention secures the golden time by detecting the acceleration of the rise even in the early stages when the temperature is low and issuing an alarm immediately. In addition, in a situation where a small ember catches on a person's clothing, prior art fails to detect the fire due to its small scale or classifies it as a low grade, whereas the present invention detects proximity to the human body and immediately raises the risk level to the highest grade to prevent casualties. Furthermore, regarding fixed heat sources such as stoves, prior art generates continuous false alarms, whereas the present invention provides a differentiated effect of maintaining a normal state by confirming that there is no acceleration of the temperature rise.

[0135] The above thermal data analysis module (130) determines a dangerous situation according to a judgment rule table. If the temperature rise acceleration exceeds a threshold, it is immediately classified as a dangerous stage regardless of the current temperature to prepare for thermal runaway. Even if the rise acceleration is low but the distance between the high-temperature area and a person is very close, it is classified as a dangerous stage to prevent casualties. On the other hand, if the temperature is high but the rise acceleration converges to zero and there is no person, it is classified as a caution stage to monitor only whether the equipment is overheating, and if both the temperature and the rise rate are low, it is maintained as a normal stage.

[0136] In the judgment rule of the above-mentioned thermal data analysis module (130), the fact that the initial temperature of a fire rises exponentially is a physical law based on the flashover theory of combustion engineering, and the fact that the risk of burns increases as the distance between the human body and the heat source increases is a basic principle of safety engineering. Therefore, this logic is an algorithm inevitably derived to achieve the clear technical objectives of protecting human life and the physical properties of fire, rather than the subjective judgment of the operator.

[0137] By adopting these components, the above-mentioned thermal data analysis module (130) solves the problem of delayed detection, which is a limitation of existing temperature sensing methods, and provides a technical effect that enables response before the spread of fire. In addition, through a logic that prioritizes the possibility of human casualties over the physical size of the fire, it achieves the effect of precisely identifying dangerous situations that could lead to actual human casualties and strengthening response capabilities.

[0138] The above thermal data analysis module (130) defines and uses key terms and threshold values ​​for clear operation. The thermal runaway threshold is defined as the acceleration inflection point at which the rate of temperature increase per unit time begins to increase non-linearly, and this is set by learning normal temperature change data of the installation environment. The human body proximity limit distance is defined as the effective radius at which the radiant heat of the heat source can burn the human body, and this is dynamically calculated in proportion to the temperature and size of the heat source.

[0139] The 'temperature rise acceleration' applied when determining fire risk tracks not a simple temperature difference, but a change amount that approximates the second derivative of real-time temperature data with respect to time.

[0140] [Logical Formula 4]

[0141] Fire Risk = Current Temperature Weight + (Sensitivity Constant × Hourly Change in Temperature Rate)

[0142] In the above logical expression, the moment the second term, the acceleration term, takes a positive value and exceeds the threshold coincides with the thermodynamic inflection point where 'thermal runaway' begins. This system detects this inflection point and performs a trigger operation to immediately saturate the risk value to the highest level.

[0143] This is data measuring how risk responds preemptively according to the heating rate in an environment with an initial temperature of 40 degrees. (Risk threshold 80 points)

[0144]

[0145] This is the result of a comparison of early detection performance before reaching the set temperature (80 degrees).

[0146]

[0148] The above response control module (150) performs compensation control based on available channels and dual safety control loop logic to prevent malfunction of the field alarm device and to strengthen field response capabilities in the event of external communication interruption. The above response control module (150) aims to maximize the transmission power of the alarm and ensure the reliability of the system, going beyond simply turning the alarm on and off according to the risk level. To this end, it defines and operates three key judgment criteria. The first criterion, grade stability, is an indicator of how consistently the calculated risk level is maintained for a certain period of time and is used as a criterion to prevent flicker phenomena, which are field confusion caused by frequent flashing or changes of the alarm device. The second criterion, communication channel availability, refers to the success of data transmission to an external control server or related agency and the response speed, and is defined as a trigger for compensation control that determines the field alarm intensity. The third criterion, alarm duration fatigue, refers to the time during which the same alarm is maintained and is used as a modulation criterion to prevent auditory and visual adaptation that occurs when exposed to the alarm for a long time.

[0149] The above-mentioned response control module (150) performs calculations to determine control parameters by applying the law of conservation of total amount of information transmission channels and the principle of preventing sensory adaptation based on the above criteria. Specifically, taking into account that frequent on / off switching of the alarm device in a transitional situation where the risk level fluctuates frequently exacerbates confusion at the site, it applies hysteresis control to maintain the mode level within the set voting window. In addition, based on the logic that if it is determined that information transmission to an external network is impossible, the loss of information transmission must be compensated at the site, it performs logic to automatically switch to an independent alarm enhancement mode that forcibly increases the flashing frequency of the warning light at the site and the sound pressure level of the speaker to the maximum. Furthermore, it increases the alarm intensity in proportion to the time the risk level is maintained, but applies a logarithmic scaling method that gradually reduces the increase in intensity or modulates the frequency pattern in consideration of auditory and visual adaptation.

[0150] The above response control module (150) performs the operation of obtaining data for each judgment criterion through system logs and hardware status monitoring. It stores the risk grade history of the last 30 frames set as the grade decision voting window in a buffer and obtains dominant grade data by counting the number of occurrences for each grade. In addition, it obtains communication availability data by measuring the response time occurring when calling an external application interface in milliseconds and counting the number of transmission failures and whether a timeout occurred in real time. Furthermore, it collects basic data for fatigue calculation by measuring the maintenance time of the currently transmitted alarm grade in seconds using a timer.

[0151] The above response control module (150) organically controls the field device and external communication based on the determination result. If the grade stability is not secured, it suspends the immediate grade change and maintains the previous grade to suppress unnecessary flashing of the warning light. In an emergency situation where the availability of the communication channel is lost, it recognizes that reporting to the outside is impossible and transmits a command to amplify the brightness of the warning light and the siren sound to a physical limit so that pedestrians around the scene can recognize the situation and report it on their behalf. In addition, if the alarm is prolonged, it performs control to alert the field personnel by changing the tone of the siren from an intermittent sound to a continuous sound or changing the frequency band.

[0152] The processing procedure of the above response control module (150) proceeds step by step according to a specific scenario. Taking a complex disaster scenario in which a network line is disconnected at the same time as a fire occurs as an example, in the first step, the risk level is calculated as Grade 3 by the fire detection logic, but a phenomenon is detected where the level fluctuates between Grade 2 and Grade 3 due to unstable smoke in the early stages of the fire. In the second step, it is confirmed that the ratio of Grade 3 within the voting window is 70 percent or more, and the level is confirmed as Grade 3 to prevent the alarm device from shaking. In the third step, an attempt is made to automatically report to the fire station server, but it is detected that the response time exceeds 500 milliseconds and fails three consecutive times, so it is determined to be in a state of communication disconnection. In the final step, the risk compensation logic due to the communication failure is activated, and a command is transmitted to execute an independent reinforcement mode that switches the warning light to strobe mode to flash at maximum light intensity and outputs the siren at a maximum sound volume of 100 decibels or more.

[0153] The above-mentioned response control module (150) performs exception processing considering equipment protection and human safety to ensure the reliability and safety of the logic. Even when the independent reinforcement mode is in operation, a limiter is applied to prevent damage to the equipment so as not to exceed the hardware allowable input of the speaker or amplifier. In addition, since maximum volume output may cause complaints or hearing damage in environments with low ambient noise, such as nighttime, environment-adaptive clipping processing is performed to dynamically limit the upper limit of the maximum output in conjunction with the illuminance sensor and noise sensor values.

[0154] The above response control module (150) provides advanced operational results compared to prior art. Unlike prior art, where the alarm device is frequently turned on and off, causing confusion when the risk level is at the boundary, the present invention maintains a stable alarm state through voting-based hysteresis control. Furthermore, unlike prior art, where external reporting fails and the on-site alarm sounds only according to default settings when the network is disconnected, thereby weakening response capabilities, the present invention maintains the overall defense capability of the system by utilizing communication failure as a trigger to reinforce the on-site alarm. In addition, to prevent desensitization to sound during prolonged alarms, the pattern is modified over time to enable continuous evacuation guidance.

[0155] The above-mentioned corresponding control module (150) determines the control method according to a judgment rule table. In the transitional period when the class stability is low, conservative control is performed to suppress class changes and maintain the current state. If the class is stable but communication is interrupted, the field alarm output is set to a weighting factor of 2 times to output the maximum output. If the class is stable and communication is smooth, the rated output according to the standard manual is output, and if the alarm duration exceeds a threshold, it enters a modulation mode.

[0156] The judgment rule of the above-mentioned response control module (150) is an engineering design to ensure the reliability of information transmission by increasing the amount of information in another channel (field alarm) to maintain the total amount when the communication channel is blocked, and modulating the stimulus to prevent sensory adaptation, in which the response decreases when the same stimulus is sustained, is based on the basic principles of cognitive psychology. Therefore, this logic is not merely a simple list of functions, but an algorithm inevitably constructed to achieve the technical objective of reliable information transmission in emergency situations.

[0157] By adopting these components, the above response control module (150) provides a technical effect that ensures independent, field-oriented survivability even in the worst-case scenario of a network failure. In addition, through clear warning delivery without malfunction and continuous alerting, it achieves the effect of maximizing evacuation efficiency and reducing casualties in actual disaster situations.

[0158] The above-mentioned corresponding control module (150) defines and uses key terms and reference values ​​for clear operation. The voting window is a sampling interval to ensure the reliability of the grade determination and is typically set to 30 frames, which correspond to 1 second. The independent reinforcement mode is defined as an emergency operation mode that raises the output of the field device to the maximum hardware specifications when external communication is impossible.

[0159] The voting window size for rating stabilization is set by converting the average time it takes for humans to perceive and react to changes in visual stimuli (about 1.0 second) into the number of camera frames.

[0160] [Setting Criteria]

[0161] Voting Window Size = Recognition Reaction Time (1.0 sec) × Camera Frame Rate

[0162] In addition, the compensation control applied in the event of communication failure follows a complementary operation logic that maximizes the internal channel capacity when the external channel capacity becomes zero, in accordance with the law of conservation of total information.

[0163] [Logical Formula 5]

[0164] Field Alert Intensity ∝ Risk Level ÷ Network Availability Index (The closer the availability index is to 0, the greater the alert intensity)

[0165] This is the result of control in an unstable situation where the risk level alternates between levels 2 and 3 five times in one second.

[0166]

[0167] This is the result of a comparison of responses in a complex disaster situation where the network was disconnected simultaneously with the outbreak of a fire.

[0168]

[0170] The analysis learning and update system of a multi-site artificial intelligence thermal imaging camera-linked security model according to one embodiment of the present invention operates organically as follows, assuming a complex disaster situation in which a fire caused by equipment overheating and a worker's suffocation accident occur simultaneously at an industrial site handling chemical substances.

[0171] In the initial stage, the composite imaging unit (110) acquires visible light images and infrared thermal images in real time targeting piping facilities and workers within the monitoring area. At this time, if a situation occurs where the arrival time of thermal images is delayed due to temporary network congestion, the image data processing module (120) immediately performs a reliability judgment logic based on spatiotemporal alignment on the received data. As a result of internal calculations, when the timestamp error between the real image frame and the thermal image frame exceeds the physical allowable limit of 33 milliseconds and a discontinuity in the frame sequence is detected, the system immediately determines that the fusion reliability is low. Accordingly, the system performs an operation to block primary false alarms by suspending fusion for about 0.5 seconds until the data is normalized and waiting in standby mode, without committing the error of overwriting past thermal image data with the current real image worker location.

[0172] Subsequently, when the network stabilizes and the synchronization error enters within 10 milliseconds, the thermal data analysis module (130) activates the dynamic calculation logic for fire risk based on the matched data. In the scenario, the surface temperature of the pipe is at the 50-degree level and does not reach the fire threshold of 80 degrees. However, the thermal data analysis module (130) detects that a rapid temperature increase acceleration of more than 10 degrees per second is occurring by differentiating the temperature change over the previous 1 second. At the same time, the real-time analysis confirms that a worker is located right next to the high-temperature pipe and that the human body proximity distance is within 10 pixels. Accordingly, the system considers this to be a precursor to thermal runaway and an imminent situation of human casualties, even though the current temperature is low, and performs a calculation to predict and set the fire risk index to the highest level.

[0173] At the same time, the video data processing module (120) performs a tracking stability-based severity determination logic in parallel to analyze the behavior of a worker staggering after inhaling toxic gas. As the worker falls to the floor, a sudden change in joints is detected, and a high motion intensity is calculated. At this time, a situation occurs where other colleagues move urgently around the worker and obstruct the view, causing the worker's object unique identifier to change momentarily. However, the tracking stability filter within the logic operates to attenuate the score of the frame where the identifier switching occurred by considering it as noise, and adopts only the motion in the section where the identifier was stably maintained as valid data. Subsequently, as the falling state persists for more than 3 seconds, a duration saturation logic is applied to finally confirm the worker's fainting state.

[0174] The risk level calculation module (140) calculates the risk level by combining the thermal runaway precursor detection and worker fainting information. At this time, due to smoke and the movement of workers, the risk level calculated for each frame fluctuates unstably between level 2 and level 3. Accordingly, the module applies hysteresis logic to create a voting window of 30 frames that takes into account human cognitive reaction time, and performs an operation to prevent unnecessary shaking of the alarm device by fixing the highest calculated level 3 within that section as the final risk level.

[0175] In the final response phase, the response control module (150) calls an application program interface to make an emergency report to the control center. However, a communication interruption situation is confirmed where the response time exceeds a threshold and transmission failures occur repeatedly due to damage to the communication line caused by the fire. Accordingly, the response control module (150) activates available channel-based compensation control logic and decides to compensate for the loss of information transmission on-site. That is, the system recognizes on its own that an external report is impossible and transmits an independent reinforcement mode command to the alarm output unit (160). As a result, the alarm output unit (160) ignores the standard manual output of 70 decibels when communication is active and activates an alarm sound of 100 decibels, which is the maximum allowed by the hardware protection circuit, and a warning light in high-speed strobe mode. In addition, to prevent workers from becoming desensitized to the sound as the alarm continues, it performs an operation of periodically modulating and transmitting the frequency pattern. Through such coordinated actions, even in isolated situations where external communication is cut off, other workers in the vicinity immediately recognize the severity of the situation and lead to the rescue and evacuation of the collapsed worker.

[0176] Finally, the learning server unit (170) separately classifies and stores identifier switching noise data generated during the above process and acceleration-based fire detection success data at low temperatures. This data is subsequently used in a retraining process to increase the stability of the tracking model and optimize the sensitivity of the fire prediction trigger, thereby continuously enhancing the progressiveness of the system.

[0177] Although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. Various modifications are possible by those skilled in the art without departing from the essence of the invention as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present invention. Explanation of the symbols

[0179] Composite shooting unit (110) Image data processing module (120) Thermal data analysis module (130) Risk grade calculation module (140) Corresponding control module (150) Alarm output unit (160) Learning server unit (170)

Claims

Claim 1 Image data processing modules and thermal data analysis modules installed in multiple surveillance zones; and a learning server unit that acquires data from the image data processing modules and thermal data analysis modules and updates an artificial intelligence model; The above includes, wherein the image data processing module classifies data determined to be ignored due to identifier switching, and the thermal data analysis module classifies data identified as a fixed heat source and processed as an exception, into filtering logs and transmits them to the learning server unit, and finally, data of an event propagated as a dangerous situation is classified as a confirmed event and transmitted, wherein the data determined to be ignored due to identifier switching includes data determined to have an unstable tracking state according to a negative feedback operation in which the motion intensity is calculated through the sum of the joint point movement vectors of an object within the image, and a switching count is calculated as the number of times the object's unique identifier (ID) changes within a preset window time, and the switching count is defined as a noise component that impairs tracking reliability and the motion intensity as an effective signal component, thereby inversely attenuating the motion intensity score as the switching count increases, and wherein the learning server unit acquires the filtering logs and the confirmed events and performs edge case-based retraining logic, wherein the data determined to be ignored due to identifier switching is classified as training data for improving the tracking model, and the data identified as a fixed heat source and processed as an exception is classified as a false alarm prevention dataset and stored, thereby improving the model Continuously improving precision and deploying a model that enhances object tracking performance during multi-person crossovers by retraining images with frequent identifier switching, or performing updates to fine-tune dead zone thresholds by learning from data where instantaneous high-temperature acceleration, such as welding sparks, occurred but was not a fire, configuring datasets at regular intervals, separating samples determined to be false alarms or missed detections to generate retraining candidates, and verifying whether validation metrics satisfy criteria for the retrained models or threshold sets,The above verification indicators include a false alarm rate, average detection time, and the number of class changes; an update is distributed only when the above criteria are satisfied, and a rollback to a previous version is performed when the above criteria are not satisfied; the update is performed in the form of correcting at least one of a weight, a sensitivity constant, a dead zone threshold, and a threshold of a class mapping table; and the update is performed by distributing the model or threshold set updated through the above retraining to the image data processing module and the thermal data analysis module. This characterizes an analysis learning and update system for a security model linked to a multi-site artificial intelligence thermal imaging camera. Claim 2 delete

Images