Integrated Safety Management System for Electric Vehicle Charging Zone Based on Artificial Intelligence

Abstract

The artificial intelligence-based integrated safety management system for an electric vehicle charging area according to the present invention comprises: a camera assembly including a video camera and a thermal camera installed in the charging area of ​​an electric vehicle to acquire images; a temperature sensor attached to an electric vehicle charging gun to detect temperature changes during charging; an AI edge device that generates a site recognition result based on data from the video camera, the thermal camera, and the temperature sensor while including an artificial intelligence model; a guidance unit including an electronic display and a speaker to output an alarm; and a controller including a risk state determination module that determines a risk state of the charging area based on the site recognition result, and an alarm control module that controls the output of an alarm from the guidance unit according to the risk state. According to the AI-based integrated safety management system for electric vehicle charging areas of the present invention, since the AI ​​edge device independently determines the fire risk status of the electric vehicle charging area on-site based on data from a video camera, a thermal imaging camera, and a temperature sensor, the detection function is maintained and real-time response is possible even in the event of a network failure, and the system has the advantage of enabling phased alarm output appropriate to the situation by differentially adjusting the alarm output level according to the risk status.

Description

Technology Field

[0001] The present invention relates to an AI-based integrated safety management system for electric vehicle charging areas, and more specifically, to a system that manages the safety of electric vehicle charging areas in an integrated manner by having an AI edge device installed on-site independently determine the fire risk status of the charging area based on data from a video camera, a thermal imaging camera, and a temperature sensor without relying on a cloud server, and differentially adjusting the alarm output level according to the risk status. Background Technology

[0002] With the rapid expansion of electric vehicle adoption recently, charging areas are being installed in various locations, including apartment complexes, public facilities, and commercial buildings.

[0003] Electric vehicles use lithium-ion batteries as a power source, but thermal runaway can occur in lithium-ion batteries due to causes such as overcharging, external impact, or cell defects. Once thermal runaway begins, it can lead to a fire in a short period of time, causing large-scale damage to surrounding vehicles and facilities.

[0004] In particular, if an electric vehicle fire occurs in a confined underground parking lot or a charging area where many vehicles are densely packed, the combustion spreads rapidly, making it highly likely to lead to casualties.

[0005] Due to these risks, there is a growing need for a safety management system that detects fires in electric vehicle charging areas early and issues alarms.

[0006] Currently, fire detection methods applied to electric vehicle charging areas include single-sensor based approaches, such as smoke and heat sensors, and methods that analyze CCTV footage.

[0007] However, smoke detection sensors have limitations in that it is difficult to detect smoke when it spreads rapidly in a well-ventilated charging area, and heat detection sensors react only after the battery temperature has risen sufficiently, resulting in a delayed detection of the precursor stage of thermal runaway.

[0008] Korean Registered Patent No. 10-2162735 discloses a technology that analyzes video captured by multiple CCTV cameras installed in a building using a deep learning model to recognize objects, flames, and smoke and determine whether there is a fire.

[0009] However, since the above technology relies solely on video cameras, it cannot utilize data from thermal imaging cameras and temperature sensors that detect minute temperature changes occurring during the precursor stages of thermal runaway in electric vehicles, making it difficult to detect the precursor stages of thermal runaway early.

[0010] Furthermore, since the structure processes fire detection results on a remote server, the detection function is interrupted in the event of a network failure, and a delay occurs in the transmission of detection results, making it unsuitable for the initial response to fires in electric vehicle charging areas.

[0011] Furthermore, it lacks the function to differentially adjust alarm output levels according to the severity of the danger state, making it impossible to provide stepwise alarm output appropriate to the situation.

[0012] Therefore, there is a need to develop novel and advanced technology in which AI edge devices, based on data from video cameras, thermal cameras, and temperature sensors, independently detect precursors to thermal runaway in electric vehicles on-site and differentially adjust alarm output levels via electronic displays and speakers according to the severity of the dangerous condition. Prior art literature

[0013] Korean Registered Patent No. 10-2162735 The problem to be solved

[0014] The present invention was devised to overcome the problems of the above technology, and its main purpose is to provide an AI-based integrated safety management system for electric vehicle charging areas, wherein an AI edge device equipped with an AI model based on data from a video camera, a thermal imaging camera, and a temperature sensor independently determines the fire risk status of the electric vehicle charging area on-site, and a controller including a risk status determination module and an alarm control module differentially adjusts the alarm output level through an electronic display and a speaker according to the risk status.

[0015] Another objective of the present invention is to calculate six coefficients of boundary alignment, area diffusion, light source variation, persistence, temperature rise, and charging abnormality to suppress false positives in the artificial intelligence model and to differentially adjust the alarm output level according to the danger state information.

[0016] Another objective of the present invention is to suppress instantaneous risk fluctuations by temporally stably updating risk state information based on the multi-risk source fusion stability, which mathematically fuses six coefficients, and the risk state information of the previous frame. means of solving the problem

[0017] To achieve the above objective, the artificial intelligence-based integrated safety management system for an electric vehicle charging area according to the present invention comprises: a camera assembly including a video camera and a thermal camera installed in the charging area of ​​an electric vehicle to acquire images; a temperature sensor attached to an electric vehicle charging gun to detect temperature changes during charging; an AI edge device that generates on-site recognition results based on data from the video camera, the thermal camera, and the temperature sensor while including an artificial intelligence model; a guidance unit that outputs an alarm including an electronic display and a speaker; and a controller including a risk state determination module that determines a risk state of the charging area based on the on-site recognition results, and an alarm control module that controls the output of an alarm from the guidance unit according to the risk state.

[0018] In addition, the AI ​​edge device further includes a vehicle type classification model that classifies the electric vehicle into a general electric vehicle and a plug-in hybrid electric vehicle based on the exterior image and license plate image of the electric vehicle acquired from the video camera, and further includes the classification result of the vehicle type classification model in the field recognition result; the controller further includes a charging allowance time calculation module that calculates the allowance time for each electric vehicle according to the classification result of the vehicle type classification model; and the alarm control module controls the output of a movement request alarm through the guidance unit for electric vehicles that have exceeded the allowance time.

[0019] In addition, the controller includes an evacuation path calculation module that calculates an evacuation path based on the location of either a person object included in the site recognition result received from the AI ​​edge device and the smoke when a fire occurs, and the alarm control module is characterized by outputting the evacuation path received from the evacuation path calculation module to the electronic display board. Effects of the invention

[0020] According to the artificial intelligence-based electric vehicle charging area integrated safety management system of the present invention,

[0021] 1) Since the AI ​​edge device independently determines the fire risk status of the electric vehicle charging area on-site based on data from video cameras, thermal cameras, and temperature sensors, detection functions are maintained and real-time response is possible even in the event of network failure, and it has the advantage of enabling phased alarm output appropriate to the situation by differentially adjusting the alarm output level according to the risk status,

[0022] 2) By comprehensively evaluating six coefficients—boundary alignment, area diffusion, light source fluctuation, persistence, temperature rise, and charging abnormalities—it suppresses false positives in the artificial intelligence model, thereby preventing unnecessary alarm issuance, and

[0023] 3) By weighted summing the fusion stability of multiple risk sources and the risk state information of the previous frame to update the risk state information in a timely manner, it suppresses instantaneous risk fluctuations and has the effect of enabling reliable risk state judgment. Brief explanation of the drawing

[0024] FIG. 1 is a block diagram illustrating the configuration of the system of the present invention. FIG. 2 is a plan view illustrating the arrangement of the system of the present invention in the configuration of an electric vehicle charging area. Figure 3 is a conceptual diagram illustrating the smoke detection and alarm output process. Figure 4 is a conceptual diagram illustrating the state of displaying an evacuation route on an electronic display board. FIG. 5 is a flowchart illustrating the process of calculating the characteristic quantity of the present invention. FIG. 6 is a flowchart illustrating the risk state correction process of the present invention. Specific details for implementing the invention

[0025] Preferred embodiments of the present invention will be described in detail below with reference to the attached drawings. The attached drawings are not drawn to scale, and the same reference numerals in each drawing refer to the same components.

[0026] The AI-based electric vehicle charging area integrated safety management system of the present invention (hereinafter referred to as the "System") is primarily intended to overcome the limitations of existing cloud server-based video analysis systems, which are unable to process data in real time and suffer from high costs and installation complexity due to BMS integration, by enabling an AI edge device to process data from video cameras, thermal cameras, and temperature sensors in real time at the site without cloud transmission, and by allowing a controller to determine a dangerous state and output an alarm through a guidance unit.

[0027] In other words, the system of the present invention minimizes human casualties by enabling the early detection of precursors to electric vehicle thermal runaway using only existing infrastructure, without the need for integration with a separate BMS, through the controller determining a dangerous state by having an AI edge device generate object recognition results on-site based on data from a video camera, a thermal camera, and a temperature sensor, and by having the controller determine the dangerous state through a dangerous state determination module and an alarm control module.

[0028] Hereinafter, the structure and function of the present invention will be described in detail with reference to the attached drawings.

[0029] FIG. 1 is a block diagram illustrating the configuration of the system of the present invention, and FIG. 2 is a plan view illustrating the arrangement of the system of the present invention in the configuration of an electric vehicle charging area.

[0030] First, the electric vehicle referred to in this invention means a vehicle that drives an electric motor by charging a battery from an external power source, and includes pure electric vehicles (BEVs, Battery Electric Vehicles) that use a battery as the sole power source, and plug-in hybrid electric vehicles (PHEVs, Plug-in Hybrid Electric Vehicles) that use an internal combustion engine and a battery in combination and are capable of external charging.

[0031] The batteries of such electric vehicles are generally composed of lithium-ion cells, and thermal runaway can occur due to causes such as overcharging, over-discharging, external impact, or cooling abnormalities.

[0032] Thermal runaway is a phenomenon in which the internal temperature of a battery cell rises rapidly and propagates in a chain reaction to adjacent cells; since smoke is generated during the precursor stage before ignition, early detection at this stage plays a crucial role in preventing casualties.

[0033] In addition, an electric vehicle charging area refers to a dedicated parking space within a parking lot where an electric vehicle charger is installed, and includes a slow charger or a fast charger, as well as a charging cable and a charging gun connected thereto.

[0034] In accordance with the Act on the Promotion of Development and Dissemination of Environmentally Friendly Vehicles and its enforcement decree, continuous parking of electric vehicles and externally charged hybrid vehicles is prohibited in charging areas for more than 1 hour in rapid charging areas, and for more than 14 hours in the case of electric vehicles and 7 hours in the case of externally charged hybrid vehicles in slow charging areas.

[0035] Since the charging gun remains connected to the vehicle while charging, there is a concern about additional heat generation caused by the charging current in the event of thermal runaway; furthermore, in enclosed underground parking environments, smoke and toxic gases can spread rapidly, potentially leading to severe casualties.

[0036] In order to prevent such risks in advance or to induce rapid evacuation in the event of an actual fire, the system of the present invention installs an image acquisition device and a detection sensor in the charging area of ​​an electric vehicle to detect the precursor stage of thermal runaway in the electric vehicle at an early stage and issue an alarm.

[0037] Specifically, as can be seen from FIG. 1, the system of the present invention is based on including a camera assembly (10), a temperature sensor (20), an AI edge device (30), a guide unit (40), and a controller (100).

[0038] The camera assembly (10) includes an image camera (11) and a thermal imaging camera (12) and is installed on the ceiling or wall within the charging area of ​​an electric vehicle. The camera assembly (10) may be installed such that the image camera (11) and the thermal imaging camera (12) are combined integrally in a single housing or housed in separate housings and positioned in adjacent locations, and may be installed in a pan-tilt type or a fixed type.

[0039] The video camera (11) is a device for capturing vehicles, people, and smoke within the charging area. The video camera (11) may be a CCTV camera, a dome camera, an IP camera, etc. The resolution of the video camera (11) may be set to, for example, Full HD (1920×1080) or higher, but may vary depending on the shooting environment and surveillance range.

[0040] A thermal imaging camera (12) is a device for capturing the temperature distribution around the vehicle surface and the charging gun. As the thermal imaging camera (12), a thermal imaging camera equipped with a non-cooled microbolometer detection element or a thermal imaging camera equipped with a cooled infrared detection element may be used. The temperature resolution of the thermal imaging camera (12) may be selected, for example, in a range of 0.05°C or more and 0.1°C or less, but may vary depending on the required detection precision.

[0041] The images captured by the video camera (11) and the thermal camera (12) of the camera assembly (10) are both transmitted to the AI ​​edge device (30).

[0042] The temperature sensor (20) is a device attached to the electric vehicle charging gun to detect temperature changes during charging.

[0043] The temperature sensor (20) can be installed by attaching it to the outer housing of the charging gun with adhesive or adhesive tape, fixing it with a clamp or band, or embedding it inside the housing of the charging gun.

[0044] Additionally, placing the temperature sensor (20) near the connector where heat from the charging gun cable is concentrated is advantageous in terms of detection precision, but depending on the installation environment, it may be placed in the middle of the cable or on the main body of the charging gun.

[0045] As the temperature sensor (20), a contact-type temperature sensor such as an NTC thermistor, a K-type thermocouple, or a PT100 RTD, or a non-contact infrared temperature sensor based on a thermopile can be used.

[0046] The measurement range of the temperature sensor (20) can be selected, for example, from -40°C to 200°C, but may vary depending on the rated temperature of the charging gun and the purpose of detecting thermal runaway.

[0047] The measurement data of the temperature sensor (20) is transmitted to the AI ​​edge device (30).

[0048] The AI ​​edge device (30) of the present invention is a device that generates field recognition results based on data from a video camera (11), a thermal camera (12), and a temperature sensor (20) while including an artificial intelligence model.

[0049] Existing cloud server-based video analysis systems transmit captured video to a remote server for processing, so there is a limitation in that processing delay of more than 500ms occurs due to transmission delay and object detection accuracy remains below 0.65 based on mAP.

[0050] In contrast, the system of the present invention processes data directly at the charging area site without transmission from the AI ​​edge device (30) to a cloud server, so, for example, it can achieve an inference delay time of 100ms or less and real-time video processing performance of 30 frames per second or more, and the object detection accuracy can be set to 0.85 or higher based on mAP.

[0051] The hardware of the AI ​​edge device (30) can be implemented as an embedded computer equipped with a processor specialized for AI computation, an industrial PC, or a device having equivalent computational performance. The processor may include a GPU (Graphics Processing Unit) specialized for parallel computation, a Neural Processing Unit (NPU) with a low-power structure optimized for deep learning inference, a Vision Processing Unit (VPU) specialized for image processing pipelines, or an AI-dedicated SoC (System on Chip) that integrates a CPU, GPU, and NPU within a single chip.

[0052] At this time, GPUs are advantageous for large-scale matrix operations and have the characteristic of increasing the inference speed of complex deep learning models, NPUs are suitable for continuous AI inference in edge environments with low power consumption, and VPUs have the advantage of being specialized for real-time preprocessing and feature extraction of image data.

[0053] The AI ​​edge device (30) can be installed in a control room, a charging facility box, or a separate waterproof and dustproof enclosure within a parking lot, and can be connected to a video camera (11), a thermal imaging camera (12), and a temperature sensor (20) via a communication means such as wired Ethernet, RS-485, or wireless Wi-Fi / LoRa.

[0054] The artificial intelligence model included in the AI ​​edge device (30) may include an object detection model that detects vehicles, people, and smoke from images captured by a video camera (11), and a temperature anomaly detection model that detects temperature anomalies from temperature distribution data of a thermal imaging camera (12).

[0055] For object detection models, for example, YOLO (You Only Look Once) series models, SSD (Single Shot MultiBox Detector) series models, and EfficientDet series models can be applied.

[0056] YOLO-based models have a structure that divides input images into a grid and simultaneously predicts bounding boxes and class probabilities in each grid cell using a single neural network, offering advantages in high processing speed and real-time object detection.

[0057] SSD (Single Shot MultiBox Detector) series models utilize anchor boxes from feature maps of various scales to detect objects of different sizes in a single pass, and have the advantage of an excellent balance between small object detection accuracy and processing speed.

[0058] EfficientDet family models feature a structure that balances network depth, width, and resolution using complex scaling techniques, enabling them to achieve high detection accuracy within limited computational resources.

[0059] For temperature anomaly detection models, examples such as ResNet-based models and U-Net-based models can be applied.

[0060] ResNet family models are structures that solve the vanishing gradient problem in deep networks through residual connection structures, and they possess characteristics advantageous for classifying abnormal temperature distribution patterns on vehicle surfaces from thermal images.

[0061] U-Net series models utilize an encoder-decoder structure and skip connections to segment abnormal temperature regions within thermal images at the pixel level, offering the advantage of precisely extracting the location and range of these abnormal temperature areas.

[0062] The artificial intelligence model applied to the AI ​​edge device (30) is not limited to this and can be selected according to the environment and required performance of the charging area.

[0063] The field recognition result generated by the AI ​​edge device (30) includes object information of vehicles, people, and smoke detected from the captured video of the video camera (11), and temperature anomaly information detected from the temperature distribution data of the thermal camera (12).

[0064] Object information for vehicles, people, and smoke may include the location, type, size, and confidence score of each object, which is derived by an object detection model performing bounding box regression and class classification on the input image. The confidence score of an object is calculated in a range of 0 to 1, and, for example, only objects with a confidence score of 0.7 or higher may be included in the field recognition results.

[0065] The smoke object information is calculated by an object detection model detecting smoke pixel clusters within the input image at the pixel level, and includes the location, area, coordinates of the circumscribed rectangle, and confidence score of the smoke pixel clusters.

[0066] For example, the location of a smoke pixel cluster is calculated using the center point coordinates of the detected cluster, the area using the number of pixels constituting the cluster, and the coordinates of the circumscribed rectangle using the vertex coordinates of the smallest rectangle containing the detected cluster.

[0067] The confidence score can be calculated as a class classification probability through a softmax function, for example, only smoke pixel clusters with a confidence score of 0.6 or higher can be included in the field recognition result.

[0068] The temperature anomaly information may include the location, area, and temperature value of the abnormal temperature area detected by the temperature anomaly detection model of the thermal imaging camera (12) analyzing the temperature distribution of the area of ​​interest around the vehicle, and the rate of change of temperature per hour of the charging gun detected by the temperature sensor (20).

[0069] In this way, the field recognition result is generated in real-time at the charging area site, including the various detailed information described above, and transmitted to the controller (100) without transmission to a cloud server. As a result, it has the advantage of detecting the precursor stage of electric vehicle thermal runaway in real-time and issuing an alarm quickly without cloud transmission delay.

[0070] In addition, as will be described later, the raw temperature distribution data of the thermal imaging camera (12) and the hourly temperature change rate data of the temperature sensor (20) can be transmitted to the controller (100) separately from the field recognition results and used for precise determination of the dangerous state.

[0071] Meanwhile, the AI ​​edge device (30) can perform additional functions such as transmitting event video captured by the camera assembly (10) to the NVR (Network Video Recorder) and delivering a real-time alarm to the administrator terminal when a fire event occurs, which will be described later.

[0072] The guidance unit (40) is a device that outputs an alarm including an electronic display (41) and a speaker (42), and is installed in a location where the user can easily recognize it, such as an entrance, ceiling, or wall within the charging area of ​​the electric vehicle.

[0073] The display board (41) is a device that visually outputs charging zone status information, fire alarms, and evacuation guidance, and, for example, an LED display board, an LCD display, or an electronic information board may be used.

[0074] Such an electronic display (41) can output an alarm using visual means such as text, color, and flashing, and the output level can be adjusted differentially according to the result of judging the danger state.

[0075] For example, when multiple risk levels are set, attention is drawn with yellow text and slow flashing as the risk level is lower, the alarm level is raised with orange or red text and fast flashing as the risk level increases, and at the confirmed fire level, evacuation guidance information including evacuation routes is updated and displayed.

[0076] The number of these risk levels and the output settings for each level may vary depending on the size of the charging area and the operating environment.

[0077] The speaker (42) is a device that audibly outputs alarm voice and evacuation guidance voice, and, for example, a general speaker, a directional speaker, or an explosion-proof speaker may be used.

[0078] The output level of such speakers (42) can also be adjusted differentially according to the judgment of the danger state. For example, when the danger level is low, a low-volume alarm sound of 70 dB or less is output, and when the danger level is high, an evacuation guidance voice is output along with a medium-volume alarm sound of 80 dB or more or a high-volume alarm sound of 90 dB or more.

[0079] The output level of the speaker (42) and the type of alarm sound may also vary depending on the size and operating environment of the charging area.

[0080] In the present invention, the alarm is a concept that includes the visual output of the electronic display board (41) and the auditory output of the speaker (42), and the output level is differentially adjusted according to the magnitude of the risk, the rate of change, and the fusion result calculated from a plurality of detection means. For example, if the risk level rises rapidly in a short period, the output level may be rapidly increased, and if the risk level decreases for a certain period of time or longer, the output level may be gradually lowered. Detailed information regarding the differential adjustment of the alarm output level will be described later in the description of the controller (100).

[0081] The controller (100) of the present invention is a device that determines the danger state of a charging area based on the field recognition result received from the AI ​​edge device (30) and controls the alarm output of the guidance unit (40) according to the determined danger state.

[0082] The AI ​​edge device (30) described above is a device optimized for real-time processing of video, thermal images, and temperature data by being equipped with high-spec AI computing processors such as GPUs and NPUs. However, when it simultaneously handles danger state judgment and alarm output control, there is a concern that the real-time performance of generating field recognition results may be degraded due to the concentration of computational load.

[0083] Accordingly, the system of the present invention adopts a structure in which the AI ​​edge device (30) is responsible for generating field recognition results, and the controller (100) is responsible for determining the danger state and controlling the alarm output, thereby having the characteristic of simultaneously securing independent optimization and real-time capability for both processes.

[0084] In addition, even if a failure occurs in the AI ​​edge device (30), the controller (100) can maintain the last risk state judgment result and continuously output an alarm, thereby improving the safety of the system, and the update of the AI ​​model and the update of the risk state judgment algorithm can be performed independently, making it possible to increase the convenience of maintenance.

[0085] The hardware of the controller (100) can be implemented as an embedded controller suitable for real-time computation processing, an industrial PC, a PLC (Programmable Logic Controller), or a device having equivalent computational performance.

[0086] The software of the controller (100) consists of a data processing program responsible for receiving and processing field recognition results, a danger state judgment algorithm, and alarm output control logic. The software can be implemented in a programming language such as, for example, C / C++ or Python, and can be operated in a real-time operating system (RTOS) or Linux-based embedded operating system environment.

[0087] The controller (100) may be implemented as an integrated unit with the AI ​​edge device (30) or as a separate device physically separated from the AI ​​edge device (30).

[0088] When implemented as an integrated unit, for example, AI inference and control processing functions can be integrated into a single embedded board to exchange data via internal bus communication, thereby enabling the reception of field recognition results and the determination of risk conditions without external communication delays.

[0089] In addition, if they are implemented as separate devices, for example, by connecting them via wired Ethernet or RS-485 communication, they can be implemented in a form that allows for independent maintenance and functional expansion of each device.

[0090] Figure 3 is a conceptual diagram illustrating the smoke detection and alarm output process.

[0091] Specifically, the controller (100) includes a danger state determination module (110) and an alarm control module (120).

[0092] The danger state determination module (110) performs the function of determining the danger state of the charging area based on the field recognition results received from the AI ​​edge device (30).

[0093] The determination of a dangerous state by the dangerous state determination module (110) can be made by considering, either individually or in combination, image-based object information included in the field recognition result, temperature anomaly information from the thermal imaging camera (12), and temperature change rate information from the temperature sensor (20).

[0094] For example, the danger state determination module (110) may be applied in such a way that it determines the danger state as a caution level when only smoke pixel clusters are detected and there are no temperature abnormalities, determines it as a warning level when smoke pixel clusters and temperature abnormalities around the vehicle are detected together, and determines it as a danger level when smoke pixel clusters, temperature abnormalities and rapid temperature change rates of the charging gun are all detected.

[0095] At this time, the risk level, which is an indicator specifying the risk state, may be set to three levels of caution, warning, and danger, four levels of normal, caution, warning, and danger, or five or more levels that are more detailed.

[0096] Transitions between risk levels can be determined by reflecting not only the detection results of the current frame but also the accumulated state of the previous frame, and a structure can also be applied in which a reverse transition to a lower risk level occurs under certain conditions when the risk state is resolved.

[0097] The danger state determination module (110) performs the function of determining the danger state based on the field recognition results and transmitting the determination result to the alarm control module (120).

[0098] Furthermore, the danger state judgment module (110) can perform an additional judgment algorithm that suppresses false positives step by step by combining and accumulating coefficients calculated from multiple detection means, and details regarding this will be described later.

[0099] The alarm control module (120) performs the function of controlling the alarm output of the display panel (41) and speaker (42) of the guide unit (40) according to the danger state received from the danger state determination module (110).

[0100] The alarm control module (120) can control the electronic display (41) in relation to the alarm output of the electronic display (41), for example, by using yellow text and slow flashing to draw attention when the risk level is low, and for another example, by using orange or red text and high-speed flashing together and displaying a fire alarm message as the risk level increases.

[0101] For example, if the risk level is set to 3, the alarm control module (120) can control the display board (41) by sequentially displaying yellow flashing in level 1, orange flashing and a warning message in level 2, and red flashing, a fire alarm message, and an evacuation route in level 3. If the risk level is set to 4, the alarm control module (120) can control the display board (41) so that a more detailed alarm output is achieved by varying the color, flashing speed, and displayed message for each level.

[0102] The alarm control module (120) can control the speaker (42) in relation to the alarm output of the speaker (42) in such a way that, for example, when the risk level is low, only an alarm sound of 70 dB or less is output, and as the risk level increases, an alarm sound of 80 dB or more is output along with voice guidance.

[0103] As another example, in a confirmed fire level, the alarm control module (120) can control the speaker (42) in a manner that repeatedly outputs a voice guiding the evacuation direction and exit location along with an alarm sound of the highest volume.

[0104] The alarm control module (120) may be operated in such a way that, when the dangerous condition is resolved, the output level of the alarm is not immediately cut off, but is gradually lowered according to the reversal of the danger level.

[0105] Furthermore, the alarm control module (120) can perform an additional control function to more precisely differentially adjust the output level of the alarm by reflecting the magnitude and rate of change of the risk, and details regarding this will be described later.

[0106] The system of the present invention may additionally include a Network Video Recorder (NVR) and an administrator terminal.

[0107] The NVR is a device that automatically stores and manages video captured by the camera assembly (10) and transmitted through the AI ​​edge device (30) when a fire event occurs, and can be installed in a control room or a separate server room within the charging area of ​​the electric vehicle.

[0108] The NVR can perform an event trigger saving function that automatically saves video before and after a fire event, and can be configured to automatically save video from 30 seconds before the event to 5 minutes after the event, but is not limited to this.

[0109] The storage capacity of the NVR can be set to, for example, 4TB or more, and the video stored in the NVR can be used for fire cause analysis, accident investigation, and legal evidence. The NVR can be connected to an AI edge device (30) and an administrator terminal via a wired Ethernet or wireless network.

[0110] The administrator terminal is a device that receives real-time information on the danger status and fire events of the charging area, and can be implemented as a smartphone, tablet PC, or laptop.

[0111] The administrator terminal can receive real-time alarm notifications by risk level, event occurrence location, captured video, and temperature data from the AI ​​edge device (30) or controller (100) through a dedicated application or web interface.

[0112] The administrator terminal can be configured, for example, to send push notifications when a warning level or higher occurs, and to simultaneously issue SMS and phone alarms when a confirmed fire level occurs, but is not limited to this. The administrator terminal links received event information with video stored in the NVR to enable real-time verification and necessary actions from a remote location.

[0113] Below, Example 1, which is an example of operation for the system of the present invention, and Comparative Example 1, which is an existing cloud server-based image analysis system, are set up and the results are compared.

[0114] <Example 1>

[0115] Assume a situation where one electric vehicle is slow-charging in an underground parking lot charging area. About 2 hours after charging began, smoke started to appear from the battery cell of the electric vehicle as a precursor to thermal runaway, and in the video captured by the video camera (11) at that time, the area of ​​the smoke pixel cluster was calculated to be 150 pixels and the reliability score was 0.72.

[0116] The thermal imaging camera (12) detected the temperature of the vehicle's undercarriage as 68℃, and the temperature sensor (20) measured the rate of temperature change of the charging gun as 0.15℃ / s.

[0117] In the system of the present invention, the AI ​​edge device (30) includes the field recognition result because the reliability score of the smoke pixel cluster 0.72 satisfies the valid detection criterion of 0.6 or higher, and transmits the field recognition result to the controller (100) within 100ms.

[0118] The danger state determination module (110) of the controller (100) determined a warning level by combining the detection of smoke pixel clusters and the temperature abnormality information from the thermal imaging camera (12), and the alarm control module (120) displayed an orange flashing and warning message on the display board (41) and output an 80dB alarm sound and voice guidance through the speaker (42).

[0119] The total time taken from the time smoke occurred until the alarm was issued was measured to be approximately 150ms.

[0120] <Comparative Example 1>

[0121] Comparative Example 1 applied the same input data as Example 1. The system of Comparative Example 1 is structured to transmit captured video from a video camera to a remote cloud server for processing, and due to transmission delay, it took more than 580ms from video reception to completion of processing.

[0122] The object detection accuracy of Comparative Example 1 was 0.63 based on mAP, and it failed to detect in the precursor stage where the area of ​​the initial smoke pixel cluster was small.

[0123] The fixed threshold-based temperature judgment system of Comparative Example 1 failed to determine that it was a fire at 68°C, which was below the set threshold of 75°C, and issued an alarm only after about 3 minutes had passed until the temperature exceeded the threshold.

[0124] As a result of comparing Example 1 and Comparative Example 1, the system of the present invention issued an alarm within 150ms during the thermal runaway precursor stage, whereas Comparative Example 1 issued an alarm only after about 3 minutes had elapsed in the same situation, confirming that the system of the present invention is significantly superior in early fire detection and rapid alarm issuance.

[0125] In summary, the system of the present invention provides the characteristic that an AI edge device (30) generates on-site recognition results in real time at the site without cloud transmission, and a controller (100) determines a dangerous state based on this, thereby detecting the precursor stage of thermal runaway in an electric vehicle early with a significantly shorter delay time compared to existing cloud-based systems and issuing a warning quickly.

[0126] Furthermore, in order to precisely detect smoke generated during the pre-heat runaway stage of an electric vehicle by distinguishing it from ordinary dust or water vapor, the AI ​​edge device (30) may further include a smoke pattern recognition model.

[0127] The smoke pattern recognition model is an artificial intelligence model that detects smoke in the pre-heat runaway stage of an electric vehicle by analyzing the shape change and movement patterns of frame-by-frame smoke pixel clusters acquired from a video camera (11).

[0128] Models such as ConvLSTM (Convolutional Long Short-Term Memory) series models, 3D CNN (3-Dimensional Convolutional Neural Network) series models, or optical flow-based models can be applied as such smoke pattern recognition models.

[0129] ConvLSTM family models are structures that combine the spatial feature extraction function of convolutional neural networks with the time-series memory function of LSTMs, and are advantageous for distinguishing pre-thermal runaway smoke by simultaneously learning the shape change and movement direction of smoke pixel clusters between frames.

[0130] 3D CNN-based models are structured to process consecutive frames as 3D inputs and extract spatiotemporal features in a single network, making them suitable for capturing patterns of thermal runaway smoke that spreads rapidly within a short period of time.

[0131] The optical flow-based model is structured to quantify the movement speed and direction of smoke pixel clusters by extracting pixel movement vectors between adjacent frames, enabling precise identification of the movement patterns of thermal runaway precursor smoke spreading toward vehicle boundaries.

[0132] The smoke pattern recognition model is not limited to this and can be selected depending on the environment of the charging area and the required detection precision.

[0133] The detection result of the smoke pattern recognition model is included in the field recognition result and transmitted to the danger state judgment module (110) of the controller (100), and the danger state judgment module (110) reflects the detection result of the smoke pattern recognition model in the danger state judgment.

[0134] As a result, the danger state determination module (110) can quickly determine the danger state by detecting early smoke in the pre-thermal runaway stage, which is difficult to distinguish with only a general object detection model.

[0135] Meanwhile, since the allowed charging times for electric vehicles and plug-in hybrid electric vehicles within the charging area are different, the AI ​​edge device (30) may further include a vehicle type classification model that distinguishes the type of electric vehicle and transmits it to the controller (100).

[0136] Electric vehicles can be classified into conventional electric vehicles and plug-in hybrid electric vehicles depending on the power source and charging method.

[0137] A standard electric vehicle is an electric vehicle that uses only a battery as a power source and charges only with an external power source, and an allowable charging time of 14 hours applies in slow charging zones and 1 hour in fast charging zones.

[0138] Plug-in hybrid electric vehicles are electric vehicles that use an internal combustion engine and a battery in combination and can be charged externally, with an allowable charging time of 7 hours in slow charging zones and 1 hour in fast charging zones.

[0139] In other words, the AI ​​edge device (30) includes a vehicle type classification model so that the controller (100) can apply different allowable charging times for each vehicle type according to the characteristics of each type of electric vehicle.

[0140] Specifically, the vehicle type classification model is an artificial intelligence model that classifies the type of electric vehicle into such general electric vehicles and plug-in hybrid electric vehicles based on the exterior image and license plate image of the electric vehicle acquired from the video camera (11).

[0141] ResNet-based models, EfficientNet-based models, or MobileNet-based models can be applied as such vehicle classification models.

[0142] ResNet-based models are advantageous for precisely classifying detailed features, such as the location and shape of charging ports on the exterior of electric vehicles and the shape of battery packs on the underside of the vehicles, through their residual connection structure.

[0143] EfficientNet family models are structures that accurately classify color and character information in license plate images using complex scaling techniques with low computational resources; for instance, they are suitable for distinguishing between standard electric vehicles and plug-in hybrid electric vehicles by extracting color information from blue license plates assigned to electric vehicles.

[0144] MobileNet family models are lightweight models that significantly reduce computational load through a depth-separated convolutional structure, making them suitable for classifying electric vehicle types in real-time processing environments.

[0145] For example, in cases where license plate recognition is difficult, such as in a nighttime environment, the shape of the charging port in the electric vehicle exterior image is applied first. If, for instance, the classification results of the exterior image and the license plate image differ, the reliability scores of the two results can be compared, and the result with the higher reliability can be confirmed as the final classification result.

[0146] The vehicle classification model is not limited to this and can be selected depending on the environment of the charging area and the required accuracy.

[0147] In response to this, the controller (100) of the present invention may further include a charging allowable time calculation module (130) that calculates an allowable charging time for each electric vehicle according to the classification result of a vehicle type classification model.

[0148] The charging allowance time calculation module (130) calculates the allowance time of the electric vehicle by matching the classification result transmitted from the vehicle type classification model with the type of charging zone (slow / fast), and records the time of entry of the electric vehicle into the charging zone to calculate the difference from the current time in real time.

[0149] For example, when a standard electric vehicle enters a slow charging zone, the allowable charging time is calculated as 14 hours, and when a plug-in hybrid electric vehicle enters a slow charging zone, it is calculated as 7 hours, and in the case of a fast charging zone, it is calculated as 1 hour regardless of the vehicle type.

[0150] Accordingly, the alarm control module (120) can determine whether the allowable charging time transmitted from the charging allowable time calculation module (130) has been exceeded by comparing the allowable charging time with the actual parking elapsed time, and can control the electric vehicle that has exceeded the allowable charging time to output a movement request alarm through the guidance unit (40).

[0151] For example, if a standard electric vehicle exceeds 14 hours in a slow charging area, the alarm control module (120) can control the display board (41) by flashing the message "Charging complete - Move request" in yellow and outputting a move request voice through the speaker (42).

[0152] As another example, if a plug-in hybrid electric vehicle exceeds 7 hours in a slow charging zone, the alarm control module (120) can control the vehicle by displaying a message indicating the allowed charging time for each vehicle type with an orange flash on the display board (41) and repeatedly outputting a voice requesting movement through the speaker (42).

[0153] As a result, the alarm control module (120) has the advantage of enabling efficient operation of the charging area by quickly issuing a movement request alarm to electric vehicles that have exceeded the allowed charging time for each vehicle type.

[0154] Figure 4 is a conceptual diagram illustrating the state of displaying an evacuation route on an electronic display board.

[0155] As can be seen from FIG. 4, in order to minimize casualties within the charging area in the event of a fire, the controller (100) may further include an evacuation path calculation module (140) that calculates an evacuation path based on the location of human objects and smoke included in the field recognition result.

[0156] The evacuation path calculation module (140) receives the location of a person object included in the field recognition result and the location of a smoke pixel cluster as input based on the planar map information of the charging area and performs the function of calculating the evacuation path to the exit.

[0157] The evacuation path calculation module (140) can calculate the evacuation path by applying a shortest path search algorithm, such as the A* (A-Star) algorithm or the Dijkstra algorithm.

[0158] The A*(A-Star) algorithm is designed to find the shortest path from a starting point to a target point. It operates on the principle of prioritizing the exploration of the node that minimizes the sum of the actual travel cost from the starting point to the current node and the estimated cost from the current node to the target point. It is advantageous for rapidly calculating the shortest path by reducing unnecessary searches in environments where the target point is clear.

[0159] Dijkstra's algorithm is designed to find the shortest path from a starting point to all adjacent nodes. It operates on the principle of prioritizing the visit of the node with the shortest distance found so far and iteratively updating the distances to adjacent nodes. It is suitable for simultaneously calculating the shortest paths to all exits in a charging area with multiple exits.

[0160] Specifically, when only a person object is detected and no smoke is detected, the evacuation path calculation module (140) calculates the shortest path to the nearest exit within the charging area starting from the location of the person object.

[0161] For example, if a person object is located in the center of the charging area, the evacuation path calculation module (140) calculates a path toward the shortest distance exit from that location.

[0162] If only smoke is detected and no human objects are detected, the evacuation path calculation module (140) sets the location of the smoke pixel cluster as a danger zone and calculates in advance an evacuation path in the direction of the exit that avoids the danger zone.

[0163] For example, if smoke is detected in the northern section of the charging area, the evacuation path calculation module (140) calculates a path to the southern or side exit, excluding the northern exit.

[0164] When both a person object and smoke are detected, the evacuation path calculation module (140) calculates the shortest path to the exit that bypasses the smoke area by setting the location of the person object as the starting point and the location of the smoke pixel cluster as the avoidance zone.

[0165] For example, if a person object is located near a smoke-generating zone, the evacuation path calculation module (140) calculates a detour path toward an exit in a direction free of smoke.

[0166] In response to this, the alarm control module (120) controls the evacuation route received from the evacuation route calculation module (140) to be displayed on the electronic display board (41).

[0167] In the example described above, when a person object is located in the center of the charging area and no smoke is detected, the alarm control module (120) controls the display board (41) to output an evacuation path toward the corresponding exit along with a phrase such as "Please move to the nearest exit."

[0168] If smoke is detected in the northern section of the charging area and no human object is detected, the alarm control module (120) can control the display panel (41) to output an evacuation path toward a safe exit along with a message such as "Northern exit danger - evacuate to the southern exit."

[0169] Additionally, when a person object is located near a smoke-generating area, the alarm control module (120) controls the display board (41) to update and output a path to bypass the smoke-generating area in real time, along with a message such as "Fire occurred - evacuate immediately."

[0170] In this way, the evacuation route calculation module (140) calculates the optimal evacuation route by reflecting the location of human objects and smoke within the charging area in real time when a fire occurs, and the alarm control module (120) outputs this to the electronic display board (41), thereby providing the advantage of inducing rapid evacuation of users of the charging area.

[0171] In the enclosed parking environment of electric vehicle charging areas, there is a possibility that AI models may misidentify smoke precursors to thermal runaway or overlook actual dangerous situations due to various environmental factors, such as rapid changes in headlight light sources during vehicle entry and exit, smoke-like patterns of vehicle exhaust and dust, and glare from vehicle surfaces.

[0172] This is an inherent limitation stemming from the AI ​​model's reliance on visual patterns within a single frame and its black-box judgment structure; when a danger state is determined solely based on the on-site recognition results described earlier, conflicting risks coexist, such as unnecessary alarm issuance due to false positives or the failure to detect an actual fire.

[0173] Accordingly, the system of the present invention does not stop at applying the field recognition results of the artificial intelligence model directly to the danger state determination, but proposes a post-processing algorithm in which the danger state determination module (110) of the controller (100) mathematically fuses coefficients calculated from a plurality of detection means and accumulates them over time to suppress false positives step by step.

[0174] That is, the risk state judgment module (110) of the controller (100) can perform the function of gradually increasing the reliability of the risk state through a series of post-processing steps of fusion, accumulation, and correction, by calculating so-called feature values ​​by combining six coefficients—boundary alignment, area diffusion, light source variation, persistence, temperature rise, and charging abnormality—in order to compensate for the uncertainty of the artificial intelligence judgment result.

[0175] Figure 5 is a flowchart illustrating the process of calculating the characteristic quantity of the present invention.

[0176] Specifically, the risk state determination module (110) includes a data receiving unit (111), a boundary alignment calculation unit (112), an area diffusion calculation unit (113), a light source fluctuation calculation unit (114), a persistence calculation unit (115), a temperature rise calculation unit (116), a charging abnormality calculation unit (117), and a risk state generation unit (118).

[0177] The data receiving unit (111) receives field recognition results from the AI ​​edge device (30) and receives measurement data from the thermal imaging camera (12) and the temperature sensor (20) of the camera assembly (10).

[0178] For example, the data receiving unit (111) can receive field recognition results from the AI ​​edge device (30) via wired Ethernet, RS-485, or USB communication, and receive measurement data from the thermal imaging camera (12) and the temperature sensor (20) via ONVIF protocol-based network communication or analog signal conversion.

[0179] The reason the data receiving unit (111) receives the field recognition result and the measurement data from the thermal imaging camera (12) and the temperature sensor (20) separately is that it is difficult to exclude false positives caused by changes in the light source of the parking lot environment or exhaust gas with only the field recognition result, and to increase the reliability of the danger state judgment by receiving the measurement data from the thermal imaging camera (12) and the temperature sensor (20) together and independently calculating the temperature rise coefficient and the charging abnormality coefficient.

[0180] The boundary alignment calculation unit (112) calculates the boundary alignment coefficient B_align[n] using the following mathematical formula 1 based on the movement vector of the smoke pixel cluster included in the field recognition result transmitted from the data receiving unit (111) and the outer normal vector of the vehicle mask boundary point.

[0181] Mathematical formula 1.

[0182]

[0183] Here, B_align[n] is the boundary alignment coefficient of the nth frame acquired from the image camera, v_smoke[n] is the movement vector between the center point of the smoke pixel cluster of the nth frame and the center point of the smoke pixel cluster of the n-1th frame, v_boundary[n] is the outer normal vector of the vehicle mask boundary point closest to the center point of the smoke pixel cluster of the nth frame, and ε is a reference value to prevent the denominator from becoming zero.

[0184] B_align[n] is a coefficient that quantifies how much the direction of movement of a smoke pixel cluster in the nth frame acquired from the video camera (11) matches the outer normal direction of the vehicle mask boundary point. Since the smoke from the thermal runaway of an electric vehicle spreads from inside the vehicle to the outside, the direction of movement of the smoke in the nth frame shows a high degree of agreement with the outer normal direction of the vehicle boundary, whereas smoke caused by external environmental factors such as exhaust gas and dust in the parking lot has the characteristic of moving regardless of the vehicle boundary.

[0185] Therefore, the boundary alignment calculation unit (112) can distinguish smoke caused by external environmental factors that the artificial intelligence model may misidentify from smoke that is a precursor to thermal runaway by calculating B_align[n].

[0186] Mathematical formula 1 is a formula that calculates the cosine similarity between the direction of movement of the smoke pixel cluster and the outer normal direction of the vehicle mask boundary point, and the value of B_align[n] is calculated in the range of -1 to 1.

[0187] When B_align[n] is close to 1, it indicates a pattern of smoke spreading from inside the vehicle to the outside, indicating a high possibility of thermal runaway; when B_align[n] is close to 0, it indicates a pattern of smoke movement unrelated to the vehicle boundary, indicating a high possibility of smoke caused by external environmental factors; and when B_align[n] is negative, it indicates a pattern of smoke flowing backward toward the vehicle, indicating smoke entering from the outside.

[0188] For a numerical example, if the motion vector of the smoke pixel cluster in the nth frame is v_smoke[n] = (3, -2) and the outer normal vector of the nearest vehicle mask boundary point is v_boundary[n] = (4, -3), the inner product is 3×4 + (-2)×(-3) = 12 + 6 = 18, and = √(9+4) = √13 = approximately 3.606, and Since = √(16+9) = √25 = 5, the boundary alignment calculation unit (112) calculates B_align[n] = 18 / max(3.606×5, 1×10-6) = 18 / 18.03 = approximately 0.998, indicating a pattern with a high probability of thermal runaway, where smoke spreads strongly outward from the vehicle boundary.

[0189] On the other hand, when the translation vector is v_smoke[n] = (3, 1) and the normal vector is v_boundary[n] = (4, -3), the inner product is 3×4 + 1×(-3) = 12 - 3 = 9, and = √(9+1) = √10 = approximately 3.162, and Since = 5, the boundary alignment calculation unit (112) calculates B_align[n] = 9 / max(3.162×5, 1×10-6) = 9 / 15.81 = approximately 0.569, indicating that the direction of movement of the smoke does not clearly coincide with the outer normal direction of the vehicle boundary, and thus indicates a pattern that may be a false positive due to external environmental factors such as changes in the light source.

[0190] The area diffusion calculation unit (113) calculates the area diffusion coefficient D_area[n] based on the area change of the smoke pixel cluster included in the field recognition result transmitted from the data receiving unit (111).

[0191] As an example, the area diffusion calculation unit (113) can calculate D_area[n] using the following formula a.

[0192] Formula a.

[0193] D_area[n] = (A[n] - A[n-1]) / max(A[n-1], ε)

[0194] Here, D_area[n] is the area diffusion coefficient of the nth frame acquired from the video camera (11), A[n] is the area (pixels) of the smoke pixel cluster of the nth frame, A[n-1] is the area (pixels) of the smoke pixel cluster of the n-1th frame, and ε is a reference value to prevent the denominator from becoming zero.

[0195] D_area[n] is a coefficient that quantifies how much the area of ​​the smoke pixel cluster in the n-th frame has spread compared to the previous frame.

[0196] Generally, the area of ​​the pre-thermal runaway smoke from electric vehicles expands rapidly immediately after occurrence, resulting in a high D_area[n], whereas smoke caused by external environmental factors such as exhaust gases and dust in parking lots shows gradual or irregular changes in area.

[0197] Therefore, the area diffusion calculation unit (113) can distinguish between smoke from thermal runaway that exhibits a rapid area diffusion pattern and smoke caused by external environmental factors by calculating D_area[n].

[0198] Formula a is not limited to this, and as another example, a form calculated using a logarithmic ratio, such as D_area[n] = log(A[n] / max(A[n-1], ε)), can also be applied. In this case, it is advantageous to improve numerical stability when calculating fusion stability by mitigating rapid fluctuations in coefficient values ​​in sections where the area increases rapidly.

[0199] For a numerical example of the formula a, if the area of ​​the smoke pixel cluster in the n-1th frame is A[n-1] = 150 pixels and the area of ​​the nth frame is A[n] = 230 pixels, the area diffusion calculation unit (113) calculates D_area[n] = (230 - 150) / max(150, 1×10-6) = 80 / 150 = approximately 0.533, indicating that the area of ​​the smoke pixel cluster is a thermal runaway pattern that has rapidly diffused compared to the previous frame.

[0200] On the other hand, when A[n-1] = 150 pixels and A[n] = 155 pixels, the area diffusion calculation unit (113) calculates D_area[n] = (155 - 150) / max(150, 1×10-6) = 5 / 150 = approximately 0.033, indicating that the area change of the smoke pixel cluster is minimal and that it is a pattern that may be a false positive due to external environmental factors such as changes in the light source.

[0201] The light source variation calculation unit (114) calculates the light source variation coefficient L_var[n] based on the brightness values ​​of the background pixels surrounding the smoke pixel cluster included in the field recognition result transmitted from the data receiving unit (111).

[0202] As an example, the light source variation calculation unit (114) can calculate L_var[n] using the following formula b.

[0203] Formula b.

[0204] L_var[n] = (1 / N) × Σ(I_bg_i[n] - μ_bg[n])²

[0205] Here, L_var[n] is the light source variation coefficient of the nth frame acquired from the video camera (11), N is the number of background pixels surrounding the smoke pixel cluster, I_bg_i[n] is the brightness value (0–255) of the i-th background pixel in the nth frame, and μ_bg[n] is the average of the background pixel brightness values ​​of the nth frame.

[0206] L_var[n] is a coefficient that quantifies the brightness variance of background pixels surrounding smoke pixel clusters in the nth frame.

[0207] In a parking lot environment, the brightness dispersion of background pixels frequently increases due to the entry and exit of vehicle headlights or sudden changes in lighting, and the artificial intelligence model may misidentify such light source changes as smoke. Therefore, the light source variation calculation unit (114) calculates L_var[n] and applies it as a basis for suppressing misidentification in frames with large light source variations.

[0208] Formula b is not limited to this, and as another example, a form calculated as the average of the brightness change of background pixels between adjacent frames, such as L_var[n] = (1 / N) × Σ|I_bg_i[n] - I_bg_i[n-1]|, can also be applied. In this case, it is advantageous to more sensitively capture light source fluctuations caused by instantaneous headlight illumination by directly reflecting temporal brightness changes rather than brightness variance within a single frame.

[0209] For example, regarding the numerical example of formula b, when the brightness values ​​of 5 background pixels in the nth frame are I_bg[n] = {120, 118, 125, 122, 119}, μ_bg[n] = 120.8, and the light source variation calculation unit (114) calculates L_var[n] = {(120-120.8)² + (118-120.8)² + (125-120.8)² + (122-120.8)² + (119-120.8)²} / 5 = {0.64 + 7.84 + 17.64 + 1.44 + 3.24} / 5 = 30.8 / 5 = approximately 6.16, indicating that the brightness variance of the background pixels is low, and the pattern has a low possibility of false positives due to light source variation.

[0210] On the other hand, when the brightness value of the background pixel is I_bg[n] = {80, 120, 95, 140, 110}, μ_bg[n] = 109, and the light source variation calculation unit (114) calculates L_var[n] = {(80-109)² + (120-109)² + (95-109)² + (140-109)² + (110-109)²} / 5 = {841 + 121 + 196 + 961 + 1} / 5 = 2120 / 5 = approximately 424.0, indicating that the brightness dispersion of the background pixel is very high, and that it is a pattern that has a possibility of false positive due to rapid light source variation such as headlights.

[0211] The persistence calculation unit (115) calculates a persistence coefficient P_exist[n] that quantifies how continuously smoke pixel clusters are detected over consecutive frames, based on the field recognition results transmitted from the data receiving unit (111).

[0212] The persistence calculation unit (115) can calculate P_exist[n] using the following formula c.

[0213] Formula c.

[0214] P_exist[n] = (1 / W) × Σ(k=n-W+1 to n) 1(A[k] > 0)

[0215] Here, P_exist[n] is the persistence coefficient of the nth frame acquired from the video camera (11), W is the number of past consecutive frames referenced for the persistence calculation based on the nth frame, for example, when W = 5, P_exist[n] is calculated by referencing 5 frames from the n-4th frame to the nth frame, 1(A[k] > 0) is an indicator function that returns 1 if a smoke pixel cluster exists in the kth frame and 0 if it does not exist, and A[k] is the area (pixels) of the smoke pixel cluster in the kth frame.

[0216] P_exist[n] is the ratio of frames in which smoke pixel clusters were detected among W past frames, calculated in the range of 0 to 1.

[0217] The smoke that signals thermal runaway in electric vehicles is continuously detected in subsequent frames once it occurs, so P_exist[n] converges to 1, whereas smoke caused by external environmental factors such as exhaust gas and dust often appears and disappears instantaneously, so P_exist[n] is calculated to be low.

[0218] Accordingly, the persistence calculation unit (115) calculates P_exist[n] and applies it as a basis for distinguishing between temporary false positives and actual thermal runaway precursor smoke from a temporal perspective.

[0219] Formula c is not limited to this, and as another example, a form calculated as an exponentially weighted moving average, such as P_exist[n] = α × P_exist[n-1] + (1-α) × 1(A[n] > 0), can also be applied. Since it exponentially reflects the detection history of past frames without a fixed window size, it has a low amount of computation and is advantageous for giving higher weight to the detection results of recent frames.

[0220] For example, if W = 5 and the indicator function value indicating whether smoke pixel clusters are detected in the last 5 frames is {1, 1, 1, 1, 1}, the persistence calculation unit (115) calculates P_exist[n] = (1+1+1+1+1) / 5 = 5 / 5 = 1.0, indicating that smoke pixel clusters are continuously detected in consecutive frames, indicating a pattern with a high probability of being a precursor to thermal runaway.

[0221] On the other hand, when the indicator function value is {0, 1, 0, 1, 0}, the persistence calculation unit (115) calculates P_exist[n] = (0+1+0+1+0) / 5 = 2 / 5 = 0.4, indicating that the smoke pixel cluster is a pattern that may be a false positive due to external environmental factors such as changes in light sources that appear and disappear irregularly.

[0222] The temperature rise calculation unit (116) calculates the temperature rise coefficient T_rise[n] based on the measurement data of the thermal imaging camera (12) transmitted from the data receiving unit (111).

[0223] For example, the temperature rise calculation unit (116) can calculate T_rise[n] using the following formula d.

[0224] Formula d.

[0225] T_rise[n] = (T_max[n] - T_max[n-1]) / Δt

[0226] Here, T_rise[n] is the temperature rise coefficient (°C / s) of the nth frame acquired from the thermal imaging camera (12), T_max[n] is the highest temperature (°C) of the region of interest around the smoke pixel cluster in the nth frame, T_max[n-1] is the highest temperature (°C) of the region of interest around the smoke pixel cluster in the n-1th frame, and Δt is the time interval (s) between adjacent frames.

[0227] T_rise[n] is a coefficient that quantifies the rate of temperature rise in the region of interest surrounding the smoke pixel cluster per unit time.

[0228] In the pre-stage of thermal runaway in an electric vehicle, the temperature of the area of ​​interest around the vehicle rises rapidly due to the heat generated by the battery cells, resulting in a high T_rise[n]. In contrast, smoke caused by external environmental factors such as exhaust gas and dust in a parking lot does not accompany a temperature rise or has a very low rise rate, resulting in a low T_rise[n]. Accordingly, the temperature rise calculation unit (116) calculates T_rise[n] and applies it as a basis for suppressing false positives caused by external environmental factors that do not accompany temperature changes.

[0229] The formula for calculating the temperature rise coefficient is not limited to this, and a form based on the moving average of the highest temperatures of past W frames, such as T_rise[n] = (T_max[n] - T_mean[nW:n-1]) / Δt, can also be applied, which is advantageous for reducing noise in temperature changes between single frames and more stably reflecting the temperature rise trend.

[0230] For example, in the case where T_max[n-1] = 45℃ and T_max[n] = 68℃ and Δt = 1s, the temperature rise calculation unit (116) calculates T_rise[n] = (68 - 45) / 1 = 23.0℃ / s, indicating a pattern with a high probability of thermal runaway, where the temperature rises rapidly per unit time.

[0231] On the other hand, when T_max[n-1] = 45℃ and T_max[n] = 46℃ and Δt = 1s, the temperature rise calculation unit (116) calculates T_rise[n] = (46 - 45) / 1 = 1.0℃ / s, indicating that the temperature rise is minimal and that there is a possibility of false detection due to external environmental factors such as exhaust gas and dust that do not accompany temperature changes.

[0232] Unlike the thermal imaging camera (12), the temperature sensor (20) attached to the charging gun measures the temperature by coming into direct contact with the charging gun, so it can detect the temperature change of the charging gun occurring during the battery thermal runaway precursor stage most sensitively.

[0233] Accordingly, the charging abnormality calculation unit (117) calculates the charging abnormality coefficient C_grad[n] using the following formula e based on the measurement data of the temperature sensor (20) transmitted from the data receiving unit (111).

[0234] Formula e.

[0235] C_grad[n] = (T_sensor[n] - T_sensor[n-1]) / Δt

[0236] Here, C_grad[n] is the charging abnormality coefficient (°C / s) of the nth sample, T_sensor[n] is the measured temperature (°C) of the temperature sensor (20) in the nth sample, T_sensor[n-1] is the measured temperature (°C) of the temperature sensor (20) in the n-1st sample, and Δt is the time interval (s) between samples.

[0237] C_grad[n] is a coefficient that quantifies the rate of temperature rise per unit time of the charging gun.

[0238] During normal charging, the temperature of the charging gun is maintained gradually, resulting in a low C_grad[n]; however, during the pre-thermal runaway stage, heat from the battery cells is transferred to the charging gun through the charging cable, causing the temperature to rise rapidly, resulting in a high C_grad[n].

[0239] The charging abnormality calculation unit (117) calculates C_grad[n] and applies it as a basis for determining whether there is an abnormality in the charging gun independently of the temperature rise coefficient T_rise[n] based on the thermal imaging camera (12).

[0240] The formula for calculating the charge abnormality factor is not limited to the formula e, and a form calculated as a moving average of the temperature change rates of past M samples, such as C_grad[n] = (1 / M) × Σ(k=n-M+1 to n) (T_sensor[k] - T_sensor[k-1]) / Δt, can also be applied. According to this, noise in temperature changes between single samples can be reduced and the temperature rising trend can be stably reflected.

[0241] For example, in the case where T_sensor[n-1] = 45℃ and T_sensor[n] = 46.5℃ and Δt = 0.1s, the charging abnormality calculation unit (117) calculates C_grad[n] = (46.5 - 45) / 0.1 = 1.5 / 0.1 = 15.0℃ / s, indicating a pattern with a high probability of thermal runaway, where the temperature of the charging gun rises rapidly.

[0242] On the other hand, when T_sensor[n-1] = 35℃ and T_sensor[n] = 35.01℃ and Δt = 0.1s, the charging abnormality calculation unit (117) calculates C_grad[n] = (35.01 - 35) / 0.1 = 0.01 / 0.1 = 0.1℃ / s, indicating a pattern in which there is a possibility of false detection due to external environmental factors that do not accompany charging abnormalities, as the temperature rise of the charging gun is minimal.

[0243] The danger state generation unit (118) generates danger state information based on six coefficients B_align[n], D_area[n], L_var[n], P_exist[n], T_rise[n], and C_grad[n] calculated from the boundary alignment calculation unit (112), area diffusion calculation unit (113), light source variation calculation unit (114), persistence calculation unit (115), temperature rise calculation unit (116), and charging abnormality calculation unit (117), and transmits the generated danger state information to the alarm control module (120).

[0244] The risk state generation unit (118) can generate risk state information in a weighted sum-based form as an example of a method for generating risk state information, by calculating a risk score by assigning weights to each of the six coefficients and summing them, and if the risk score exceeds a preset threshold, the risk level is raised.

[0245] As another example, risk status information can be generated based on rules that combine whether each coefficient exceeds a threshold value. For instance, it can be applied in a form where the level is determined as caution if B_align[n] ≥ 0.7 and P_exist[n] ≥ 0.6, the level is determined as warning if B_align[n] ≥ 0.7, D_area[n] ≥ 0.3 and T_rise[n] ≥ 5.0℃ / s, and the level is determined as danger if B_align[n] ≥ 0.9, D_area[n] ≥ 0.5, T_rise[n] ≥ 10.0℃ / s and C_grad[n] ≥ 5.0℃ / s.

[0246] For example, if the danger state generation unit (118) generates danger state information using a rule-based method by applying the numerical example presented earlier, B_align[n] = 0.998, D_area[n] = 0.533, L_var[n] = 6.16, P_exist[n] = 1.0, T_rise[n] = 23.0℃ / s, and C_grad[n] = 15.0℃ / s, then all conditions of B_align[n] ≥ 0.9, D_area[n] ≥ 0.5, T_rise[n] ≥ 10.0℃ / s, and C_grad[n] ≥ 5.0℃ / s are satisfied, so it is determined to be a danger level, and L_var[n] = 6.16, so the light source fluctuation is minimal, confirming that it is not a false positive caused by a change in the light source.

[0247] On the other hand, when B_align[n] = 0.569, D_area[n] = 0.033, L_var[n] = 424.0, P_exist[n] = 0.4, T_rise[n] = 1.0℃ / s, and C_grad[n] = 0.1℃ / s, the condition B_align[n] ≥ 0.7 is not satisfied, and since L_var[n] = 424.0 indicates a very large fluctuation in the light source, it is determined to be a false positive caused by external environmental factors and no alarm is issued.

[0248] Below, Example 2 and Comparative Example 2 are set up and the results are compared to verify the effect of the six coefficient-based post-processing algorithm of the risk state judgment module (110) in suppressing false positives in the artificial intelligence model.

[0249] <Example 2>

[0250] Example 2 applies a six-coefficient-based post-processing algorithm of a risk state determination module (110) to the system of the present invention, wherein the risk state determination module (110) calculates six coefficients, B_align[n], D_area[n], L_var[n], P_exist[n], T_rise[n], and C_grad[n], based on the field recognition result generated by the AI ​​edge device (30) and the measurement data of the thermal imaging camera (12) and the temperature sensor (20), and generates risk state information.

[0251] When applying the values ​​of the thermal runaway precursor situation presented earlier, the danger state judgment module (110) calculates B_align[n] = 0.998, D_area[n] = 0.533, L_var[n] = 6.16, P_exist[n] = 1.0, T_rise[n] = 23.0℃ / s, and C_grad[n] = 15.0℃ / s to determine the danger level, and the alarm control module (120) outputs an alarm of the danger level through the guide unit (40).

[0252] On the other hand, when applying the values ​​of the false positive situation presented earlier, the danger state judgment module (110) calculates B_align[n] = 0.569, D_area[n] = 0.033, L_var[n] = 424.0, P_exist[n] = 0.4, T_rise[n] = 1.0℃ / s, and C_grad[n] = 0.1℃ / s, and does not satisfy the condition B_align[n] ≥ 0.7, and L_var[n] = 424.0, so the light source fluctuation is very large, so it is judged to be a false positive caused by external environmental factors and does not issue an alarm.

[0253] <Comparative Example 2>

[0254] Comparative Example 2 is the same as the system applied in Example 1 above, in which the controller (100) determines the danger state based only on the field recognition result of the AI ​​edge device (30) without applying the six coefficient-based post-processing algorithm of the danger state determination module (110).

[0255] When the same input data as in Example 2 is applied, in a thermal runaway precursor situation, the AI ​​edge device (30) detects a smoke pixel cluster and the controller (100) issues an alarm, and in a false alarm situation, the AI ​​edge device (30) misidentifies a smoke-like pattern caused by a change in light source as actual smoke, and the controller (100) issues an unnecessary alarm.

[0256] In response to this, the alarm control module (120) differentially adjusts the alarm output level of the guidance unit (40) according to the danger state information received from the danger state generation unit (118).

[0257] In the case where the values ​​in the previously presented numerical example are calculated as B_align[n] = 0.998, D_area[n] = 0.533, L_var[n] = 6.16, P_exist[n] = 1.0, T_rise[n] = 23.0℃ / s, and C_grad[n] = 15.0℃ / s and are determined to be a risk level, the alarm control module (120) controls the display panel (41) to output a red flashing light and a fire alarm message, and to output an alarm sound of 90dB or higher and evacuation guidance voice through the speaker (42).

[0258] On the other hand, if B_align[n] = 0.569, D_area[n] = 0.033, L_var[n] = 424.0, P_exist[n] = 0.4, T_rise[n] = 1.0℃ / s, and C_grad[n] = 0.1℃ / s, and it is determined to be a false positive due to external environmental factors, the alarm control module (120) does not issue an alarm.

[0259] In summary, the danger state judgment module (110) provides a characteristic that prevents unnecessary alarm issuance by suppressing smoke-like patterns caused by external environmental factors that can cause false detection by the artificial intelligence model through a complex judgment of six coefficients.

[0260] Furthermore, since combining the previously calculated six coefficients based only on simple rules has limitations in that it fails to reflect the mutually complementary relationship between each coefficient and the judgment result fluctuates sensitively depending on the threshold setting, the controller (100) may further include a fusion stability calculation module (150) that mathematically fuses the six coefficients to calculate a single fusion stability.

[0261] In other words, the fusion stability calculation module (150) generates a single indicator by integrating six coefficients into a single formula so that the fusion stability is calculated to be higher when boundary alignment, area diffusion, persistence, temperature rise, and charging abnormality are high, and the fusion stability is calculated to be lower when light source fluctuation is high.

[0262] Specifically, the fusion stability calculation module (150) calculates the fusion stability F_fusion[n] using the following mathematical formula 2.

[0263] Mathematical formula 2.

[0264]

[0265] Here, F_fusion[n] is the n-th multiple hazard source fusion stability, B_align[n] is the n-th boundary alignment coefficient, D_area[n] is the n-th area diffusion coefficient, P_exist[n] is the n-th persistence coefficient, T_rise[n] is the n-th temperature rise coefficient, C_grad[n] is the n-th fill abnormality coefficient, L_var[n] is the n-th light source variation coefficient, τ is a weighting coefficient that adjusts the reflection ratio of the temperature-based coefficient and is greater than 0, and γ is a light source variation attenuation coefficient and is greater than 0.

[0266] F_fusion[n] is a value that quantifies the possibility of the presence of thermal runaway smoke as a single numerical value by fusing five coefficients of boundary alignment, area diffusion, persistence, temperature rise, and charging, and corrects it with a light source variation coefficient, where a higher F_fusion[n] indicates a higher possibility of thermal runaway, and a lower F_fusion[n] indicates a higher possibility of false positives due to external environmental factors.

[0267] Mathematical Equation 2 has the following functional properties.

[0268] The molecule is designed with a product structure of B_align[n], D_area[n], and P_exist[n], so that if any of the three coefficients converge to 0, F_fusion[n] converges to 0. Therefore, it has the characteristic that high fusion stability is produced only when the diffusion directionality, area diffusion, and persistence of the smoke are all satisfied.

[0269] In addition, the (1+τ(T_rise[n]+C_grad[n])) term of the molecule converges to 1 when there is no temperature rise or charging abnormality, so the fusion stability is calculated only to a limited extent based on smoke detection alone, and the fusion stability can be amplified as the temperature abnormality increases.

[0270] The 1+γ·L_var[n]² term in the denominator reflects the squared coefficient of light source variation, so as the light source variation increases, the denominator increases rapidly, which has the advantage of strongly suppressing F_fusion[n].

[0271] For example, when τ = 0.1 and γ = 0.001 are set and the values ​​of the thermal runaway precursor conditions presented earlier are applied, the fusion stability calculation module (150) calculates F_fusion[n] = {0.998×0.533×1.0×(1+0.1×(23.0+15.0))} ​​ / {1+0.001×6.16²} = {0.532×4.8} / 1.038 = 2.554 / 1.038 = approximately 2.46.

[0272] On the other hand, when applying the value of a false positive situation, the fusion stability calculation module (150) calculates F_fusion[n] = {0.569×0.033×0.4×(1+0.1×(1.0+0.1))} / {1+0.001×424.0²} = {0.00834} / 180.776 = approximately 0.000046, indicating that the multi-hazard source fusion stability converges to 0 in a false positive situation where the light source variation is very large.

[0273] In response to this, the risk state generation unit (118) updates the risk state information using the following mathematical formula 3 based on the multi-risk source fusion stability F_fusion[n] of the nth frame received from the fusion stability calculation module (150) and the risk state information R_fusion[n-1] of the previous frame.

[0274] Mathematical formula 3.

[0275]

[0276] Here, R'_fusion[n] is the updated risk state information of the n-th frame, R_fusion[n-1] is the risk state information of the n-1-th frame, F_fusion[n] is the multi-risk source fusion stability of the n-th frame, L_var[n] is the light source variation coefficient of the n-th frame, η is the previous state retention ratio with a value greater than 0 and less than 1, and δ is the environment noise attenuation coefficient with a value greater than 0.

[0277] R'_fusion[n] is a value that quantifies updated risk state information reflecting temporal persistence by weighting the multi-risk source fusion stability F_fusion[n] of the current frame and the risk state information R_fusion[n-1] of the previous frame.

[0278] If the danger state is determined solely by F_fusion[n], a problem may arise where the risk level fluctuates rapidly due to instantaneous smoke detection or changes in the light source.

[0279] Accordingly, the risk state generation unit (118) calculates R'_fusion[n] using mathematical formula 3 and reflects the risk state information of the previous frame, thereby suppressing rapid fluctuations in risk and enabling a time-stable risk state determination.

[0280] In mathematical equation 3, as η increases, the weight of the risk state information R_fusion[n-1] from the previous frame increases, making the change in risk gentler, and as η decreases, the F_fusion[n] from the current frame is reflected more strongly, making the change in risk more sensitive.

[0281] In addition, the e^(-δ·L_var[n]) term exponentially reduces the contribution of F_fusion[n] of the current frame as light source variation increases, thereby suppressing instantaneous false positives caused by light source changes from being reflected in the updated risk state information. Since both η and δ are set to positive, R'_fusion[n] converges stably in the range of 0 or greater.

[0282] For example, regarding the numerical example for mathematical formula 3, when η = 0.7 and δ = 0.005 are set and the numerical value of the thermal runaway precursor condition is applied in the initial state where R_fusion[n-1] = 0, the danger state generating unit (118) calculates R'_fusion[n] over consecutive frames as follows. When n = 1, R'_fusion[1] = 0.7×0 + 0.3×2.46×e^(-0.005×6.16) = 0.738×0.970 = approximately 0.716, when n = 2, R'_fusion[2] = 0.7×0.716 + 0.3×2.46×0.970 = 0.501 + 0.716 = approximately 1.217, and when n = 3, R'_fusion[3] = 0.7×1.217 + 0.716 = approximately 1.568, indicating that the updated risk state information in consecutive frames gradually increases.

[0283] On the other hand, when applying the value of a false positive situation, the risk state generation unit (118) calculates R'_fusion[n] = 0.7×0 + 0.3×0.000046×e^(-0.005×424.0) = 0.0000138×0.120 = approximately 0.0000017, indicating that the updated risk state information converges to 0.

[0284] In response to this, the alarm control module (120) differentially adjusts the alarm output level of the guide unit (40) according to the updated danger state information R'_fusion[n] in Equation 3.

[0285] In the example of the numerical values ​​presented earlier, if R'_fusion[3] is calculated to be approximately 1.568 and is determined to be a risk level, the alarm control module (120) controls the display panel (41) to output a red flashing light and a fire alarm message, and to output an alarm sound of 90 dB or higher and evacuation guidance voice through the speaker (42). On the other hand, if R'_fusion[n] is calculated to be approximately 0.0000017 and the updated risk status information converges to 0, the alarm control module (120) does not issue an alarm.

[0286] In the following, Example 3 and Comparative Example 3 are set up and the results are compared to verify the effect of the fusion stability calculation module (150) in increasing the temporal stability of the danger state judgment.

[0287] <Example 3>

[0288] Example 3 is an additional application of a fusion stability calculation module (150) to the system of the present invention, wherein the risk state generation unit (118) updates the risk state information by calculating R'_fusion[n] using Equation 3 based on F_fusion[n] received from the fusion stability calculation module (150) and the risk state information R_fusion[n-1] of the previous frame.

[0289] In the numerical example presented earlier, R'_fusion[1] = approximately 0.716, R'_fusion[2] = approximately 1.217, and R'_fusion[3] = approximately 1.568 gradually increase over three consecutive frames to determine a risk level, and the alarm control module (120) outputs a warning of the risk level through the guide unit (40). When applying the numerical value of the false positive situation, R'_fusion[n] = approximately 0.0000017 converges to 0, so no alarm is issued.

[0290] <Comparative Example 3>

[0291] Comparative Example 3 is the same as the system applied in Example 2 above, and determines the danger state independently in each frame using only rule-based judgment of six coefficients without applying the fusion stability calculation module (150). When the same input data as in Example 3 is applied, the danger level is determined by individually comparing the six coefficients in each frame, so the danger level fluctuates rapidly depending on whether instantaneous smoke is detected and the cumulative effect over time is not reflected.

[0292] As a result of comparing Example 3 and Comparative Example 3, Example 3 stably determined the risk level as R'_fusion[n] gradually increased to 0.716, 1.217, and 1.568 in consecutive frames, whereas Comparative Example 3 determined the risk level independently in each frame, so the risk level fluctuated rapidly depending on whether instantaneous smoke was detected, making it difficult to determine a consistent risk state.

[0293] In summary, the fusion stability calculation module (150) provides highly reliable risk state judgment characteristics by stably accumulating risk levels in consecutive frames compared to independent judgment per frame of Comparative Example 3 through the weighted fusion structure of F_fusion[n] and the temporal accumulation structure of R'_fusion[n], and suppressing instantaneous risk fluctuations.

[0294] FIG. 6 is a flowchart illustrating the risk state correction process of the present invention.

[0295] Furthermore, when determining risk levels using a fixed threshold, the rate of change in the risk state is not reflected, which may result in delayed risk level transitions in situations where updated risk state information rises rapidly, or unnecessary level transitions in situations where it changes gradually.

[0296] Accordingly, the controller (100) may further include a risk class transition threshold calculation module (160) that calculates the risk class transition threshold in real time according to the rate of change of the updated risk state information R'_fusion[n].

[0297] In other words, the risk level transition threshold calculation module (160) increases the risk level transition threshold as the rate of change of the updated risk state information increases to suppress excessive level transition, and as the rate of change slows, maintains the risk level transition threshold at a reference value to realize stable level transition.

[0298] Specifically, the risk level transition threshold calculation module (160) calculates the risk level transition threshold θ_k[n] using the following mathematical formula 4.

[0299] Mathematical formula 4.

[0300]

[0301] Here, θ_k[n] is the k-th risk level transition threshold of the n-th frame, θ_0k is the reference threshold of the k-th risk level, μ is the threshold change sensitivity, which is a value greater than 0, R'_fusion[n] is the updated risk state information of the n-th frame, R'_fusion[n-1] is the updated risk state information of the n-1-th frame, and Δt is the time interval between frames (s).

[0302] θ_k[n] is a value that adjusts the risk level transition threshold in real time by reflecting the rate of change per unit time of the updated risk state information.

[0303] When determining risk levels using a fixed threshold, excessive level transitions may occur in sections where risk levels rise rapidly, as the threshold may not reflect the rate of change.

[0304] Accordingly, the risk level transition threshold calculation module (160) calculates θ_k[n] using Equation 4 and adjusts the threshold according to the rate of change in risk, thereby suppressing excessive level transitions and enabling stable risk level determination.

[0305] Mathematical Equation 4 has the following functional properties.

[0306] When the updated risk state information rises, i.e., when R'_fusion[n] > R'_fusion[n-1], (R'_fusion[n]-R'_fusion[n-1]) / Δt becomes positive, so the tanh value is calculated as positive, and thus θ_k[n] becomes higher than the reference threshold θ_0k, suppressing excessive class transitions. When the updated risk state information remains stable, i.e., when R'_fusion[n] = R'_fusion[n-1], the tanh value becomes 0, so the reference threshold is maintained at θ_k[n] = θ_0k.

[0307] When the updated risk state information decreases, i.e., when R'_fusion[n] < R'_fusion[n-1], the tanh value is calculated as negative, causing θ_k[n] to become lower than θ_0k and facilitating the transition to a lower risk level. Additionally, due to the saturation characteristics of the tanh function, θ_k[n] converges stably in the range greater than 0, and changes in θ_k[n] become more sensitive as μ increases.

[0308] For a numerical example of mathematical formula 4, when θ_0k = 1.0, μ = 2.0, and Δt = 1s are set and the values ​​of the thermal runaway precursor situation presented earlier are applied, R'_fusion[n] = 1.568 and R'_fusion[n-1] = 1.217, so the rate of change is (1.568-1.217) / 1 = 0.351, and the risk class transition threshold calculation module (160) calculates θ_k[n] = 1.0×{1+tanh(2.0×0.351)} = 1.0×{1+tanh(0.702)} = 1.0×(1+0.604) = approximately 1.604, indicating that the threshold is raised compared to the reference threshold of 1.0, thereby suppressing excessive class transition.

[0309] On the other hand, when applying the value of the false positive situation, R'_fusion[n] = approximately 0 and R'_fusion[n-1] = 0, so the rate of change converges to 0, and the risk level transition threshold calculation module (160) indicates that the reference threshold is maintained as θ_k[n] = 1.0×{1+tanh(0)} = 1.0×(1+0) = approximately 1.0.

[0310] The risk state generation unit (118) calculates the second updated risk state information R''_fusion[n] based on the θ_k[n] received from the risk grade transition threshold calculation module (160) and the updated risk state information R'_fusion[n] using the following mathematical formula 5.

[0311] Mathematical formula 5.

[0312]

[0313] Here, R''_fusion[n] is the second updated risk state information of the n-th frame, R'_fusion[n] is the updated risk state information of the n-th frame, and θ_k[n] is the k-th risk level transition threshold of the n-th frame.

[0314] R''_fusion[n] is the maximum k value at which the updated risk state information R''_fusion[n] satisfies the risk level transition threshold θ_k[n], and is a value that quantifies the current risk level. Since θ_k[n] calculated by the risk level transition threshold calculation module (160) is a threshold adjusted to reflect the rate of change in risk, the risk state generation unit (118) calculates R''_fusion[n] using Equation 5 and determines the risk level that is adaptively adjusted according to the rate of change in risk.

[0315] Mathematical formula 5 determines the maximum grade that satisfies the conditions with a max{k} structure, so it accurately determines the risk grade closest to the current risk state.

[0316] In addition, when the rate of change in risk is large, the threshold value is raised in conjunction with θ_k[n] calculated in mathematical formula 4 to suppress excessive grade transitions, and when the rate of change in risk is small, the threshold value is kept low to realize stable grade transitions. Furthermore, since the continuous value R'_fusion[n] is converted into a discrete risk grade, the alarm control module (120) has the advantage of providing risk state information in a form suitable for differentially adjusting the alarm output level according to the risk grade.

[0317] For a numerical example of mathematical formula 5, the risk level is set to three levels: k=1 (caution), k=2 (warning), and k=3 (danger), and the threshold values ​​for each are set to θ_0_1 = 0.5, θ_0_2 = 1.0, and θ_0_3 = 1.5.

[0318] Applying (1+tanh(0.702)) = 1.604 calculated in Equation 4 above, θ_1[n] = 0.5×1.604 = approximately 0.802, θ_2[n] = 1.0×1.604 = approximately 1.604, and θ_3[n] = 1.5×1.604 = approximately 2.406.

[0319] When R'_fusion[n] = 1.568 is applied in a thermal runaway precursor situation, the danger state generating unit (118) satisfies the condition R'_fusion[n] ≥ θ_1n but does not satisfy the condition R'_fusion[n] ≥ θ_2n, so it calculates R''_fusion[n] = 1 (caution level). When R'_fusion[n] = approximately 0 is applied in a false positive situation, the danger state generating unit (118) satisfies no θ_k[n] condition, so it calculates R''_fusion[n] = 0 and does not issue an alarm.

[0320] Below, Example 4 and Comparative Example 4 are set up and the results are compared to verify the effect of the risk grade transition threshold calculation module (160) in suppressing excessive grade transitions by reflecting the rate of change in risk.

[0321] <Example 4>

[0322] Example 4 additionally applies a risk level transition threshold calculation module (160) to the system of the present invention, wherein the risk level transition threshold calculation module (160) calculates θ_k[n] using Equation 4 and the risk state generation unit (118) calculates R''_fusion[n] using Equation 5 to update the risk state information a second time.

[0323] In the example of the numerical values ​​presented earlier, when R'_fusion[n] = 1.568 and θ_1[n] = 0.802, θ_2[n] = 1.604, and θ_3[n] = 2.406 are calculated, the danger state generating unit (118) calculates R''_fusion[n] = 1 (caution level) because the condition that R'_fusion[n] is greater than or equal to θ_1[n] is satisfied, but the condition that R'_fusion[n] is greater than or equal to θ_2[n] is not satisfied.

[0324] <Comparative Example 4>

[0325] Comparative Example 4 is the same as the system applied in Example 3 above, but without applying the risk level transition threshold calculation module (160), the risk level is determined with fixed thresholds θ_0_1 = 0.5, θ_0_2 = 1.0, and θ_0_3 = 1.5.

[0326] When R'_fusion[n] = 1.568 is applied to the same input data as in Example 4, the risk state generating unit (118) determines R''_fusion[n] = 3 (risk level) because R'_fusion[n] satisfies all conditions of being greater than or equal to θ_0_1 (0.5), θ_0_2 (1.0), and θ_0_3 (1.5), resulting in an excessive level transition.

[0327] As a result of comparing Example 4 and Comparative Example 4, Example 4 suppressed excessive grade transitions by calculating R''_fusion[n] = 1 (caution grade) with an adjusted threshold reflecting the rate of change in risk, whereas Comparative Example 4 calculated R''_fusion[n] = 3 (risk grade) with a fixed threshold, confirming that excessive grade transitions occurred in the rapid increase in risk.

[0328] In summary, the risk class transition threshold calculation module (160) adjusts the risk class transition threshold in real time according to the rate of change of the updated risk status information, thereby suppressing excessive class transitions compared to the fixed threshold-based judgment of Comparative Example 4 and providing stable risk class judgment characteristics.

[0329] Furthermore, since there is a possibility of false positives if the double-lock condition of the thermal imaging camera and temperature sensor is not satisfied even if the updated risk status information has risen above the fire confirmation standard, the controller (100) may further include a risk correction module (170) that corrects the updated risk status information depending on whether the double-lock condition is satisfied.

[0330] In other words, the risk correction module (170) maintains the updated risk state information as is when the double lock condition is satisfied, and corrects the updated risk state information by gradually reducing it as the double lock condition is not satisfied.

[0331] Specifically, the risk correction module (170) calculates the risk state information R_adjusted[n] corrected by the following mathematical formula 6.

[0332] Mathematical formula 6.

[0333]

[0334] Here, R_adjusted[n] is the corrected risk state information of the nth frame, R'_fusion[n] is the updated risk state information of the nth frame, C_dual is a value indicating whether the double lock condition is satisfied, κ is a value greater than 0, and Δt_dual is the duration (s) of the double lock condition not being satisfied when the updated risk state information is greater than or equal to the fire confirmation criterion.

[0335] R_adjusted[n] is a value that corrects the risk state information updated based on whether the double lock condition is satisfied and the duration of non-satisfaction, reflecting the simultaneous detection by the thermal imaging camera and temperature sensor in the risk state determination.

[0336] That is, even if the updated risk status information rises above the fire confirmation standard, there is still a possibility of false detection if only one of the thermal imaging camera and the temperature sensor detects it. Therefore, the risk correction module (170) calculates R_adjusted[n] using Equation 6 and gradually reduces the risk status information according to the duration of non-compliance with the double lock condition, thereby preventing unnecessary issuance of the highest grade alarm.

[0337] In Equation 6, when the double lock condition is satisfied, i.e., when C_dual = 1, Equation 6 becomes R_adjusted[n] = R'_fusion[n]·{1+(1-1)·e^(-κ·Δt_dual)} = R'_fusion[n], and the updated risk state information is maintained.

[0338] When the double lock condition is not satisfied, i.e., when C_dual = 0, Equation 6 becomes R_adjusted[n] = R'_fusion[n]·e^(-κ·Δt_dual), and as the duration of the double lock condition not being satisfied Δt_dual increases, R_adjusted[n] decreases exponentially.

[0339] The risk correction module (170) can control the rate of decrease of R_adjusted[n] by adjusting κ, and the rate of decrease becomes faster as κ increases.

[0340] Equation 6 provides the characteristic of maintaining the updated risk status information as is when the double lock condition is satisfied, so that the risk level is immediately reflected at the highest level in an actual fire situation, and gradually correcting the risk level without a sudden drop in risk by exponentially decreasing the risk status information according to the duration of non-satisfaction of the double lock condition, thereby preventing unnecessary highest level alarm issuance due to false positives.

[0341] For a numerical example of mathematical formula 6, κ is set to 0.1 and R'_fusion[n] = 1.568 is applied. When the double lock condition is satisfied (C_dual = 1), the risk correction module (170) calculates R_adjusted[n] = 1.568 and maintains the updated risk state information. When the double lock condition is not satisfied for 5 seconds (C_dual = 0, Δt_dual = 5s), the risk correction module (170) calculates R_adjusted[n] = 1.568×e^(-0.1×5) = 1.568×0.607 = approximately 0.952, indicating that the risk state information is reduced.

[0342] When the double lock condition is not met for 10 seconds (C_dual = 0, Δt_dual = 10s), the risk correction module (170) calculates R_adjusted[n] = 1.568×e^(-0.1×10) = 1.568×0.368 = approximately 0.577, indicating that the risk state information gradually decreases as the duration of the double lock condition not met increases.

[0343] In response to this, the risk state generation unit (118) calculates the third-level updated risk state information R'''_fusion[n] based on the corrected risk state information R_adjusted[n] received from the risk level correction module (170) using the following mathematical formula 7.

[0344] Mathematical formula 7.

[0345]

[0346] Here, R'''_fusion[n] is the third-order updated risk state information of the n-th frame, R_adjusted[n] is the corrected risk state information of the n-th frame, and θ_k[n] is the k-th risk grade transition threshold determined from the second-order updated risk state information.

[0347] R'''_fusion[n] is a value that quantifies the relative level of the current risk state by dividing the corrected risk state information R_adjusted[n], which reflects double lock correction, by the risk level transition threshold θ_k[n].

[0348] That is, mathematical formula 7 provides risk state information in a form that can be directly utilized by the alarm control module (120) to differentially adjust the alarm output level according to the risk level by normalizing the absolute value R_adjusted[n] relative to the risk level transition threshold to express the relative severity of the risk state as a single index.

[0349] Looking at the function characteristics of Equation 7, when R_adjusted[n] is greater than θ_k[n], it indicates that R'''_fusion[n] is greater than 1, meaning the risk level transition threshold is exceeded; when R_adjusted[n] is equal to θ_k[n], it indicates that R'''_fusion[n] is 1, meaning the risk level transition threshold is exactly matched; and when R_adjusted[n] is less than θ_k[n], it indicates that R'''_fusion[n] is less than 1, meaning the risk level transition threshold is not met.

[0350] In addition, as the duration of non-satisfaction of the double lock condition increases, R_adjusted[n] decreases, and thus R'''_fusion[n] also gradually decreases, quantitatively reflecting the lowering of the severity of the risk state.

[0351] Equation 7 normalizes R_adjusted[n] to θ_k[n] to provide the relative level of the current risk state relative to the risk level transition threshold as a single index, which has the advantage of enabling more intuitive judgment than the absolute value of the risk state. In addition, by calculating final risk state information that simultaneously reflects double lock correction and the risk level transition threshold, it simultaneously achieves false positive suppression and stable grade determination.

[0352] As a numerical example for Equation 7, since k was determined to be 1 (caution level) in Equation 5 above, θ_k[n] = θ_1[n] = 0.802 is applied. When the double lock condition is satisfied (C_dual = 1), R_adjusted[n] = 1.568, so the danger state generating unit (118) calculates R'''_fusion[n] = 1.568 / 0.802 = approximately 1.955.

[0353] When the double lock condition is not satisfied for 5 seconds (Δt_dual = 5s), R_adjusted[n] = 0.952, so the danger state generating unit (118) calculates R'''_fusion[n] = 0.952 / 0.802 = approximately 1.187.

[0354] When the double lock condition is not satisfied for 10 seconds (Δt_dual = 10s), R_adjusted[n] = 0.577, so the danger state generation unit (118) calculates R'''_fusion[n] = 0.577 / 0.802 = approximately 0.719, indicating that as the duration of the double lock condition not being satisfied increases, the third updated danger state information gradually decreases.

[0355] In the following, Example 5 and Comparative Example 5 are set up and the results are compared to verify the effect of the risk correction module (170) improving the precision of the risk state judgment by reflecting whether the double lock condition is satisfied.

[0356] <Example 5>

[0357] Example 5 additionally applies a risk correction module (170) to the system of the present invention, wherein the risk correction module (170) calculates R_adjusted[n] using Equation 6 and the risk state generation unit (118) calculates R'''_fusion[n] using Equation 7 to update the risk state information for the third time.

[0358] In the numerical example presented above, when the double lock condition is satisfied (C_dual = 1), the risk state generating unit (118) calculates R'''_fusion[n] = 1.955 to indicate that the risk level transition threshold is exceeded, when the double lock condition is not satisfied for 5 seconds (Δt_dual = 5s), it calculates R'''_fusion[n] = 1.187, and when the double lock condition is not satisfied for 10 seconds (Δt_dual = 10s), it calculates R'''_fusion[n] = 0.719 to indicate that the risk level transition threshold is not met.

[0359] <Comparative Example 5>

[0360] Comparative Example 5 is the same as the system applied in Example 4 above, and the risk state is determined as R''_fusion[n] = 1 (caution level) regardless of whether the double lock condition is satisfied without applying the risk correction module (170).

[0361] When the same input data as in Example 5 is applied, the risk level remains the same regardless of the duration of non-compliance with the double lock condition, so the same alarm is issued even in situations where false positives are possible.

[0362] As a result of comparing Example 5 and Comparative Example 5, it was confirmed that Example 5 issued an immediate alarm when the double lock condition was satisfied, exceeding the risk level transition threshold with R'''_fusion[n] = 1.955, and when the double lock condition was not satisfied for 10 seconds, the risk level was lowered to R'''_fusion[n] = 0.719, whereas Comparative Example 5 maintained the same risk level regardless of whether the double lock condition was satisfied, thereby issuing unnecessary alarms even in situations where false positives are possible.

[0363] In summary, the risk correction module (170) reflects whether the double lock condition is satisfied and the duration of non-satisfaction in the risk status information, thereby providing the characteristic of simultaneously realizing immediate alarm issuance in actual fire situations and downward adjustment of the risk level in false alarm situations compared to Comparative Example 5.

[0364] As explained above, the configuration and operation of the AI-based integrated safety management system for electric vehicle charging areas according to the present invention have been described in the above description and drawings; however, this is merely an example, and the concept of the present invention is not limited to the above description and drawings. It is understood that various changes and modifications are possible within the scope of the technical concept of the present invention. Explanation of the symbols

[0365] 10: Camera assembly 11: Video camera 12: Thermal imaging camera 20: Temperature sensor 30: AI Edge Device 40: Guide 41: Electronic display 42: Speaker 100: Controller 110: Risk State Determination Module 111: Data receiving unit 112: Boundary alignment calculation unit 113: Area diffusion calculation unit 114: Light source fluctuation calculation unit 115: Durability Calculator 116: Temperature Rise Calculator 117: Charging Anomaly Calculation Unit 118: Danger State Generation Unit 120: Alarm control module 130: Allowable charging time calculation module 140: Evacuation Route Calculation Module 150: Fusion Stability Calculation Module 160: Risk Grade Transition Threshold Calculation Module 170: Risk Correction Module

Claims

Claim 1 An AI-based integrated safety management system for electric vehicle charging areas, comprising: a camera assembly including a video camera and a thermal imaging camera installed in an electric vehicle charging area to acquire images; a temperature sensor attached to an electric vehicle charging gun to detect temperature changes during charging; an AI edge device that generates on-site recognition results based on data from the video camera, the thermal imaging camera, and the temperature sensor while including an AI model; a guidance unit that outputs an alarm including an electronic display and a speaker; and a controller including a risk state determination module that determines a risk state of the charging area based on the on-site recognition results, and an alarm control module that controls the output of an alarm from the guidance unit according to the risk state; wherein the risk state determination module comprises: a data receiving unit that receives an object recognition result received from the AI ​​edge device and measurement data from the thermal imaging camera of the camera assembly and the temperature sensor; a boundary alignment calculation unit that calculates a boundary alignment coefficient according to the following mathematical formula 1 based on the smoke pixel cluster of the nth frame acquired from the video camera included in the object recognition result received from the data receiving unit; and mathematical formula 1. (Here, B_align[n] is the boundary alignment coefficient of the nth frame acquired from the video camera, v_smoke[n] is the movement vector between the center point of the smoke pixel cluster of the nth frame and the center point of the smoke pixel cluster of the (n-1)th frame, v_boundary[n] is the outer normal vector of the vehicle mask boundary point closest to the center point of the smoke pixel cluster of the nth frame, and ε is a reference value to prevent the denominator from becoming zero) An area diffusion calculation unit that calculates an area diffusion coefficient based on the growth rate of the smoke pixel cluster area of ​​the nth frame acquired from the video camera included in the object recognition result received from the data receiving unit relative to the previous frame, a light source variation calculation unit that calculates a light source variation coefficient based on the variance values ​​of the recent k frames for the average luminance value of the region of interest set based on the circumscribed rectangle of the smoke pixel cluster of the nth frame acquired from the video camera included in the object recognition result received from the data receiving unit, and the IoU of the smoke pixel cluster between the nth frame and the (n-1)th frame acquired from the video camera included in the object recognition result received from the data receiving unit An integrated safety management system comprising: a persistence calculation unit that calculates a persistence coefficient based on the number of consecutive frames maintained above a reference value; a temperature rise calculation unit that calculates a temperature rise coefficient based on the rate of increase relative to the average of the last k frames of the highest temperature of the nth frame acquired from the thermal imaging camera of the camera assembly received from the data receiving unit; a charging abnormality calculation unit that calculates a charging abnormality coefficient based on the rate of change of temperature per hour at the nth measurement point of the temperature sensor received from the data receiving unit; and a risk state generation unit that generates risk state information based on the six calculated coefficients, wherein the alarm control module is characterized by differentially adjusting the output level of the alarm output from the guidance unit according to the risk state information. Claim 2 An integrated safety management system according to claim 1, wherein the AI ​​edge device includes a smoke pattern recognition model that detects smoke in the thermal runaway precursor stage of the electric vehicle based on the shape change and movement pattern of a frame-by-frame smoke pixel cluster acquired from the video camera. Claim 3 An integrated safety management system according to claim 1, wherein the AI ​​edge device further includes a vehicle type classification model that classifies the electric vehicle into a general electric vehicle and a plug-in hybrid electric vehicle based on the exterior image and license plate image of the electric vehicle acquired from the video camera, and further includes the classification result of the vehicle type classification model in the field recognition result; the controller further includes a charging allowable time calculation module that calculates the allowable charging time for each electric vehicle according to the classification result of the vehicle type classification model; and the alarm control module controls the output of a movement request alarm through the guidance unit for electric vehicles that have exceeded the allowable charging time. Claim 4 In claim 2, the controller includes an evacuation path calculation module that calculates an evacuation path based on the location of either a person object included in the site recognition result received from the AI ​​edge device at the time of fire, and the alarm control module outputs the evacuation path received from the evacuation path calculation module to the electronic display board, the integrated safety management system. Claim 5 delete Claim 6 In claim 1, the controller includes a fusion stability calculation module that calculates the multi-risk source fusion stability through the following mathematical formula 2 based on coefficients calculated from each calculation unit of the risk state determination module, and mathematical formula 2. (Here, F_fusion[n] is the n-th multiple hazard source fusion stability, B_align[n] is the n-th boundary alignment coefficient, D_area[n] is the n-th area diffusion coefficient, P_exist[n] is the n-th persistence coefficient, T_rise[n] is the n-th temperature rise coefficient, C_grad[n] is the n-th charging abnormality coefficient, L_var[n] is the n-th light source fluctuation coefficient, τ is a weighting coefficient that adjusts the reflection ratio of the temperature-based coefficient and is greater than 0, and γ is a light source fluctuation attenuation coefficient and is greater than 0) The risk state generation unit is characterized by updating the risk state information through the following Equation 3 based on the multiple hazard source fusion stability, in an integrated safety management system. Equation 3. (Here, R'_fusion[n] is the updated hazard state information of the n-th frame, R_fusion[n-1] is the hazard state information of the (n-1)-th frame, F_fusion[n] is the multi-hazard source fusion stability of the n-th frame, L_var[n] is the light source variation coefficient of the n-th frame, η is the previous state retention ratio with a value greater than 0 and less than 1, and δ is the environmental noise attenuation coefficient with a value greater than 0) Claim 7 In claim 6, the controller further includes a risk level transition threshold calculation module that calculates a risk level transition threshold through the following mathematical formula 4, and mathematical formula 4. (wherein θ_k[n] is the k-th risk level transition threshold of the n-th frame, θ_0k is the reference threshold of the k-th risk level, μ is the threshold change sensitivity with a value greater than 0, R'_fusion[n] is the updated risk state information of the n-th frame, R'_fusion[n-1] is the updated risk state information of the n-1-th frame, and Δt is the time interval between frames (s)) The above-described risk state generation unit is characterized by secondarily updating the risk state information through the following Equation 5 based on the above-described risk level transition threshold, in an integrated safety management system. Equation 5. (Here, R''_fusion[n] is the second-updated risk state information of the n-th frame, R'_fusion[n] is the updated risk state information of the n-th frame, and θ_k[n] is the k-th risk level transition threshold of the n-th frame) Claim 8 In claim 7, the controller further includes a risk correction module that calculates a corrected cumulative risk through the following mathematical formula 6, and mathematical formula 6. (wherein R_adjusted[n] is corrected risk state information of the n-th frame, R'_fusion[n] is updated risk state information of the n-th frame, C_dual is a value indicating whether the double lock condition is satisfied, κ is a value greater than 0, and Δt_dual is the duration (s) of non-satisfaction of the double lock condition when the updated risk state information is greater than or equal to the fire confirmation standard) The risk state generation unit is characterized by updating the risk state information a third time through the following mathematical formula 7 based on the corrected cumulative risk level, in an integrated safety management system. Mathematical formula 7. (Here, R'''_fusion[n] is the 3rd-order updated risk state information of the n-th frame, R_adjusted[n] is the corrected risk state information of the n-th frame, and θ_k[n] is the k-th risk class transition threshold determined from the 2nd-order updated risk state information)

Images