Fire detection apparatus and method

Abstract

Fire detection devices and methods are provided. A fire detection device can include a camera mounted on a vehicle, a processor, and a memory. The memory can store at least one instruction configured to, upon execution by the processor in communication with the memory, cause the fire detection device to: obtain one or more images of an environment of the vehicle via the camera; detect an object in the one or more images based on at least one of an object detection model or a feature point extraction algorithm; determine whether a fire event exists based on a target classification type of the object; cause the vehicle to perform one or more remedial measures based on a determination that the fire event exists.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit and priority of Korean Patent Application No. 10-2025-0014794, filed with the Korean Intellectual Property Office on February 5, 2025, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This invention relates to a fire detection device and method, and more specifically, to a technique for detecting vehicle-related fires. Background Technology

[0004] One of the design challenges facing electric vehicles is the risk of battery-related fires, which may be small but cannot be ignored.

[0005] For example, if a fire breaks out while an electric vehicle is parked in a garage, it may be difficult to detect and report the incident to authorities in a timely manner, and the fire may be difficult to respond to. Delayed response can further increase the extent of damage that the fire may cause.

[0006] Therefore, there is a need for a more effective technology that can immediately detect fires (e.g., fires caused by batteries) occurring in or around parked electric vehicles.

[0007] The matters described in this background section are only intended to enhance the understanding of the background of the invention and should not be construed as an admission that they correspond to prior art known to those skilled in the art. Summary of the Invention

[0008] The present invention aims to solve the above-mentioned problems that occur in at least some of the embodiments, while fully retaining the advantages achieved by those embodiments.

[0009] One aspect of the present invention provides a fire detection device and method utilizing a camera installed on a vehicle.

[0010] The technical problems to be solved by the present invention are not limited to those described above, and any other technical problems not mentioned herein will be clearly understood by those skilled in the art from the following description.

[0011] According to one or more exemplary embodiments of the present invention, a fire detection device may include: a camera device mounted on a vehicle; a processor; and a memory. The memory may store at least one instruction configured to, when executed by the processor communicating with the memory, cause the fire detection device to: acquire one or more images of the vehicle environment via the camera device; detect targets in the one or more images based on at least one target detection model or feature point extraction algorithm; determine whether a fire event exists based on the target classification type of the targets; and, based on the determination that a fire event exists, cause the vehicle to perform one or more remedial measures.

[0012] At least one instruction can be configured to, when executed by a processor communicating with memory, cause the fire detection device to acquire one or more images in such a way as to acquire one or more images when the vehicle is parked.

[0013] At least one instruction can be configured to, when executed by a processor communicating with memory, cause the fire detection device to acquire one or more images by acquiring one or more images at predetermined time intervals via a camera device.

[0014] Object detection models can include AI-based models that have been pre-trained to detect objects in photographic images.

[0015] At least one instruction can be configured, when executed by a processor communicating with memory, to cause the fire detection device to detect a target by inputting one or more images into a target detection model.

[0016] At least one instruction can be configured, when executed by a processor communicating with memory, to cause the fire detection device to detect a target by: extracting feature points and feature point descriptors from one or more images using a feature point extraction algorithm; and detecting a target corresponding to the feature points and descriptors by comparing predetermined reference target information with the feature points and descriptors.

[0017] At least one instruction can be configured, when executed by a processor communicating with the memory, to further cause the fire detection device to determine, after detecting a target, whether at least one battery parameter of the battery is within a predetermined range via the vehicle's battery management system.

[0018] At least one instruction can be configured, when executed by a processor communicating with the memory, to cause the fire detection device to determine whether a fire event exists by further determining whether a fire event exists based on whether at least one battery parameter of the battery is within a predetermined range.

[0019] The vehicle may be a first vehicle. At least one instruction may be configured, when executed by a processor communicating with the memory, to cause the fire detection device to determine that a fire event exists by: determining that the fire event exists in a second vehicle different from the first vehicle based on at least one battery parameter of the battery being within a predetermined range and the target classification type being smoke or flame.

[0020] At least one instruction can be configured, when executed by a processor communicating with memory, to cause a fire detection device to determine that a fire event exists in a vehicle by determining that at least one battery parameter of the battery is outside a predetermined range and the target classification type is smoke or flame.

[0021] According to one or more exemplary embodiments of the present invention, a method performed by a vehicle device may include: acquiring one or more images of the vehicle environment via a camera device mounted on the vehicle; detecting targets in the one or more images based on at least one target detection model or feature point extraction algorithm; determining whether a fire event exists based on the target classification type of the targets; and, based on determining that a fire event exists, causing the vehicle to perform one or more remedial measures.

[0022] Obtaining one or more images may include obtaining one or more images while the vehicle is parked.

[0023] Acquiring one or more images may include acquiring one or more images via a camera device at predetermined time intervals.

[0024] Object detection models can include AI-based models that have been pre-trained to detect objects in photographic images.

[0025] The detection target can include: inputting one or more images into the target detection model.

[0026] The detection target may include: extracting feature points and feature point descriptors from one or more images using a feature point extraction algorithm; and detecting the target corresponding to the feature points and descriptors by comparing predetermined reference target information with the feature points and descriptors.

[0027] The method may further include: after detecting the target, determining whether at least one battery parameter of the battery is within a predetermined range via the vehicle's battery management system.

[0028] Determining whether a fire event exists may include further determining whether a fire event exists based on whether at least one battery parameter of the battery is within a predetermined range.

[0029] The vehicle can be the first vehicle. Determining the existence of a fire event can include: determining that the fire event exists in a second vehicle, different from the first vehicle, based on at least one battery parameter being within a predetermined range and the target classification type being smoke or flame.

[0030] According to one or more exemplary embodiments of the present invention, the vehicle may include a camera device, a temperature sensor, a processor, and a memory. The memory may store at least one instruction configured to, when executed by the processor communicating with the memory, cause the vehicle to: acquire one or more images of the vehicle environment via the camera device; detect targets in the one or more images based on at least one target detection model or feature point extraction algorithm; determine whether a fire event exists based on a temperature measurement from the temperature sensor and based on the target's target classification type; and, based on the determination that a fire event exists, cause the vehicle to perform autonomous driving operations. Attached Figure Description

[0031] The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings:

[0032] Figure 1 This is a block diagram illustrating the configuration of an example fire detection device;

[0033] Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6 , Figure 7 and Figure 8 This is a flowchart illustrating a fire detection method performed by an example fire detection device;

[0034] Figure 9 This is a schematic diagram illustrating the results of detecting targets included in an image by utilizing an example algorithm for target recognition;

[0035] Figure 10 This is a schematic diagram illustrating the results of detecting targets included in an image by utilizing an example algorithm for target recognition;

[0036] Figure 11 An example computing system is shown. Detailed Implementation

[0037] In the following description, one or more exemplary embodiments of the invention will be described in detail with reference to the accompanying drawings. When adding reference numerals to components in each drawing, it should be noted that the same components are designated by the same numerals even if they are shown in other drawings. Furthermore, in describing exemplary embodiments of the invention, detailed descriptions of well-known features or functions may be omitted to avoid unnecessarily obscuring the spirit of the invention.

[0038] In describing components according to exemplary embodiments of the invention, terms such as first, second, "A", "B", (a), (b), etc., may be used. These terms are merely intended to distinguish one component from another, and they do not limit the nature, order, or sequence of the respective components. Furthermore, unless otherwise defined, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning equivalent to that in the context of the relevant field and should not be interpreted as having an ideal or overly formal meaning unless expressly defined as having such a meaning in this application.

[0039] For the purposes of this application and claims, the exemplary phrases “at least one of A, B, or C” or “at least one of A, B, or C” are used, which means “at least one A, or at least one B, or at least another C, or any combination of at least one A, at least one B, and at least one C.” Furthermore, exemplary phrases such as “A, B, or C,” “at least one of A, B, and C,” “at least one of A, B, or C,” etc., as used herein, may refer to each of the listed items or all possible combinations of the listed items. For example, “at least one of A or B” may refer to (1) at least one A; (2) at least one B; or (3) at least one A and at least one B.

[0040] According to the Society of Automotive Engineers (SAE), the automation levels of autonomous vehicles can be classified as follows: Level 0, corresponding to "No Automation," involves the autonomous driving system temporarily engaging and / or only providing warnings (e.g., blind spot warning, lane departure warning) in emergency situations (e.g., automatic emergency braking), requiring the driver to operate the vehicle. Level 1, corresponding to "Driver Assistance," involves the system performing some driving functions (e.g., steering, acceleration, braking, lane centering, adaptive cruise control), while the driver operates the vehicle in normal driving conditions. The driver should determine the system's operating status and / or timing, perform other driving functions, and respond to (e.g., resolve) emergency situations. Level 2, corresponding to "Partial Automation," involves the system performing steering, acceleration, and / or braking under driver supervision. The driver should determine the system's operating status and / or timing, perform other driving functions, and respond to (e.g., resolve) emergency situations. At Level 3 of autonomous driving, the SAE classification standard can correspond to "conditional automation," where the system drives the vehicle under defined conditions (e.g., performing driving functions such as steering, acceleration, and / or braking), but transfers driving control to the driver when the required conditions are not met. The driver is responsible for determining the system's operating state and / or timing, and taking over control in emergency situations, but not operating the vehicle (e.g., steering, acceleration, and / or braking) under other circumstances. At Level 4 of autonomous driving, the SAE classification standard can correspond to "high automation," where the system performs all driving functions, and the driver should only take control of the vehicle in emergency situations. At Level 5 of autonomous driving, the SAE classification standard can correspond to "full automation," where the system performs all driving functions without any driver assistance (including in emergency situations), and the driver does not need to perform any driving functions other than determining the system's operating state. While this invention can apply the SAE classification standard to autonomous driving classification, other classification methods and / or algorithms may be used in one or more configurations described herein.

[0041] One or more features related to autonomous driving control can be activated based on configured autonomous driving control settings (e.g., based on at least one of: autonomous driving classification, selection of vehicle autonomous driving level, etc.). Based on one or more features described herein (e.g., detecting an emergency such as a fire), vehicle operation can be controlled. Vehicle control can include various vehicle-related operational controls (e.g., autonomous driving control, sensor control, braking control, braking timing control, acceleration control, rate of change of acceleration control, warning timing control, forward collision warning timing control, etc.). For example, one or more auxiliary devices (e.g., engine brakes, exhaust brakes, hydraulic retarders, electromagnetic retarders, regenerative brakes, etc.) can also be controlled based on one or more features described herein (e.g., detecting an emergency such as a fire).

[0042] For example, one or more communication devices (e.g., modems, network adapters, radio transceivers, antennas, etc., capable of communicating via one or more wired or wireless communication protocols, such as Ethernet, Wi-Fi, Near Field Communication (NFC), Bluetooth, Long Term Evolution (LTE), 5G New Radio (5G NR), Vehicle-to-Everything (V2X) etc.) can also be controlled based on one or more features described herein (e.g., detecting emergencies such as fire incidents).

[0043] For example, minimum risk strategy (MRM) operations can also be controlled based on one or more features described herein (e.g., detecting emergency situations such as fires). Minimum risk strategy operations (e.g., risk minimization strategy, minimum risk policy) can be strategy operations that minimize (e.g., reduce) the risk of collision between the vehicle and surrounding vehicles, achieving a reduced (e.g., minimum) risk state. Minimum risk strategies can be operations activated during autonomous driving when the driver is unable to respond to intervention requests. During a minimum risk strategy, one or more processors in the vehicle can control the vehicle's driving operations for a set time period.

[0044] For example, biased driving operations can also be controlled based on one or more features described herein (e.g., detecting an emergency such as a fire). The driving control unit can perform biased driving control. To perform biased driving, the driving control unit can control the vehicle to travel within the lane by maintaining a lateral distance between the vehicle's center position and the lane center. For example, the driving control unit can control the vehicle to remain within the lane but not in the lane center. The driving control unit can identify or determine a target lateral distance for biased driving control. For example, during strategies such as lane changes, the target lateral distance may include an intentionally adjusted lateral distance that the vehicle may aim to maintain relative to a reference point (e.g., the lane center or another vehicle). Such adjustments can be made to improve the vehicle's stability, safety, and / or performance under different driving conditions. For example, during lane changes, the driving control system can bias the lateral distance to maintain a safer distance from adjacent vehicles, taking into account factors such as vehicle speed, road conditions, and / or the presence of obstacles.

[0045] For example, one or more sensors (e.g., IMU sensors, cameras, LiDAR, RADAR, blind spot monitoring sensors, lane departure warning sensors, parking sensors, light sensors, rain sensors, traction control sensors, anti-lock braking system sensors, tire pressure monitoring sensors, seat belt sensors, airbag sensors, fuel sensors, emission sensors, throttle position sensors, inverters, converters, motor controllers, power distribution units, high-voltage lines and connectors, auxiliary power modules, charging interfaces, etc.) can be controlled based on one or more features described herein (e.g., detecting an emergency such as a fire). Operational control for autonomous driving of a vehicle can include various driving controls of the vehicle by vehicle control units (e.g., acceleration, deceleration, steering control, gear shifting control, braking system control, traction control, stability control, cruise control, lane keeping assist control, collision avoidance system control, emergency braking assist control, traffic sign recognition control, adaptive headlight control, etc.). Based on the determination of a fire event, operational control of autonomous driving (e.g., remedial measures) can be, for example, leaving (e.g., escaping) the area or location (e.g., parking lot) associated with the fire event by activating one or more autonomous driving functions. Operational controls (e.g., remedial measures) may also include, for example, cooling the battery to reduce its temperature, sending emergency messages, etc.

[0046] A vehicle equipped with a fire detection device may be referred to as the vehicle itself or the main vehicle. The vehicle itself may be, for example, an autonomous vehicle (also known as an autonomous driving vehicle, self-driving car, driverless vehicle, robotaxi, robotic vehicle, or intelligent vehicle). A vehicle in front of the vehicle (e.g., in the same lane as the vehicle) may be referred to as the preceding vehicle (e.g., the vehicle directly in front), the lead vehicle, the leading vehicle, or the preceding vehicle. A vehicle following the vehicle (e.g., in the same lane as the vehicle) may be referred to as the following vehicle, the trailing vehicle, the following vehicle, or the vehicle after it. An adjacent vehicle may refer to any vehicle (e.g., moving or parked) located in any direction (e.g., in front, behind, to the left, to the right, diagonally, etc.) of the vehicle, provided that there are no other vehicles (e.g., intermediate vehicles) between it and the vehicle (e.g., regardless of distance). Alternatively, in some cases, only those vehicles located within a threshold distance of the vehicle (e.g., the detection range and / or detection limit of one or more sensors of the vehicle) may be referred to as adjacent vehicles. The target vehicle can be any vehicle (including any parked vehicle) near this vehicle (e.g., within a threshold distance of this vehicle). The target vehicle can be any vehicle that the fire detection device actively or passively monitors, identifies, confirms, tracks, and / or analyzes, once or multiple times, occasionally or continuously. The threshold distance can be, for example, the detection range and / or detection limit of one or more sensors on this vehicle, but the threshold distance can be a value smaller than the detection range and / or detection limit of one or more sensors on this vehicle (e.g., an adjustable value). The target vehicle can be, for example, a vehicle in front, a vehicle behind, a parked vehicle, a passing vehicle, an adjacent vehicle (e.g., regardless of its distance from this vehicle and / or regardless of whether there is an intermediate vehicle between the target vehicle and this vehicle), etc. The target vehicle can also be referred to as surrounding vehicles, nearby vehicles, external vehicles, another vehicle (other vehicles), etc.

[0047] In the following text, reference will be made to Figures 1 to 11 One or more exemplary embodiments of the present invention will be described in detail.

[0048] Figure 1 This is a block diagram illustrating the configuration of an example fire detection device.

[0049] The fire detection device 100 can be a computing device capable of performing calculations. For example, the fire detection device 100 can be installed on a smartphone, robot, vehicle, and / or mobile device. For example, the fire detection device 100 can be implemented inside a vehicle. In this case, the fire detection device 100 can be integrated with the vehicle's internal control unit; alternatively, the fire detection device 100 can be implemented as a separate device and can be connected to the vehicle's control unit via a separate connection device.

[0050] Reference Figure 1 The fire detection device 100 may include a camera device 110, a memory 120, and a processor 130. The configuration of the fire detection device 100 is not limited to... Figure 1 The example shown. For example, the fire detection device 100 may further include some components, or some components of the fire detection device 100 may be omitted. The fire detection device 100 may further include an output device (not shown). For example, the output device of the fire detection device 100 may include a display device included in a vehicle. The display device may visually display vehicle-related information (e.g., information from a camera device installed in the vehicle or images captured by the camera device).

[0051] The imaging device 110 may include at least one image sensor, such as a charge-coupled device (CCD) image sensor, a complementary metal-oxide-semiconductor (CMOS) image sensor, a charge-initiating device (CPD) image sensor, and / or a charge-injecting device (CID) image sensor. The imaging device 110 may include an image processor that performs image processing on the image acquired by the image sensor, such as noise reduction, color reproduction, file compression, image quality control, and / or saturation control.

[0052] Memory 120 may store information in any suitable format generated or determined by processor 130, and / or information in a suitable format received by a communication device (not shown). For example, memory 120 may store algorithms for target recognition (e.g., AI-based target detection models and / or feature point extraction algorithms). As another example, memory 120 may store reference target information. Memory 120 may operate under the control of processor 130.

[0053] The memory 120 may include at least one type of storage medium, such as flash memory, hard disk memory, multimedia card micro-memory, card-type memory (e.g., Security Digital (SD) cards and / or XD cards), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), programmable ROM (PROM), magnetic storage, magnetic disk, and / or optical disk. The fire detection device 100 may operate in conjunction with a network storage device that performs the storage functions of the memory 120 over the Internet. The above description relating to the memory 120 is provided by way of example only, and the invention is not limited thereto.

[0054] The processor 130 may be implemented with one or more cores and may include a processor for performing operations associated with data processing of the central processing unit (CPU), general-purpose graphics processing unit (GPGPU), tensor processing unit (TPU), etc. of the fire detection device 100.

[0055] Processor 130 can perform computations for learning an artificial intelligence-based model. For example, processor 130 can perform computations for learning an artificial intelligence-based model, such as processing input data for learning in deep learning (DL), extracting features from the input data, calculating errors, and updating the weights of the artificial intelligence-based model using backpropagation. Processor 130 can handle the learning of network functions. Furthermore, processor 130 can perform both the learning of network functions and the data processing of network functions using processors that utilize multiple computing devices simultaneously.

[0056] The processor 130 typically controls all operations of the fire detection device 100. The processor 130 can process signals, data, information, etc., input or output through components included in the fire detection device 100, or can run applications stored in the memory 120, thereby providing user-friendly information or processing user-friendly functions.

[0057] This article refers to Figures 2 to 8 Describe in detail one or more example fire detection methods. Figures 2 to 8 This is a flowchart illustrating a fire detection method performed by an example fire detection device.

[0058] In this article, it is assumed that Figure 1 Fire detection device 100 performs Figures 2 to 8 The process. Furthermore, in reference... Figures 2 to 8 The description given, which describes the operation performed by the device, can be understood as being controlled by the processor 130 of the fire detection device 100.

[0059] Reference Figure 2 The fire detection device 100 can obtain one or more images (e.g., photographic images) from the camera device 110 installed on the vehicle (S200). The one or more images can be the environment of the vehicle (e.g., the interior environment or the exterior environment).

[0060] An image can be a representation of a target formed by the refraction or reflection of light. An image can be a visual representation formed by light rays arranged on a plane or in space, or by a medium that visualizes information. For example, images can include still images, moving images, etc. However, the types of images are not limited to these. An image can correspond to information obtained by a camera device.

[0061] The camera device 110 may include a built-in CAM and / or an autonomous driving camera device installed in the vehicle.

[0062] Reference Figure 3 If the vehicle is in a power-off state (IG OFF), the fire detection device 100 can obtain images by using a camera device at predetermined time intervals (e.g., at predetermined time intervals, such as every minute) (S210).

[0063] A power-off state can refer to a state where the vehicle's power is disconnected or reduced. In a power-off state, only minimal or reduced power can be used, and the vehicle's main electrical systems (such as the drivetrain, dashboard, infotainment system, air conditioning system, etc.) may be inactive and / or in low-power mode. For example, for internal combustion engine vehicles, the ignition system may be turned off when the vehicle is in a power-off state. For example, a power-off state can be activated when the vehicle is parked. A power-off state can also be referred to as a low-power state, low-power mode, parking mode, parking state, etc. If the vehicle is in a power-off state, the vehicle may be parked. Accordingly, when the vehicle is parked, the fire detection device 100 can be activated to detect a fire (also known as a fire incident, fire accident, fire outbreak, runaway fire, etc.) by acquiring images using a camera device at given time intervals and detecting targets in the images.

[0064] The fire detection device 100 can operate without acquiring an image while the vehicle is in motion. For example, refer to... Figure 4 Before obtaining an image from the camera installed on the vehicle (e.g., before S200), if the vehicle is in a powered-on state (IG ON), the fire detection device 100 can wait until the vehicle enters a powered-off state (IG OFF) (S100).

[0065] The energized state (which is the state in which the vehicle's power is on) can be a state in which most (e.g., a predetermined group) of the vehicle's electrical devices are activated. For example, in the energized state, the vehicle may be ready for driving operations (e.g., the vehicle is ready to move by pressing the accelerator pedal). For example, for an internal combustion engine vehicle, if the vehicle is in the energized state, the ignition system can be started. The energized state can also be referred to as a ready-to-drive state, a ready state, a driving mode, a driving state, etc. Accordingly, if the vehicle is in motion, the fire detection device 100 may not perform the fire detection process; the fire detection device 100 may only perform the fire detection process when the vehicle is parked.

[0066] Back Figure 2 The fire detection device 100 can detect targets included in an image by utilizing an algorithm for target recognition (S300).

[0067] Algorithms for target recognition may include at least one of an AI-based target detection model and a feature point extraction algorithm.

[0068] Reference Figure 5 The fire detection device 100 can detect targets included in the image by inputting the image into the target detection algorithm (S310).

[0069] In the specification, an AI-based model can be implemented using a set of interconnected computational units, typically referred to as "nodes." Nodes can be called "neurons." An AI-based model is implemented by including at least one or more nodes. The nodes (or neurons) constituting the AI-based model can be interconnected via one or more links.

[0070] In AI-based models, one or more nodes connected via links can form relative relationships between input and output nodes. The concepts of input and output nodes can be relative. For example, any node that has an output node relationship with one node can have an input node relationship with any other node, and vice versa. As mentioned above, relationships between input and output nodes can be generated based on links. One or more output nodes can be connected to an input node via a link, and vice versa.

[0071] In a relationship between input and output nodes connected by a link, the value of the output node's data can be determined based on the data input to the input nodes. In this paper, the links connecting input and output nodes can have weights. These weights can be variable and can be altered by the user or an algorithm based on an AI model to perform the desired function. For example, if one or more input nodes are connected to an output node via corresponding links, the output node's value can be determined based on the values ​​input to the input nodes connected to it, as well as the weights assigned to the links corresponding to each input node.

[0072] As mentioned above, in an AI-based model, one or more nodes can be interconnected via one or more links to form relationships between input and output nodes. The characteristics of an AI-based model can be determined based on the number of nodes, the number of links, the relationships between nodes and links, or the weights assigned to each link. For example, if two AI-based models exist with the same number of nodes and links but different link weights, these two models can be identified as different from each other.

[0073] Artificial intelligence-based models can be implemented using a set of one or more nodes. A subset of nodes constituting an AI-based model can form a layer. Some nodes constituting an AI-based model can form a layer based on their distance from the initial input node. For example, a set of nodes each a distance "n" from the initial input node can form the nth layer. The distance from the initial input node can be defined by the minimum number of links required for information to travel from the initial input node to the corresponding node. However, the layer definition provided in this paper is an example; the order of layers in an AI-based model can be defined in other ways. For example, a layer of nodes can be defined by its distance from the final output node.

[0074] A collection of neurons or nodes can be defined by an expression called a layer.

[0075] An initial input node can refer to one or more nodes in an AI-based model that are not directly input through linked data in relation to any other nodes. Alternatively, in an AI-based model network, an initial input node can refer to a node that, in relation to linked nodes, does not include any other input nodes connected by links. As mentioned above, a final output node can refer to one or more nodes in an AI-based model that do not have an output relationship with any other nodes. Furthermore, a hidden node can refer to any node in the AI-based model other than the initial input node and the final output node.

[0076] In AI-based models, the number of nodes in the input layer can be the same as the number of nodes in the output layer, and the number of nodes can decrease and then increase as you move from the input layer to the hidden layer.

[0077] In AI-based models, the number of nodes in the input layer can be less than the number of nodes in the output layer, and the number of nodes can decrease as you move from the input layer to the hidden layer.

[0078] In AI-based models, the number of nodes in the input layer can be greater than the number of nodes in the output layer, and the number of nodes can increase as you move from the input layer to the hidden layer.

[0079] Artificial intelligence-based models can be combinations of the aforementioned artificial intelligence-based models.

[0080] Artificial intelligence-based models can include deep neural networks (DNNs) (or models based on deep artificial intelligence). A deep neural network can refer to an AI-based model that includes multiple hidden layers in addition to input and output layers. Using deep neural networks, the underlying structure of data can be identified. That is, the underlying structure of an image can be identified (e.g., whether there are any objects in the image). Deep neural networks can include convolutional neural networks (CNNs), recurrent neural networks (RNNs), transformers, etc. The above description of deep neural networks is provided as an example only, and the invention is not limited thereto.

[0081] Artificial intelligence-based models can be represented by any suitable network structure described above, which includes an input layer, hidden layers, and an output layer.

[0082] Neural networks that can be used in artificial intelligence-based models can be trained through at least one of supervised learning, unsupervised learning, semi-supervised learning, transfer learning, active learning, and / or reinforcement learning. Training a neural network can correspond to the process of applying the knowledge required for the neural network to perform specific operations.

[0083] A neural network can be trained to minimize the error in its output. The training of a neural network can correspond to the process of updating the weights of each node of the neural network by: (1) iteratively inputting training data into the neural network, (2) calculating the error of the neural network output and the target in relation to the training data, and (3) propagating the error of the neural network backward from the output layer to the input layer in the direction of reducing the error.

[0084] In supervised learning, training data with the correct answers labeled in each training data set can be used (e.g., using labeled training data); in unsupervised learning, the correct answers may not be labeled in each training data set.

[0085] For example, in supervised learning related to data classification, the training data can be data labeled with categories for each training data point. The error can be calculated by feeding the labeled training data into a neural network and comparing the network's output (the category) with the labels on the training data.

[0086] As another example, in unsupervised learning related to data classification, error can be calculated by comparing the training data as input with the output of the neural network. The calculated error can be backpropagated in the neural network in the opposite direction (e.g., from the output layer to the input layer), and the connection weights of each node in each layer of the neural network can be updated through backpropagation. The amount of change in the updated connection weights of each node can be determined based on the learning rate. The computation of the neural network's input data and the backpropagation of the error constitute a training cycle. The learning rate can be applied differently depending on the number of iterations in the neural network's training cycle. For example, a high learning rate can be used in the early stages of neural network training to improve efficiency, that is, to enable the neural network to quickly secure a given performance level; a low learning rate can be used in the later stages of training to improve accuracy.

[0087] In training neural networks, training data is generally a subset of the actual data (e.g., data processed using the trained neural network), which can reduce the error of the training data. However, there may be training cycles where the error of the actual data increases. Overfitting is the phenomenon where excessive learning on the training data leads to an increase in the error of the actual data. For example, a neural network that learns cats by showing yellow cats may fail to recognize other cats besides yellow cats after seeing cats, which could be an example of overfitting. Overfitting can lead to an increase in the error of machine learning algorithms. Various optimization methods can be used to prevent overfitting. Increasing the training data can prevent overfitting; alternatively, regularization, dropout (disabling some nodes of the network during the learning process), and batch normalization layers can be used.

[0088] Object detection models can correspond to artificial intelligence-based models that are pre-trained to detect objects in images (such as photographic images) using a dataset.

[0089] A dataset can refer to a collection of data used to train and validate a neural network. A dataset can include a training dataset and / or a validation dataset. A training dataset can be a collection of data that includes training images and information about objects in the training images. For example, a training dataset can be a collection of data used in the process of training an object detection model. A validation dataset can be a collection of data used to evaluate an object detection model.

[0090] Object detection models can include AI-based models that are pre-trained to receive images and detect objects within them. For example, object detection models can include AI-based models from the R-CNN (Region-Based CNN) family and the You Only Look Once (YOLO) family. However, the types of object detection models are not limited to these.

[0091] Object detection models can generate bounding boxes corresponding to objects. For example, an object detection model can be trained or operated to receive an image and output bounding boxes corresponding to objects in the image.

[0092] A target can refer to something captured by a camera device. A target can refer to something that exists or can be imagined in the real or virtual world. For example, a target can include smoke, flames, vehicles, license plates, etc. However, the types of targets are not limited to these.

[0093] Target information, which is related to the target, may include information about the category (also known as classification type, target classification type, or target type) of the target (e.g., smoke, fireworks, vehicle, or license plate), location information including target coordinates (e.g., bounding box), size information about the target size, area information about the target area, shape information about the target shape, identification (ID) information for identifying the target (e.g., identifier), etc.

[0094] Reference Figure 6 The fire detection device 100 can extract feature points and feature point descriptors from the image by using a feature point extraction algorithm (S320).

[0095] Feature point extraction algorithms can include scale-invariant feature transform (SIFT).

[0096] Feature points can be coordinates corresponding to each part represented in an image. Feature points can be represented by vectors that include information about direction, magnitude, gradient, etc. The gradient can be the vector obtained by taking the partial derivative of the loss function with respect to each parameter. The gradient can indicate the upward direction of the steepest slope on the surface represented by the loss function.

[0097] A feature point descriptor can refer to information used to describe each feature point. For example, for each feature point, the feature point descriptor can include information about orientation, magnitude, and / or relationships with surrounding pixels.

[0098] The fire detection device 100 can detect targets corresponding to feature points and descriptors by comparing pre-stored (e.g., predetermined) reference target information with feature points and descriptors (S330).

[0099] Reference target information, which serves as information mapped to the target, may include information about the target, such as its kind (e.g., type), location, size, shape, and / or ID.

[0100] Reference Figure 7After detecting targets included in an image by using an algorithm for target recognition (S300), the fire detection device 100 can determine whether the battery (e.g., the vehicle's battery) is abnormal by using a battery management system (BMS).

[0101] A battery management system can refer to a system that monitors one or more attributes (such as temperature, voltage, current, etc.) of a battery installed in a vehicle. The battery management system can be installed in a vehicle. The battery management system can be controlled by a processor 130.

[0102] If the results of checking the battery using the battery management system indicate that all values ​​(e.g., battery parameters such as temperature, voltage, current, state of charge (SoC), state of health (SoH), impedance, battery cell balance, etc.) are within the normal range (e.g., within the predetermined, expected, or safe range of values), then the fire detection device 100 can determine that the battery is normal (not abnormal).

[0103] If the results of checking the battery using the battery management system indicate that at least one value (e.g., battery temperature, voltage, or current) is outside the normal range, the fire detection device 100 can determine that the battery is abnormal.

[0104] An abnormal battery condition can indicate the potential for a fire hazard caused by the battery. In other words, an abnormal battery may have (e.g., can be determined to have) a fire risk level above a threshold. An abnormal battery condition may indicate that one or more attribute values ​​(e.g., temperature, voltage, current, etc.) are outside of a predetermined (e.g., normal, expected, safe, etc.) range.

[0105] Back Figure 2 The fire detection device 100 can determine whether any fire event exists based on the category of the target (S500).

[0106] Reference Figure 8 The fire detection device 100 can determine whether any fire event exists based on whether the battery is abnormal and the category of the target (S510).

[0107] For example, if the battery is normal and the target category is smoke or flame, the fire detection device 100 can determine that the fire occurred in any other vehicle and not in its own vehicle.

[0108] In another example, if the battery is abnormal and the category of the target corresponds to smoke or flame, the fire detection device 100 can determine that a fire has occurred in the vehicle.

[0109] References in this article Figure 9 and Figure 10 Describe the target detection results in detail. Figure 9This is a schematic diagram illustrating the results of detecting targets included in an image by utilizing an example algorithm for target recognition. Figure 10 This is a schematic diagram illustrating the results of detecting targets included in an image by utilizing an example algorithm for target recognition.

[0110] Reference Figure 9 The fire detection device 100 can detect targets 220 and 230 included in image 210 by using an algorithm for target recognition. The first target 220 can be classified as smoke, and the second target 230 can be classified as a vehicle. The fire detection device 100 can determine whether the battery is abnormal by using a battery management system.

[0111] Based on the determination that there is no abnormality in the battery, the fire detection device 100 can determine that the fire occurred (e.g., a fire event) in the vehicle corresponding to the second target 230 (e.g., the target vehicle), rather than in the vehicle equipped with the camera device 110.

[0112] Based on the determination of a battery malfunction, the fire detection device 100 can determine that the fire occurred in a vehicle equipped with a camera device 110.

[0113] The fire detection device 100 can send a notification of a fire emergency (e.g., message-type data, image-type data, or voice-type data) along with the location information of the vehicle corresponding to the vehicle equipped with the camera device 110 and / or the location information of the vehicle corresponding to the second target 230 to an external device (e.g., a device corresponding to a police station, fire station, management office, emergency room of a nearby major institution, or an identifiable consumer).

[0114] Reference Figure 10 The fire detection device 100 can detect targets 320, 330, and 340 included in image 310 by utilizing an algorithm for target recognition. The third target 320 can be categorized as flame, the fourth target 330 as a vehicle, and the fifth target 340 as a license plate. The fire detection device 100 can determine whether the battery is abnormal by using a battery management system.

[0115] Based on the determination that there is no abnormality in the battery, the fire detection device 100 can determine that the fire occurred in the vehicle corresponding to the fourth target 330, rather than in the vehicle equipped with the camera device 110.

[0116] Based on the determination of a battery malfunction, the fire detection device 100 can determine that the fire occurred in a vehicle equipped with a camera device 110.

[0117] Based on the determination of the existence of a fire event, the fire detection device 100 may take one or more measures. For example, the fire detection device 100 may take one or more remedial measures, such as sending notifications (e.g., alarms, emergency messages, etc.), controlling one or more devices or components of a vehicle (e.g., locking or unlocking doors, raising or lowering windows, turning fans on or off, opening or closing ventilation, etc.). For example, the fire detection device 100 may send a notification of the fire outbreak (e.g., message-type data, image-type data, or voice-type data) along with the location information of the vehicle corresponding to the vehicle equipped with camera device 110 and / or the location information of the vehicle corresponding to the fourth target 330 to an external device (e.g., a device corresponding to a police station, fire station, management office, emergency room of a nearby major institution, or a device associated with the vehicle owner and / or driver). For example, the fire detection device 100 may identify the numbers described in the fifth target 340. The fire detection device 100 may obtain the phone number of the corresponding consumer by comparing pre-stored consumer information (e.g., the consumer's vehicle license plate and phone number) with the numbers identified from the fifth target 340. The fire detection device 100 can send a notification of a fire emergency along with the location information of the vehicle equipped with the camera device 110 and / or the location information of the vehicle corresponding to the fourth target 330 by using the consumer's telephone number.

[0118] Figure 11 An example computing system is shown. One or more components of the fire detection device configuration can be configured as follows: Figure 11 The computing system 1000 shown is used to implement this.

[0119] Reference Figure 11 The computing system 1000 may include at least one processor 1100, a memory 1300, a user interface input device 1400, a user interface output device 1500, a storage device 1600, and a network interface 1700 connected via a bus 1200.

[0120] Processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in memory 1300 and / or storage device 1600. Memory 1300 and storage device 1600 may include various types of volatile or non-volatile storage media. For example, memory 1300 may include read-only memory (ROM) 1310 and random access memory (RAM) 1320.

[0121] Therefore, the operation of the methods or algorithms described herein can be directly embodied in one or more hardware modules, one or more software modules executed by processor 1100, or a combination thereof. The software modules can reside in storage media (e.g., memory 1300 and / or storage device 1600), such as RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disks, removable disks, and CD-ROMs.

[0122] Example storage media can be coupled to processor 1100. Processor 1100 can read information from and write information to the storage media. Alternatively, the storage media can be integrated with processor 1100. The processor and storage media can reside in an application-specific integrated circuit (ASIC). The ASIC can reside within the user terminal. Alternatively, the processor and storage media can reside as separate components in the user terminal.

[0123] According to one aspect of the invention, the fire detection device may include a camera, a processor, and a memory installed on a vehicle.

[0124] The processor can acquire images from a camera device, detect targets included in the images by utilizing a target detection algorithm (including at least one target detection model and feature point extraction algorithm), and determine whether a fire has occurred based on the category of the target.

[0125] If the vehicle is powered on (IG ON), the processor can wait until the vehicle enters the powered-off state (IG OFF).

[0126] If the vehicle is in a power-off state (IG OFF), the processor can acquire images by using a camera device at predetermined time intervals.

[0127] Object detection models can include artificial intelligence-based models that are pre-trained to detect objects in images.

[0128] The processor can detect objects in an image by feeding the image into an object detection model.

[0129] The processor can extract feature points and feature point descriptors from an image by using a feature point extraction algorithm, and can detect targets corresponding to feature points and descriptors by comparing pre-stored reference target information with the feature points and descriptors.

[0130] After detecting the target, the processor can use the battery management system (BMS) to determine if the battery is abnormal.

[0131] The processor can determine whether a fire has occurred based on whether the battery is abnormal or the category of the target.

[0132] If the battery is normal and the target category is smoke or fire, the processor can determine that the fire occurred in another vehicle that is not this vehicle.

[0133] If the battery malfunctions and the target category is smoke or flame, the processor can determine that a fire occurred in the vehicle.

[0134] According to one aspect of the invention, a fire detection method may include: acquiring an image from a camera device installed in a vehicle; detecting targets included in the image by utilizing a target detection algorithm (including at least one of a target detection model and a feature point extraction algorithm); and determining whether a fire has occurred based on the category of the target.

[0135] Before acquiring an image from a camera installed in the vehicle, the method may further include: if the vehicle is in a powered-on state (IG ON), waiting until the vehicle enters a powered-off state (IG OFF).

[0136] Obtaining images from a camera installed in a vehicle may include: if the vehicle is in a power-off state (IG OFF), obtaining images by using the camera at predetermined time intervals.

[0137] Detecting targets in an image by using algorithms for object recognition can include: detecting targets in an image by inputting the image into an object detection model.

[0138] Detecting targets in an image using algorithms for target recognition can include: extracting feature points and descriptors from the image using a feature point extraction algorithm; and detecting targets corresponding to feature points and descriptors by comparing pre-stored reference target information with the feature points and descriptors.

[0139] After detecting targets in an image using an algorithm for target recognition, the method may further include: determining whether the battery is malfunctioning by using a battery management system (BMS).

[0140] Determining whether a fire has occurred based on the category of the target can include: determining whether a fire has occurred based on whether the battery is abnormal or the category of the target.

[0141] Determining whether a fire has occurred based on whether the battery is abnormal or the type of target can include: if the battery is not abnormal and the type of target is smoke or flame, then the fire is determined to have occurred in another vehicle that is not this vehicle.

[0142] Determining whether a fire has occurred based on whether the battery is abnormal or the type of the target can include: if the battery is abnormal and the type of the target is smoke or flame, then it is determined that the fire occurred in the vehicle.

[0143] This technology can minimize the potential damage from a fire by using cameras installed on the vehicle to detect fires and contacting external sources, such as the fire department, the owner of the vehicle on fire, or the management office.

[0144] In the foregoing, although the invention has been described with reference to one or more exemplary embodiments and accompanying drawings, the invention is not limited thereto, but can be modified and altered by those skilled in the art without departing from the spirit and scope of the invention as claimed in the appended claims.

Claims

1. A fire detection device, comprising: The camera device is installed on the vehicle; processor; as well as A memory storing at least one instruction configured to, when executed by a processor communicating with the memory, cause the fire detection device to: One or more images of the vehicle environment are obtained via the camera device; Detect targets in one or more images based on at least one target detection model or feature point extraction algorithm; Based on the target classification type, determine whether a fire event exists; Based on the determination of a fire incident, the vehicle is instructed to perform one or more remedial measures.

2. The fire detection device according to claim 1, wherein, The at least one instruction is configured to, when executed by a processor communicating with the memory, cause the fire detection device to acquire the one or more images in the following manner: The one or more images are obtained while the vehicle is parked.

3. The fire detection device according to claim 1, wherein, The at least one instruction is configured to, when executed by a processor communicating with the memory, cause the fire detection device to acquire the one or more images in the following manner: One or more images are acquired via the camera device at predetermined time intervals.

4. The fire detection device according to claim 1, wherein, The target detection model includes an AI-based model that has been pre-trained to detect targets in photographic images.

5. The fire detection device according to claim 1, wherein, The at least one instruction is configured, when executed by a processor communicating with the memory, to cause the fire detection device to detect the target in the following manner: The one or more images are input into the target detection model.

6. The fire detection device according to claim 1, wherein, The at least one instruction is configured, when executed by a processor communicating with the memory, to cause the fire detection device to detect the target in the following manner: Feature points and feature point descriptors are extracted from one or more images using a feature point extraction algorithm; The target corresponding to the feature point and descriptor is detected by comparing the predetermined reference target information with the feature point and descriptor.

7. The fire detection device according to claim 1, wherein, The at least one instruction is configured, when executed by a processor communicating with the memory, to further cause the fire detection device to: after detecting a target, determine, via the vehicle's battery management system, whether at least one battery parameter of the battery is within a predetermined range.

8. The fire detection device according to claim 7, wherein, The at least one instruction is configured, when executed by a processor communicating with the memory, to cause the fire detection device to determine whether a fire event exists in the following manner: Further, the presence of a fire event is determined based on whether at least one battery parameter is within a predetermined range.

9. The fire detection device according to claim 8, wherein, The vehicle is a first vehicle, wherein the at least one instruction is configured, when executed by a processor communicating with the memory, to cause the fire detection device to determine the presence of a fire event in such a way as follows: Based on at least one battery parameter being within a predetermined range and the target classification type being smoke or flame, it is determined that the fire event exists in a second vehicle different from the first vehicle.

10. The fire detection device according to claim 8, wherein, The at least one instruction is configured, when executed by a processor communicating with the memory, to cause the fire detection device to determine the presence of a fire event in the following manner: A fire event is determined to exist in the vehicle if at least one battery parameter is outside a predetermined range and the target classification type is smoke or flame.

11. A method performed by a device of a vehicle, the method comprising: Obtain one or more images of the vehicle's environment via a camera device installed on the vehicle; Detect targets in one or more images based on at least one target detection model or feature point extraction algorithm; Based on the target classification type, determine whether a fire event exists; Based on the determination of a fire incident, the vehicle is instructed to perform one or more remedial measures.

12. The method according to claim 11, wherein, Obtaining one or more images includes: The one or more images are obtained while the vehicle is parked.

13. The method according to claim 11, wherein, Obtaining one or more images includes: One or more images are acquired via the camera device at predetermined time intervals.

14. The method according to claim 11, wherein, The target detection model includes an AI-based model that has been pre-trained to detect targets in photographic images.

15. The method according to claim 11, wherein, Target detection includes: The one or more images are input into the target detection model.

16. The method according to claim 11, wherein, Target detection includes: Feature points and feature point descriptors are extracted from one or more images using a feature point extraction algorithm; The target corresponding to the feature point and descriptor is detected by comparing the predetermined reference target information with the feature point and descriptor.

17. The method of claim 11, further comprising: After the target is detected, the vehicle's battery management system determines whether at least one battery parameter is within a predetermined range.

18. The method according to claim 17, wherein, Determining whether a fire has occurred includes: Further, the presence of a fire event is determined based on whether at least one battery parameter is within a predetermined range.

19. The method according to claim 18, wherein, The vehicle is the first vehicle, and the determination of a fire incident includes: Based on at least one battery parameter being within a predetermined range and the target classification type being smoke or flame, it is determined that the fire event exists in a second vehicle different from the first vehicle.

20. A vehicle comprising: Camera device; Temperature sensor; processor; as well as A memory storing at least one instruction configured to, when executed by a processor in communication with the memory, cause the vehicle to: One or more images of the vehicle environment are obtained via the camera device; Detect targets in one or more images based on at least one target detection model or feature point extraction algorithm; Based on temperature measurements from temperature sensors and the target classification type of the target, it is determined whether a fire event exists; Based on the determination of a fire incident, the vehicle is instructed to perform autonomous driving operations.

Images