Tree hole smoldering fire monitoring method and device based on thermal infrared
Through infrared image recognition methods and equipment, image fusion and processing of tree cave smoldering fires is solved, and the problem of traditional smoke detectors being difficult to detect tree cave fires in a timely manner is achieved, achieving higher detection accuracy and reliability.
Patent Information
- Application Number
- CN202411672814.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-21
- Publication Date
- 2025-03-25
AI Technical Summary
Traditional smoke detectors are difficult to detect fire sources in time in the early stages of tree hole fires, and are susceptible to environmental factors, resulting in false alarms or missed alarms, reducing the accuracy and reliability of the detection system.
Infrared image recognition methods and equipment are used to detect the target video area, extract the tree hole and thermal abnormality features, and perform image fusion and processing to achieve accurate detection of the smoldering fire in the tree hole.
It improves the detection accuracy and success rate of tree cave smoldering fire, reduces the possibility of misjudgment and misreport, and enhances the reliability of the fire detection system.
Smart Images
Figure CN119693867A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular to an infrared image and visible light image processing method, an infrared temperature detection method and a device. Background Art
[0002] Forest fires are a major threat to the global ecosystem, bringing profound impacts to the environment, society and economy every year. Forest fires not only destroy a large amount of forest resources, but also damage biodiversity, affect air quality, and aggravate climate change. In particular, tree hole fires, as a special type of forest fire, have unique burning characteristics. The hidden and chronic characteristics of the fire make it difficult for traditional smoke detection systems to effectively identify them.
[0003] Tree cavity fires usually occur inside the hollows of trees, which are filled with dead branches, fallen leaves and other combustibles. When lightning strikes or other natural factors occur, the combustibles inside the tree cavity may be ignited. Due to the closed nature of the tree cavity, when the flame burns inside, the oxygen supply is insufficient, the fire is small but the temperature is extremely high, and the burning process is slow and long. Compared with open forest fires, tree cavity fires have less smoke and lower smoke particle concentrations, so traditional smoke detectors have difficulty in detecting the fire source in time in the early stages of the fire.
[0004] At present, traditional fire warning systems mainly rely on smoke detectors such as photoelectric sensors and visible light cameras to detect smoke. Although these systems are effective in some cases, they have significant limitations when dealing with tree hole fires. First, because tree hole fires produce less smoke, traditional smoke detectors are slow to respond to trace smoke and cannot detect the source of the fire in time. Secondly, tree hole fires usually occur inside trees, and smoke diffuses slowly to the outside, further delaying detection and response time. Furthermore, existing smoke detection methods are easily affected by environmental factors such as humidity and airflow, resulting in false alarms or missed alarms, reducing the accuracy and reliability of the detection system.
[0005] Therefore, a new fire detection method and equipment are needed that can more effectively identify tree hole fires and issue an alarm in time. Summary of the invention
[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.
[0007] Some embodiments of the present disclosure propose an infrared image recognition method and device to solve the technical problems mentioned in the above background technology section.
[0008] In the first aspect, some embodiments of the present disclosure provide a thermal infrared tree hole smoldering fire monitoring method, the method comprising: performing area detection on a target video to obtain a set of visible light image frames and infrared image frames reflecting the foreground area; using deep learning to extract tree hole and thermal anomaly features from the visible light image and infrared image frames respectively; determining the intersection of the foreground area in the target image frame in the image frame set to obtain a set of image frames of suspected tree hole smoldering fire; performing image fusion on the visible light image and the infrared image to generate a fused infrared image; performing image processing on the fused infrared image to expand the color contrast in the infrared image to obtain a set of infrared images with more prominent abnormal conditions, and through image processing, retaining the portion of the image with the largest contrast with the environment to achieve accurate detection of tree hole smoldering fire.
[0009] In the second aspect, some embodiments of the present disclosure provide a thermal infrared tree hole smoldering fire monitoring device, the device comprising: an image detection device, configured to perform foreground area detection on a target video to obtain an infrared image frame set and a visible light image frame set; a computing device, configured to detect whether the suspicious tree hole smoldering fire set is an empty set, if it is an empty set, determining that the target video does not include a tree hole smoldering fire, if it is a non-empty set, proceeding to the next step; an image fusion device, configured to fuse the visible light tree hole image set with the infrared thermal anomaly set; an image processing device, configured to expand the color contrast in the infrared image, and retain the portion of the image that has the largest contrast with the environment.
[0010] One of the above-mentioned embodiments of the present disclosure has the following beneficial effects: the thermal infrared tree hole smoldering fire monitoring method of some embodiments of the present disclosure can detect tree hole smoldering fire more accurately, which makes up for the problem that the existing visible light images are difficult to identify to a certain extent, and makes full use of the temperature and appearance characteristics of tree hole smoldering fire, while reducing the difficulty of detection as much as possible. Specifically, the existing infrared image recognition method will produce misjudgment of the fire source and fire condition due to its own relative low temperature and the interference of other high-temperature heat sources in the forest, so visible light and infrared fusion detection technology is used as a supplement. Based on this, the thermal infrared tree hole smoldering fire monitoring method of some embodiments of the present disclosure adopts a variety of image comparison methods, fusing abnormal infrared images and visible light images, which can greatly improve the success rate of detecting tree hole smoldering fire and make the judgment of abnormal frames more accurate. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. The accompanying drawings are schematic, so the components and elements are not necessarily drawn to scale.
[0012] Figure 1 is a schematic diagram of an application scenario of a thermal infrared tree hole smoldering fire monitoring method according to some embodiments of the present disclosure;
[0013] Figure 2 is a flow chart of some embodiments of the thermal infrared tree hole smoldering fire monitoring method according to the present disclosure;
[0014] Figure 3 is a flow chart of visible light and infrared image extraction according to the thermal infrared tree hole smoldering fire monitoring method disclosed herein;
[0015] Figure 4 is a flow chart of image fusion and processing according to the thermal infrared tree hole smoldering fire monitoring method disclosed herein;
[0016] Figure 5 is a diagram of equipment according to some embodiments of the thermal infrared tree hole smoldering fire monitoring method of the present disclosure; DETAILED DESCRIPTION
[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.
[0018] For ease of description, only parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.
[0019] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
[0020] Figure 1 A schematic diagram showing an application scenario of the thermal infrared tree hole smoldering fire monitoring method to which some embodiments of the present disclosure can be applied.
[0021] exist Figure 1 In the application scenario shown, first, an image processing device 101 is loaded on a computer, which is composed of an infrared camera, a visible light camera, a memory and a processor. It can receive an infrared image set 102 and a visible light image set 103 taken in a target tree hole smoldering fire 105. Among them, there is an identifiable heat source 104 that may appear in the tree hole smoldering fire 105. At the same time, the image processing device also performs image recognition and anomaly analysis on two target video data, an infrared image set and a visible light image set, and obtains whether there is an abnormal image frame based on whether there is an intersection with the target feature set to be detected, and then intercepts the abnormal video clip, performs anomaly extraction through deep learning, and obtains an abnormal infrared image set 106 and an abnormal visible light image set 107. In this embodiment, the initial abnormal image frame set is an empty set that does not include any elements.
[0022] Next, the image to be detected is determined. The image processing device 101 is used to extract the pictures of the same frame in the same video from the abnormal infrared image set 106 and the abnormal visible light image set 107 and stack them to obtain a fused infrared image 108 .
[0023] Therefore, it is determined that there is a high probability of an abnormal situation in the set of the abnormal infrared image 106 and the abnormal visible light image 107 of the target video, and the final result is fed back to the image processing device 101.
[0024] Continue to refer Figure 2 , showing some specific steps of the thermal infrared tree hole smoldering fire monitoring method 200, including the following steps:
[0025] Step 201 , performing region detection on a target video to obtain a set of infrared and visible light image frames reflecting a foreground region.
[0026] In an optional implementation of some embodiments, the execution subject may perform region detection on the video by background subtraction. As an example, the execution subject may use a Gaussian mixture model to model the background, further subtract the background, and obtain a set of infrared and visible light image frames reflecting the foreground area.
[0027] Step 202: Use deep learning to extract tree hole and thermal anomaly features from visible light images and infrared image frames respectively.
[0028] In some embodiments, the execution body of the thermal infrared tree hole smoldering fire monitoring method (such as Figure 1 The image processing device shown in the figure can monitor the thermal anomaly and tree hole of the target infrared video and visible light video respectively through the image recognition model in deep learning to obtain the infrared image and visible light image set. In addition, the execution subject can sequentially determine whether each image frame in the infrared image set belongs to the category with abnormal conditions. If so, the current frame is intercepted and stored in the abnormal video set.
[0029] Step 203: Fusing the visible light image and the infrared image to generate a fused infrared image.
[0030] In some embodiments, the above-mentioned execution entity needs to extract a set of image frames of smoldering fire in a tree hole, and perform image processing on the part where the tree hole overlaps with the thermal anomaly feature, so as to generate a visible light infrared fusion image.
[0031] In some optional implementations of some embodiments, the execution subject may identify overlapping pixels that need to be eliminated through program settings to generate a static infrared image.
[0032] Step 204, performing image processing on the visible light infrared fusion image, expanding the color contrast in the infrared image, obtaining a set of infrared images with more prominent abnormal conditions, and retaining the portion of the image with the greatest contrast with the environment through image processing, so as to accurately detect the smoldering fire in the tree hole.
[0033] In some embodiments, the execution subject may use color processing software to amplify the contrast of the image, thereby obtaining a clearer and more accurate result.
[0034] Further references Figure 3 , which shows a flowchart 300 of visible light and infrared image extraction of a thermal infrared tree hole smoldering fire monitoring method, including the following contents and steps:
[0035] 301 is a step of extracting the entire visible light image.
[0036] Step 302, detect each video in the visible light video set and extract a video set of tree hole detection results.
[0037] In some embodiments, the above-mentioned execution entity can determine the tree hole position of each image in the above-mentioned visible light image set in turn through program settings.
[0038] Step 303: extract the captured video frame by frame into pictures, use deep learning to extract anomalies from the pictures, and obtain feature information corresponding to the tree hole pictures.
[0039] In some embodiments, the above-mentioned execution entity can perform frame extraction processing on the video through software, and use deep learning to extract the feature values of abnormal tree holes.
[0040] 304 is a step of extracting the entire dynamic image.
[0041] Step 305 , detect each video in the infrared video set, and capture a video set of infrared video thermal anomaly detection results.
[0042] In some embodiments, the execution subject can transmit the video to the storage device after interception through wireless network transmission or wired network. As an example, the visible light camera can use DJI's 3D modeling camera.
[0043] In some embodiments, the execution subject may also select video frames in the video stream at a certain interval to obtain the target video frame set. As an example, video frames whose position numbers are integer multiples of the set value may be further processed, and the remaining video frames may not be processed.
[0044] In some embodiments, the execution subject may perform anomaly detection on each video frame in the target video frame set through a target detection algorithm, thereby achieving the purpose of detection.
[0045] Step 306: extract the captured video frame by frame, use deep learning to extract abnormalities from the images, and obtain feature information corresponding to the thermal abnormality images.
[0046] In some embodiments, the above-mentioned execution entity can perform frame extraction processing on the video through software, and use deep learning to extract the thermal anomaly feature values of the image.
[0047] from Figure 3 It can be seen that Figure 2 Compared with the description of some corresponding embodiments, Figure 3 The process 300 of the thermal infrared tree hole smoldering fire monitoring method in some corresponding embodiments embodies the processing method of visible light and infrared image extraction. Therefore, the schemes described in these embodiments can better detect the tree hole smoldering fire accurately when it occurs.
[0048] Further references Figure 4 , which shows a flowchart 400 of image fusion and processing of a thermal infrared tree hole smoldering fire monitoring method, including the following contents and steps:
[0049] Step 401 : Fusing the extracted visible light and infrared images to generate a fused visible light infrared image.
[0050] Step 402 , performing image processing on the fused visible light infrared image to increase the color contrast in the infrared image and obtain a set of infrared images with more prominent abnormal conditions.
[0051] Step 405: retain the portion of the image with the greatest contrast with the environment through image processing.
[0052] Reference below Figure 5 , which shows an image processing device (eg, Figure 1 The electronic devices in some embodiments of the present disclosure may include but are not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0053] like Figure 5As shown, the computer device 500 may include an image processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to the program stored in a read-only memory (ROM) 504. Various programs and data required for the operation of the device 500 are also stored in the ROM 504. The infrared camera 502 and the processor 505 are connected to each other, and the visible light camera 503 is also connected to the processor 505. The input / output (I / O) interface is also connected to the bus.
[0054] Typically, the following devices can be connected to the I / O interface: input devices such as touch screens, touch pads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices such as liquid crystal displays (LCDs), speakers, vibrators, etc. The above electronic devices can communicate with other devices wirelessly or by wire to exchange data. Figure 5 The computer device 500 having the main devices is shown, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead.
[0055] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from storage 504.
[0056] It should be noted that the computer-readable medium described in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0057] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0058] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: detects the target video frame to obtain a set of image frames reflecting the detection results of the monitoring area; determines whether there is a target to be detected in the image frames in the image frame set to obtain an abnormal image frame set; determines whether the abnormal image frame set is an empty set. If it is a non-empty set, it is determined that the target video frame includes an abnormal situation. If it is an empty set, it proceeds to the next step.
[0059] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0060] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0061] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including a visual detection unit, a first determination unit, an audio detection unit, a second determination unit, a third determination unit, and a fourth determination unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the visual detection unit may also be described as a "unit for detecting visual information of a monitoring area".
[0062] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0063] According to one or more embodiments of the present disclosure, the visual detection unit is further configured to: perform detection through a target detection model in deep learning to obtain a set of target image frames.
[0064] According to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: one or more processors; and a storage device on which one or more programs are stored, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any of the above methods.
[0065] According to one or more embodiments of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, wherein when the program is executed by a processor, any of the above methods is implemented.
[0066] The methods provided by some embodiments of the present disclosure can more effectively detect smoldering forest fires, compensating for the problems of missed alarms and false alarms to a certain extent.
[0067] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.
Claims
1. A tree hole smoldering fire monitoring method based on thermal infrared, comprising: Perform region detection on the target video to obtain a set of infrared and visible light image frames; Extract tree hole features from visible light images and obtain a visible light target set; Extract thermal anomaly features from infrared images to obtain infrared target sets; The infrared target set and the visible light target set are image fused and the image color contrast is enhanced to accurately detect smoldering fires in tree holes.
2. The method according to claim 1, wherein: Before detecting each target video frame to obtain an image set reflecting the abnormal area detection result, the method further includes: performing image preprocessing such as filtering, noise reduction, and scaling on each video frame in the video frame set.
3. The method according to claim 1, wherein: The method of detecting each target video frame in the visible light tree hole image set to obtain a target set reflecting the detection results of the monitoring area includes: using a target detection model in deep learning to detect each target video frame in the target video frame set to obtain the target set.
4. The method according to claim 1, wherein: The method of detecting each target video frame in the visible light tree hole image set to obtain a target set reflecting the detection results of the monitoring area includes: using a target detection model in deep learning to detect each target video frame in the target video frame set to obtain the target set.
5. The method according to claim 1, wherein: When extracting the infrared and visible light videos, before obtaining the new video frame set, the method also needs to pay attention to: the number of frames captured for the collected infrared image must be the same as the number of frames captured for the visible light tree hole image set.
6. The method according to claim 1, wherein: When fusing the image data, before obtaining the new fused image, the method further includes: performing image preprocessing such as noise reduction and sampling on the collected infrared image and visible light image data.
7. The method according to claim 1, wherein: When the color contrast of the image data is enhanced, before the final determination of the smoldering fire in the tree hole is obtained, the method also includes: performing pre-processing such as noise reduction and sampling on the infrared data collected in the abnormal area, and then converting the image into the HSV color space and processing only the brightness channel.
8. A tree hole smoldering fire device based on thermal infrared, comprising: The visual detection unit is configured to detect each target video frame in the target video frame set to obtain an infrared and visible light image frame set reflecting the detection result of the monitoring area; the first determination unit is configured to determine whether there is a tree hole feature in the image frame in the image frame set to obtain a tree hole feature set; A second determining unit is configured to determine whether there is a thermal abnormality feature in the image frames in the image frame set, to obtain a thermal abnormality image frame set; A visible light and infrared image fusion unit is configured to fuse the visible light image and the infrared image to generate a fused infrared image, thereby eliminating interference from non-tree hole heat sources; The color contrast enhancement unit is configured to perform color contrast enhancement on the final fused infrared image, thereby highlighting the smoldering fire with a higher temperature and excluding other hot smoke-like interferences.
9. An electronic device, comprising: one or more processors; A storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-7.
10. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.