An early warning and automatic fire extinguishing system based on image fire identification and sound wave positioning

Abstract

The application provides an early warning and automatic fire extinguishing system based on image fire identification and sound wave positioning, relates to the technical field of fire prevention and control, and comprises a multispectral image acquisition module, a sound wave detection array module, a central fusion control unit, a hierarchical warning module, a fire extinguishing execution module, a fire extinguishing strategy database and a cloud data processing module, the multispectral image acquisition module is used for collecting infrared thermal imaging images and visible light high-definition images of a monitoring area in real time; the application adopts a flame early identification algorithm of multispectral image and dynamic feature fusion, carries out double verification through infrared image high-temperature abnormal area screening and visible light image flame dynamic feature extraction, combines time sequence continuous matching and area growth trend judgment, greatly reduces the false alarm rate caused by interference factors such as strong light reflection, welding sparks and high-temperature steam, can accurately identify the early stage of smoldering fire and open fire, improves the identification accuracy, and gains valuable time for early fire disposal.

Description

Technical Field

[0001] This invention relates to the field of fire prevention and control technology, and in particular to an early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning. Background Technology

[0002] Existing fire early warning systems primarily rely on single detection methods for fire identification and warning. Mainstream technologies include flame recognition based on visible light images, high-temperature detection based on infrared thermal imaging, and single-mode audio-visual alarm linkage. Some high-end systems incorporate multispectral acquisition equipment or acoustic detection modules, but these often employ simple signal superposition for data fusion, failing to establish an effective multimodal collaborative mechanism. For example, existing forest fire monitoring systems acquire images using pan-tilt-zoom cameras, identifying smoke and fire points based on spectral characteristics. However, this is significantly affected by light and smoke obstruction and lacks seamless integration with fire suppression systems. While some sound source localization devices can achieve three-dimensional positioning, they are not integrated with image recognition technology, making them unsuitable for the positioning needs of complex fire scenarios. Furthermore, existing systems are mostly equipped with basic fire suppression equipment, lacking personalized fire suppression strategies for different scenarios and fire conditions, and failing to establish comprehensive data review and prevention mechanisms.

[0003] Specifically, in existing technologies, flame recognition accuracy is low; single image acquisition or simple fusion methods are easily interfered with by strong light reflection, welding sparks, high-temperature steam, etc., resulting in a high false alarm rate and difficulty in identifying early fires such as smoldering fires; fire source location is limited by the environment; visual positioning fails in scenarios with dense smoke, obstruction, or drastic changes in light; single acoustic positioning is not accurate enough to achieve continuous positioning in the entire environment; early warning methods are limited, mostly relying on on-site audible and visual alarms, without graded early warning based on the severity of the fire, resulting in untimely transmission of early warning information, poor linkage with emergency measures, and inability to effectively guide personnel evacuation; fire extinguishing strategies... The lack of specificity and failure to match the optimal fire extinguishing plan with parameters such as combustible material type and combustion scale results in limited fire loss control effectiveness. Furthermore, the lack of a data closed-loop mechanism limits the system to a single "early warning-response" process, making it impossible to optimize protection plans through historical data review and hindering proactive prevention. These deficiencies prevent existing systems from meeting the needs of early fire prevention in complex scenarios, easily leading to fire spread and increasing the risk of casualties and property damage. Therefore, this invention proposes an early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning to address the problems existing in the prior art. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes an early warning and automatic fire suppression system based on image fire recognition and acoustic localization. This system employs a flame early recognition algorithm that fuses multispectral images with dynamic features. Through dual verification using infrared image high-temperature anomaly region screening and visible light image flame dynamic feature extraction, combined with continuous time series matching and area growth trend judgment, it significantly reduces the false alarm rate caused by interference factors such as strong light reflection, welding sparks, and high-temperature steam. It can accurately identify the early stages of smoldering fires and open flames, with an accuracy rate more than 85% higher than existing systems, thus gaining valuable time for early fire response. Furthermore, the algorithm has adaptive adjustment capabilities, dynamically correcting the temperature threshold according to ambient temperature, further improving recognition stability in different scenarios.

[0005] To achieve the objectives of this invention, the following technical solution is provided: an early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning, comprising a multispectral image acquisition module, an acoustic detection array module, a central fusion control unit, a graded early warning module, a fire extinguishing execution module, a fire extinguishing strategy database, and a cloud data processing module. The multispectral image acquisition module is used to acquire infrared thermal imaging images and visible light high-definition images of the monitored area in real time and transmit the image data to the central fusion control unit. The acoustic detection array module is used to passively receive infrasound and acoustic signals in the 20Hz~2000Hz range generated by fire combustion and transmit the acoustic signals to the central fusion control unit. The central fusion control unit is used to process the image data and acoustic signals, execute early flame recognition, three-dimensional fire source positioning, and multimodal data fusion algorithms, and output recognition results, positioning coordinates, and fire level determination signals. The graded early warning module is used to determine the signal based on the fire level and execute corresponding graded early warning and emergency response actions; the fire extinguishing strategy database is used to store fire extinguishing strategy parameters for different scenarios and fire conditions, providing decision support for the fire extinguishing execution module; the fire extinguishing execution module is used to execute automatic fire extinguishing actions based on the instructions of the central integrated control unit and the parameters of the fire extinguishing strategy database; the cloud data processing module is used to store full-dimensional fire data, build risk models, push hazard warnings and protection plan optimization suggestions, and realize a data closed loop of early warning-response-review-prevention.

[0006] Further improvements include: the multispectral image acquisition module includes an infrared thermal imaging camera and a visible light high-definition camera. The infrared thermal imaging camera has a resolution of ≥640×480 pixels, a temperature measurement range of -20℃ to 1500℃, a frame rate of ≥30fps, and features automatic focusing and temperature calibration functions; the visible light high-definition camera has a resolution of ≥1920×1080 pixels, a frame rate of ≥25fps, and is equipped with an anti-glare lens; the two cameras are symmetrically distributed and installed, covering the entire monitoring area, and the image data is transmitted via Ethernet with a transmission delay of ≤50ms.

[0007] A further improvement lies in the fact that the early flame identification algorithm executed by the central fusion control unit is a spatiotemporal dynamic feature fusion algorithm, the specific steps of which include: S1: Preprocess the infrared thermal imaging image, remove noise by Gaussian filtering, and use an adaptive threshold segmentation algorithm to screen high temperature abnormal areas. The temperature threshold T is calculated as: T=T0+ΔT, where T0 is the normal ambient temperature of the monitored area, ΔT is the temperature correction coefficient, and the value range is 5℃~15℃. When the average temperature of the area is ≥T, it is determined to be a high temperature abnormal area, and its two-dimensional pixel coordinates are recorded. S2: Extract the dynamic features of the flame corresponding to the coordinates of the high-temperature anomaly area from the visible light image, including the flicker frequency of 10Hz~20Hz, the edge contour roundness <0.6, and the RGB-HSV color conversion features: hue 0°~60°, saturation 50%~100%, and brightness 40%~100%. S3: Perform continuous time series matching verification, set a preset number of frames N, corresponding to 3 seconds, N=90 frames; when the number of consecutive matching frames between the infrared anomaly area and the visible light flame dynamic characteristics is ≥N, and the area growth rate of the high temperature anomaly area is ≥5% / s, it is determined to be the initial stage of fire. The formula for calculating the area growth rate is as follows: r is the area growth rate, S t S represents the area of ​​the high-temperature anomaly region in the current frame. t-1 This represents the area of ​​the high-temperature anomaly region in the previous frame.

[0008] Further improvements include: the temperature correction coefficient ΔT can be automatically adjusted according to seasonal changes, with ΔT=10℃~15℃ in winter, ΔT=5℃~10℃ in summer, and ΔT=8℃~12℃ in spring and autumn; the preset frame number N is adjusted according to the fire risk level of the monitoring scenario, with N=60 frames for high-risk scenarios, corresponding to 2 seconds; N=90 frames for medium-risk scenarios, corresponding to 3 seconds; and N=120 frames for low-risk scenarios, corresponding to 4 seconds.

[0009] Further improvements include: the acoustic wave detection array module consists of no fewer than four high-sensitivity microphones arranged in a three-dimensional geometric distribution at the vertices of a regular tetrahedron. The microphones have a frequency response range of 20Hz to 2000Hz, a sensitivity of -38dB ± 2dB, a sampling rate ≥ 44.1kHz, and noise suppression capabilities. The central fusion control unit uses an improved generalized cross-correlation time delay estimation algorithm to calculate the time difference (TDOA) of sound waves arriving at different microphones, and combines this with a particle swarm optimization algorithm for three-dimensional coordinate calculation. The improved generalized cross-correlation function formula is as follows: , in, Let τ be the generalized cross-correlation function between microphone i and microphone j, and τ be the time difference. For SCOT weighting functions, , These are the Fourier transforms of the sound wave signals collected by microphones i and j, respectively. for The conjugate complex number, f, is the sound wave frequency; the three-dimensional coordinate solution equations are: , Where (x,y,z) are the three-dimensional coordinates of the fire source, (xi,yi,zi) are the three-dimensional coordinates of the i-th microphone, i=2,3,4, (x1,y1,z1) are the coordinates of the reference microphone, and c=340m / s is the speed of sound. Let be the time difference between the arrival of the sound wave at the i-th microphone and the reference microphone.

[0010] Further improvements are made by setting the parameters of the particle swarm optimization algorithm as follows: particle swarm size 50, number of iterations 100, inertia weight 0.8, learning factor c1=c2=2, and solving the optimal solution of the three-dimensional coordinates of the fire source through iterative calculation, achieving a positioning accuracy of ±3cm.

[0011] A further improvement lies in the fact that the central fusion control unit constructs a visual-acoustic multimodal spatiotemporal calibration and confidence-weighted fusion localization mechanism, specifically including: The two-dimensional pixel coordinates (u,v) of image recognition are mapped to three-dimensional spatial coordinates (x,v) through a coordinate transformation matrix. vis ,y vis ,z vis The transformation formula is: , where M is a 3×4 coordinate transformation matrix, obtained from the camera's intrinsic and extrinsic parameters; Perform spatiotemporal alignment of image positioning coordinates and acoustic positioning coordinates to ensure consistent timestamps and a unified spatial coordinate system; Dynamically assign confidence weights w1 for visual localization and w2 for acoustic localization, where w1 + w2 = 1. The optimal fire source coordinates (X, Y, Z) are then output through weighted fusion. The fusion formula is as follows: , Among them, (x) snd ,y snd ,z snd The three-dimensional coordinates for sound wave localization are w1 and w2, which are dynamically adjusted according to the image quality evaluation index Q: when Q≥0.8, w1=0.7, w2=0.3; when 0.5≤Q<0.8, w1=0.5, w2=0.5; when Q<0.5, w1=0.2, w2=0.8.

[0012] Further improvements include: the graded early warning module sets three warning thresholds, corresponding to the smoldering stage, small fire stage, and open flame stage of a fire. It automatically determines the fire level by combining infrared temperature, flame area, and smoke concentration: Level 1 warning is for the smoldering stage: infrared temperature 30℃~50℃, flame area ≤0.5m². 2 No obvious smoke was detected, triggering a low-intensity audible and visual alarm on-site, sending a notification to on-site management personnel via mobile app, and activating the smoke extraction system; Level II warning indicates a small fire stage: infrared temperature 50℃~100℃, flame area 0.5m²~5m². 2 A small amount of smoke triggers a high-intensity audible and visual alarm on-site, sends a notification to all relevant personnel via a mobile app, and activates emergency lighting and evacuation guidance signs; Level III warning is the open flame stage: infrared temperature > 100℃, flame area > 5m². 2 A large amount of smoke triggered the highest intensity audible and visual alarm, mobile APP push notification, fire platform report, and linkage of fire exit unlocking and all emergency measures.

[0013] Further improvements include: the fire extinguishing strategy database stores fire extinguishing strategy parameters for 15 typical scenarios, covering special scenarios such as factories, warehouses, and laboratories. The parameters include combustible material type, extinguishing medium, spray angle, spray pressure, and pre-action actions; the fire extinguishing execution module includes an extinguishing medium storage tank, spraying device, power-off switch, and isolation door. Based on the combustible material type, combustion scale, and personnel distribution identified by image recognition, it matches the optimal fire extinguishing strategy from the database and executes automatic fire extinguishing actions.

[0014] Further improvements include: the cloud data processing module uses a cloud server and connects to the central integrated control unit via a 5G network to automatically record full-dimensional data such as fire time, location coordinates, combustion process, response time, and fire extinguishing effect; based on historical data, a risk model is constructed using a BP neural network, and hazard warnings and protection scheme optimization suggestions are pushed regularly to achieve closed-loop data management.

[0015] The beneficial effects of this invention are as follows: 1. This invention employs an early flame recognition algorithm that fuses multispectral images and dynamic features. Through dual verification of high-temperature anomaly region screening in infrared images and dynamic flame feature extraction in visible light images, combined with continuous time series matching and area growth trend judgment, it greatly reduces the false alarm rate caused by interference factors such as strong light reflection, welding sparks, and high-temperature steam. It can accurately identify the early stages of smoldering fires and open flames, with an accuracy rate more than 85% higher than existing systems, thus gaining valuable time for early fire response. At the same time, the algorithm has adaptive adjustment capabilities and can dynamically correct the temperature threshold according to the ambient temperature, further improving the recognition stability in different scenarios.

[0016] 2. This invention introduces a passive three-dimensional spatial precise positioning technology for fire sources based on acoustic arrays. It uses a detection array composed of no fewer than four high-sensitivity microphones arranged in a three-dimensional geometric distribution. Combined with an improved generalized cross-correlation time delay estimation algorithm and particle swarm optimization algorithm, it realizes the three-dimensional coordinate calculation of the sound source. It does not depend on light or line of sight and can work independently in extreme scenarios such as dense smoke, obstructed visual sensors, or rapid changes in light. It forms a redundant complement to image positioning, and the positioning accuracy can reach ±3cm. It solves the pain point of existing positioning technologies being limited by the environment and ensures the continuity and accuracy of fire source positioning in all scenarios.

[0017] 3. This invention constructs a visual-acoustic multimodal spatiotemporal calibration and confidence-weighted fusion positioning mechanism. By using a coordinate transformation matrix, the two-dimensional pixel coordinates of image recognition are mapped to three-dimensional space, achieving spatiotemporal alignment with the three-dimensional coordinates of acoustic positioning. Confidence weights are dynamically allocated according to environmental conditions, increasing the visual positioning weight in good lighting and unobstructed conditions, and increasing the acoustic positioning weight in visual failure scenarios. The optimal fire source coordinates are output through weighted fusion. This mechanism ensures the stability of positioning accuracy under different environmental conditions, avoids the limitations of a single positioning method, and results in small positioning errors.

[0018] 4. This invention designs a three-level graded early warning and multi-channel linkage mechanism. It sets corresponding early warning thresholds according to the early stage of a fire, and automatically determines the fire level by combining flame / smoke intensity, fire source range, and temperature data. Differentiated early warning and emergency linkage actions are executed for different levels, realizing multi-channel coordination of on-site audible and visual alarms, mobile APP push, fire platform reporting, and emergency system activation. It effectively guides personnel evacuation, and the early warning information transmission coverage reaches 100%. It solves the problems of single early warning method and poor linkage of existing systems, and reduces the risk of casualties.

[0019] 5. This invention constructs a database of fire extinguishing strategies for 15 typical scenarios, covering special scenarios such as factories, warehouses, and laboratories. Combining data on combustible material types, combustion scale, and personnel distribution obtained through image recognition, it automatically matches the optimal fire extinguishing medium and spray angle. For special scenarios such as electrical fires and chemical fires, it triggers preemptive actions such as power outages and area isolation to reduce fire losses. At the same time, it creates a data closed loop of "early warning-response-review-prevention". Through cloud data processing modules, it constructs risk models and pushes hazard warnings and optimization suggestions for protection plans, realizing the upgrade of fire protection mode from passive fire extinguishing to active prevention, further improving the initiative and scientific nature of fire prevention and control. Attached Figure Description

[0020] Figure 1 This is a diagram illustrating the composition of the present invention. Detailed Implementation

[0021] To enhance understanding of the present invention, the present invention will be further described in detail below with reference to embodiments. These embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0022] Example 1 according to Figure 1 As shown, this embodiment proposes an early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning. It specifically implements the functions of multispectral image acquisition and early flame recognition, and is suitable for factory workshop scenarios. These scenarios are prone to interference factors such as welding operations and high-temperature equipment operation, which can easily lead to false alarms, and there is an urgent need for early recognition of smoldering fires.

[0023] The multispectral image acquisition module employs a combination of an infrared thermal imaging camera and a visible light high-definition camera. The infrared thermal imaging camera has a resolution of 640×480 pixels, a temperature measurement range of -20℃ to 1500℃, a frame rate of 30fps, and features autofocus and temperature calibration. The visible light high-definition camera has a resolution of 1920×1080 pixels, a frame rate of 25fps, and is equipped with an anti-glare lens to effectively suppress strong light reflection interference. The two cameras are symmetrically distributed on the factory roof, covering the entire factory workshop. The acquired infrared and visible light images are transmitted in real-time to the central fusion control unit via Ethernet, with a transmission latency of ≤50ms.

[0024] The central fusion control unit executes a spatiotemporal dynamic feature fusion algorithm. The specific steps are as follows: First, the infrared image is preprocessed. Image noise is removed by Gaussian filtering. An adaptive threshold segmentation algorithm is used to screen for high-temperature abnormal areas. The temperature threshold T is dynamically adjusted according to the ambient temperature of the factory. The calculation formula is: T=T0+ΔT, where T0 is the normal ambient temperature of the factory (preset to 25℃), and ΔT is the temperature correction coefficient (the value range is 5℃~15℃, which is automatically adjusted according to seasonal changes). When the average temperature of a certain area in the infrared image is ≥T, it is determined to be a high-temperature abnormal area. The two-dimensional pixel coordinates (x1,y1)~(x2,y2) of the area are recorded. The second step involves extracting the flame dynamics of the corresponding coordinate regions from the synchronously acquired visible light images based on the pixel coordinates of the infrared anomaly areas. This includes flicker frequency, irregular changes in edge contours, and RGB-HSV color conversion characteristics. The flicker frequency is calculated using the inter-frame difference method, statistically analyzing the brightness variation cycle of the region over 30 consecutive frames. The flame flicker frequency range is set to 10Hz~20Hz. Irregular changes in edge contours are extracted using the Canny edge detection algorithm, and the roundness of the contours is calculated (roundness = 4π × area / perimeter). 2 When the circularity is less than 0.6, it is determined to meet the characteristics of flame edge. The RGB-HSV color conversion feature converts the RGB color space of the visible light image to the HSV space and extracts the hue (0°~60°), saturation (50%~100%), and brightness (40%~100%). When all three parameters meet the above ranges, it is determined to meet the characteristics of flame color. The third step is to perform continuous time series matching verification. The preset frame number is set to 90 frames (corresponding to 3 seconds, frame rate 30fps). When the number of consecutive matching frames between the infrared abnormal area and the visible light flame dynamic features is ≥90 frames, and the area growth rate of the high temperature abnormal area is ≥5% / s, the central fusion control unit determines that it is the initial stage of fire and outputs flame identification signal and preliminary positioning coordinates. If the number of matching frames is insufficient or the area does not show a growth trend, it is determined to be an interference signal and no subsequent action is triggered.

[0025] In this embodiment, the algorithm can effectively distinguish between welding sparks (flicker frequency > 20Hz, circularity close to 1), high-temperature steam (no obvious edge contour, HSV parameters do not meet flame characteristics) and early flames, with a false alarm rate controlled below 0.5%, smoldering fire identification time ≤ 10s, and open flame identification time ≤ 3s, meeting the early identification requirements of factory scenarios.

[0026] Example 2 according to Figure 1As shown, this embodiment proposes an early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning. It specifically implements the acoustic detection array and the three-dimensional positioning function of the fire source, which is suitable for warehouse scenarios. In such scenarios, a large number of goods are piled up, which can easily cause visual obstruction. Furthermore, visual positioning is prone to failure in dense smoke environments, so the accuracy and stability of acoustic positioning are required to be high.

[0027] The acoustic wave detection array module consists of four high-sensitivity microphones arranged in a tetrahedral configuration at the vertices of the warehouse ceiling. The microphones are spaced 5 meters apart and utilize condenser microphones with a frequency response range of 20Hz to 2000Hz, a sensitivity of -38dB±2dB, and a sampling rate of 44.1kHz. These microphones feature noise suppression capabilities, effectively filtering out irrelevant acoustic interference from the environment (such as the sounds of goods being moved or equipment running). The acoustic signals collected by the microphones are amplified by a differential amplifier circuit, then converted into digital signals by an A / D converter before being transmitted to the central fusion control unit.

[0028] The central fusion control unit uses an improved generalized cross-correlation time delay estimation algorithm (GCC) to calculate the time difference (TDOA) of sound waves arriving at different microphones, and combines it with particle swarm optimization (PSO) to solve for three-dimensional spatial coordinates. Specifically, the implementation is as follows: First, the sound wave signals collected by the four microphones are preprocessed. The SCOT weighting function is used to filter the signals and suppress environmental noise. The improved generalized cross-correlation function formula is: , in, Let τ be the generalized cross-correlation function between microphone i and microphone j, and τ be the time difference. For SCOT weighting functions, , These are the Fourier transforms of the sound wave signals collected by microphones i and j, respectively. for The conjugate complex number of , where f is the frequency of the sound wave. When When the maximum value is reached, the corresponding τ is the time difference between the sound waves arriving at microphones i and j. .

[0029] Then, using one of the microphones as a reference (let's call it microphone 1, coordinates (x1, y1, z1)), a three-dimensional spatial coordinate system is established, with the sound source (fire source) coordinates as (x, y, z), and the sound wave propagation speed c = 340 m / s. The equations for calculating the sound source coordinates according to TDOA are as follows: , Where i = 2, 3, 4, and (xi, yi, zi) are the three-dimensional coordinates of the i-th microphone. Let be the time difference between the arrival of the sound wave at the i-th microphone and microphone 1. Since this system of equations is overdetermined, a particle swarm optimization algorithm is used to solve for the optimal solution. The particle swarm size is set to 50, the number of iterations is 100, the inertia weight is 0.8, and the learning factors are c1=c2=2. The three-dimensional coordinates (x,y,z) of the sound source are obtained through iterative calculation, and the positioning accuracy can reach ±3cm.

[0030] In this embodiment, when smoldering turns into open flame in the warehouse, the 100Hz~500Hz sound waves generated by the flames are received by the microphone array. Even when the visual sensor is obscured by dense smoke, the acoustic positioning module can independently output the three-dimensional coordinates of the fire source. The positioning response time is ≤1s, ensuring accurate positioning even in extreme environments and providing accurate basis for subsequent fire extinguishing actions.

[0031] Example 3 according to Figure 1 As shown, this embodiment proposes an early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning. It specifically implements the multimodal fusion positioning function and is suitable for laboratory scenarios where lighting changes frequently (such as light switching and sunlight) and there are experimental equipment obstructions, requiring extremely high continuity and accuracy in positioning.

[0032] The central fusion control unit first constructs a spatiotemporal calibration model, mapping the two-dimensional pixel coordinates (u,v) of the flame identified by the multispectral image acquisition module to the actual three-dimensional spatial coordinates (x,v) through a coordinate transformation matrix. vis ,y vis ,z vis The coordinate transformation formula is: , Where M is the coordinate transformation matrix (3×4 matrix), calibrated from the camera's intrinsic parameters (focal length, pixel size) and extrinsic parameters (installation position, attitude), u and v are the two-dimensional pixel coordinates of the flame in the visible light image, and x... vis y vis z vis These are the mapped three-dimensional spatial coordinates.

[0033] Then, locate the coordinates (x) of the image. vis ,y vis ,z vis ) and sound wave positioning coordinates (x snd ,y snd ,z snd Spatiotemporal alignment is performed using a time synchronization algorithm (based on the NTP protocol) to ensure that the timestamps of the two positioning data are consistent. Spatial alignment is achieved through coordinate translation and rotation to eliminate the installation position deviation of the two modules, so that the two coordinates are in the same three-dimensional coordinate system.

[0034] Next, the confidence weights are dynamically configured. The central fusion control unit detects image quality parameters (sharpness, contrast, smoke density) in real time and sets the image quality evaluation index Q. When Q ≥ 0.8 (good lighting, no obstruction), the visual localization confidence weight w1 = 0.7, and the acoustic localization confidence weight w2 = 0.3; when 0.5 ≤ Q < 0.8 (slight blur, small amount of smoke), w1 = 0.5, w2 = 0.5; when Q < 0.5 (severe blur, dense smoke obstruction), w1 = 0.2, w2 = 0.8. The confidence weighted fusion formula is: , Where (X,Y,Z) are the optimal three-dimensional coordinates of the fire source in the final output, and w1 and w2 are the confidence weights of visual localization and acoustic localization, respectively, and satisfy w1+w2=1.

[0035] In this embodiment, when the laboratory lights are on and there are no obstructions, the visual positioning accuracy is high, and the positioning error after fusion is ≤2cm. When the experiment produces smoke that causes the image to blur (Q=0.4), the acoustic positioning weight is automatically increased, and the positioning error after fusion is still ≤5cm. This ensures the continuity and high accuracy of fire source positioning under different lighting and obstruction conditions, meeting the positioning requirements of laboratory scenarios.

[0036] Example 4 according to Figure 1 As shown, this embodiment proposes an early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning. It specifically implements the functions of graded early warning and emergency linkage, and is suitable for the scenario of a comprehensive office building. This scenario has high requirements for the timeliness of early warning information transmission and the effectiveness of evacuation guidance, and requires differentiated emergency measures based on the fire level.

[0037] The tiered early warning module includes an audible and visual alarm device, a mobile app push module, a fire platform reporting module, and an emergency system linkage module, all electrically connected to the central integrated control unit. Based on flame identification results, fire source range, and temperature data, the central integrated control unit sets three levels of early warning thresholds according to the early stage of a fire, as follows: Level 1 Warning (Smoldering Stage): Infrared temperature 30℃~50℃, flame area ≤0.5m² 2 No obvious smoke; Level II warning (small fire stage): Infrared temperature 50℃~100℃, flame area 0.5m²~5m² 2 Small amount of smoke; Level 3 warning (open flame stage): Infrared temperature > 100℃, flame area > 5m² 2 The presence of a large amount of smoke meets the preliminary criteria for general or above fires as stipulated in management regulations.

[0038] When the central integrated control unit determines that a Level 1 warning has been issued, the following actions are triggered: a low-intensity audible and visual alarm is triggered on-site (volume 60dB~80dB, light flashing frequency 1Hz), a mobile APP is pushed to remind the building property management personnel (including the location of the fire source and temperature data), the building smoke exhaust system is activated, and the smoke exhaust fans in the corresponding areas are turned on to reduce the smoke concentration and prevent the spread of smoldering fire.

[0039] When a Level 2 warning is issued, the following actions will be triggered: a high-intensity audible and visual alarm will be set up on-site (volume 80dB~100dB, light flashing frequency 2Hz), a mobile APP will be pushed to remind all staff in the building (including evacuation route guidance), the emergency lighting system and evacuation guidance signs will be activated to ensure sufficient lighting in evacuation passages, guide personnel to evacuate in an orderly manner, and at the same time shut down the air conditioning system in the corresponding area and cut off non-emergency power.

[0040] When a Level 3 warning is issued, the following actions are triggered: the highest intensity audible and visual alarm is triggered (volume > 100dB, light flashing frequency 3Hz), a mobile APP reminder is pushed to all personnel, and a report is simultaneously uploaded to the local fire platform (including detailed fire information and location coordinates), fire exit unlocking is activated (fire doors and evacuation passage door locks are automatically opened), all emergency measures are activated, including sprinkler systems, fire hydrant systems, and emergency broadcasts (playing evacuation instructions in a loop), and the elevator is forced to descend to the first floor and the elevator power is cut off to prevent people from being trapped.

[0041] In this embodiment, the response time of the graded early warning module is ≤1s, the mobile APP push delay is ≤3s, the fire platform reporting delay is ≤5s, the emergency system linkage delay is ≤10s, and the early warning information transmission coverage reaches 100%, which can effectively guide personnel evacuation, buy time for fire fighting, and reduce the risk of casualties.

[0042] Example 5 according to Figure 1 As shown, this embodiment proposes an early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning. It specifically implements the functions of fire extinguishing strategy matching and data closed loop, and is suitable for chemical laboratory scenarios. These scenarios contain a variety of chemicals and electrical equipment, and the types of combustibles are complex. The requirements for the targeting and safety of fire extinguishing strategies are extremely high, and proactive prevention needs to be achieved through data review.

[0043] The fire suppression strategy database stores fire suppression strategy parameters for 15 typical scenarios, including chemical laboratories, factories, warehouses, and office buildings. The strategy parameters for the chemical laboratory scenario include: combustible material type (flammable liquids, flammable solids, electrical equipment, chemicals), extinguishing medium (dry powder, carbon dioxide, foam, water mist), spray angle (30°~60°), spray pressure (0.3MPa~0.5MPa), and pre-emptive actions (power outage, area isolation, ventilation). The database uses SQL Server for storage, supports real-time updates and queries, and the central integrated control unit can access the database parameters in real time via Ethernet.

[0044] The fire suppression execution module includes a fire extinguishing medium storage tank, a spraying device, a power-off switch, and an isolation door, all electrically connected to the central fusion control unit. When the central fusion control unit outputs a fire suppression command, it first analyzes the type of combustible material (e.g., identifying flammable liquid containers and electrical equipment by color and shape), the scale of the fire (flame area and temperature), and the distribution of personnel through an image recognition module. Then, it matches the optimal fire suppression strategy from the fire suppression strategy database: If it is an electrical fire (e.g., a short circuit fire in experimental equipment), it first triggers the power-off switch to cut off the power to the corresponding area, then selects carbon dioxide fire extinguishing medium at a spray angle of 45° and a spray pressure of 0.4MPa to avoid the risk of electrical conduction; if it is a chemical fire (e.g., a fire caused by a flammable liquid leak), it first closes the chemical container valve, activates the area isolation door, selects foam fire extinguishing medium at a spray angle of 30° and a spray pressure of 0.5MPa to cover the surface of the fire source and isolate oxygen; if it is a common solid fire (e.g., a fire caused by paper or wood), it selects water mist fire extinguishing medium at a spray angle of 60° and a spray pressure of 0.3MPa to reduce the fire temperature.

[0045] The cloud-based data processing module utilizes a cloud server and connects to the central integrated control unit via a 5G network. It automatically records comprehensive fire data, including the time of fire occurrence, fire source location coordinates, combustion process (temperature changes, flame area changes), early warning response duration, fire suppression execution process, and fire suppression effectiveness. Based on historical data, the cloud platform employs a backpropagation (BP) neural network to construct a risk model, analyzing fire hazards in different areas (such as aging laboratory equipment and improper chemical storage). It regularly pushes hazard warnings and suggestions for optimizing protection plans to management personnel (such as regular equipment maintenance and categorized chemical storage). Simultaneously, management personnel can view historical fire data and the effectiveness of fire suppression strategies through the cloud platform, reviewing and optimizing fire suppression strategies and early warning thresholds to achieve a data closed loop of "early warning-response-review-prevention."

[0046] In this embodiment, the response time of the fire extinguishing execution module is ≤15s, the accuracy rate of fire extinguishing medium matching is over 98%, and the fire loss can be reduced by more than 90%. Through the data closed-loop mechanism, the fire hazard incidence rate in the chemical laboratory has been reduced by 60% compared to before, realizing the upgrade from passive fire extinguishing to active prevention fire protection mode.

[0047] Validation data: The system of this invention has undergone repeated testing in multiple scenarios, and all performance indicators have met the design requirements. Specific data are as follows: Flame recognition performance: Smoldering fire recognition time ≤10s, open flame recognition time ≤3s, recognition accuracy ≥99.5%, false alarm rate ≤0.5%, effectively distinguishing interference signals such as strong light reflection, welding sparks, and high-temperature steam. Fire source positioning performance: 3D positioning accuracy ±3cm, positioning response time ≤1s, positioning error ≤2cm in unobstructed scenarios, positioning error ≤5cm in dense smoke-obstructed scenarios, and 100% positioning continuity across all environments. Early warning and linkage performance: Graded early warning response time ≤1s, mobile APP push delay ≤3s, fire platform reporting delay ≤5s, emergency system linkage delay ≤10s, and early warning information transmission coverage 100%. Fire extinguishing performance: Fire extinguishing response time ≤15s, fire extinguishing medium matching accuracy ≥98%, fire loss reduction ≥90%, and 100% accuracy in pre-emptive actions in special scenarios (electrical and chemical fires). Data closed-loop performance: 100% integrity of fire data recording, cloud-based risk model early warning accuracy ≥78%, reduction of scene hazard occurrence rate ≥60%, and effectiveness of protection scheme optimization suggestions ≥85%. Environmental adaptability: Operating temperature range -40℃ to +50℃, humidity range 10% to 90%, adaptable to complex environments such as strong light, dense smoke, obstruction, and drastic changes in light, with continuous operation stability ≥99.8%.

[0048] This early warning and automatic fire suppression system based on image fire recognition and acoustic positioning employs a flame early recognition algorithm that fuses multispectral images with dynamic features. Through dual verification of high-temperature anomaly area screening in infrared images and dynamic flame feature extraction in visible light images, combined with continuous time series matching and area growth trend judgment, it greatly reduces the false alarm rate caused by interference factors such as strong light reflection, welding sparks, and high-temperature steam. It can accurately identify the early stages of smoldering fires and open flames, with an accuracy rate more than 85% higher than existing systems, saving valuable time for early fire response. At the same time, the algorithm has adaptive adjustment capabilities and can dynamically adjust the temperature threshold according to the ambient temperature, further improving the recognition stability in different scenarios. This invention introduces a passive 3D spatial precise positioning technology for fire sources based on acoustic arrays. It employs a detection array composed of at least four high-sensitivity microphones arranged in a three-dimensional geometric distribution. Combined with an improved generalized cross-correlation time delay estimation algorithm and particle swarm optimization algorithm, it achieves 3D coordinate calculation of the sound source. This technology is independent of light and line-of-sight, and can operate independently in extreme scenarios such as dense smoke, obstructed visual sensors, or rapidly changing lighting conditions. It forms a redundant complement to image positioning, achieving a positioning accuracy of ±3cm. This solves the pain point of existing positioning technologies being limited by the environment, ensuring the continuity and accuracy of fire source positioning in all scenarios. This invention constructs a visual-acoustic multimodal spatiotemporal calibration and confidence-weighted fusion positioning mechanism. Through a coordinate transformation matrix, the 2D pixel coordinates of image recognition are mapped to 3D space, achieving spatiotemporal alignment with the 3D coordinates of acoustic positioning. Confidence weights are dynamically allocated according to environmental conditions: visual positioning weights are increased in good lighting and unobstructed conditions, while acoustic positioning weights are increased in scenarios where visual positioning fails. The optimal fire source coordinates are output through weighted fusion. This mechanism ensures the stability of positioning accuracy under different environmental conditions, avoids the limitations of single positioning methods, and minimizes positioning errors. This invention designs a three-level graded early warning and multi-channel linkage mechanism. It sets corresponding early warning thresholds according to the early stage of a fire, and automatically determines the fire level by combining flame / smoke intensity, fire source range, and temperature data. Differentiated early warning and emergency linkage actions are executed for different levels, realizing multi-channel coordination of on-site audible and visual alarms, mobile APP push, fire platform reporting, and emergency system activation. It effectively guides personnel evacuation, and the early warning information transmission coverage reaches 100%. It solves the problems of single early warning method and poor linkage of existing systems, and reduces the risk of casualties.This invention constructs a database of fire extinguishing strategies for 15 typical scenarios, covering special scenarios such as factories, warehouses, and laboratories. By combining data on combustible material types, combustion scale, and personnel distribution obtained through image recognition, it automatically matches the optimal fire extinguishing medium and spray angle. For special scenarios such as electrical fires and chemical fires, it triggers preemptive actions such as power outages and area isolation to reduce fire losses. At the same time, it creates a data closed loop of "early warning-response-review-prevention". Through cloud data processing modules, it constructs risk models and pushes hazard warnings and optimization suggestions for protection plans, realizing an upgrade of fire protection mode from passive fire extinguishing to proactive prevention, further enhancing the initiative and scientific nature of fire prevention and control.

[0049] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. An early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning, comprising a multispectral image acquisition module, an acoustic detection array module, a central fusion control unit, a graded early warning module, a fire extinguishing execution module, a fire extinguishing strategy database, and a cloud data processing module, characterized in that: The multispectral image acquisition module is used to acquire infrared thermal imaging images and visible high-definition images of the monitored area in real time, and transmit the image data to the central fusion control unit. The acoustic wave detection array module is used to passively receive infrasound and acoustic signals in the range of 20Hz~2000Hz generated by fire combustion, and transmit the acoustic signals to the central fusion control unit; the central fusion control unit is used to process image data and acoustic signals, execute early flame identification, three-dimensional fire source localization and multimodal data fusion algorithms, and output identification results, localization coordinates and fire level determination signals. The graded early warning module is used to determine the signal based on the fire level and execute corresponding graded early warning and emergency response actions; the fire extinguishing strategy database is used to store fire extinguishing strategy parameters for different scenarios and fire conditions, providing decision support for the fire extinguishing execution module; the fire extinguishing execution module is used to execute automatic fire extinguishing actions based on the instructions of the central integrated control unit and the parameters of the fire extinguishing strategy database; the cloud data processing module is used to store full-dimensional fire data, build risk models, push hazard warnings and protection plan optimization suggestions, and realize a data closed loop of early warning-response-review-prevention.

2. The early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning according to claim 1, characterized in that: The multispectral image acquisition module includes an infrared thermal imaging camera and a visible light high-definition camera. The infrared thermal imaging camera has a resolution of ≥640×480 pixels, a temperature measurement range of -20℃ to 1500℃, a frame rate of ≥30fps, and features autofocus and temperature calibration. The visible light high-definition camera has a resolution of ≥1920×1080 pixels, a frame rate of ≥25fps, and is equipped with an anti-glare lens. The two cameras are symmetrically distributed and installed, covering the entire monitoring area. Image data is transmitted via Ethernet with a transmission delay of ≤50ms.

3. The early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning according to claim 1, characterized in that: The flame early identification algorithm executed by the central fusion control unit is a spatiotemporal dynamic feature fusion algorithm, and its specific steps include: S1: Preprocess the infrared thermal imaging image, remove noise by Gaussian filtering, and use an adaptive threshold segmentation algorithm to screen high temperature abnormal areas. The temperature threshold T is calculated as: T=T0+ΔT, where T0 is the normal ambient temperature of the monitored area, ΔT is the temperature correction coefficient, and the value range is 5℃~15℃. When the average temperature of the area is ≥T, it is determined to be a high temperature abnormal area, and its two-dimensional pixel coordinates are recorded. S2: Extract the dynamic features of the flame corresponding to the coordinates of the high-temperature anomaly area from the visible light image, including the flicker frequency of 10Hz~20Hz, the edge contour roundness <0.6, and the RGB-HSV color conversion features: hue 0°~60°, saturation 50%~100%, and brightness 40%~100%. S3: Perform continuous time series matching verification, set a preset number of frames N, corresponding to 3 seconds, N=90 frames; when the number of consecutive matching frames between the infrared anomaly area and the visible light flame dynamic characteristics is ≥N, and the area growth rate of the high temperature anomaly area is ≥5% / s, it is determined to be the initial stage of fire. The formula for calculating the area growth rate is as follows: r is the area growth rate, S t S represents the area of ​​the high-temperature anomaly region in the current frame. t-1 This represents the area of ​​the high-temperature anomaly region in the previous frame.

4. The early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning according to claim 3, characterized in that: The temperature correction coefficient ΔT can be automatically adjusted according to seasonal changes. In winter, ΔT = 10℃~15℃, in summer, ΔT = 5℃~10℃, and in spring and autumn, ΔT = 8℃~12℃. The preset frame number N is adjusted according to the fire risk level of the monitoring scene. In high-risk scenes, N = 60 frames, corresponding to 2 seconds. In a medium-risk scenario, N=90 frames, corresponding to 3 seconds; In a low-risk scenario, N=120 frames, corresponding to 4 seconds.

5. The early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning according to claim 1, characterized in that: The acoustic wave detection array module consists of no fewer than four high-sensitivity microphones arranged in a three-dimensional geometric distribution at the vertices of a regular tetrahedron. The microphones have a frequency response range of 20Hz to 2000Hz, a sensitivity of -38dB ± 2dB, a sampling rate ≥ 44.1kHz, and noise suppression capabilities. The central fusion control unit uses an improved generalized cross-correlation time delay estimation algorithm to calculate the time difference (TDOA) of sound waves arriving at different microphones, and combines this with a particle swarm optimization algorithm for three-dimensional coordinate calculation. The improved generalized cross-correlation function formula is as follows: ， in, Let τ be the generalized cross-correlation function between microphone i and microphone j, and τ be the time difference. For SCOT weighting function, , These are the Fourier transforms of the sound wave signals collected by microphones i and j, respectively. for The conjugate complex number, f, is the sound wave frequency; the three-dimensional coordinate solution equations are: ， Where (x,y,z) are the three-dimensional coordinates of the fire source, (xi,yi,zi) are the three-dimensional coordinates of the i-th microphone, i=2,3,4, (x1,y1,z1) are the coordinates of the reference microphone, and c=340m / s is the speed of sound. Let be the time difference between the arrival of the sound wave at the i-th microphone and the reference microphone.

6. The early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning according to claim 5, characterized in that: The parameters of the particle swarm optimization algorithm are set as follows: particle swarm size 50, number of iterations 100, inertia weight 0.8, learning factor c1=c2=2. The optimal solution of the three-dimensional coordinates of the fire source is obtained through iterative calculation, and the positioning accuracy reaches ±3cm.

7. The early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning according to claim 2, characterized in that: The central fusion control unit constructs a visual-acoustic multimodal spatiotemporal calibration and confidence-weighted fusion localization mechanism, specifically including: The two-dimensional pixel coordinates (u,v) of image recognition are mapped to three-dimensional spatial coordinates (x,v) through a coordinate transformation matrix. vis ,y vis ,z vis The transformation formula is: , where M is a 3×4 coordinate transformation matrix, obtained from the camera's intrinsic and extrinsic parameters; Perform spatiotemporal alignment of image positioning coordinates and acoustic positioning coordinates to ensure consistent timestamps and a unified spatial coordinate system; Dynamically assign confidence weights w1 for visual localization and w2 for acoustic localization, where w1 + w2 = 1. The optimal fire source coordinates (X, Y, Z) are then output through weighted fusion. The fusion formula is as follows: ， Among them, (x) snd ,y snd ,z snd The three-dimensional coordinates for sound wave localization are w1 and w2, which are dynamically adjusted according to the image quality evaluation index Q: when Q≥0.8, w1=0.7, w2=0.3; when 0.5≤Q<0.8, w1=0.5, w2=0.5; when Q<0.5, w1=0.2, w2=0.

8.

8. The early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning according to claim 1, characterized in that: The graded early warning module sets three warning thresholds, corresponding to the smoldering stage, small fire stage, and open flame stage of a fire. It automatically determines the fire level based on infrared temperature, flame area, and smoke concentration: Level 1 warning is for the smoldering stage: infrared temperature 30℃~50℃, flame area ≤0.5m². 2 No obvious smoke was detected, triggering a low-intensity audible and visual alarm on-site, sending a notification to on-site management personnel via mobile app, and activating the smoke extraction system; Level II warning indicates a small fire stage: infrared temperature 50℃~100℃, flame area 0.5m²~5m². 2 A small amount of smoke triggers a high-intensity audible and visual alarm on-site, sends a notification to all relevant personnel via a mobile app, and activates emergency lighting and evacuation guidance signs; Level III warning is the open flame stage: infrared temperature > 100℃, flame area > 5m². 2 A large amount of smoke triggered the highest intensity audible and visual alarm, mobile APP push notification, fire platform report, and linkage of fire exit unlocking and all emergency measures.

9. The early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning according to claim 1, characterized in that: The fire extinguishing strategy database stores fire extinguishing strategy parameters for 15 typical scenarios, covering special scenarios such as factories, warehouses, and laboratories. The parameters include combustible material type, extinguishing medium, spray angle, spray pressure, and pre-action. The fire extinguishing execution module includes an extinguishing medium storage tank, spraying device, power-off switch, and isolation door. Based on the combustible material type, combustion scale, and personnel distribution identified by image recognition, it matches the optimal fire extinguishing strategy from the database and executes automatic fire extinguishing actions.

10. The early warning and automatic fire extinguishing system based on image fire recognition and acoustic positioning according to claim 1, characterized in that: The cloud-based data processing module uses a cloud server and connects to the central integrated control unit via a 5G network. It automatically records full-dimensional data such as fire time, location coordinates, combustion process, response time, and fire extinguishing effect. Based on historical data, it uses a BP neural network to build a risk model and regularly pushes hazard warnings and protection plan optimization suggestions to achieve closed-loop data management.

Images