An intelligent management system for fire fighting equipment based on image processing

Abstract

The present invention relates to the technical field of fire-fighting equipment management, and in particular to an intelligent management system for fire-fighting equipment based on image processing; the present invention adopts an early fire detection method based on the fusion of thermal imaging and visible light, and realizes more accurate and earlier detection of fire by fusing multiple sensory data; combines thermal imaging with visible light images, uses a deep learning algorithm for real-time analysis, and provides more comprehensive environmental information, including temperature anomalies and signs of fire under visible light, including open flames and smoke; analyzes changes in visible light and infrared spectra at the same time, detects early signs of fire, including local temperature anomalies and smoke generation, before the fire spreads on a large scale, and realizes earlier warning; uses an algorithm for fusion data to distinguish real fires from non-fire heat sources or optical phenomena, reduces false alarms, and improves system reliability.

Description

Technical Field

[0001] The present invention relates to the technical field of fire-fighting equipment management, and in particular to an intelligent management system for fire-fighting equipment based on image processing. Background Art

[0002] In the current intelligent management of fire-fighting equipment, early fire detection usually uses sensor perception, such as smoke detectors and temperature sensors. Smoke detectors rely on changes in the concentration of smoke particles in the air to determine the occurrence of fire and thus provide early warning. At the same time, in the management and maintenance of fire-fighting equipment, regular manual inspections and monitoring of sensor data are required to monitor the operating status of the equipment and ensure the normal operation of the fire-fighting equipment.

[0003] However, in the initial stage of a fire, smoke will not spread quickly to the location of the detector, especially in a large space or well-ventilated environment, which will cause the detector to fail to respond in time. Similarly, temperature sensors only work in the middle and late stages of a fire, that is, when the temperature rises significantly, an alarm is triggered. Relying on temperature sensors for early fire detection will miss the best time to respond. In terms of daily maintenance, manual inspections are relatively inefficient and require a lot of time. At the same time, when a fire disaster occurs, equipment in the disaster scene and equipment with harsh installation environments (such as high-altitude fire extinguishers, sensors in pipelines, etc.) are difficult to inspect, take a long time, and cannot detect problems in time. Therefore, there is an urgent need for an intelligent management system for fire equipment based on image processing to solve such problems. Summary of the invention

[0004] To this end, the present invention provides an intelligent management system for fire-fighting equipment based on image processing to overcome the problems in the prior art that the sensor perception effect is delayed and fire scenes cannot be warned in time; manual inspection efficiency is relatively low, and the inspection of equipment in disaster scenes and equipment in harsh installation environments is difficult and time-consuming, and problems cannot be discovered in time.

[0005] To achieve the above objectives, on the one hand, the present invention provides an intelligent management system for fire fighting equipment based on image processing, comprising:

[0006] The multimodal image fusion module combines thermal imaging data with visible light image data and uses a convolutional neural network (CNN) to perform real-time analysis on the fused multimodal data to capture temperature anomalies and signs of fire under visible light, enabling early detection of fires and identifying potential fire threats before smoke or temperature changes significantly.

[0007] The early fire warning module uses spectral data extraction and analysis to extract visible light and infrared spectrum information from the fused image data, analyze the spectral change rate at each pixel, and capture early signs of fire, including local temperature anomalies and the generation of initial smoke;

[0008] The fire phenomenon differentiation module is used for feature extraction and fusion. It extracts features from visible light images and infrared images respectively, and performs weighted fusion on the feature vectors to generate representations in a high-dimensional feature space.

[0009] The fire source location and identification module is used to locate the fire source. It uses multimodal fusion data and visible light image information to locate the location of the fire source.

[0010] Equipment status monitoring module: The virtual model of fire-fighting equipment simulates the current status of the equipment and performs equipment monitoring.

[0011] On the other hand, the present invention also provides a method for intelligent management of fire-fighting equipment based on image processing, comprising the following steps:

[0012] Step S1, thermal imaging visible light image fusion, combines thermal imaging data with visible light image data, uses a convolutional neural network with a deep learning algorithm to perform real-time analysis on the fused multimodal data, and comprehensively captures temperature anomalies and signs of fire under visible light, including open flames and smoke; achieves earlier detection of fires, and can identify potential fire threats before smoke or temperature changes significantly;

[0013] Step S2, early fire warning, analyzes the changes in visible light and infrared spectra based on fused data to detect early signs of fire, including local temperature anomalies and initial smoke. Step S2 uses the aforementioned image fusion data to improve the sensitivity of fire detection and the accuracy of early warning, and provides earlier fire warning than a single sensor, identifying danger signals in advance before the fire spreads on a large scale.

[0014] Step S3, multimodal data fusion, using a fusion algorithm to distinguish between real fire and non-fire phenomena, including sunlight reflection and thermal radiation from mechanical equipment; in step S3, the multimodal information extracted in steps S1 and S2 is used to perform fire detection;

[0015] Step S4, fire source location, using multimodal fusion data and visible light information to locate the fire source; in step S4, the specific nature of the fire is simultaneously identified, and electrical fires and combustible material fires are distinguished; based on multimodal analysis, the fire nature distinction is further improved, providing firefighters with accurate fire source location information;

[0016] Step S5, virtual model reconstruction, using the virtual model of the fire equipment to simulate the current status of the equipment, and combining it with the multimodal data extracted previously; through the reconstruction of the virtual model, the equipment status is continuously monitored and the points prone to failure are identified.

[0017] Furthermore, the thermal imaging visible light image fusion analysis method in step S1 is:

[0018] Input data includes thermal imaging data and visible light image data ,in Represents a thermal image, containing the temperature information of each pixel. Represents a visible light image of RGB three channels, including the light intensity of each pixel. Indicates the image pixel height and width;

[0019] To ensure that multimodal data are aligned in the same coordinate system, and Perform image registration, ,in, For the aligned data, represents the registration transformation matrix, which is determined by minimizing the optimal registration loss function;

[0020] Carry out convolutional neural network modeling, and assume that the fused image is , As the input of CNN, it is defined as: ,in, Represents the splicing operation on the channel. At this time, , the first convolutional layer of CNN is Perform a convolution operation: ,in, represents the output feature map of the first convolutional layer, is the convolution kernel weight matrix, size is , is the bias vector of the convolutional layer, is the ReLU activation function;

[0021] In the In the convolutional layer, higher-level features are extracted: ,in, For the The output feature map of the convolutional layer, For the Layer convolution kernel weight matrix, For the The output feature map of the layer.

[0022] Furthermore, in step S1, a preliminary fire sign detection is performed:

[0023] The output of the last convolutional layer After the fully connected layer, the fire signs are classified. ,in, is the probability distribution of fire classification results, is the weight matrix of the fully connected layer, is the bias vector of the fully connected layer, Indicates that the feature map Flatten into vectors;

[0024] Using the cross entropy loss function Measures the difference between the predicted result and the actual label, ,in is the loss function value, is the one-hot encoding of the actual label, is the predicted probability distribution, when When the preset threshold is exceeded, a fire alarm is triggered.

[0025] Furthermore, the early fire warning method in step S2 is:

[0026] From the fused image data In the process of extracting visible light and infrared spectrum information, set the pixel point The visible light spectrum at , infrared spectrum is ,but:

[0027] ;

[0028] ,in, represents the wavelength of the visible light spectrum, Represents RGB three channels, is the weight factor of each channel, is the weighted sum based on RGB channels, representing the pixel point The visible light spectrum intensity at Represents pixel The infrared spectrum intensity at A nonlinear mapping function representing the extraction of temperature information from infrared images;

[0029] Calculate the rate of change of the spectrum in continuous time frames and The rate of change of the visible light spectrum is calculated to capture the early generation of smoke and abnormal changes in local temperature: ,in, is a small positive number;

[0030] Calculate the rate of change of the infrared spectrum to detect abnormal increases in local temperature: ,in, Represents pixel At wavelength The visible light spectrum change rate under Represents pixel The infrared spectrum change rate at Represents a small positive number.

[0031] Furthermore, the early fire warning method in step S2 also includes:

[0032] Perform anomaly detection by integrating spectral features, integrating the change rates of visible light and infrared spectra to form comprehensive early signs of fire : ,in, and represents the adjustment parameters that control the weights of visible light and infrared spectra;

[0033] Perform anomaly detection using weighted features With the preset threshold Compare to see if there are early signs of fire:

[0034] Early fire detection and warning based on multimodal spectral data can trigger alarms in time in the early stages of fire by detecting temperature anomalies and initial smoke.

[0035] Furthermore, in step S3, the multimodal data fusion and fire phenomenon differentiation are performed in the following manner:

[0036] For each pixel , from the visible light image Extract a set of features from Represents the intensity information of the RGB three channels and their second-order statistical characteristics:

[0037] ,in Represents pixels Place The intensity value of each channel (R, G, B), Represents pixels The average intensity value of the surrounding local area, Represents pixels The intensity variance of the surrounding local area;

[0038] From infrared images Extract feature vectors from Representing temperature information and its second-order statistical characteristics :

[0039] ,in Represents pixels The infrared intensity value at Represents pixels The average infrared intensity value of the surrounding local area, Represents pixels Variance of infrared intensity in the surrounding local area;

[0040] The visible light and infrared feature vectors are fused to obtain the joint feature vector :

[0041] ,in Represents pixels The fused feature vector at , represents the weighting coefficients of visible and infrared features;

[0042] The fusion feature vector Perform nonlinear mapping to generate feature representations in high-dimensional feature space :

[0043] ,in Represents a nonlinear activation function, using ReLU and Sigmoid functions. represents the weight matrix of the fused feature vector, represents the bias vector of the fused feature vector, Represents pixels High-dimensional feature representation at .

[0044] Furthermore, in step S3, the multimodal data fusion and fire phenomenon differentiation are performed in the following manner:

[0045] For each pixel , from the visible light image Extract a set of features from Represents the intensity information of the RGB three channels and their second-order statistical characteristics:

[0046] ,in Represents pixels Place The intensity value of each channel (R, G, B), Represents pixels The average intensity value of the surrounding local area, Represents pixels The intensity variance of the surrounding local area;

[0047] From infrared images Extract feature vectors from Representing temperature information and its second-order statistical characteristics :

[0048] ,in Represents pixels The infrared intensity value at Represents pixels The average infrared intensity value of the surrounding local area, Represents pixels Variance of infrared intensity in the surrounding local area;

[0049] The visible light and infrared feature vectors are fused to obtain the joint feature vector :

[0050] ,in Represents pixels The fused feature vector at , represents the weighting coefficients of visible and infrared features;

[0051] The fusion feature vector Perform nonlinear mapping to generate feature representations in high-dimensional feature space :

[0052] ,in Represents a nonlinear activation function, using ReLU and Sigmoid functions. represents the weight matrix of the fused feature vector, represents the bias vector of the fused feature vector, Represents pixels High-dimensional feature representation at .

[0053] Furthermore, in step S5, a virtual model of the equipment is constructed using a generative adversarial network (GANs), and the virtual model data comes from historical image data and sensor data of the firefighting equipment;

[0054] In step S5, virtual reconstruction technology is used to identify and simulate minor changes in the equipment, including fine cracks in the appearance, rust expansion, and coating aging. By continuously comparing the current state with the virtual model, possible equipment failures can be warned in advance.

[0055] Furthermore, the virtual model reconstruction method in step S5 is:

[0056] Use generative adversarial networks (GANs) to build virtual models of firefighting equipment;

[0057] Input data includes historical images of the equipment (photos of the equipment from different angles) and operational data recorded by sensors (temperature, vibration);

[0058] The generative network and the discriminative network are used to compete with each other to optimize the model. The generative network is responsible for creating a new virtual device model, while the discriminative network evaluates the authenticity of the model.

[0059] After the virtual model is built, the real-time acquired device images and sensor data are compared with the virtual model to identify the differences between the current state of the device and the virtual model.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] The present invention adopts an early fire detection method based on the fusion of thermal imaging and visible light, and realizes more accurate and earlier detection of fire by fusing multiple sensory data. It combines thermal imaging with visible light images and uses deep learning algorithms (convolutional neural networks) for real-time analysis. The fusion provides more comprehensive environmental information, including temperature anomalies and signs of fire under visible light, including open flames and smoke.

[0062] The present invention analyzes the changes in visible light and infrared spectra to detect early signs of fire, including local temperature anomalies and smoke generation, before the fire spreads on a large scale, thereby achieving earlier warning.

[0063] The present invention adopts an algorithm for fusing data to distinguish between real fire and non-fire heat sources or optical phenomena, thereby reducing false alarms and improving the reliability of the system.

[0064] The present invention, combined with visible light information, can more intuitively locate the location of the fire source and identify the specific nature of the fire.

[0065] The present invention collects historical image data and sensor data of a device and uses a generative adversarial network (GANs) to create a virtual model of the device; the model simulates the current state of the device.

[0066] The present invention uses virtual reconstruction technology to identify and simulate minor changes in equipment, including fine cracks in appearance, rust expansion, and coating aging; by continuously comparing the current state with the virtual model, it can provide early warning of possible equipment failure points. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a flow chart of the intelligent management method of fire-fighting equipment based on image processing of the present invention. DETAILED DESCRIPTION

[0068] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0069] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0070] The present invention is further described in detail below in conjunction with the accompanying drawings: Example 1

[0071] See also Figure 1 The present invention provides an intelligent management system for fire fighting equipment based on image processing, comprising:

[0072] The multimodal image fusion module combines thermal imaging data with visible light image data and uses a convolutional neural network (CNN) to perform real-time analysis on the fused multimodal data to capture temperature anomalies and signs of fire under visible light, enabling early detection of fires and identifying potential fire threats before smoke or temperature changes significantly.

[0073] The early fire warning module uses spectral data extraction and analysis to extract visible light and infrared spectrum information from the fused image data, analyze the spectral change rate at each pixel, and capture early signs of fire, including local temperature anomalies and the generation of initial smoke;

[0074] The fire phenomenon differentiation module is used for feature extraction and fusion. It extracts features from visible light images and infrared images respectively, and performs weighted fusion on the feature vectors to generate representations in a high-dimensional feature space.

[0075] The fire source location and identification module is used to locate the fire source. It uses multimodal fusion data and visible light image information to locate the location of the fire source.

[0076] Equipment status monitoring module: The virtual model of fire-fighting equipment simulates the current status of the equipment and performs equipment monitoring.

[0077] See also Figure 1 The present invention also provides a method for intelligent management of fire-fighting equipment based on image processing, comprising:

[0078] Step S1, thermal imaging visible light image fusion, combines thermal imaging data with visible light image data, uses a convolutional neural network with a deep learning algorithm to perform real-time analysis on the fused multimodal data, and comprehensively captures temperature anomalies and signs of fire under visible light, including open flames and smoke; achieves earlier detection of fires, and can identify potential fire threats before smoke or temperature changes significantly;

[0079] The method for performing thermal imaging visible light image fusion analysis in step S1 is:

[0080] Input data includes thermal imaging data and visible light image data ,in Represents a thermal image, containing the temperature information of each pixel. Represents a visible light image of RGB three channels, including the light intensity of each pixel. Indicates the image pixel height and width;

[0081] To ensure that multimodal data are aligned in the same coordinate system, and Perform image registration, ,in, For the aligned data, represents the registration transformation matrix, which is determined by minimizing the optimal registration loss function;

[0082] Carry out convolutional neural network modeling, and assume that the fused image is , As the input of CNN, it is defined as: ,in, Represents the splicing operation on the channel. At this time, , the first convolutional layer of CNN is Perform a convolution operation: ,in, represents the output feature map of the first convolutional layer, is the convolution kernel weight matrix, size is , is the bias vector of the convolutional layer, is the ReLU activation function;

[0083] In the In the convolutional layer, higher-level features are extracted: ,in, For the The output feature map of the convolutional layer, For the Layer convolution kernel weight matrix, For the Output feature map of the layer;

[0084] In step S1, a preliminary fire sign detection is performed:

[0085] The output of the last convolutional layer After the fully connected layer, the fire signs are classified. ,in, is the probability distribution of fire classification results, is the weight matrix of the fully connected layer, is the bias vector of the fully connected layer, Indicates that the feature map Flatten into vectors;

[0086] Using the cross entropy loss function Measures the difference between the predicted result and the actual label, ,in is the loss function value, is the one-hot encoding of the actual label, is the predicted probability distribution, when When the preset threshold is exceeded, a fire alarm is triggered;

[0087] Step S2, early fire warning, analyzes the changes in visible light and infrared spectra based on fused data to detect early signs of fire, including local temperature anomalies and initial smoke. Step S2 uses the aforementioned image fusion data to improve the sensitivity of fire detection and the accuracy of early warning, and provides earlier fire warning than a single sensor, identifying danger signals in advance before the fire spreads on a large scale.

[0088] The early fire warning method in step S2 is:

[0089] From the fused image data In the process of extracting visible light and infrared spectrum information, set the pixel point The visible light spectrum at , infrared spectrum is ,but:

[0090] ;

[0091] ,in, represents the wavelength of the visible light spectrum, Represents RGB three channels, is the weight factor of each channel, is the weighted sum based on RGB channels, representing the pixel point The visible light spectrum intensity at Represents pixel The infrared spectrum intensity at A nonlinear mapping function representing the extraction of temperature information from infrared images;

[0092] Calculate the rate of change of the spectrum in continuous time frames and The rate of change of the visible light spectrum is calculated to capture the early generation of smoke and abnormal changes in local temperature: ,in, is a small positive number;

[0093] Calculate the rate of change of the infrared spectrum to detect abnormal increases in local temperature: ,in, Represents pixel At wavelength The visible light spectrum change rate under Represents pixel The infrared spectrum change rate at Represents a small positive number;

[0094] The early fire warning method in step S2 also includes:

[0095] Perform anomaly detection by integrating spectral features, integrating the change rates of visible light and infrared spectra to form comprehensive early signs of fire : ,in, and represents the adjustment parameters that control the weights of visible light and infrared spectra;

[0096] Perform anomaly detection using weighted features With the preset threshold Compare to see if there are early signs of fire:

[0097] Early fire detection and warning based on multimodal spectral data can trigger alarms in time in the early stages of a fire by detecting temperature anomalies and initial smoke;

[0098] Specifically, in terms of early fire detection, the system can identify potential fire threats before smoke spreads to the detector or the temperature rises significantly by combining thermal imaging data and visible light image data with a single sensor and using a convolutional neural network (CNN) for real-time analysis. The fusion of multimodal data not only improves the sensitivity of early fire detection, but also effectively reduces the risk of missed and false alarms.

[0099] Step S3, multimodal data fusion, using a fusion algorithm to distinguish between real fire and non-fire phenomena, including sunlight reflection and thermal radiation from mechanical equipment; in step S3, the multimodal information extracted in steps S1 and S2 is used to perform fire detection;

[0100] The method of multimodal data fusion and fire phenomenon distinction in step S3 is as follows:

[0101] For each pixel , from the visible light image Extract a set of features from Represents the intensity information of the RGB three channels and their second-order statistical characteristics:

[0102] ,in Represents pixels Place The intensity value of each channel (R, G, B), Represents pixels The average intensity value of the surrounding local area, Represents pixels The intensity variance of the surrounding local area;

[0103] From infrared images Extract feature vectors from Representing temperature information and its second-order statistical characteristics :

[0104] ,in Represents pixels The infrared intensity value at Represents pixels The average infrared intensity value of the surrounding local area, Represents pixels Variance of infrared intensity in the surrounding local area;

[0105] The visible light and infrared feature vectors are fused to obtain the joint feature vector :

[0106] ,in Represents pixels The fused feature vector at , represents the weighting coefficients of visible and infrared features;

[0107] The fusion feature vector Perform nonlinear mapping to generate feature representations in high-dimensional feature space :

[0108] ,in Represents a nonlinear activation function, using ReLU and Sigmoid functions. represents the weight matrix of the fused feature vector, represents the bias vector of the fused feature vector, Represents pixels High-dimensional feature representation at ;

[0109] In step S3, fire phenomena are distinguished and classified:

[0110] The mapped features Input into a pre-trained classifier (support vector machine and neural network are used) to classify fire and non-fire phenomena: ,in Represents pixels The classification results at Indicates fire, Indicates non-fire;

[0111] Multi-scale analysis is performed on the image to improve the detection accuracy of fire phenomena. , calculate the classification results after multi-scale fusion : ,in Represents pixels The multi-scale fusion classification results at represents the weighting coefficient of scale s, Represents the classification result at scale s;

[0112] Finally, by setting the threshold The classification results after fusion Perform binary processing to determine whether there is a fire phenomenon;

[0113] Step S4, fire source location, using multimodal fusion data and visible light information to locate the fire source; in step S4, the specific nature of the fire is simultaneously identified, and electrical fires and combustible material fires are distinguished; based on multimodal analysis, the fire nature distinction is further improved, providing firefighters with accurate fire source location information;

[0114] Specifically, the system accurately locates the source of fire and identifies the nature of the fire. It combines multimodal data and visible light information to precisely locate the source of fire and distinguish the specific nature of the fire, including electrical fires or fires caused by combustible materials, allowing firefighters to respond to fires more quickly and accurately, reducing casualties and property losses.

[0115] Step S5, virtual model reconstruction, using the virtual model of the fire equipment to simulate the current state of the equipment, and combining it with the multimodal data extracted previously; through the reconstruction of the virtual model, the equipment state is continuously monitored and the points prone to failure are identified;

[0116] In step S5, a virtual model of the equipment is constructed using a generative adversarial network (GANs), and the virtual model data comes from historical image data and sensor data of the firefighting equipment;

[0117] In step S5, virtual reconstruction technology is used to identify and simulate minor changes in the equipment, including minor cracks in the appearance, corrosion expansion, and coating aging. By continuously comparing the current state with the virtual model, possible equipment failures can be warned in advance.

[0118] The virtual model reconstruction method in step S5 is:

[0119] Use generative adversarial networks (GANs) to build virtual models of firefighting equipment;

[0120] Input data includes historical images of the equipment (photos of the equipment from different angles) and operational data recorded by sensors (temperature, vibration);

[0121] The generative network and the discriminative network are used to compete with each other to optimize the model. The generative network is responsible for creating a new virtual device model, while the discriminative network evaluates the authenticity of the model.

[0122] After the virtual model is built, the real-time collected device images and sensor data are compared with the virtual model to identify the differences between the current state of the device and the virtual model;

[0123] Specifically, in terms of equipment management and fault warning, a virtual model of fire-fighting equipment is built through generative adversarial networks (GANs) to improve the accuracy of equipment status monitoring. Historical image data and sensor data are used to simulate the current status of the equipment and compare it with the data collected in real time. It can effectively identify minor changes in the appearance of the equipment, such as cracks, rust, or coating aging, greatly reducing the potential risks caused by equipment failure.

[0124] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. An intelligent management system for fire fighting equipment based on image processing, characterized in that: include: The multimodal image fusion module combines thermal imaging data with visible light image data, and uses the convolutional neural network (CNN) to perform real-time analysis on the fused multimodal data to capture temperature anomalies and signs of fire under visible light. The early fire warning module uses spectral data extraction and analysis to extract visible light and infrared spectrum information from the fused image data, analyze the spectral change rate at each pixel, and capture early signs of fire, including local temperature anomalies and the generation of initial smoke; The fire phenomenon differentiation module is used for feature extraction and fusion. It extracts features from visible light images and infrared images respectively, and performs weighted fusion on the feature vectors to generate representations in a high-dimensional feature space. The fire source location and identification module is used to locate the fire source. It uses multimodal fusion data and visible light image information to locate the location of the fire source. Equipment status monitoring module: the virtual model of fire equipment simulates the current status of the equipment and performs equipment monitoring; The management method of the fire-fighting equipment intelligent management system based on image processing includes: Step S1, thermal imaging visible light image fusion, combining thermal imaging data with visible light image data, using a convolutional neural network with a deep learning algorithm to perform real-time analysis on the fused multimodal data, and comprehensively capturing temperature anomalies and signs of fire under visible light, including open flames and smoke; Step S2, early fire warning, analyzes the changes in visible light and infrared spectra based on the fused data to detect early signs of fire, including local temperature anomalies and incipient smoke; Step S3, multimodal data fusion, using a fusion algorithm to distinguish between real fire and non-fire phenomena, including sunlight reflection and thermal radiation from mechanical equipment; in step S3, the multimodal information extracted in steps S1 and S2 is used to perform fire detection; Step S4, fire source location, using multimodal fusion data combined with visible light information to locate the fire source; in step S4, the specific nature of the fire is simultaneously identified to distinguish between electrical fires and combustible material fires; Step S5, virtual model reconstruction, using the virtual model of the fire equipment to simulate the current state of the equipment and combining it with the previously extracted multimodal data; The method for performing thermal imaging and visible light image fusion analysis in step S1 is: Input data includes thermal imaging data and visible light image data ,in Represents a thermal image, containing the temperature information of each pixel. Represents a visible light image of RGB three channels, including the light intensity of each pixel. Indicates the image pixel height and width; right and Perform image registration, ,in, For the aligned data, represents the registration transformation matrix, which is determined by minimizing the optimal registration loss function; Carry out convolutional neural network modeling, and assume that the fused image is , As the input of CNN, it is defined as: ,in, Represents the splicing operation on the channel. At this time, , the first convolutional layer of CNN is Perform a convolution operation: ,in, represents the output feature map of the first convolutional layer, is the convolution kernel weight matrix, size is , is the bias vector of the convolutional layer, is the ReLU activation function; In the In the convolutional layer, higher-level features are extracted: ,in, For the The output feature map of the convolutional layer, For the Layer convolution kernel weight matrix, For the The output feature map of the layer.

2. The fire-fighting equipment intelligent management system based on image processing according to claim 1 is characterized in that: In step S1, a preliminary fire sign detection is performed: The output of the last convolutional layer After the fully connected layer, the fire signs are classified. ,in, is the probability distribution of fire classification results, is the weight matrix of the fully connected layer, is the bias vector of the fully connected layer, Indicates that the feature map Flatten into vectors; Using the cross entropy loss function Measures the difference between the predicted result and the actual label, ,in is the loss function value, is the one-hot encoding of the actual label, is the predicted probability distribution, when When the preset threshold is exceeded, a fire alarm is triggered.

3. The fire-fighting equipment intelligent management system based on image processing according to claim 2 is characterized in that: The early fire warning method in step S2 is: From the fused image data In the process of extracting visible light and infrared spectrum information, set the pixel point The visible light spectrum at , infrared spectrum is ,but: ； ,in, represents the wavelength of the visible light spectrum, Represents RGB three channels, is the weight factor of each channel, is the weighted sum based on RGB channels, representing the pixel point The visible light spectrum intensity at Represents pixel The infrared spectrum intensity at A nonlinear mapping function representing the extraction of temperature information from infrared images; Calculate the rate of change of the spectrum in continuous time frames and The rate of change of the visible light spectrum is calculated to capture the early generation of smoke and abnormal changes in local temperature: ,in, is a small positive number; Calculate the rate of change of the infrared spectrum to detect abnormal increases in local temperature: ,in, Represents pixel At wavelength The visible light spectrum change rate under Represents pixel The infrared spectrum change rate at Represents a small positive number.

4. The fire-fighting equipment intelligent management system based on image processing according to claim 3 is characterized in that: The early fire warning method in step S2 also includes: Perform anomaly detection by integrating spectral features, integrating the change rates of visible light and infrared spectra to form comprehensive early signs of fire : ,in, and represents the adjustment parameters that control the weights of visible light and infrared spectra; Perform anomaly detection using weighted features With the preset threshold Make a comparison to determine if there are early signs of a fire.

5. The fire-fighting equipment intelligent management system based on image processing according to claim 4 is characterized in that: The method of multimodal data fusion and fire phenomenon distinction in step S3 is as follows: For each pixel , from the visible light image Extract a set of features from Represents the intensity information of the RGB three channels and their second-order statistical characteristics: ,in Represents pixels Place The intensity value of each channel (R, G, B), Represents pixels The average intensity value of the surrounding local area, Represents pixels The intensity variance of the surrounding local area; From infrared images Extract feature vectors from Representing temperature information and its second-order statistical characteristics : ,in Represents pixels The infrared intensity value at Represents pixels The average infrared intensity value of the surrounding local area, Represents pixels Variance of infrared intensity in the surrounding local area; The visible light and infrared feature vectors are fused to obtain the joint feature vector : ,in Represents pixels The fused feature vector at , represents the weighting coefficients of visible and infrared features; The fusion feature vector Perform nonlinear mapping to generate feature representations in high-dimensional feature space : ,in Represents a nonlinear activation function, using ReLU and Sigmoid functions. represents the weight matrix of the fused feature vector, represents the bias vector of the fused feature vector, Represents pixels High-dimensional feature representation at .

6. The fire-fighting equipment intelligent management system based on image processing according to claim 5 is characterized in that: In step S3, fire phenomena are distinguished and classified: The mapped features Input into a pre-trained classifier to classify fire and non-fire phenomena: ,in Represents pixels The classification results at Indicates fire, Indicates non-fire; Perform multi-scale analysis on the image, and for each scale s, calculate the classification result after multi-scale fusion ,in Represents pixels The multi-scale fusion classification results at represents the weighting coefficient of scale s, Represents the classification result at scale s; Finally, by setting the threshold The classification results after fusion Perform binary processing to determine whether there is a fire phenomenon.

7. The fire-fighting equipment intelligent management system based on image processing according to claim 6 is characterized in that: In step S5, a virtual model of the equipment is constructed using a generative adversarial network (GANs), and the virtual model data comes from historical image data and sensor data of the firefighting equipment; In step S5, virtual reconstruction technology is used to identify and simulate minor changes in the equipment, including fine cracks in the appearance, rust expansion, and coating aging. By continuously comparing the current state with the virtual model, possible equipment failures can be warned in advance.

8. The fire-fighting equipment intelligent management system based on image processing according to claim 7 is characterized in that: The virtual model reconstruction method in step S5 is: Use generative adversarial networks (GANs) to build virtual models of firefighting equipment; Input data includes historical images of equipment and operational data recorded by sensors; The generative network and the discriminative network are used to compete with each other to optimize the model. The generative network is responsible for creating a new virtual device model, while the discriminative network evaluates the authenticity of the model. After the virtual model is built, the real-time acquired device images and sensor data are compared with the virtual model to identify the differences between the current state of the device and the virtual model.

Images