Fire confidence assessment method based on multi-modal deep collaboration in complex industrial environment

Abstract

The application provides a kind of fire confidence assessment method of multi-modal deep collaboration under complex industrial environment, it is related to fire confidence assessment technical field. Including: the multi-modal data of the region to be measured is collected, the corresponding temperature mutation rate and diffusion entropy are determined according to the thermal imaging image data;Temperature mutation rate and diffusion entropy are input into the constructed smoldering probability model to obtain the smoldering probability;Determine the smoke confidence according to the visible light image data;Determine the fire confidence according to the smoldering probability and the smoke confidence;Fire confidence is used to evaluate the fire of the region to be measured.The application solves the problem of high false alarm rate and poor real-time performance of the prior art fire warning technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fire confidence assessment, in particular to a multi-modal deep collaboration fire confidence assessment method in a complex industrial environment. BACKGROUND

[0002] In the industrial scene of cotton and linen textile, chemical fiber processing and other equipment-intensive, high-temperature normalization, the traditional fire warning technology has long been facing severe challenges, and there are problems of high false alarm rate, smoldering detection lag and weak anti-interference ability. Smoldering is the initial stage of fire, and the traditional thermal imaging sensor relies on a fixed temperature threshold to trigger the alarm device. However, the normal operating temperature of equipment such as motors and pressing machines in the workshop can reach 50-70℃, resulting in a high false alarm rate. Visible light smoke detection is completely ineffective when there is no visible smoke in the early stage of smoldering or when there is dust shielding. Gas sensors (such as CO detectors) are affected by the ventilation in the workshop, and the gas concentration is not evenly distributed, which significantly increases the risk of false negatives.

[0003] More importantly, the temperature field in a complex environment is non-uniformly distributed: normal equipment heating is uniformly diffused; and in the early stage of smoldering, the material absorbs heat and shows a centripetal convergence gradient difference. The traditional static threshold method cannot capture the spatiotemporal dynamic characteristics, resulting in an increased risk of missing early fire detection. In addition, the height and shape of the material pile in the industrial scene are variable, further increasing the error of thermal imaging temperature measurement. Some studies have used a multi-sensor fusion scheme, but most of these technologies only use simple signal stacking, lack the ability to suppress high-temperature noise, and the traditional machine learning algorithms (SVM, random forest) have a large delay in reasoning, making it difficult to meet the real-time requirements. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide a multi-modal deep collaboration fire confidence assessment method in a complex industrial environment, which solves the problems of high false alarm rate and poor real-time performance of the fire warning technology in the prior art.

[0005] To achieve the above-mentioned purpose, the present application provides the following scheme:

[0006] A multi-modal deep collaboration fire confidence assessment method in a complex industrial environment, comprising:

[0007] acquiring multi-modal data of a to-be-measured region, wherein the multi-modal data includes thermal imaging image data and visible light image data;

[0008] determining a corresponding temperature mutation rate and diffusion entropy according to the thermal imaging image data;

[0009] inputting the temperature mutation rate and diffusion entropy into a constructed smoldering probability model to obtain a smoldering probability;

[0010] determining a smoke confidence according to the visible light image data;

[0011] determining a fire confidence according to the smoldering probability and the smoke confidence;

[0012] evaluating a fire in the to-be-tested region by using the fire confidence.

[0013] Preferably, the step of calculating the temperature mutation rate comprises:

[0014] setting a time variation interval;

[0015] determining device temperature data according to the temperature score data;

[0016] determining a temperature mutation rate in a current time variation interval according to the time variation interval and the device data, wherein the expression of the temperature mutation rate is:

[0017] ;

[0018] wherein, is the temperature mutation rate, is the time variation interval, is device temperature data corresponding to the current time, is the current time.

[0019] Preferably, the step of calculating the diffusion entropy comprises:

[0020] determining a corresponding image gradient according to the thermal imaging image data;

[0021] calculating a corresponding gradient amplitude and direction according to the image gradient;

[0022] determining a thermal imaging gradient direction histogram according to a direction interval index and the gradient amplitude and direction;

[0023] determining a diffusion entropy according to the gradient direction histogram;

[0024] The expression for calculating the diffusion entropy is:

[0025] ;

[0026] wherein, is the diffusion entropy, is the normalized gradient direction probability distribution, and u is the index of the gradient direction.

[0027] Preferably, the network framework of the smoldering probability model is a three-layer structure of LSTM layer, full connection layer and activation function.

[0028] Preferably, determining a smoke confidence according to the visible light image data comprises:

[0029] Develop an improved lightweight smoke detection model;

[0030] The visible light image data is input into the improved lightweight smoke detection model to determine the smoke confidence level. The improved lightweight smoke detection model is constructed based on an improved YOLOv8n network with 96 channels and an activation function of either ReLU or SiLU.

[0031] Preferably, determining the fire confidence level based on the smoldering probability and the smoke confidence level includes:

[0032] The evidence set is determined based on the smoldering probability and the smoke confidence level;

[0033] The weights of each piece of evidence in the evidence set are dynamically allocated based on historical data to determine the weights corresponding to each piece of evidence. The historical data includes historical thermal imaging image data and historical visible light image data.

[0034] The fire confidence level is determined based on the evidence with predetermined weights.

[0035] Preferably, determining the weights corresponding to each piece of evidence includes:

[0036] Determine the accuracy rate corresponding to each piece of evidence;

[0037] The normalization weights are determined based on the accuracy rate;

[0038] The detection results corresponding to each piece of evidence are determined and it is determined whether there is a conflict. If there is a conflict, uniformly distributed uncertain evidence is introduced and the normalization weights are corrected.

[0039] Preferably, determining the fire confidence level based on the weighted evidence includes:

[0040] Determine the coefficient of conflict of evidence;

[0041] The fire confidence level is determined using the Dempster rule and the pre-weighted evidence.

[0042] The expression for the fire confidence level is as follows:

[0043] ;

[0044] Where A represents the subset allocated to the synthesized evidence body, and B and C are subsets of the first and second evidence bodies, respectively. For fire confidence level, The conflict coefficient, and These are the weights corresponding to the first and second pieces of evidence, respectively.

[0045] The present invention discloses the following technical effects:

[0046] This invention provides a multimodal deep collaborative fire confidence assessment method in complex industrial environments, comprising: collecting multimodal data of the area to be tested, wherein the multimodal data includes thermal imaging image data and visible light image data; determining the corresponding temperature abrupt change rate and diffusion entropy based on the thermal imaging image data; inputting the temperature abrupt change rate and diffusion entropy into a pre-constructed smoldering probability model to obtain the smoldering probability; determining the smoke confidence level based on the visible light image data; determining the fire confidence level based on the smoldering probability and the smoke confidence level; and using the fire confidence level to assess the fire in the area to be tested. This invention utilizes a combination of a track-mounted mobile robot and fixed cameras (thermal imaging sensors and visible light cameras) to achieve multimodal data acquisition and preprocessing. In dynamic temperature gradient discrimination, the spatiotemporal characteristics of temperature abrupt change rate and diffusion entropy are used to distinguish between normal equipment heating and the initial abnormality of smoldering. An LSTM network is used to model the smoldering probability of the temperature abrupt change rate and diffusion entropy time series, capturing nonlinear and long-term dependent dynamic changes. For visible light smoke detection, the YOLOv8n network is improved through backbone network compression optimization, detector head optimization, and anti-interference training to enhance the accuracy and real-time performance of smoke detection in complex environments. Based on DS evidence theory, dual-modal confidence scores are fused to construct an evidence body and perform dynamic weight allocation, resolving sensor conflicts and uncertainty interference. Practical verification shows that this method reduces the false alarm rate to below 5%, advances smoldering detection by 10 minutes, and controls the system response delay to within 300 ms, achieving accurate and real-time fire confidence assessment and providing a reliable fire early warning solution for high-risk industrial environments. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 The flowchart illustrates a fire confidence assessment method based on multimodal deep collaboration in a complex industrial environment, as provided in this embodiment of the invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] like Figure 1 As shown, this invention provides a multimodal deep collaborative fire confidence assessment method for complex industrial environments, including:

[0052] Step 100: Collect multimodal data of the area to be tested, wherein the multimodal data includes: thermal imaging image data and visible light image data;

[0053] Specifically, to obtain comprehensive and accurate fire-related information in complex industrial environments, we deploy sensors using a combination of rail-mounted mobile robots and fixed cameras. The rail-mounted mobile robots can move flexibly within the workshop, expanding the monitoring range, while the fixed cameras can continuously monitor specific areas.

[0054] Specifically, the sensors equipped with two modes include:

[0055] Thermal imaging sensor: Operating in the 8-14μm wavelength band, its temperature measurement range is -20℃ to 150℃, with an accuracy of ±2℃. This sensor can capture the temperature distribution in the environment in real time, providing basic data for subsequent temperature analysis.

[0056] Visible light camera: With a resolution of 1920×1080, it can capture visible light images of the workshop in real time, which can be used to continuously detect smoke signals and detect early signs of fire in a timely manner.

[0057] Step 200: Determine the corresponding temperature abrupt change rate and diffusion entropy based on the thermal imaging image data;

[0058] Step 300: Input the temperature mutation rate and diffusion entropy into the constructed smoldering probability model to obtain the smoldering probability;

[0059] Step 400: Determine the smoke confidence level based on the visible light image data;

[0060] Step 500: Determine the fire confidence level based on the smoldering probability and the smoke confidence level;

[0061] Step 600: Assess the fire situation in the area to be tested using the fire confidence level.

[0062] Furthermore, by combining the spatiotemporal characteristics of temperature abrupt change rate and diffusion entropy, the normal heating of equipment and the abnormality in the early stage of smoldering are distinguished. The specific steps are as follows:

[0063] Set the time range for variation;

[0064] Determine the equipment temperature data based on the temperature fraction data;

[0065] Among them, the temperature fraction data is a temperature feature matrix obtained by normalizing and calibrating thermal imaging image data.

[0066] The temperature abrupt change rate within the current time interval is determined based on the time variation interval and the device data, wherein the expression for the temperature abrupt change rate (time dimension) is:

[0067] ;

[0068] in, The temperature mutation rate, For the time range, This refers to the device temperature data at the current moment. ,and At that time, spatial dimension analysis is triggered; Indicates ambient temperature. Set the current time .

[0069] The temperature rises relatively slowly in the initial stage of smoldering in cotton and linen, so a lower temperature change rate threshold (0.2℃ / min) needs to be set because although the temperature change is not obvious in the initial stage of smoldering, it is persistent over a long period. At the same time, the ambient temperature becomes higher due to the operation of machines in the workshop, so the temperature change threshold is set to be 10℃ higher than the ambient temperature.

[0070] Furthermore, the calculation steps for the diffusion entropy include:

[0071] Determine the corresponding image gradient based on the thermal imaging image data;

[0072] Calculate the corresponding gradient magnitude and direction based on the image gradient;

[0073] The thermal imaging gradient direction histogram is determined based on the direction interval index and the gradient magnitude and direction.

[0074] The diffusion entropy is determined based on the gradient direction histogram.

[0075] Specifically, calculate the gradient of the thermal imaging image:

[0076] The gradient reflects the rate of change of image pixel values, and the Sobel operator is used to calculate the approximate gradient of the image grayscale function. Assuming this applies to thermal imaging images... Calculate its gradient in the horizontal (x-direction) and vertical (y-direction) directions respectively:

[0077] ;

[0078] ;

[0079] In the formula, * denotes the convolution operation. and These represent thermal imaging images. Gradients in the horizontal and vertical directions.

[0080] Calculate the gradient magnitude and direction:

[0081] The gradient magnitude reflects the degree of drastic change in pixel values ​​in an image, while the gradient direction indicates the direction of change in pixel values. The calculation is as follows:

[0082] ;

[0083] ;

[0084] In the formula, and These represent the magnitude and direction of the image gradient, respectively. .

[0085] For ease of calculation, Adjust to :

[0086] ;

[0087] Divide the gradient direction interval:

[0088] To quantize the gradient direction, The range is divided into 8 gradient direction intervals of equal width. The width of each interval is... for:

[0089] ;

[0090] It can be seen that the starting angle of the i-th interval is: The ending angle is: ,in: .

[0091] For any image pixel, the gradient direction can be determined by its gradient direction interval index. :

[0092] ;

[0093] In the formula, Floor(·) is the floor operator.

[0094] Extract and normalize the histogram of gradient directions (HOG) from thermal imaging:

[0095] Iterate through all pixels in the image, and accumulate the gradient magnitude of each pixel into the histogram value of its corresponding gradient direction interval, finally obtaining a HOG feature vector containing 8 elements.

[0096] ;

[0097] In the above formula, Corresponding to 8 directional intervals, ; This represents the gradient histogram value, where u is the index of the gradient direction; Represents the coordinates of all pixels in the image. This represents the gradient direction interval index to which the pixel belongs.

[0098] Based on the calculated gradient histogram, the normalized probability distribution is as follows:

[0099] ;

[0100] Calculate diffusion entropy:

[0101] ;

[0102] in, This represents the normalized gradient direction probability distribution. .

[0103] Furthermore, fusion using LSTM networks and Time series, output smoldering probability .

[0104] In complex industrial environments, the temperature abrupt change rate during the initial stage of smoldering and diffusion entropy value It exhibits nonlinear and long-term dependent dynamic changes. Traditional machine learning models struggle to capture the temporal correlations within its time series data. Therefore, a Long Short-Term Memory (LSTM) network is used to construct a smoldering probability model to achieve dynamic prediction of the smoldering state. The specific steps are as follows:

[0105] Determine the input features of the LSTM:

[0106] and Both are time series, with two features for each time interval. Assumption: Taking the past... A time window, that is: a duration of The input feature sequence is defined as follows:

[0107] ;

[0108] This represents the starting point of N time windows, with the ending point being t;

[0109] * represents the last hidden state set output by the LSTM layer, and * represents the dimension.

[0110] In the input feature sequence, It is a dimensionless quantity. To eliminate the difference in dimensions, the data needs to be preprocessed as follows:

[0111] ;

[0112] in, and These are the mean and standard deviation of the training set, respectively.

[0113] Define status labels ,in This indicates that there is a risk of smoldering at the present moment; the label is generated based on manually labeled data.

[0114] The model adopts a three-layer structure of "LSTM layer + fully connected layer + activation function", as follows:

[0115] The LSTM layer uses a single layer of 128 units to capture dependencies in long time series and returns the hidden state of the last time step. Then it is processed through a fully connected layer.

[0116] The fully connected layer consists of 64 neurons and performs dimensionality reduction on the high-dimensional features output by the LSTM. The activation function used is ReLU, i.e.:

[0117] ;

[0118] in, For the activation function, the weight matrix These are the core parameters of the fully connected layer; It is a 128-dimensional vector, which is the hidden state output at the last time step after the LSTM layer has processed the entire time series; This is the bias vector used to adjust the threshold of the activation function.

[0119] in, , The output layer consists of one neuron, which outputs the smoldering probability. Map the output to the probability space:

[0120] ;

[0121] in, , ,enter It is a 64-dimensional vector used to map the output layer to 1 dimension; The weight matrix of the model output layer; The bias term of the model output layer. Among them, , . , maps any real number to the interval [0,1].

[0122] Training process and optimization:

[0123] The loss function used is binary cross-entropy loss, which measures the difference between the predicted probability and the true label.

[0124] ;

[0125] Where M is the batch size. Let i be the true label of the i-th sample. For loss function, To predict probabilities.

[0126] The optimizer uses the Adam optimizer to adaptively adjust the learning rate.

[0127] ;

[0128] in, For model parameters, , The initial learning rate, and This is a correction term for the deviation of momentum and second moment. These are the model parameters at the t-th iteration. These are the model parameters after the (t+1)th update. To avoid the denominator being zero, let it be a very small constant. =1e -8 .

[0129] To prevent overfitting, a Dropout layer is added after the LSTM layer to randomly deactivate some neurons; secondly, early stopping is used to terminate training when the validation set loss does not decrease for 5 consecutive epochs.

[0130] Real-time data collection for the current and nine past time intervals. and Determine the input features for LSTM generation ;

[0131] Calculate using the trained LSTM model ,Right now:

[0132] ;

[0133] in, This represents the mapping relationship of the trained model. * represents the optimal set of parameters.

[0134] Set smoldering threshold The following rules determine whether the current state is in a smoldering state:

[0135] ;

[0136] If the current state is determined to be a high probability state of smoldering, then the subsequent DS evidence theory fusion module is triggered.

[0137] Furthermore, determining the smoke confidence level based on the visible light image data includes:

[0138] Develop an improved lightweight smoke detection model;

[0139] The visible light image data is input into the improved lightweight smoke detection model to determine the smoke confidence level. The improved lightweight smoke detection model is constructed based on an improved YOLOv8n network with 96 channels and an activation function of either ReLU or SiLU.

[0140] Specifically, in complex industrial environments, visible light smoke detection is a key auxiliary means for fire early warning. Addressing challenges such as dust obstruction and low-contrast smoke in cotton and linen workshops, an improved lightweight smoke detection model was designed and implemented as follows:

[0141] To balance detection accuracy and real-time performance, this module addresses challenges in industrial scenarios such as dust obstruction and low contrast in smoke by employing a deeply optimized YOLOv8n network. Through lightweight model design, detector head optimization, and anti-interference strategies, highly robust smoke detection is achieved. The implementation is divided into the following three layers:

[0142] Backbone network compression optimization:

[0143] The number of channels in the original YOLOv8n backbone network was reduced from 128 to 96 to reduce redundant feature extraction; the activation function used was SiLU.

[0144] ;

[0145] In the formula, This is the Sigmoid function.

[0146] Detection head optimization:

[0147] A single-scale detection head is used, retaining only the P3 layer (80×80 feature map) for detection, focusing on small target smoke particles (diameter <10 pixels) to avoid the computational overhead caused by multi-scale fusion.

[0148] Furthermore, based on the smoke morphology (slender diffusion) in the cotton and linen workshop, the re-clustering anchor frame size is... , and To improve the positioning accuracy of the smoke boundary box.

[0149] Anti-interference training:

[0150] To simulate a dusty workshop environment, dust noise was injected to enhance the data. Specifically, Gaussian noise and salt-and-pepper noise were mixed, and the noise intensity parameters were adjusted. This improves detection robustness under low signal-to-noise ratio conditions. Adaptive histogram equalization (CLAHE) is applied to the visible light image to enhance the contrast of the smoke region, as shown in the formula:

[0151] ;

[0152] in, For adaptive histogram equalization operators, This is the image after contrast enhancement.

[0153] Based on the above improvements, the reasoning process is described as follows:

[0154] A 640×640 image is input and processed by a backbone network to generate 80×80 feature maps. Each grid cell predicts three anchor boxes, and the bounding box coordinates are obtained through regression. With confidence level .

[0155] Smoke classification: The detector outputs two categories (smoke and non-smoke), and the smoke confidence score is obtained after Sigmoid activation. ,in, .

[0156] Non-maximum suppression (NMS) technology is used to retain high-confidence, non-overlapping detection boxes and avoid repeated detection in the same smoke area. For the Sigmoid function, This formula aims to detect the confidence score of the smoke class in smoke classification (smoke, non-smoke). The purpose of this formula is to use the Sigmoid activation function to... Mapping to the [0,1] space, and then transforming it into probability. .

[0157] The acquisition of logitsmoke involves preprocessing visible light image data (such as resizing), inputting it into a YOLOv8n network with 96 channels, and finally generating it through linear mapping by the output layer.

[0158] Use a multi-frame sliding window and define the window length. Only when consecutive frames Smoke confidence level was detected in all cases. When the smoke is detected, it is considered valid smoke. The multi-frame sliding window detection method can suppress instantaneous interference such as dust movement and equipment reflection, thereby reducing the false alarm rate.

[0159] Furthermore, determining the fire confidence level based on the smoldering probability and the smoke confidence level includes:

[0160] The evidence set is determined based on the smoldering probability and the smoke confidence level;

[0161] The weights of each piece of evidence in the evidence set are dynamically allocated based on historical data to determine the weights corresponding to each piece of evidence. The historical data includes historical thermal imaging image data and historical visible light image data.

[0162] The fire confidence level is determined based on the evidence with predetermined weights.

[0163] Specifically, by integrating the smoldering probability of thermal imaging with the confidence level of visible light smoke detection, the system addresses sensor conflicts and uncertainties in complex industrial environments, enabling dynamic assessment of fire confidence.

[0164] Furthermore, determining the weights corresponding to each piece of evidence includes:

[0165] Determine the accuracy rate corresponding to each piece of evidence;

[0166] The normalization weights are determined based on the accuracy rate;

[0167] The detection results corresponding to each piece of evidence are determined and it is determined whether there is a conflict. If there is a conflict, uniformly distributed uncertain evidence is introduced and the normalization weights are corrected.

[0168] Furthermore, determining the fire confidence level based on the weighted evidence includes:

[0169] Determine the coefficient of conflict of evidence;

[0170] The fire confidence level is determined using the Dempster rule and the pre-weighted evidence.

[0171] Specifically, the construction of evidence:

[0172] The basic probability distribution of thermal imaging and visible light;

[0173] The output of multimodal sensors is converted into a probability distribution, thereby quantifying the confidence level of fire, non-fire, and uncertain states.

[0174] First, define the recognition framework:

[0175] ;

[0176] in: This indicates an uncertain state, reflecting a situation where the sensor cannot clearly determine the situation due to factors such as obstruction or noise. An identification framework for describing fire confidence assessment includes a set of all possible states.

[0177] Smoldering probability The smoldering probability has been dynamically determined based on LSTM output using temperature mutation rate and diffusion entropy. Assuming that thermal imaging detection of smoldering is directly related to fire risk, with no additional uncertainty:

[0178] ;

[0179] in, Smoldering probability based on thermal imaging The created thermal imaging evidence, The probability of detecting smoldering by thermal imaging is The probability of thermal imaging detecting a non-fire condition is: . Since thermal imaging is unaffected by the environment, the detection results are deterministic and do not require an uncertain state. Assigning probabilities.

[0180] Considering the possibility of missed detections under dense obstruction of visible light, a fixed confidence level is assigned to the non-fire category, with the remaining uncertainty being:

[0181] ;

[0182] To avoid misjudgment in the absence of smoke, a confidence level of 0.2 is fixed for non-fire scenarios, and the remaining... The uncertainty represents the proportion of detection uncertainty caused by occlusion or low signal-to-noise ratio.

[0183] For smoldering probability based on visible light The created visible light evidence, To analyze the probability of detecting smoke based on visible light images . To avoid incorrect judgments caused by environmental factors (dust, ambient light) affecting visible light in the absence of smoke, the probability of no fire in visible light analysis is fixed at 0.2 to improve the model's anti-interference ability. To account for the significant susceptibility of visible light to environmental influences and interference, uncertain states that depend on... Dynamic settings.

[0184] Dynamic weight allocation:

[0185] By dynamically adjusting the weights based on the historical accuracy of the sensor in the target scene, the detection bias between thermal imaging (resistant to occlusion but susceptible to high temperature interference) and visible light (intuitively detects smoke but is easily obstructed) can be resolved.

[0186] The accuracy rates for thermal imaging and visible light are defined as follows:

[0187] ;

[0188] ;

[0189] In thermal imaging, artificially calibrated smoldering is used as a positive example to calculate... Detection accuracy at time In visible light smoke testing, statistics were collected. Smoke detection accuracy at that time .

[0190] Weight normalization:

[0191] ;

[0192] Weighted adjustment of evidence:

[0193] When there is a conflict between the high smoldering probability of thermal imaging and the smoke-free detection results from visible light, the evidence is weighted and then synthesized to avoid a single sensor dominating the decision-making process. It is the weight of the thermal imaging smoldering detection results. It is the weight of the visible light smoke detection results.

[0194] To prevent the excessive influence of low-weighted sensors, uniformly distributed uncertain evidence is introduced, and for each piece of evidence... Weighted:

[0195] ;

[0196] in, and These are the weighted averages corresponding to the first piece of evidence (thermal imaging) and the second piece of evidence (visible light), respectively. This is uncertain evidence that is uniformly distributed.

[0197] Evidence composition rules:

[0198] The main function is to integrate and weight the evidence, handle sensor conflicts, and output the final fire confidence level.

[0199] Conflict coefficient of evidence The degree of discrepancy between thermal imaging and visible light detection results was measured and calculated as follows:

[0200] ;

[0201] when When the situation is not highly conflicted, Dempster's rules are used for synthesis:

[0202] ;

[0203] Where A represents the subset of evidence allocated to the synthesized evidence body, and B and C are from two independent evidence bodies (thermal imaging). and visible light A subset of ) whose intersection satisfies . The conflict coefficient, and These are the weighted averages corresponding to the first and second pieces of evidence, respectively. Assign probability to subset A of the synthesized evidence body.

[0204] After Dempster rules are synthesized, the probability of the fire subset is calculated as follows:

[0205] ;

[0206] in, This represents the final fire probability after synthesis.

[0207] when At that time, when there is a high degree of conflict, conflicting evidence is allocated to uncertainty:

[0208] ;

[0209] Normalized to:

[0210] ;

[0211] in, This represents the final synthesized fire probability. This represents the final non-fire probability after synthesis. This represents the uncertainty after final synthesis.

[0212] The synthesized evidence includes confidence levels for "fire," "non-fire," and "uncertain," defining the fire confidence level. This is a weighted sum of the confidence level and partial uncertainty of a subset of fire-related data. For example, in an industrial setting, with a focus on risk warning, 70% of the uncertainty is allocated to fire-related data.

[0213] ;

[0214] Decision-making logic:

[0215] Level 1 Alert;

[0216] Level 2 Alert;

[0217] : Safe status.

[0218] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0219] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A fire confidence assessment method based on multimodal deep collaboration in complex industrial environments, characterized in that, include: Collect multimodal data of the area to be tested, wherein the multimodal data includes: thermal imaging image data and visible light image data; The corresponding temperature abrupt change rate and diffusion entropy are determined based on the thermal imaging image data. The temperature mutation rate and diffusion entropy are input into the constructed smoldering probability model to obtain the smoldering probability; The confidence level of the smoke is determined based on the visible light image data; The fire confidence level is determined based on the smoldering probability and the smoke confidence level. The fire confidence level is used to assess the fire situation in the area to be tested. Determining the fire confidence level based on the smoldering probability and the smoke confidence level includes: The evidence set is determined based on the smoldering probability and the smoke confidence level; The weights of each piece of evidence in the evidence set are dynamically allocated based on historical data to determine the weights corresponding to each piece of evidence. The historical data includes historical thermal imaging image data and historical visible light image data. The fire confidence level is determined based on the evidence with predetermined weights. The determination of the weights corresponding to each piece of evidence includes: Determine the accuracy rate corresponding to each piece of evidence; The normalization weights are determined based on the accuracy rate; The detection results corresponding to each piece of evidence are determined and it is determined whether there is a conflict. If there is a conflict, uniformly distributed uncertain evidence is introduced and the normalization weight is corrected. Determining the fire confidence level based on the weighted evidence includes: Determine the coefficient of conflict of evidence; The fire confidence level is determined using the Dempster rule and the pre-weighted evidence. The expression for the fire confidence level is as follows: ； Where A represents the subset allocated to the synthesized evidence body, and B and C are subsets of the first and second evidence bodies, respectively. For fire confidence level, The conflict coefficient, and These are the weights corresponding to the first and second pieces of evidence, respectively.

2. The fire confidence assessment method based on multimodal deep collaboration in a complex industrial environment according to claim 1, characterized in that, The calculation steps for the temperature mutation rate include: Set the time range for variation; Determine the equipment temperature data based on the temperature fraction data; The temperature abrupt change rate within the current time interval is determined based on the time variation interval and the device temperature data, wherein the expression for the temperature abrupt change rate is: ； in, Temperature mutation rate, For the time range of variation, This refers to the device temperature data at the current moment. This refers to the current time.

3. The fire confidence assessment method based on multimodal deep collaboration in a complex industrial environment according to claim 1, characterized in that, The steps for calculating the diffusion entropy include: Determine the corresponding image gradient based on the thermal imaging image data; Calculate the corresponding gradient magnitude and direction based on the image gradient; The thermal imaging gradient direction histogram is determined based on the direction interval index and the gradient magnitude and direction. The diffusion entropy is determined based on the gradient direction histogram. The expression for calculating the diffusion entropy is: ； in, For diffusion entropy, Let be the normalized gradient direction probability distribution, and u be the index of the gradient direction.

4. The fire confidence assessment method based on multimodal deep collaboration in a complex industrial environment according to claim 1, characterized in that, The network framework of the smoldering probability model is a three-layer structure consisting of an LSTM layer, a fully connected layer, and an activation function.

5. The fire confidence assessment method based on multimodal deep collaboration in a complex industrial environment according to claim 1, characterized in that, Determining the smoke confidence level based on the visible light image data includes: Develop an improved lightweight smoke detection model; The visible light image data is input into the improved lightweight smoke detection model to determine the smoke confidence level. The improved lightweight smoke detection model is constructed based on an improved YOLOv8n network with 96 channels, ReLU activation function, and a single-scale detection head.

Images