A fire-fighting hidden danger monitoring system based on artificial intelligence

Abstract

The application relates to the technical field of image data processing, and discloses a fire-fighting hidden danger monitoring system based on artificial intelligence; the system is composed of an image acquisition module, a data processing and communication module, a cloud analysis and early warning response module, and combines a coordinate-level adaptive step YOLOv10 model and a YOLOv8-seg segmentation network to realize accurate detection of open flames, inflammable material stacking, passage obstruction and equipment abnormalities, and to identify temperature abnormalities and overheating trends through thermal imaging analysis; a multi-modal analysis unit extracts features by using CNN and Transformer, introduces a ranking perception PPO to optimize candidate results, and improves real-time performance and accuracy; finally, the system triggers sound-light alarms, information pushing and equipment linkage according to hidden danger grades, and builds an intelligent closed-loop monitoring mechanism from perception to disposal.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of image data processing, and in particular to a fire hazard monitoring system based on artificial intelligence. BACKGROUND

[0002] With the acceleration of urbanization and the expansion of building scale, fire safety problems are increasingly prominent; the existing intelligent fire hazard monitoring system mainly relies on video monitoring, infrared thermal imaging or smoke sensors and other single or limited sensing means, which, although achieving automatic monitoring to a certain extent, still has obvious shortcomings in complex building environments; on the one hand, the single modal monitoring method is easily affected by factors such as light changes, environmental obstructions, climate conditions and equipment sensitivity, resulting in high false alarm rate and missed alarm rate, and it is difficult to timely and accurately identify key hazards such as fire sources, illegal stacking of flammable materials, blocked fire passages and abnormal equipment states; on the other hand, the existing system generally lacks the ability to fuse and comprehensively analyze multiple types of monitoring data, and can only identify single risk events, making it difficult to comprehensively judge the relevance and evolution trend between different hazards; at the same time, the early warning and response mechanism is relatively lagging, and most systems can only trigger a simple alarm signal after detecting a fire, lacking a hierarchical early warning based on hazard levels and a linkage control of emergency equipment, and it is difficult to meet the high requirements of real-time and reliability in complex places. SUMMARY

[0003] The application aims to solve the problems of existing fire hazard monitoring systems, such as being easily limited by single modal in complex building environments, insufficient recognition accuracy and lagging early warning response, and proposes a fire hazard monitoring system based on artificial intelligence; the system integrates image acquisition, multi-modal data processing, cloud intelligent analysis and hierarchical early warning linkage mechanism in architecture, the core innovation is to introduce a coordinate-level adaptive step YOLOv10 model and a YOLOv8-seg instance segmentation network to realize high-precision detection of multiple hazard targets such as open flames, flammable material stacking, fire passage obstruction and equipment abnormalities, and to improve the convergence stability in complex non-convex optimization through a coordinate-by-coordinate dynamic step factor and momentum accumulation mechanism; at the same time, the thermal imaging analysis unit combines abnormal temperature rise trend identification to realize early detection of early thermal hazards; at the multi-modal analysis level, a deep feature extraction network combining CNN and Transformer is constructed, and a ranking perception grouping PPO model is introduced, which pushes the correct hazard result to the top of the beam through beam search and ranking reward optimization, taking into account real-time and accuracy; the system finally outputs multi-level early warning signals through the cloud intelligent analysis and hierarchical response module and links emergency equipment, realizing the whole-process intelligent closed-loop monitoring from hazard perception, intelligent analysis to emergency disposal, and significantly improving the comprehensiveness, robustness and practicality of fire hazard monitoring.

[0004] The application provides a fire hazard monitoring system based on artificial intelligence, which comprises an image acquisition module, a data processing and communication module, a cloud analysis server and an early warning and response module.

[0005] The image acquisition module is arranged in a monitoring area and is used for collecting multi-modal environment data of the monitoring area in real time, wherein the multi-modal environment data comprises video image data, thermal imaging data, temperature and humidity data, gas sensing data and smoke concentration data.

[0006] The data processing and communication module is in communication connection with the image acquisition module, pre-processes and unifies the format of the multi-modal environment data, and transmits the multi-modal environment data to the cloud analysis server.

[0007] The cloud analysis server is in communication connection with the data processing and communication module, and an AI hazard identification engine is integrated in the cloud analysis server.

[0008] The image recognition unit processes the video image data based on a deep learning target detection algorithm to identify one or more first hazard feature characteristics such as open flames, illegal stacking of flammable materials, blocked fire passages and abnormal fire equipment states, and outputs a hazard identification result.

[0009] The thermal imaging analysis unit processes the thermal imaging data to identify one or more second hazard feature characteristics such as temperature abnormal points, overheated equipment and abnormal temperature rising trends, and obtains thermal anomaly feature data.

[0010] The multi-modal data analysis unit constructs multi-modal fusion data by combining the hazard identification result, the thermal anomaly feature data and the multi-modal environment data, processes the multi-modal fusion data through a hazard identification model, generates a comprehensive hazard identification result and a corresponding confidence, and the result comprises one or more hazard events such as open flames, illegal stacking of flammable materials, blocked fire passages, abnormal fire equipment states, temperature abnormal points, overheated equipment and abnormal temperature rising trends.

[0011] The early warning and response module is in communication connection with the cloud analysis server, receives the comprehensive hazard identification result, triggers different levels of early warning signals according to the hazard level and a preset threshold, and the early warning signals comprise sound and light alarms, sending of early warning information to a specified terminal and linkage control of emergency fire equipment.

[0012] Further, the image recognition unit outputs the hazard identification result through the deep learning target detection algorithm, and the process specifically comprises the following contents:

[0013] Step S1: processing the video image data by using a coordinate-level adaptive step YOLOv10 model to output candidate hidden danger target information, the candidate hidden danger target information including candidate target position and category information of the open fire, flammable material and fire-fighting equipment; the coordinate-level adaptive step YOLOv10 model including a feature extraction network, a feature fusion network and a YOLOv10 model;

[0014] Step S2: performing pixel-level segmentation on the video image data by using a YOLOv8-seg instance segmentation network to generate a fire-fighting passage segmentation mask and extract fire-fighting passage occlusion feature information;

[0015] Step S3: inputting the candidate hidden danger target information and the fire-fighting passage occlusion feature information into a time sequence association module, processing the information by using a ByteTrack algorithm to generate a unique identification target track;

[0016] Step S4: judging whether one or more of the following fire-fighting hidden dangers exists according to the unique identification target track and the fire-fighting passage segmentation mask: open fire, illegal stacking of flammable materials, occlusion of a fire-fighting passage and abnormal state of equipment, and outputting a hidden danger identification result.

[0017] Further, step S1 specifically includes the following steps:

[0018] Step S11: inputting the video image data into the feature extraction network to generate multi-scale feature data;

[0019] Step S12: processing the multi-scale feature data by using the feature fusion network to generate fusion feature data;

[0020] Step S13: processing the fusion feature data by using an adaptive convolution to generate an initial detection candidate box;

[0021] Step S14: in the training stage of the YOLOv10 model, using a multi-head prediction structure, processing the initial detection candidate box by using a double assignment strategy to generate a main assignment sample and an auxiliary assignment sample, and merging the two to form a positive sample set; in the training process, introducing a coordinate-level adaptive step optimization method to update the parameters of the YOLOv10 model based on the positive sample set supervision signal, and finally obtaining a trained YOLOv10 model;

[0022] Step S15: the trained YOLOv10 model using a single-head prediction structure to generate candidate hidden danger target information.

[0023] Further, the process of introducing the coordinate-level adaptive step optimization method to update the parameters of the YOLOv10 model specifically includes the following steps:

[0024] Step T1: under the supervision constraint of the positive sample set, a multi-dimensional supervision loss function is constructed, the multi-dimensional supervision loss function comprising a bounding box regression loss function, a target classification loss function and an instance segmentation loss function;

[0025] Step T2: under the constraint of the multi-dimensional supervision loss function, YOLOv10 model parameter gradient information is obtained through back propagation;

[0026] Step T3: second-order momentum estimation is introduced to square-weight average the parameter gradient information, suppress the transient fluctuation of the gradient, and generate an estimation result; a diagonal precondition matrix is constructed based on the estimation result to constrain the parameter dimension update scale in the YOLOv10 model; on this basis, a regularized first-order Taylor expansion approximation objective function is introduced to locally approximate the multi-dimensional supervision loss function, and a parameter update direction is constructed;

[0027] Step T4: based on the parameter update direction, a coordinate-by-coordinate adaptive step factor and a momentum accumulation mechanism are introduced to perform iterative update on the YOLOv10 model parameters.

[0028] Further, the multi-modal data analysis unit generates a comprehensive hidden danger identification result, specifically including the following contents:

[0029] Step B1: input the multi-modal fusion data into the CNN to generate first fusion features;

[0030] Step B2: input the first fusion features into the Transformer to generate second fusion features;

[0031] Step B3: the second fusion features are processed by using a beam search decoding method, the conditional probability distribution of the hidden danger class is gradually expanded and cumulative probability is sorted to generate a candidate hidden danger identification result;

[0032] Step B4: a grouping strategy optimization reinforcement learning method with ranking awareness is introduced to update the strategy of the candidate hidden danger identification result, and the correct candidate hidden danger identification result is pushed to the front of the beam search through the grouping baseline variance reduction and ranking reward, to obtain an optimized candidate hidden danger identification result;

[0033] Step B5: according to the optimized candidate hidden danger identification result, a comprehensive hidden danger identification result is output.

[0034] By using the above scheme, the application has the following beneficial effects:

[0035] Firstly, the present application realizes multi-modal comprehensive perception and high-precision identification of fire hazards, improves the monitoring coverage and accuracy of multiple types of hidden danger events in complex building environments; by introducing the coordinate level adaptive step YOLOv10 model and the YOLOv8-seg instance segmentation network, not only the high-precision detection of multiple key hidden dangers such as open fire, illegal stacking of flammable materials, fire passage obstruction and equipment abnormalities is realized, but also the recall rate and stability are maintained under the condition of small target and complex background, solving the problem of false alarm and missed alarm caused by the limitation of traditional monitoring methods based on single video or sensor to environmental light, obstruction and equipment sensitivity, thereby enhancing the practical value of the system in real scenes.

[0036] Secondly, the present application realizes early detection and dynamic analysis of early thermal hazards, improves the prediction and prevention and control ability of fire precursors; by setting up a thermal imaging analysis unit and combining an abnormal temperature rise trend identification mechanism, the system can capture risk features in time when equipment overheating and temperature anomalies occur, effectively solving the problem of the existing system that can only alarm after the fire has occurred; this improvement makes the system not only have a post-alarm function, but also can provide early warning in the early stage of fire hazards, enhancing the proactivity and initiative of prevention and control, and gaining valuable time for emergency response.

[0037] Finally, the present application realizes intelligent fusion and hierarchical response linkage of multi-modal features, improves the real-time performance and response efficiency of the whole monitoring; by jointly modeling CNN and Transformer, combining the ranking perception grouping PPO model for candidate hidden danger result optimization, the recognition result can quickly converge and be accurately sorted, thereby significantly reducing the computational burden and solving the problem of insufficient real-time performance of the existing system; at the same time, the system triggers hierarchical warning signals according to the hidden danger level, can output sound and light alarm, push terminal warning information, and link the automatic start of fire emergency equipment, enhancing the closed-loop ability of the fire hazard monitoring system from hidden danger identification to emergency disposal, ensuring that it has fast and reliable linkage prevention and control effect in high-risk environments. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 A module schematic diagram of a fire hazard monitoring system based on artificial intelligence is provided for the present application.

[0039] Figure 2 The YOLOv10 model training convergence curve graph proposed in embodiment four. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application; based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0041] In an embodiment, the present application provides a fire-fighting hidden danger monitoring system based on artificial intelligence, comprising an image acquisition module, a data processing and communication module, a cloud analysis server and an early warning and response module. Figure 1

[0042] The image acquisition module is arranged in a monitoring area and is configured to acquire multi-modal environment data of the monitoring area in real time, wherein the multi-modal environment data comprises video image data, thermal imaging data, temperature and humidity data, gas sensing data and smoke concentration data.

[0043] The data processing and communication module is in communication connection with the image acquisition module, configured to pre-process and unify the format of the multi-modal environment data, and transmit the multi-modal environment data to the cloud analysis server.

[0044] The cloud analysis server is in communication connection with the data processing and communication module, and has an AI hidden danger identification engine integrated therein; the AI hidden danger identification engine comprises an image recognition unit, a thermal imaging analysis unit and a multi-modal data analysis unit.

[0045] The image recognition unit is configured to process the video image data based on a deep learning target detection algorithm, to identify one or more first hidden danger features such as open fire, illegal stacking of flammable materials, blocked fire passageway and abnormal state of fire-fighting equipment, and output a hidden danger identification result.

[0046] The thermal imaging analysis unit is configured to process the thermal imaging data, to identify one or more second hidden danger features such as temperature abnormal point, overheated equipment and abnormal temperature rising trend, and obtain thermal abnormal feature data.

[0047] The multi-modal data analysis unit is configured to construct multi-modal fusion data in combination with the hidden danger identification result, the thermal abnormal feature data and the multi-modal environment data, process the multi-modal fusion data through a hidden danger identification model, and generate a comprehensive hidden danger identification result and a corresponding confidence level, wherein the result comprises one or more hidden danger events such as open fire, illegal stacking of flammable materials, blocked fire passageway, abnormal state of fire-fighting equipment, temperature abnormal point, overheated equipment and abnormal temperature rising trend; the hidden danger identification model comprises a CNN, a Transformer and a ranking-aware grouping PPO model; the hidden danger identification model is constructed in combination with multi-modal feature extraction of the CNN and the Transformer, candidate generation of beam search decoding and a ranking-aware grouping strategy optimization reinforcement learning method.​

[0048] The early warning and response module is in communication connection with the cloud analysis server, receives the comprehensive hidden danger identification result, and triggers different levels of early warning signals according to the hidden danger level and the preset threshold value; the early warning signal includes an audible and visual alarm, sending of early warning information to a specified terminal, and linkage control of starting of an emergency fire-fighting device.

[0049] In the second embodiment, based on the first embodiment, in the embodiment, the image recognition unit outputs the hidden danger identification result based on a deep learning target detection algorithm, and the process specifically includes the following contents:

[0050] Step S1: The coordinate-level adaptive step YOLOv10 model is used to process the video image data, and candidate hidden danger target information is output, the candidate hidden danger target information including candidate target position and category information of open fire, flammable material and fire-fighting equipment; the coordinate-level adaptive step YOLOv10 model is constructed in the following way: a YOLOv10 model is established, and a coordinate-level adaptive step optimization method is introduced to optimize the parameter update mechanism of the YOLOv10 model for construction; the coordinate-level adaptive step YOLOv10 model includes a feature extraction network, a feature fusion network and a YOLOv10 model;

[0051] Step S2: The YOLOv8-seg instance segmentation network is used to perform pixel-level segmentation on the video image data, to generate a fire-fighting passage segmentation mask, and to extract fire-fighting passage occlusion feature information;

[0052] Step S3: The candidate hidden danger target information and the fire-fighting passage occlusion feature information are input into a time sequence association module, the time sequence association module is processed by a ByteTrack algorithm, and a unique identification target track is generated;

[0053] Step S4: According to the unique identification target track and the fire-fighting passage segmentation mask, it is judged whether there is one or more of the following fire-fighting hidden dangers: open fire, illegal stacking of flammable materials, fire-fighting passage occlusion and equipment state abnormality, and a hidden danger identification result is output.

[0054] In the third embodiment, based on the first embodiment, in the embodiment, the image recognition unit outputs the hidden danger identification result based on a deep learning target detection algorithm, and the process specifically includes the following contents:

[0055] Step R1: The YOLOv10 model is used to process the video image data, and candidate hidden danger target information is output;

[0056] Step R2: The YOLOv8-seg instance segmentation network is used to perform pixel-level segmentation on the video image data, to generate a fire-fighting passage segmentation mask, and to extract fire-fighting passage occlusion feature information;

[0057] Step R3: input the candidate hidden danger target information and the fire passage blocking feature information into the time sequence association module, the time sequence association module processes through the ByteTrack algorithm, and generates a unique identification target track;

[0058] Step R4: according to the unique identification target track and the fire passage segmentation mask, it is judged whether one or more of the following fire hazards exist: open fire, illegal stacking of flammable materials, fire passage blocking, and equipment state abnormality, and the hidden danger identification result is output.

[0059] In this embodiment, step S1 specifically comprises the following steps: Figure 2

[0060] Step S11: input the video image data into the feature extraction network to generate multi-scale feature data; the feature extraction network is based on the Cross Stage Partial (CSP) structure, and a coordinate attention mechanism is introduced therein, which realizes feature enhancement of complex background and small targets by modeling the channel while preserving the spatial position information in the horizontal and vertical directions, thereby providing high-quality initial features for subsequent multi-scale feature fusion;

[0061] Step S12: process the multi-scale feature data through the feature fusion network to generate fusion feature data; the feature fusion network adopts the path aggregation structure (PANet), which establishes efficient information transmission between cross-layer features, ensuring that the fine-grained small target features extracted by the Backbone can be fused with global semantic features, thereby forming a more complete and multi-scale consistent feature representation;

[0062] Step S13: process the fusion feature data using adaptive convolution to generate initial detection candidate boxes;

[0063] Step S14: in the training phase of the YOLOv10 model, a multi-head prediction structure is adopted, the initial detection candidate boxes are processed through a double assignment strategy to generate main assignment samples and auxiliary assignment samples, and the two are combined to form a positive sample set, thereby improving the stability of hidden danger target matching and the recall rate of small target detection; in the training process, the positive sample set supervision signal is introduced into the coordinate level adaptive step optimization method to update the parameters of the YOLOv10 model, ensuring that the model can more accurately and comprehensively learn the features of fire hazards, thereby outputting stable and reliable detection results in the inference stage, and finally obtaining the trained YOLOv10 model;

[0064] In the training phase, a YOLOv10 model training convergence curve is generated, Figure 2 ​In the figure, the left vertical axis is loss, showing the gradual decline of training loss and validation loss with the training round; the right vertical axis is score, showing the gradual increase of mAP@50 and recall with the training; the horizontal axis: training round (Epoch);

[0065] Step S15: The trained YOLOv10 model adopts a single-head prediction structure to generate candidate hidden danger target information.

[0066] Embodiment five, based on embodiment four, in this embodiment, the coordinate level adaptive step optimization method is introduced to update the parameters of the YOLOv10 model, which specifically includes the following steps:

[0067] Step T1: Under the supervision constraint of the positive sample set, a multi-dimensional supervision loss function is constructed, which includes a bounding box regression loss function, a target classification loss function and an instance segmentation loss function;

[0068] Step T2: Under the constraint of the multi-dimensional supervision loss function, the YOLOv10 model parameter gradient information is obtained by back propagation;

[0069] Step T3: Introduce the second-order momentum estimation to square-weight average the parameter gradient information, suppress the transient fluctuation of the gradient, and generate an estimation result; based on the estimation result, construct a diagonal precondition matrix to constrain the parameter dimension update scale in the YOLOv10 model; on this basis, introduce the regularization first-order Taylor expansion approximation target function to locally approximate the multi-dimensional supervision loss function and construct the parameter update direction;

[0070] The formula used for introducing the regularization first-order Taylor expansion approximation target function is as follows:

[0071] ;

[0072] Wherein, represents the iteration index, represents the variable introduced parameter update direction, represents the th iteration, the optimal direction obtained by solving the first-order Taylor expansion regularization approximation target of the supervision loss function; represents the th iteration of the improved version of the YOLOv10 model parameter vector; represents the th iteration, the local approximation value of the multi-dimensional supervision loss function based on the current model parameter vector ; represents the th iteration, the local approximation value of the multi-dimensional supervision loss function to the model parameter vector The transpose of the gradient vector; This represents a global adjustment hyperparameter used to control the strength of the regularization term, preventing excessively large update directions. Its value range is [value range missing]. [ 10 -4 , 10 2 ] ; This represents a diagonal preconditioning matrix, which serves to suppress gradients that are too large or too small in certain dimensions, thus avoiding oscillations or slow convergence during training. The innovation of this invention lies in using it as a coordinate-level dynamic adjuster to assign differentiated learning rates to different parameters.

[0073] The method for obtaining this is as follows: during model training, pre-set the global tuning hyperparameters. The range of values ​​for the loss function is determined, and the loss function is adaptively updated in each iteration based on its current trend and gradient stability. Specifically, a training stability evaluation index is constructed by comparing the magnitude of changes in the multidimensional supervised loss and the degree of gradient fluctuation between adjacent iterations. When the stability index indicates that there is a risk of oscillation or instability in the training process, the training stability is improved. The value of is chosen to enhance regularization constraints and suppress parameter update amplitude; when the stability index indicates that the training process converges smoothly and the performance index continues to improve, the value of is reduced. The value of is chosen to relax the regularization constraint and accelerate the model convergence speed, thereby achieving dynamic and controllable adjustment of the parameter update intensity.

[0074] Step T4: Based on the parameter update direction, an adaptive step size factor and momentum accumulation mechanism are introduced to iteratively update the YOLOv10 model parameters. The adaptive step size factor applies differentiated update magnitudes to parameters in different dimensions. The momentum accumulation mechanism is superimposed on the update rule, combining the update amounts of historical iteration parameters with the current gradient direction in a weighted manner, thereby balancing the stability and convergence speed of the training process. The update rule is as follows:

[0075] ;

[0076] in, Indicates the first The parameter vector of the improved YOLOv10 model in the next iteration, that is, the new parameter values ​​obtained after this optimization; Indicates the first The parameter vector of the improved YOLOv10 model in the next iteration; represents the momentum coefficient, with a value range of (0, 1); Indicates the first In the next iteration, the multidimensional supervised loss function For the current model parameter vector The gradient vector; This represents the adaptive step size factor for each coordinate, with a value range of [value range missing]. [ 10 -6 , 10 -2 ] This indicates that during the update, the direction is... The magnitude of the movement directly affects training speed and stability;

[0077] The momentum coefficient is obtained by initializing it before training begins based on the model size and the temporal stability of the object detection task, and keeping it fixed during training. It is used to accumulate the historical parameter update direction with weights to balance the stability of parameter updates and the convergence speed.

[0078] How to obtain the adaptive step size factor for each coordinate: during the model training... During each iteration, based on the gradient information and second-order momentum estimation results corresponding to each dimension of the current model parameters, a corresponding step size adjustment factor is adaptively generated for each dimension of the parameter vector. Under the constraint of the diagonal precondition matrix, the gradient magnitude and trend of different parameter dimensions are adaptively evaluated. When the gradient value of a certain parameter dimension is large or fluctuates violently, the corresponding step size factor is automatically reduced to suppress update oscillations. When the gradient value of a certain parameter dimension is small or changes slowly, the corresponding step size factor is automatically increased to accelerate the convergence speed. The generated step size factor is limited to a preset value range, thereby realizing the adjustment of the parameter update amplitude by coordinate and dynamically controllable.

[0079] Specifically, the coordinate-wise adaptive step size factor differentiates the gradients of different parameter dimensions by applying weights under the constraint of the diagonal preconditioning matrix, enabling each parameter dimension to have independent step size control during iterative updates.

[0080] When the gradient value of a certain parameter dimension is large or the local curvature is steep, the corresponding step size factor is automatically reduced to avoid training oscillations caused by excessive parameter correction.

[0081] When the gradient value of a certain parameter dimension is small or the local curvature is gentle, the corresponding step size factor is automatically increased to accelerate the convergence speed of that dimension.

[0082] Through the above-mentioned coordinate-by-coordinate dynamic adjustment mechanism, the problem of insufficient adaptability of the unified global step size to different dimensions in complex non-convex optimization problems is effectively solved, thereby improving the training stability and convergence accuracy of the improved YOLOv10 model in multi-objective and complex scenarios.

[0083] Therefore, this coordinate-wise adaptive step size factor not only enables fine-grained control of parameter updates, but also further improves the robustness and global convergence performance of the optimization process when combined with the momentum accumulation mechanism.

[0084] Example 6, based on Example 5, describes the process by which the multimodal data analysis unit generates a comprehensive hazard identification result, specifically including the following:

[0085] Step B1: Input the multimodal fusion data into the CNN, and use the local feature extraction capability of the CNN to model the spatial correlation between different modalities to generate the first fusion feature;

[0086] Step B2: Input the first fused feature into the Transformer and use the multi-head self-attention mechanism to model the long-range dependencies across modalities and time to generate the second fused feature; this feature can capture the potential patterns of hidden events at the temporal evolution and modal interaction levels.

[0087] Step B3: The second fusion feature is processed using the beam search decoding method. By progressively expanding and cumulatively ranking the conditional probability distribution of the hazard categories, candidate hazard identification results are generated.

[0088] Step B4: Introduce a ranking-aware grouping strategy to optimize the reinforcement learning method. Update the strategy for the candidate hazard identification results. By reducing variance through the grouping baseline and ranking reward, push the correct candidate hazard identification results to the front of the bundle search, so that the correct hazard results converge to the top of the bundle more quickly, significantly reduce the computational overhead and improve the real-time early warning capability, and obtain the optimized candidate hazard identification results.

[0089] The ranking-aware grouping strategy optimization reinforcement learning method updates the policy based on the candidate hazard identification results as follows:

[0090] Step 1 (Grouped Baseline Variance Reduction): Group the candidate hazard results as one output group and calculate the reward value for each candidate result. The reward function is defined as:

[0091] ;

[0092] in, Indicates the first The advantage value of each candidate hazard identification result Indicates the first The reward value for each candidate hazard identification result, which ranges from (0, 1], is calculated by the sorting position of the candidate result in the beam search and the beam width. This represents the bundle width, and its value is a positive integer ranging from 2 to 20. This represents the set of rewards for all candidate results within the same group; represents the average value of the set of candidate result rewards, i.e., the baseline;

[0093] Step 2 (ranking reward): The reward value of the correct result further introduces a beam search ranking position (rank) weight, using an exponential decay function:

[0094] ;

[0095] wherein, represents the ranking position of the candidate hazard identification result in the beam search, and the value range is 1- ;

[0096] Step 3 (policy update objective function): Construct an objective function containing the ranking reward:

[0097] ;

[0098] wherein, represents the parameters of the current policy model, represents the optimization objective function (target expected value), represents the th candidate hazard identification result (generated by beam search), represents the input of the model, i.e., the second fusion feature, represents the probability of the current policy model generating result under input , represents the probability of the old policy (last round of training parameters) generating result ; represents the clipping function, represents the clipping parameter, and the value range is (0, 1); represents the weight coefficient, which controls the influence of the KL divergence regular term, and the value is [ 10 -4 , 10 -1 ] ; represents the KL divergence, which is used to measure the difference between the current policy distribution and the reference policy ;

[0099] Through the ranking-aware reward mechanism, the correct result is pushed to the top position of the beam, allowing the beam width to be reduced while maintaining accuracy, thereby reducing the computational load in the reasoning phase and improving system real-time performance;

[0100] Step B5: Comprehensive judgment is performed on the candidate outputs of different hazard categories according to the optimized candidate hazard identification result, a high-confidence result located at the top of the beam after optimization of the ordered perception reward mechanism is taken as the final output, and thus an output comprehensive hazard identification result is obtained.

[0101] In this embodiment, the multi-modal data analysis unit generates the process of the comprehensive hazard identification result, and specifically includes the following contents:

[0102] Step Q1: The multi-modal fusion data is input into the CNN, the local feature extraction capability of the CNN is used to model the spatial correlation between different modalities, and the first fusion feature is generated;

[0103] Step Q2: The first fusion feature is input into the Transformer, and the multi-head self-attention mechanism is used to model the long-range dependency relationship across modalities and time, and the second fusion feature is generated; this feature can capture the potential patterns of hazard events in the time evolution and modal interaction level;

[0104] Step Q3: The beam search decoding method is used to process the second fusion feature, the conditional probability distribution of the hazard category is gradually expanded and cumulative probability is sorted, and the candidate hazard identification result is generated;

[0105] Step Q5: According to the candidate hazard identification result, the comprehensive hazard identification result is output.

[0106] The above describes the present application and its embodiments, which are not restrictive, and the drawings shown are only one of the embodiments of the present application, and the actual structure is not limited thereto; in general, if a person skilled in the art is inspired thereby, without departing from the spirit of the present application, without creative design, similar structure and embodiments of the technical solution can be obtained, which should belong to the protection scope of the present application.

Claims

1. A fire hazard monitoring system based on artificial intelligence, characterized in that: The system includes an image acquisition module and an AI-powered hazard identification engine; The image acquisition module collects multimodal environmental data, including video image data, thermal imaging data, temperature and humidity data, gas sensing data, and smoke concentration data. The AI-powered hazard identification engine includes: The image recognition unit processes video image data using a deep learning-based target detection algorithm and outputs hazard identification results. The thermal imaging analysis unit processes the thermal imaging data to obtain thermal anomaly characteristic data; The multimodal data analysis unit combines hazard identification results, thermal anomaly characteristic data, and multimodal environmental data to construct multimodal fusion data. The multimodal fusion data is then processed by a hazard identification model to generate a comprehensive hazard identification result. The hazard identification model includes CNN, Transformer, and sorting-aware grouped PPO model. The image recognition unit, in its process of outputting hazard identification results, specifically includes the following: Step S1: Use the coordinate-level adaptive step size YOLOv10 model to process the video image data and output candidate hidden danger target information; Step S2: Use the YOLOv8-seg instance segmentation network to perform pixel-level segmentation on the video image data, generate a fire lane segmentation mask, and extract the fire lane occlusion feature information; Step S3: Input the candidate hazard target information and fire lane obstruction feature information into the time series correlation module. The time series correlation module processes the information using the ByteTrack algorithm to generate a unique target trajectory. Step S4: Divide the mask according to the unique identifier target trajectory and fire lane, and output the hazard identification results; The coordinate-level adaptive step size YOLOv10 model includes a feature extraction network, a feature fusion network, and the YOLOv10 model. The YOLOv10 model parameters are updated by introducing a coordinate-level adaptive step-size optimization method, which includes the following steps: Step T1: Construct a multidimensional supervised loss function based on the set of positive samples; Step T2: Under the constraint of the multidimensional supervised loss function, obtain the parameter gradient information of the YOLOv10 model through backpropagation; Step T3: Introduce second-order momentum estimation to perform squared weighted averaging of parameter gradient information to generate estimation results; construct a diagonal preconditioning matrix based on the estimation results to constrain the parameter dimension update scale in the YOLOv10 model; on this basis, introduce a regularized first-order Taylor expansion approximate objective function to perform local approximate modeling of the multidimensional supervised loss function and construct the parameter update direction. Step T4: Based on the parameter update direction, introduce a coordinate-wise adaptive step size factor and momentum accumulation mechanism to perform iterative updates on the parameters of the YOLOv10 model.

2. The fire hazard monitoring system based on artificial intelligence according to claim 1, characterized in that: Step S1 specifically includes the following steps: Step S11: Input the video image data into the feature extraction network to generate multi-scale feature data; Step S12: Process multi-scale feature data through a feature fusion network to generate fused feature data; Step S13: Use adaptive convolution to process the fused feature data to generate initial detection candidate boxes; Step S14: During the training phase of the YOLOv10 model, a multi-head prediction structure is adopted, and the initial detection candidate boxes are processed through a dual allocation strategy to form a positive sample set. During the training process, the positive sample set is used as a supervision signal, and a coordinate-level adaptive step size optimization method is introduced to update the parameters of the YOLOv10 model, thus obtaining the trained YOLOv10 model. Step S15: After training, the YOLOv10 model adopts a single-head prediction structure to generate candidate potential hazard target information.

3. The fire hazard monitoring system based on artificial intelligence according to claim 1, characterized in that: The multimodal data analysis unit generates comprehensive hazard identification results, specifically including the following: Step B1: Input the multimodal fusion data into the CNN to generate the first fused feature; Step B2: Input the first fused feature into the Transformer to generate the second fused feature; Step B3: The second fusion feature is processed using the beam search decoding method to generate candidate hazard identification results; Step B4: Introduce the ranking-aware grouped PPO model to update the candidate hazard identification results. By reducing variance through grouping baseline and ranking reward, the correct candidate hazard identification results are pushed to the forefront of the bundle search, thereby optimizing the candidate hazard identification results. Step B5: Based on the optimized candidate hazard identification results, output the comprehensive hazard identification results.

Images