A multispectral synergistic detection and target identification device for fire ground environments
Through the multispectral collaborative detection and target recognition device, multi-dimensional collaborative detection and accurate identification in fire environment are realized, solving the problems of adaptability and data reliability of traditional equipment in dense smoke and high temperature environment, and providing efficient and reliable rescue support.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIV OF SCI & TECH OF CHINA
- Filing Date
- 2026-01-13
- Publication Date
- 2026-05-29
AI Technical Summary
Existing fire detection equipment struggles to achieve multi-dimensional collaborative detection and accurate identification in environments with dense smoke and high temperatures, resulting in poor adaptability of rescue equipment, low data reliability, and an inability to provide comprehensive and reliable decision support.
Employing a multispectral collaborative detection and target recognition device, it integrates a multispectral data acquisition module, a cross-modal data fusion module, a target intelligent recognition module, and a wireless data transmission module. Through the collaborative acquisition of visible light, near-infrared, and thermal infrared spectra, combined with cross-modal data fusion technology, it achieves the complementarity of temperature characteristics and visual details. Furthermore, through an environmental parameter linkage adaptation module, it calibrates the sensor accuracy in real time, thus constructing a full-link collaborative system.
It has achieved accurate detection and identification in complex fire environments, improved the comprehensiveness of detection and environmental adaptability, ensured real-time data transmission and accuracy, shortened rescue response time, reduced rescue risks and improved rescue efficiency.
Smart Images
Figure CN122115823A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of fire scene collaborative detection equipment technology, and in particular to a multispectral collaborative detection and target identification device for fire scene environment. Background Technology
[0002] With the increasing prevalence of high-rise buildings and new energy facilities, the complexity of fire accidents and the difficulty of rescue have significantly increased. Traditional "human wave tactics" are no longer sufficient to meet the needs of efficient and safe rescue. Firefighting and rescue are transforming into a "smart era" empowered by technology. The extreme environments in a fire, such as dense smoke, high temperatures, and obstructed visibility, place stringent demands on the penetration, accuracy, and stability of detection equipment. Accurately identifying the fire source, locating trapped personnel, and predicting dangerous areas have become core prerequisites for reducing rescue risks and improving rescue efficiency.
[0003] Existing fire detection equipment has significant limitations: single visible light cameras are easily obscured by dense smoke and cannot provide an effective field of view; traditional infrared thermal imagers, although able to penetrate some smoke, are susceptible to reflection interference, have limited field of view, and struggle to capture both temperature information and environmental details; at the same time, high-temperature environments can cause sensor accuracy drift, equipment adaptability is insufficient under different smoke concentrations, and various data lack effective fusion and real-time transmission, failing to provide comprehensive and reliable decision support for rescue command. Therefore, there is an urgent need for a technical solution that can adapt to complex fire environments and achieve multi-dimensional collaborative detection and accurate identification. Summary of the Invention
[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a multispectral collaborative detection and target identification device for fire environments.
[0005] To achieve the above objectives, the present invention adopts the following technical solution: a multispectral collaborative detection and target identification device for fire scene environments, comprising a housing and a sealing plate installed at the bottom of the housing, a quick-release mechanism installed at the connection between the housing and the sealing plate, and an intelligent processing mechanism for detecting and identifying the environment installed inside the housing. The intelligent processing mechanism integrates a multispectral collaborative sensing and dynamic adaptation system for realizing multispectral collaborative detection and dynamic adaptation. The multispectral collaborative sensing and dynamic adaptation system includes a multispectral data acquisition module, a cross-modal data fusion module, a target intelligent identification module, an environmental parameter linkage adaptation module, and a wireless data transmission module. Each module is used to realize the functions of multispectral image and environmental parameter acquisition, accurate cross-modal data fusion, fire source / trapped personnel / dangerous area identification, dynamic adjustment of detection parameters, and real-time data transmission.
[0006] Preferably, the quick-release mechanism includes a sealing strip installed on the inner side of the sealing plate and multiple damping rods fixed to the four corners of the inner side of the sealing plate. The bottom of the housing is provided with sealing grooves and positioning holes that match the sealing strip and damping rods.
[0007] Preferably, the intelligent processing mechanism includes a vision module installed at the front end inside the housing. A power module, a control module, a sensor module, and a heat dissipation module are installed on the top surface of the inner wall of the housing. The heat dissipation module is located at the rear end of the power module. The sensor module and the control module are located on one side of the power module and the heat dissipation module, respectively. The vision module, the sensor module, and the heat dissipation module are all connected to the control module through wires, and the control module is connected to the power module through wires.
[0008] Preferably, a visible light camera, a near-infrared camera, and a thermal infrared camera are equidistantly mounted on the vision module, with all three cameras located at the front end of the housing.
[0009] Preferably, the heat dissipation module has a corresponding circular mounting slot on the housing. A mounting box is installed on the inner wall of the mounting slot via a cross mounting bracket. A servo motor is installed inside the mounting box. A cooling fan is installed on the output end of the servo motor via a coupling. Multiple heat dissipation fins are installed at equal intervals on the inner wall of the mounting slot, and the heat dissipation fins are located above the cooling fan.
[0010] Preferably, the multispectral data acquisition module is connected to the vision module and the sensor module to synchronously acquire visible light image data, near-infrared image data, thermal infrared image data, as well as toxic and harmful gas concentration data and ambient temperature data in the fire scene. The acquired multi-source raw data is then processed in a standardized format and transmitted to the cross-modal data fusion module.
[0011] Preferably, the cross-modal data fusion module receives standardized data transmitted from the multispectral data acquisition module, aligns the spatial positions of image data of different types and sources through a preset multimodal data registration algorithm, and then uses a weighted fusion algorithm to fuse the temperature feature information in the thermal infrared image with the visual detail feature information in the visible light image and the near-infrared image to generate integrated fused data containing temperature information and visual details, which is then transmitted to the target intelligent recognition module.
[0012] As a preferred option, the target intelligent recognition module has a built-in trained fire scene target recognition model. It extracts features from the integrated fusion data transmitted by the cross-modal data fusion module, and compares the target features with the fire source feature library, human body feature library, and dangerous area feature library in the preset model to achieve accurate identification and positioning of the fire source location, the outline of trapped personnel, and the range of dangerous areas, and generates target recognition result data.
[0013] As a preferred embodiment, the environmental parameter linkage adaptation module includes a multi-source sensor accuracy calibration unit and a detection channel dynamic switching unit; The multi-source sensor accuracy calibration unit receives ambient temperature data and gas concentration data transmitted from the multispectral data acquisition module in real time, performs linkage calculations with the preset accuracy calibration model, dynamically generates parameter correction instructions based on the calculation results, and transmits them to the vision module and sensor module to correct their detection parameters in real time and compensate for the detection accuracy drift caused by the high temperature environment. The detection channel dynamic switching unit automatically adjusts the working parameters of the visible light camera, near-infrared camera, and thermal infrared camera, as well as the data fusion weight in the cross-modal data fusion module, based on the smoke concentration correlation data monitored by the multispectral data acquisition module. In high-concentration smoke environments, it enhances the working gain and data fusion weight of the thermal infrared camera and near-infrared camera, and optimizes the exposure parameters and detail acquisition gain of the visible light camera in low-concentration smoke environments, thereby improving the visual detail presentation effect.
[0014] Preferably, the wireless data transmission module is signal-connected to the control module, receives target recognition result data transmitted by the target intelligent recognition module and raw monitoring data transmitted by the multispectral data acquisition module, and transmits the data in real time to the fire command center terminal or firefighter handheld terminal through a wireless communication protocol. At the same time, it receives control commands issued by the terminal and feeds them back to the control module, realizing two-way data interaction.
[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. This solution utilizes the coordinated acquisition of visible light, near-infrared, and thermal infrared spectra, combined with cross-modal data fusion technology, to achieve complementary acquisition of temperature characteristics and visual details. The multispectral data acquisition module standardizes various types of raw data, while the fusion module generates comprehensive integrated data through spatial alignment and weighted fusion algorithms, effectively addressing the pain points of limited field of view and incomplete information from single devices. Whether in dimly lit environments obscured by dense smoke or in complex, high-temperature fire scenes, the equipment can stably capture effective information, significantly improving the comprehensiveness of detection and environmental adaptability.
[0016] 2. This solution achieves dynamic calibration through an environmental parameter linkage adaptation module. A multi-source sensor accuracy calibration unit, combining fire scene temperature and gas concentration data, generates correction coefficients using a preset model to compensate for equipment errors in real time. The target intelligent recognition module, based on a deep learning model, extracts multi-dimensional features and matches them with a standard feature library to accurately distinguish between fire sources, trapped personnel, and dangerous areas. This design ensures accuracy throughout the entire process from data acquisition to recognition output, avoiding misjudgments and omissions caused by environmental interference or equipment drift, providing a reliable basis for rescue decisions.
[0017] 3. This solution constructs a full-link collaborative system consisting of data acquisition, fusion, identification, and transmission. The wireless data transmission module supports multiple communication protocols, enabling real-time uploading of identification results and raw data, while simultaneously receiving terminal control commands to dynamically adjust parameters. The device's quick-release design facilitates rapid deployment and maintenance, and the heat dissipation module ensures continuous operation in high-temperature environments, forming a closed loop of real-time perception, precise decision-making, and dynamic adjustment. This design significantly shortens rescue response time, allowing command centers and frontline personnel to promptly grasp the dynamics of the fire scene, reducing rescue risks and improving rescue efficiency.
[0018] In summary, this solution comprehensively addresses the core pain points of traditional fire detection equipment—poor environmental adaptability, low data reliability, and weak rescue coordination—through an integrated design that combines multispectral collaborative acquisition, dynamic accuracy calibration, and end-to-end real-time transmission. It not only achieves accurate detection and identification in complex fire environments but also constructs an efficient and reliable fire monitoring support system through real-time data interaction and flexible equipment deployment, providing comprehensive technical support for fire rescue and significantly improving the safety and efficiency of rescue operations. Attached Figure Description
[0019] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the overall appearance of the device proposed in this invention; Figure 2 This is a schematic diagram of the internal structure of the device proposed in this invention; Figure 3 This is a schematic diagram of the sealing plate structure proposed in this invention; Figure 4 This is a schematic diagram of the heat dissipation module structure proposed in this invention; Figure 5 The present invention proposes Figure 3 Enlarged schematic diagram of the structure at part A in the middle; Figure 6 This is a block diagram showing the overall module connection relationship of the device proposed in this invention; Figure 7 This is a block diagram illustrating the principle of the multispectral collaborative sensing and dynamic adaptation system proposed in this invention. Figure 8 This is a block diagram of the core control logic proposed in this invention.
[0020] The components in the diagram are numbered as follows: 1. Housing; 2. Sealing plate; 3. Vision module; 4. Power module; 5. Control module; 6. Sensor module; 7. Heat dissipation module; 8. Visible light camera; 9. Near-infrared camera; 10. Thermal infrared camera; 11. Servo motor; 12. Cooling fan; 13. Heat dissipation fins; 14. Sealing strip; 15. Damping rod. Detailed Implementation
[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0022] See Figures 1 to 8 This invention discloses a multispectral collaborative detection and target identification device for fire environments, comprising a housing 1 and a sealing plate 2 installed at the bottom of the housing 1. A quick-release mechanism is installed at the connection between the housing 1 and the sealing plate 2. Due to the complex and rapidly changing nature of fire environments, the plug-in quick-release mechanism provides installation stability while allowing for quick disassembly and maintenance, saving time. Both the housing 1 and the sealing plate 2 are constructed from galvanized steel plates filled with fire-resistant and heat-insulating material and silicone sealing rings to form the device shell, achieving fire and water resistance in fire environments and ensuring normal function. The device can be quickly locked to the mounting base using a mechanical structure or Velcro, enabling rapid deployment, replacement, and maintenance. An intelligent processing mechanism for detecting and identifying the environment is installed inside the housing 1. This intelligent processing mechanism integrates a multispectral collaborative sensing and dynamic adaptation system for achieving multispectral collaborative detection and dynamic adaptation. The intelligent processing mechanism and the multispectral collaborative sensing and dynamic adaptation system facilitate rapid analysis of the fire environment, accurate target identification, and timely instructions from back-end personnel.
[0023] Specifically, the quick-release mechanism includes a sealing strip 14 installed on the inner side of the sealing plate 2 and multiple damping rods 15 fixed to the four corners of the inner side of the sealing plate 2. The sealing strip 14 and the damping rods 15 can be disassembled and assembled from the housing 1 simply by aligning them with the sealing groove and the positioning hole. The bottom of the housing 1 is provided with a sealing groove and a positioning hole that cooperate with the sealing strip 14 and the damping rods 15.
[0024] The intelligent processing mechanism includes a vision module 3 installed at the front end inside the housing 1. A power module 4, a control module 5, a sensor module 6, and a heat dissipation module 7 are installed on the top surface of the inner wall of the housing 1. The power module 4 provides a stable and uninterrupted power supply to the entire module, ensuring its independence. The control module 5 serves as the core control unit, coordinating the orderly operation of various modules and other components of the multispectral collaborative sensing and dynamic adaptation system. The sensor module 6 is equipped with sensors for carbon monoxide (CO), carbon dioxide (CO), and other pollutants. ), hydrogen sulfide ( The system includes sensors for gases such as toxic and harmful gases, ambient temperature, and smoke concentration, enabling real-time monitoring of parameters related to these parameters in the fire scene. This further enhances safety and provides richer information for decision-making regarding fire attack. The heat dissipation module 7 facilitates heat dissipation for the control module 5 and other heat-generating components. The heat dissipation module 7 is located at the rear of the power module 4, while the sensor module 6 and control module 5 are located on opposite sides of the power module 4 and heat dissipation module 7, respectively. The vision module 3, sensor module 6, and heat dissipation module 7 are all connected to the control module 5 via wires, and the control module 5 is also connected to the power module 4 via wires.
[0025] Specifically, the vision module 3 is equipped with a visible light camera 8, a near-infrared camera 9, and a thermal infrared camera 10, which are installed at equal intervals. The visible light camera 8 can be used to acquire conventional visual information about the fire scene environment, serving as a basic observation channel. In areas with no smoke or thin smoke, it provides realistic environmental images that conform to human visual habits for environmental identification and information archiving. The near-infrared camera 9 can utilize its ability to penetrate some smoke and flames to observe the outlines of people, object details, and structural states inside or behind flames and smoke, providing auxiliary imaging in fire and smoke environments. The thermal infrared camera 10 can be used to sense the thermal radiation of objects in the fire scene, obtain temperature distribution maps, and effectively detect and locate the body temperature thermal signals of trapped personnel in a wide area of dense smoke far from the core of the flames, identifying fire sources, hot spots, and personnel. The visible light camera 8, near-infrared camera 9, and thermal infrared camera 10 are all located at the front end of the housing 1.
[0026] Specifically, the heat dissipation module 7 and the housing 1 are respectively provided with circular mounting slots. The inner wall of the mounting slot is equipped with a mounting box through a cross mounting bracket. The mounting box prevents dust from entering the servo motor 11 and causing damage. The servo motor 11 is installed inside the mounting box, which facilitates the driving of the cooling fan 12. The output end of the servo motor 11 is equipped with the cooling fan 12 through a coupling. The cooling fan 12 facilitates the extraction of hot air from the inside of the device, thereby achieving the cooling function of components such as the control module 5. In addition, multiple heat dissipation fins 13 are installed at equal intervals on the inner wall of the mounting slot. The heat dissipation fins 13 can assist the cooling fan 12 to enhance the heat dissipation effect. The heat dissipation fins 13 are located at the upper end of the cooling fan 12.
[0027] Specifically, the multispectral data acquisition module establishes a bidirectional signal connection with the vision module 3 and the sensor module 6 through a wired or wireless communication link. This connection is used to synchronously acquire visible light image data output by the visible light camera 8, near-infrared image data output by the near-infrared camera 9, thermal infrared image data output by the thermal infrared camera 10, as well as data on the concentration of toxic and harmful gases in the fire scene, ambient temperature data, and smoke concentration correlation data output by the sensor module 6. The acquired multi-source raw data is then processed to standardize the format and transmitted to the cross-modal data fusion module after unifying the data transmission format. The acquisition triggering method of the multispectral data acquisition module is the synchronous clock signal output by the control module 5, with a synchronization accuracy of ≤10ms, to ensure the consistency of timestamps for visible light image data, near-infrared image data, thermal infrared image data and environmental parameter data. It should be noted that the resolution of the visible light image data acquisition is adapted to the hardware parameters of the visible light camera 8 in the vision module 3, and the acquisition format is RAW format, which retains the original pixel information; the near-infrared image data acquisition covers the near-infrared spectral band, and the acquisition format is also RAW format, recording the light intensity distribution information; the thermal infrared image data acquisition focuses on the temperature quantization value, and the acquisition format is 16-bit grayscale value format, with the grayscale value having a linear correspondence with the actual temperature; The data acquisition of toxic and harmful gas concentrations at the fire scene is based on the gas sensors configured in sensor module 6, such as carbon monoxide, carbon dioxide, and hydrogen sulfide. The data type of the acquired data is the concentration value after voltage signal conversion. The ambient temperature data is acquired from the temperature sensor in sensor module 6. The data type of the acquired data is the Celsius quantized value. Format standardization involves two steps: data normalization and format encapsulation. The first step, data normalization, uses a linear normalization algorithm to map the original data of different magnitudes to the [0,1] interval, as shown in the formula: Where x is the original collected data, These represent the preset maximum and minimum effective thresholds for this type of data, respectively. These are the normalized standard data; The second step involves encapsulating the data in JSON format. The encapsulated fields include data type identifier, timestamp, normalized data array, and data verification code. The encapsulated data is then transmitted to the cross-modal data fusion module via the data bus, with the transmission baud rate adapted to the communication baud rate of the control module 5.
[0028] Specifically, the cross-modal data fusion module includes a data receiving unit, a multimodal registration unit, a weighted fusion unit, and a data output unit, with each unit operating in series. The data receiving unit receives standardized JSON format data transmitted from the multispectral data acquisition module and extracts the visible light image matrix by parsing the fields. Near-infrared image matrix Thermal infrared image matrix The dimension of the image matrix is M×N (M is the number of pixels in the image height, and N is the number of pixels in the image width). The modal registration unit uses SIFT feature point matching combined with homography matrix transformation to achieve spatial alignment. The specific steps are as follows: To each Perform SIFT feature point detection to extract feature point sets from each image. Each feature point contains coordinate information (x, y) and a feature descriptor; by Using the baseline image, find a matching image using the Euclidean distance criterion. , Zhongyu The corresponding matching feature point pairs, the Euclidean distance formula is: ,when ( When the distance threshold is less than the preset distance threshold, it is determined to be a valid matching point pair; Solving the homography matrix based on effective matching point pairs ( Compared to (transformation matrix) ( Compared to The transformation matrix), and the homography matrix is a 3×3 matrix: ; Solving for matrix elements using the least squares method ensures that after transformation... and The spatial positions are perfectly aligned; The weighted fusion unit performs pixel-level fusion based on the environmental validity weights of each modality image. The fusion formula is as follows: ,in, To fuse the feature set of the image at the (m,n) pixel position, it includes 3 dimensions (visible light visual component, near-infrared visual component, and temperature information component). Let be the gray value of the visible light image at position (m,n). This represents the grayscale value of the near-infrared image at position (m,n) after registration. The actual temperature value obtained by converting the registered thermal infrared image is given by the following conversion relationship: , , The temperature calibration coefficient for the thermal infrared camera is preset by the hardware parameters. , , These are the fusion weights for visible light, registered near-infrared, and registered thermal infrared images, respectively. The integrated fusion data is a multi-feature matrix of M×N×3 dimensions (M is the number of pixels in the image height, and N is the number of pixels in the image width). Each pixel contains three types of information: "visible light details, near-infrared details, and temperature value," achieving synchronous encapsulation of visual details and temperature information. The data output unit encapsulates this multi-feature matrix in binary stream format, with encapsulation fields including matrix dimension identifier, weight information of each component, temperature calibration coefficient, and data check code, and transmits it to the target intelligent recognition module via the data bus.
[0029] Specifically, the target intelligent recognition module includes a feature extraction unit, a feature matching unit, and a result generation unit. The built-in fire scene target recognition model is a lightweight deep learning model based on CNN. The model is trained and converged using a fire scene dataset. The model input is an integrated fused image matrix transmitted from the cross-modal data fusion module. The output is the target recognition result; Feature extraction unit pairs Deep feature extraction is performed by sequentially processing convolutional layers, pooling layers, and fully connected layers. The convolutional layer uses a 3×3 convolutional kernel for feature mapping, and the convolution operation formula is as follows: ; in, The output feature map of the l-th convolutional layer. For the l-th layer convolution kernel, For the l-th layer bias term, It is the ReLU activation function; The pooling layer uses max pooling, with a pooling kernel size of 2×2, and the calculation formula is as follows: ; After multiple convolutional and pooling processes, the two-dimensional feature map is transformed into a one-dimensional feature vector through a fully connected layer. , where d is the feature dimension; The feature matching unit compares the feature vector F with a preset fire source feature library. Human Feature Database Hazardous area feature database Cosine similarity matching is performed. The formula for cosine similarity is: ,in, These are the standard feature vectors in the feature library (K represents 1, 2, and 3 respectively). Let L2 be the norm of the vector; when ( When a preset similarity threshold is set, the target of the corresponding type is identified. Based on the matching results and the image region coordinates corresponding to the feature vector F, the result generation unit generates a result containing the target type (fire source / trapped person / dangerous area) and the coordinates of the top-left corner. lower right corner coordinates Confidence level (i.e., corresponding The target identification result data (value) is transmitted to the wireless data transmission module in JSON format.
[0030] Specifically, the environmental parameter linkage adaptation module includes a multi-source sensor accuracy calibration unit and a detection channel dynamic switching unit. The environmental parameter linkage adaptation module has a modular structure, with the multi-source sensor accuracy calibration unit and the detection channel dynamic switching unit operating in parallel, both establishing signal connections with the multispectral data acquisition module and control module 5. The collected concentration values of carbon monoxide (CO), carbon dioxide (CO2), and hydrogen sulfide are... The multi-source sensor accuracy calibration unit receives ambient temperature data T and gas concentration data C (normalized combination of the collected concentration values of carbon monoxide, carbon dioxide, and hydrogen sulfide) from the multispectral data acquisition module in real time, and outputs parameter correction coefficients. (Correction coefficients corresponding to the visible light camera 8, near-infrared camera 9, thermal infrared camera 10 of vision module 3 and sensor module 6, respectively), the regression formula is: Where k0 represents Any one of them, , , , The preset regression coefficients were obtained by fitting sensor performance test data under high temperature and complex gas environments. The parameter correction command includes various correction coefficients. After receiving it, vision module 3 will convert the original detection parameters of the camera. Revised to 0. After receiving the data, sensor module 6 will collect the original gas concentration value. Revised to This achieves accuracy drift compensation; The dynamic switching unit for detection channels adjusts the operating parameters and fusion weights based on the smoke concentration correlation data S (normalized value range [0,1]) monitored by the multispectral data acquisition module through a piecewise function: when ( When the smoke concentration threshold is reached: the operating gain of the visible light camera 8 (Minimum gain), operating gain of near-infrared camera 9 (Maximum gain), Thermal infrared camera 10 operating gain (Maximum gain); The fusion weights of the cross-modal data fusion module are adjusted to... ,and ; when ( When the smoke concentration threshold is low: Visible light camera 8 exposure parameters (Optimal exposure value), detail acquisition gain (Maximum detail gain), Near-infrared camera 9 operating gain (Optimal gain), operating gain of thermal infrared camera 10 (Optimal gain); The fusion weights of the cross-modal data fusion module are adjusted to ,and ; when hour: The operating parameters of each camera and the fusion weights are dynamically adjusted using a linear interpolation algorithm. The interpolation formula is as follows: Where X represents the parameter or weight to be adjusted. for The value at time, for The value at time; It should be noted that, The threshold and initial parameter values were calibrated using measured data under different smoke concentration scenarios.
[0031] Specifically, the wireless data transmission module is a functional module with full-duplex communication capability, consisting of four sub-units: a data receiving interface, a data encoding unit, a wireless communication unit, and an instruction feedback unit. Each sub-unit works in sequence according to the data flow direction. The data receiving interface establishes a physical connection with the universal asynchronous transceiver interface (UART interface) of the control module 5 to receive two types of data: first, the target identification result data output by the target intelligent identification module, which includes the type, location, and confidence information of the identified fire sources, trapped personnel, and dangerous areas in the fire scene; second, the raw monitoring data output by the multispectral data acquisition module, which includes the raw information of visible light, near-infrared, and thermal infrared images, as well as the raw monitoring values of the concentration of toxic and harmful gases, ambient temperature, and smoke concentration in the fire scene. The data transmission rate of this interface is not less than 115200 baud rate to ensure the real-time performance of data transmission. The data encoding unit is responsible for performing reliability encoding processing on the two types of received data. It adopts a 32-bit cyclic redundancy check (CRC-32) encoding method. It performs calculations on the data body through a preset standard generator polynomial to generate a 32-bit check code and appends it to the end of the data. This enables error detection during data transmission and ensures that the receiving end can identify whether the data has been distorted during transmission. The wireless communication unit supports multiple communication protocols, including Wi-Fi, 4G, 5G, and fire-specific radio communication protocols. The specific protocol used can be switched through configuration commands issued by control module 5. Its operating frequency band complies with the communication frequency band specifications of the fire protection industry, and the transmission power does not exceed 1 watt, which ensures both the anti-interference capability of data transmission and meets the compliance requirements of radio communication. The core function of this unit is to transmit the encoded integrated data to the fire command center terminal or firefighter handheld terminal in real time. The data delay from sending to receiving end does not exceed 500 milliseconds, ensuring that rescue decisions are based on the latest fire scene information. Meanwhile, the wireless communication unit also has a receiving function, which can obtain control commands sent by the terminal, including commands to adjust the device detection parameters and modify the data acquisition frequency. These control commands adopt ASCII encoding format and include command type identifier, specific parameter value and check bit, which facilitates module parsing and verification. The instruction feedback unit is responsible for processing the received control instructions: first, it verifies the check bit in the instruction to confirm that no errors occurred during the transmission; if the verification passes, it parses the type and parameter value of the instruction and feeds the parsing result back to the control module 5, which then performs the corresponding adjustment operation; if the verification fails, it returns a retransmission request to the terminal that sent the instruction to ensure the accurate execution of the control instructions, thereby realizing the reliability of two-way data interaction between the device and the terminal. The wireless data transmission module is powered directly by the device's power module. Its current consumption during operation does not exceed 500 mA. It is synchronized with the control module 5 through an interrupt signal to ensure that the data transmission and reception operations are consistent with the overall working rhythm of the device.
[0032] The working principle proposed in this invention is as follows: When using this multispectral collaborative detection and target identification device for fire environments, the installation and deployment of the device are first completed through the quick-release mechanism. Align the damping rod 15 on the inner side of the sealing plate 2 with the positioning hole at the bottom of the housing 1, and simultaneously align the sealing strip 14 precisely with the sealing groove. Pressing the sealing plate 2 will achieve quick assembly. The housing 1 and the sealing plate 2 together form a fireproof and waterproof outer shell, which can effectively resist the complex environment of the fire scene, ensure the normal operation of internal components, and the quick-locking structure or Velcro design between the device and the mounting base can realize rapid deployment, replacement and maintenance.
[0033] After the device is installed, power module 4 starts up and provides stable and uninterrupted power support to all components of the entire device, ensuring that each module works independently and collaboratively. Control module 5, as the core control unit, starts up synchronously and coordinates the orderly operation of each module of the intelligent processing mechanism and the multispectral collaborative sensing and dynamic adaptation system.
[0034] Visible light camera 8, near-infrared camera 9, and thermal infrared camera 10 in vision module 3 are activated simultaneously to collect conventional visual information of the fire scene environment, imaging information penetrating some smoke and flames, and temperature distribution information corresponding to the thermal radiation of objects, respectively. At the same time, various sensors on sensor module 6 work synchronously to collect real-time data on the concentration of toxic and harmful gases such as carbon monoxide, carbon dioxide, and hydrogen sulfide in the fire scene, as well as ambient temperature data and smoke concentration correlation data.
[0035] The multispectral data acquisition module maintains signal linkage with vision module 3 and sensor module 6. After receiving all the raw data, it first performs format standardization processing to unify the data transmission format, and then transmits the processed standardized data to the cross-modal data fusion module. After receiving the data, the cross-modal data fusion module uses a preset algorithm to spatially align the image data of different types and sources to ensure the spatial consistency of each image data. Then, based on the effectiveness of each type of image data under different environments, it adopts a weighted fusion method to deeply fuse the temperature feature information of thermal infrared images with the visual detail feature information of visible light and near-infrared images, generating integrated fused data that combines temperature information and visual details, and then transmits it to the target intelligent recognition module.
[0036] The target intelligent recognition module has a built-in fire scene target recognition model trained by deep learning. It performs deep feature extraction on the integrated data, extracting key features such as the shape, temperature, and texture of the target. Then, it performs similarity matching and comparison with the preset fire source feature library, human body feature library, and dangerous area feature library to accurately identify and locate the fire source, the outline of trapped personnel, and the range of dangerous areas, generating target recognition result data that includes target type, coordinate position, and range size.
[0037] During this process, the two units of the environmental parameter linkage adaptation module work in parallel. The multi-source sensor accuracy calibration unit receives ambient temperature and gas concentration data in real time, combines them with a preset accuracy calibration model, determines the degree of sensor accuracy drift, dynamically generates parameter correction instructions, and sends them to the vision module 3 and sensor module 6 to correct their detection parameters in real time and compensate for accuracy drift caused by the high-temperature environment. The detection channel dynamic switching unit automatically adjusts the working parameters of the three cameras and the fusion weight of the cross-modal data fusion module based on smoke concentration correlation data. In high-concentration smoke environments, it enhances the working gain and fusion weight of the thermal infrared camera 10 and the near-infrared camera 9, and optimizes the exposure parameters and detail acquisition gain of the visible light camera 8 in low-concentration smoke environments to ensure detection performance in different environments.
[0038] The wireless data transmission module connects to the control module 5 via a signal link, receiving target identification results and raw monitoring data. After encoding and processing, it transmits the data in real time to the fire command center terminal or firefighter handheld terminal using a compatible wireless communication protocol. Simultaneously, this module can receive control commands from the terminal, such as parameter adjustment and acquisition frequency adjustment. After verification, these commands are fed back to the control module 5, enabling two-way data interaction and providing real-time data support for rescue decision-making.
[0039] In addition, the heat dissipation module 7 operates continuously, with the servo motor 11 driving the cooling fan 12 to rotate. Together with the heat dissipation fins 13, the fan extracts hot air from inside the housing 1, cooling heat-generating components such as the control module 5 and ensuring stable operation of all components in high-temperature fire environments. When the device requires maintenance, the sealing plate 2 can be quickly disassembled for operation, ensuring timely maintenance and continued functionality of the detection and identification functions even in complex fire environments.
[0040] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A multispectral collaborative detection and target identification device for fire scene environment, comprising a housing (1) and a sealing plate (2) installed at the bottom of the housing (1), characterized in that: A quick-release mechanism is installed at the connection between the housing (1) and the sealing plate (2). An intelligent processing mechanism for detecting and identifying the environment is installed inside the housing (1). The intelligent processing mechanism integrates a multispectral collaborative sensing and dynamic adaptation system for realizing multispectral collaborative detection and dynamic adaptation. The multispectral collaborative sensing and dynamic adaptation system includes a multispectral data acquisition module, a cross-modal data fusion module, a target intelligent identification module, an environmental parameter linkage adaptation module, and a wireless data transmission module. Each module is used to realize the functions of multispectral image and environmental parameter acquisition, cross-modal data accurate fusion, fire source / trapped personnel / dangerous area identification, detection parameter dynamic adjustment, and real-time data transmission.
2. A multispectral collaborative detection and target identification device for fire scene environments according to claim 1, characterized in that: The quick-release mechanism includes a sealing strip (14) installed on the inner side of the sealing plate (2) and multiple damping rods (15) fixed to the four corners of the inner side of the sealing plate (2). The bottom of the housing (1) is provided with sealing grooves and positioning holes that match the sealing strip (14) and the damping rods (15).
3. A multispectral collaborative detection and target identification device for fire scene environments according to claim 1, characterized in that: The intelligent processing mechanism includes a vision module (3) installed at the front end inside the housing (1). A power module (4), a control module (5), a sensor module (6) and a heat dissipation module (7) are installed on the top surface of the inner wall of the housing (1). The heat dissipation module (7) is located at the rear end of the power module (4). The sensor module (6) and the control module (5) are located on one side of the power module (4) and the heat dissipation module (7), respectively. The vision module (3), the sensor module (6) and the heat dissipation module (7) are all connected to the control module (5) through wires, and the control module (5) is connected to the power module (4) through wires.
4. A multispectral collaborative detection and target identification device for fire scene environment according to claim 3, characterized in that: Visible light camera (8), near-infrared camera (9) and thermal infrared camera (10) are installed at equal intervals on the vision module (3). The visible light camera (8), near-infrared camera (9) and thermal infrared camera (10) are all located at the front end of the housing (1).
5. A multispectral collaborative detection and target identification device for fire scene environment according to claim 3, characterized in that: The heat dissipation module (7) and the housing (1) are respectively provided with circular mounting slots. The mounting slot is equipped with a mounting box through a cross mounting bracket. The mounting box is equipped with a servo motor (11). The output end of the servo motor (11) is equipped with a cooling fan (12) through a coupling. Multiple heat dissipation fins (13) are installed at equal intervals on the inner wall of the mounting slot. The heat dissipation fins (13) are located above the cooling fan (12).
6. A multispectral collaborative detection and target identification device for fire scene environment according to claim 3, characterized in that: The multispectral data acquisition module is connected to the vision module (3) and the sensor module (6) for synchronous acquisition of visible light image data, near-infrared image data, thermal infrared image data, and toxic and harmful gas concentration data and ambient temperature data in the fire scene. The acquired multi-source raw data is processed in a standardized format and then transmitted to the cross-modal data fusion module.
7. A multispectral collaborative detection and target identification device for fire scene environment according to claim 1, characterized in that: The cross-modal data fusion module receives standardized data transmitted from the multispectral data acquisition module. It uses a preset multimodal data registration algorithm to spatially align image data of different types and sources. Then, it uses a weighted fusion algorithm to fuse temperature feature information in thermal infrared images with visual detail feature information in visible light and near-infrared images, generating integrated fused data containing temperature information and visual details, which is then transmitted to the target intelligent recognition module.
8. A multispectral collaborative detection and target identification device for fire scene environment according to claim 1, characterized in that: The target intelligent recognition module has a built-in trained fire scene target recognition model. It extracts features from the integrated fusion data transmitted by the cross-modal data fusion module. By matching and comparing the target features with the fire source feature library, human body feature library, and dangerous area feature library in the preset model, it can accurately identify and locate the fire source location, the outline of trapped personnel, and the range of dangerous areas, and generate target recognition result data.
9. A multispectral collaborative detection and target identification device for fire scene environment according to claim 4, characterized in that: The environmental parameter linkage adaptation module includes a multi-source sensor accuracy calibration unit and a detection channel dynamic switching unit; The multi-source sensor accuracy calibration unit receives ambient temperature data and gas concentration data transmitted by the multispectral data acquisition module in real time, performs linkage calculation with the preset accuracy calibration model, dynamically generates parameter correction instructions based on the calculation results, and transmits them to the vision module (3) and sensor module (6) to correct their detection parameters in real time and compensate for the detection accuracy drift caused by the high temperature environment. The detection channel dynamic switching unit automatically adjusts the working parameters of the visible light camera (8), near-infrared camera (9), and thermal infrared camera (10) and the data fusion weight in the cross-modal data fusion module based on the smoke concentration correlation data monitored by the multispectral data acquisition module. In high-concentration smoke environment, it enhances the working gain and data fusion weight of thermal infrared camera (10) and near-infrared camera (9), and optimizes the exposure parameters and detail acquisition gain of visible light camera (8) in low smoke concentration environment.
10. A multispectral collaborative detection and target identification device for fire scene environment according to claim 3, characterized in that: The wireless data transmission module is connected to the control module (5) by signal. It receives the target recognition result data transmitted by the target intelligent recognition module and the original monitoring data transmitted by the multispectral data acquisition module. It transmits the data to the fire command center terminal or the firefighter's handheld terminal in real time through the wireless communication protocol. At the same time, it receives the control instructions issued by the terminal and feeds them back to the control module (5).