Infrared thermal imaging-based intelligent search and rescue method and device for fire personnel
By simultaneously collecting data using infrared thermal imagers and lidar, and combining it with deep learning technology, the infrared thermal radiation and three-dimensional spatial features of the fire scene are extracted. This enables the perception of the correlation between the heat distribution and three-dimensional space at the fire scene, solving the problems of accuracy and efficiency in human target detection in existing technologies, and improving the precision of search and rescue and the utilization rate of information.
Patent Information
- Application Number
- CN202510991204.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-18
- Publication Date
- 2025-10-28
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing infrared thermal imaging technology suffers from multiple sources of thermal radiation interference and blurred temperature gradients at fire scenes, resulting in reduced contrast between human targets and high-temperature backgrounds, making it difficult to accurately detect human targets. LiDAR data cannot distinguish temperature characteristics, leading to information fragmentation and affecting the accuracy and efficiency of search and rescue.
By simultaneously acquiring data using an infrared thermal imager and lidar, and combining it with deep learning technology, infrared thermal radiation and three-dimensional spatial features are extracted. Through cross-modal feature alignment, the thermal distribution and three-dimensional spatial correlation perception of the fire scene are realized, thereby enhancing the detection of human targets.
It effectively overcomes the problem of the separation between thermal radiation information and spatial structure information, improves the accuracy and efficiency of human target recognition, and enhances information utilization.
Smart Images

Figure CN120856961A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of infrared thermal imaging technology, and more specifically, to a method and apparatus for intelligent search and rescue of people in fires based on infrared thermal imaging. Background Technology
[0002] In fire accidents, the rapid and accurate search for trapped personnel is crucial for reducing casualties and property damage. However, the complex environment at fire scenes, filled with adverse factors such as high temperatures, dense smoke, and darkness, greatly hinders rescue efforts and poses a severe challenge to traditional fire rescue methods. For example, dense smoke, a common phenomenon at fire scenes, severely reduces visibility, making it difficult for rescuers to spot trapped individuals with the naked eye.
[0003] With the development of technology, search and rescue methods based on infrared thermal imaging technology have gradually emerged. Infrared thermal imaging technology detects the infrared thermal radiation emitted by objects and converts it into a visible image, which is widely used in the field of fire search and rescue. Because the human body temperature is higher than the surrounding environment, even in adverse conditions such as dense smoke and darkness, infrared thermal imagers can detect the thermal signals emitted by the human body, helping search and rescue personnel to determine the location of trapped persons.
[0004] However, while infrared thermal imaging has the advantage of penetrating smoke, it suffers from numerous sources of thermal radiation interference and blurred temperature gradients in high-temperature fire environments. This significantly reduces the contrast between human targets and the high-temperature background, affecting the accurate detection of human targets. To compensate for the shortcomings of infrared technology, lidar (LiDAR) has been gradually introduced into the field of complex disaster environment perception due to its high-precision 3D point cloud reconstruction capabilities. However, lidar data only reflects the geometric information of objects and cannot distinguish the temperature characteristics of objects, making it difficult to accurately identify human targets in complex environments. Currently, existing research mostly adopts a method of independently acquiring data from multiple sensors, such as identifying high-temperature areas through infrared images and then using lidar point cloud data to construct a 3D spatial model. However, this approach fails to fully explore the potential correlation between thermal radiation characteristics and spatial geometric features, resulting in a separation between thermal radiation information and spatial structure information, insufficient information utilization, and thus affecting the accuracy and efficiency of search and rescue.
[0005] Therefore, there is a need for an optimized intelligent search and rescue method and device for personnel in fires based on infrared thermal imaging. Summary of the Invention
[0006] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a method and apparatus for intelligent fire scene search and rescue based on infrared thermal imaging. It utilizes an infrared thermal imager and a lidar to simultaneously acquire fire scene data, obtaining infrared thermal imaging data and three-dimensional point cloud data of the fire scene. Then, it further introduces deep learning-based data processing technology to extract features from the infrared thermal imaging data and the three-dimensional point cloud data of the fire scene, capturing the infrared thermal radiation distribution characteristics and three-dimensional spatial structure characteristics of the fire scene. Furthermore, through fine-grained feature alignment and joint perception of both, it captures the deep-level correlation between the thermal distribution and three-dimensional space of the fire scene, thereby achieving intelligent detection of human targets. This approach effectively overcomes the problem of the separation of thermal radiation information and spatial structure information in traditional methods, improving information utilization and enhancing the accuracy and efficiency of human target recognition.
[0007] According to one aspect of this application, a fire-related intelligent search and rescue method based on infrared thermal imaging is provided, comprising: Infrared thermal imager and lidar are used to simultaneously collect fire scene data to obtain infrared thermal imaging data and three-dimensional point cloud data of the fire scene; Infrared features are extracted from the infrared thermal imaging data to obtain an infrared thermal imaging feature map; Three-dimensional spatial features are extracted from the three-dimensional point cloud data of the fire scene to obtain a three-dimensional spatial feature map of the fire scene; The infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene are subjected to cross-modal joint sensing based on feature dynamic search alignment to obtain the fire scene thermal distribution-three-dimensional spatial joint sensing feature map. Based on the joint sensing feature map of the fire scene's thermal distribution and three-dimensional space, the detection results of human targets are determined.
[0008] According to another aspect of this application, a fire rescue intelligent personnel search and rescue device based on infrared thermal imaging is provided, comprising: The fire scene data acquisition module is used to simultaneously acquire fire scene data using an infrared thermal imager and a lidar to obtain infrared thermal imaging data and three-dimensional point cloud data of the fire scene. An infrared feature extraction module is used to extract infrared features from the infrared thermal imaging data to obtain an infrared thermal imaging feature map. The three-dimensional spatial feature extraction module is used to extract three-dimensional spatial features from the three-dimensional point cloud data of the fire scene to obtain a three-dimensional spatial feature map of the fire scene. A cross-modal joint sensing module is used to perform cross-modal joint sensing based on feature dynamic search alignment on the infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene to obtain a fire scene thermal distribution-three-dimensional spatial joint sensing feature map. The human target detection module is used to determine the human target detection result based on the joint perception feature map of the fire scene's thermal distribution and three-dimensional space.
[0009] Compared with existing technologies, the intelligent fire rescue method and device based on infrared thermal imaging provided in this application utilizes an infrared thermal imager and a lidar to simultaneously acquire fire scene data, obtaining infrared thermal imaging data and three-dimensional point cloud data of the fire scene. Then, it further introduces deep learning-based data processing technology to extract features from the infrared thermal imaging data and the three-dimensional point cloud data of the fire scene, capturing the infrared thermal radiation distribution characteristics and three-dimensional spatial structure characteristics of the fire scene. Furthermore, through fine-grained feature alignment and joint perception of both, it captures the deep-level correlation between the thermal distribution and three-dimensional space of the fire scene, thereby achieving intelligent detection of human targets. This approach effectively overcomes the problem of the separation of thermal radiation information and spatial structure information in traditional methods, improves information utilization, and enhances the accuracy and efficiency of human target recognition. Attached Figure Description
[0010] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.
[0011] Figure 1 This is a flowchart of an intelligent search and rescue method for personnel in a fire based on infrared thermal imaging, according to an embodiment of this application.
[0012] Figure 2 This is a schematic diagram of data flow in an intelligent fire rescue method based on infrared thermal imaging, according to an embodiment of this application.
[0013] Figure 3 This is a flowchart of sub-step S2 of the intelligent fire rescue method based on infrared thermal imaging according to an embodiment of this application.
[0014] Figure 4 This is a flowchart of sub-step S4 of the intelligent search and rescue method for fire personnel based on infrared thermal imaging according to an embodiment of this application.
[0015] Figure 5 This is a flowchart of sub-step S41 of the intelligent fire rescue method based on infrared thermal imaging according to an embodiment of this application.
[0016] Figure 6 This is a flowchart of sub-step S42 of the intelligent fire rescue method based on infrared thermal imaging according to an embodiment of this application.
[0017] Figure 7 This is a block diagram of an intelligent fire rescue device based on infrared thermal imaging, according to an embodiment of this application. Detailed Implementation
[0018] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.
[0019] While this application makes various references to certain modules of the systems according to embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The modules described are merely illustrative, and different aspects of the systems and methods may use different modules.
[0020] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.
[0021] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.
[0022] It is worth noting that all data acquisition actions in this application were carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.
[0023] To address the technical problems described in the background, this application proposes an intelligent fire rescue method based on infrared thermal imaging. This method utilizes an infrared thermal imager and lidar to simultaneously acquire fire scene data, obtaining infrared thermal imaging data and three-dimensional point cloud data of the fire scene. Then, it further introduces deep learning-based data processing technology to extract features from the infrared thermal imaging data and the three-dimensional point cloud data of the fire scene, capturing the infrared thermal radiation distribution characteristics and three-dimensional spatial structure characteristics of the fire scene. Furthermore, by performing fine-grained feature alignment and joint perception on both, it captures the deep-level correlation between the thermal distribution and three-dimensional space of the fire scene, thereby achieving intelligent detection of human targets. This approach effectively overcomes the problem of the separation of thermal radiation information and spatial structure information in traditional methods, improving information utilization and enhancing the accuracy and efficiency of human target recognition.
[0024] Figure 1 This is a flowchart of an intelligent search and rescue method for personnel in a fire based on infrared thermal imaging, according to an embodiment of this application. Figure 2 This is a schematic diagram of data flow in an intelligent fire rescue method based on infrared thermal imaging, according to an embodiment of this application. Figure 1 and Figure 2 As shown, the intelligent fire rescue method based on infrared thermal imaging includes the following steps: S1, simultaneously acquiring fire scene data using an infrared thermal imager and a lidar to obtain infrared thermal imaging data and three-dimensional point cloud data of the fire scene; S2, extracting infrared features from the infrared thermal imaging data to obtain an infrared thermal imaging feature map; S3, extracting three-dimensional spatial features from the three-dimensional point cloud data of the fire scene to obtain a three-dimensional spatial feature map of the fire scene; S4, performing cross-modal joint perception based on feature dynamic search alignment on the infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene to obtain a fire scene thermal distribution-three-dimensional space joint perception feature map; S5, determining the human target detection result based on the fire scene thermal distribution-three-dimensional space joint perception feature map.
[0025] In the aforementioned intelligent fire rescue method based on infrared thermal imaging, step S1 involves simultaneously acquiring fire scene data using an infrared thermal imager and a lidar to obtain infrared thermal imaging data and three-dimensional point cloud data of the fire scene. It should be understood that the fire scene environment is complex; dense smoke, high-temperature airflow, and burning materials can significantly interfere with the perception capabilities of a single sensor. Infrared thermal imagers penetrate smoke by receiving human thermal radiation signals, but high-temperature environments (such as flames and high-temperature obstacles) increase thermal radiation interference sources, reducing the temperature difference between the human target and the background (especially when the ambient temperature is close to human body temperature). LiDAR generates three-dimensional point clouds by emitting laser pulses and measuring reflection time, accurately constructing spatial structures, but it cannot distinguish the temperature characteristics of objects, potentially leading to the misidentification of objects with similar geometric features as human targets in complex scenes. Therefore, to improve the accuracy and robustness of human target detection, this application simultaneously acquires infrared thermal imaging data and three-dimensional point cloud data of the fire scene, complementing the shortcomings of both in perception capabilities and improving the comprehensiveness and accuracy of overall information perception.
[0026] Specifically, in the implementation process, precise configuration of infrared thermal imagers and lidar is required to ensure seamless collaboration between the two. First, suitable infrared thermal imagers and lidar equipment for fire rescue missions must be selected. For the selection of infrared thermal imagers, factors such as resolution, sensitivity, and the ability to penetrate dense smoke must be considered. For lidar selection, the focus is on scanning speed, accuracy, and the ability to quickly generate high-quality 3D point cloud data. Furthermore, considering the potential presence of large amounts of metal or other highly reflective materials in a fire scene, the lidar must also possess a certain degree of anti-interference capability.
[0027] Next, the installation location of the equipment also needs careful planning. Ideally, the infrared thermal imaging device and lidar should be integrated into a robust and portable unit, allowing search and rescue teams to move flexibly within the fire scene. Simultaneously, the device design must also consider fire resistance and waterproofing to withstand harsh rescue environments. In practice, the installation height and angle of the equipment may need to be adjusted according to the characteristics of different fire scenes to ensure both maximum search coverage and high-quality data acquisition.
[0028] To achieve synchronous data acquisition, an efficient communication mechanism is needed to enable the infrared thermal imager and lidar to exchange information in real time and transmit the collected data to the central processing unit. During this process, it is crucial to ensure data time synchronization; that is, the acquisition frequencies of both devices must be consistent to avoid data mismatch issues caused by time deviations. To this end, GPS or other high-precision time synchronization technologies can be used to calibrate the time difference between the two devices, ensuring that the acquired data can be compared and analyzed on the same timeline.
[0029] Furthermore, in the specific implementation process, it is also necessary to consider how to deal with different types of fire scene environments. For example, in fires occurring in large warehouses or industrial facilities, due to the large space, it may be necessary to use drones equipped with infrared thermal imagers and lidar for high-altitude detection; while in small-scale fires in residential areas, it is more suitable for ground rescue teams to directly operate portable equipment for search and rescue. Regardless of the method adopted, the key is how to effectively integrate the advantages of infrared thermal imagers and lidar, overcome their respective technical limitations, and provide more comprehensive and accurate information support for search and rescue operations.
[0030] In the aforementioned intelligent search and rescue method for fire personnel based on infrared thermal imaging, step S2 involves extracting infrared features from the infrared thermal imaging data to obtain an infrared thermal imaging feature map. Wherein, Figure 3 This is a flowchart of sub-step S2 of the intelligent fire rescue method based on infrared thermal imaging according to an embodiment of this application. Figure 3 As shown, step S2 includes the following steps: S21, performing image denoising and edge detection processing on the infrared thermal imaging data to obtain enhanced infrared thermal imaging data; S22, performing infrared feature extraction based on the MobileNet model on the enhanced infrared thermal imaging data to obtain the infrared thermal imaging feature map.
[0031] Specifically, step S21 involves performing image denoising and edge detection processing on the infrared thermal imaging data to obtain enhanced infrared thermal imaging data. It should be understood that infrared images in fire scenes are easily affected by high-temperature gas turbulence and reflections from burning particles, leading to image noise (such as salt-and-pepper noise and Gaussian noise) and thermal gradient blurring. Furthermore, in some scenarios, the temperature difference between the human target and the high-temperature background may not be significant, resulting in blurred target edges. Directly using the original infrared data may reduce the robustness of subsequent feature extraction. Therefore, this application further performs image denoising and edge detection processing on the infrared thermal imaging data to suppress noise interference, preserve the thermal radiation contour of the human target, and enhance the contrast between the target and the background. In the embodiments of this application, image denoising employs a Gaussian filtering algorithm, which, based on the weighted average principle of the Gaussian function, performs weighted calculations on the neighborhood of each pixel in the infrared thermal imaging data, effectively removing Gaussian noise and salt-and-pepper noise from the image. Edge detection employs the Canny algorithm, which calculates the gradient magnitude and direction of the image, finds the local maximum of the gradient magnitude, and accurately locates the edges through double threshold detection and edge connection algorithms, thereby effectively highlighting the contour information of the object and providing high-quality input data for subsequent feature extraction.
[0032] Specifically, in step S22, infrared feature extraction based on the MobileNet model is performed on the enhanced infrared thermal imaging data to obtain the infrared thermal imaging feature map. Specifically, this application considers that in fire scene data processing, the computing resources and processing time of equipment are often limited, while traditional convolutional networks (such as ResNet) have a large number of parameters, making it difficult to meet the real-time requirements of fire search and rescue. Therefore, this application uses the lightweight MobileNet model to extract infrared features from the enhanced infrared thermal imaging data. Specifically, the MobileNet model achieves a lightweight design through depthwise separable convolution, significantly reducing the number of parameters and computational load while ensuring feature extraction capabilities, making it suitable for deployment on edge devices. In this application, the MobileNet model first performs independent convolution operations on each input channel of the enhanced infrared thermal imaging data through depthwise convolution to extract the spatial features of each channel and capture information such as the thermal radiation distribution pattern and spatial correlation of heat sources in the image. Then, it linearly combines the output channels of the depthwise convolution through pointwise convolution to further fuse the feature information between different channels and generate an infrared thermal imaging feature map, thereby achieving efficient and accurate infrared feature extraction.
[0033] In the aforementioned intelligent search and rescue method for fire personnel based on infrared thermal imaging, step S3 involves extracting three-dimensional spatial features from the three-dimensional point cloud data of the fire scene to obtain a three-dimensional spatial feature map of the fire scene. In a specific example of this application, step S3 includes: extracting three-dimensional spatial features from the three-dimensional point cloud data of the fire scene based on the VoxelNet model to obtain a three-dimensional spatial feature map of the fire scene. It should be understood that the collapsed structures and obstacles at the fire scene lead to complex three-dimensional spatial information. Traditional methods often face problems such as large computational load and difficulty in feature extraction when processing such complex point cloud data. The VoxelNet model, as a deep learning model specifically designed for point cloud data, converts disordered point clouds into regular three-dimensional meshes through voxelization and extracts spatial geometric features (such as human posture and spatial position) by combining 3D convolution. It can effectively model target information in complex spatial structures at the fire scene and has the ability to efficiently process large-scale point cloud data. In this application, the VoxelNet model first divides the three-dimensional point cloud data of the fire scene into multiple three-dimensional voxels, each voxel containing a certain number of point cloud data. Then, feature encoding is performed on the point cloud data within each voxel. A voxel feature encoding layer transforms the point cloud data into a compact voxel feature representation, effectively reducing the dimensionality of the point cloud data and decreasing the computational load of subsequent processing. Next, a 3D convolutional neural network is used to further process the encoded voxel features. Stacked 3D convolutional layers progressively extract spatial geometric features to capture the three-dimensional spatial structure information of the fire scene, such as the shape, size, and positional relationships of objects. This generates a three-dimensional spatial feature map of the fire scene, providing rich three-dimensional spatial information for subsequent target recognition and cross-modal feature fusion.
[0034] In the aforementioned intelligent fire rescue method based on infrared thermal imaging, step S4 involves performing cross-modal joint sensing on the infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene based on dynamic feature search alignment to obtain a joint sensing feature map of the fire scene's thermal distribution and three-dimensional space. It should be understood that traditional multi-sensor data processing methods often analyze data from different sensors independently, resulting in the inability to effectively fuse thermal radiation information and spatial structure information, leading to low information utilization. Furthermore, infrared thermal imaging features and point cloud features may exhibit spatial misalignment due to modal differences, and simple feature fusion methods such as concatenation or addition cannot effectively model the feature correlations between cross-modal data. To address this, this application proposes a cross-modal joint sensing method based on dynamic feature search and alignment. This method utilizes a dynamic search algorithm to calculate the phase correlation between the infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene. It then searches for phase-matching feature point pairs in the feature space, achieving fine-grained alignment of thermal radiation distribution and three-dimensional spatial structure to ensure precise alignment of feature points. Furthermore, by incorporating an attention mechanism, adaptive joint sensing is performed on the aligned feature point pairs to enhance the expression of key features and suppress redundant information. This generates a joint sensing feature map of fire scene thermal distribution and three-dimensional space, providing a richer and more accurate information foundation for subsequent human target detection. Figure 4 This is a flowchart of sub-step S4 of the intelligent fire rescue method based on infrared thermal imaging according to an embodiment of this application. Figure 4 As shown, step S4 includes the following steps: S41, performing dynamic search alignment based on local decoupling features on the infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene to obtain a set of phase-aligned feature pairs {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix}; S42, performing cross-modal sensing convergence based on attention mechanism on the set of phase-aligned feature pairs {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix} to obtain the fire scene thermal distribution-three-dimensional space joint sensing feature map.
[0035] Specifically, in step S41, a dynamic search alignment based on local decoupling features is performed on the infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene to obtain a set of phase-aligned feature pairs {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix}. Figure 5 This is a flowchart of sub-step S41 of the intelligent fire rescue method based on infrared thermal imaging according to an embodiment of this application. Figure 5As shown, step S41 includes the following steps: S411, performing orthogonal decomposition on the infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene to obtain a set of infrared thermal imaging local feature coding matrices and a set of three-dimensional spatial local feature coding matrices of the fire scene; S412, calculating the feature phase alignment degree between the i-th infrared thermal imaging local feature coding matrix in the set of infrared thermal imaging local feature coding matrices and each of the three-dimensional spatial local feature coding matrices of the fire scene in the set of three-dimensional spatial local feature coding matrices of the fire scene to obtain a set of infrared thermal imaging-fire scene three-dimensional spatial feature phase alignment degrees; S413, extracting the fire scene three-dimensional spatial local feature coding matrix corresponding to the maximum value in the set of infrared thermal imaging-fire scene three-dimensional spatial feature phase alignment degrees, and forming a phase alignment feature pair with the i-th infrared thermal imaging local feature coding matrix to obtain the phase alignment {infrared thermal imaging local feature coding matrix, fire scene three-dimensional spatial local feature coding matrix} feature pair.
[0036] More specifically, step S411 is expressed by the formula:
[0037]
[0038] in, Represents infrared thermal imaging feature map, This represents a three-dimensional spatial feature map of a fire scene. Represents eigenvalue orthogonal decomposition. , , and These represent the 1st, 2nd, and 3rd elements in the set of local feature coding matrices for infrared thermal imaging, respectively. The and the first A local feature encoding matrix for infrared thermal imaging, The number of local feature encoding matrices in the infrared thermal imaging. , , and The first, second, and third elements in the set of three-dimensional spatial local feature encoding matrices of a fire scene are respectively represented. The and the first A three-dimensional spatial local feature encoding matrix of a fire scene.
[0039] In other words, through orthogonal feature decomposition, the human thermal radiation features in the infrared thermal imaging feature map are separated from the background thermal noise. Simultaneously, geometric features related to human morphology are extracted from the three-dimensional spatial feature map of the fire scene, forming a more discriminative set of infrared thermal imaging local feature coding matrices and a set of three-dimensional spatial local feature coding matrices of the fire scene. This orthogonal feature decomposition significantly improves the robustness of cross-modal feature matching. The local alignment of infrared thermal radiation features and three-dimensional spatial structures is no longer affected by global background temperature fluctuations. Furthermore, the decoupled set of infrared thermal imaging local feature coding matrices and the set of three-dimensional spatial local feature coding matrices of the fire scene can be quantitatively characterized by matrix operations to represent their phase consistency in local regions.
[0040] More specifically, in a specific example of this application, step S412 includes: calculating the product of the transpose of the i-th infrared thermal imaging local feature coding matrix and the fire scene three-dimensional spatial local feature coding matrix to obtain the infrared thermal imaging-fire scene three-dimensional spatial local feature joint sensing coding matrix; calculating the Frobenius norm of the infrared thermal imaging-fire scene three-dimensional spatial local feature joint sensing coding matrix to obtain the infrared thermal imaging-fire scene three-dimensional spatial feature phase alignment, expressed by the formula:
[0041] in, This represents the phase alignment measurement function. Represents the transpose of a matrix. This represents the Frobenius norm of the matrix.
[0042] In other words, by constructing an interaction space for cross-modal features through matrix operations, the dynamic changes in infrared thermal radiation distribution are mathematically correlated with the geometric topology of the three-dimensional spatial structure, thereby capturing the correlation and interaction between the two in local regions. Furthermore, by introducing the Frobenius norm of the joint sensing encoding matrix of infrared thermal imaging and three-dimensional spatial local features of the fire scene, the feature correlation between infrared thermal imaging local features and three-dimensional spatial local features of the fire scene can be quantified, generating the phase alignment degree of infrared thermal imaging and three-dimensional spatial features of the fire scene, providing a reliable basis for subsequent feature phase alignment screening.
[0043] More specifically, step S413 is expressed by the formula:
[0044] in, This indicates taking the index corresponding to the maximum value. Indicates the first The index of the three-dimensional spatial local feature encoding matrix of the fire scene with the highest feature phase alignment of the infrared thermal imaging local feature encoding matrix. That is, in relation to the first The local feature coding matrices of the fire scene in three dimensions are composed of a phase-aligned feature pair.
[0045] Here, a dynamic search mechanism effectively overcomes the limitations of traditional static registration methods. Even under complex interference such as high-temperature disturbances and smoke deformation, it accurately selects phase-aligned feature pairs with physical semantic relationships (infrared thermal imaging local feature encoding matrix, fire scene 3D spatial local feature encoding matrix). This allows subsequent attention aggregation to focus on the thermal-structural coupling features that truly reflect the human target. This constraint method, which maximizes phase alignment, effectively suppresses background noise interference and strengthens the complementarity between infrared thermal radiation features and 3D spatial topological features, thus laying the foundation for constructing a highly discriminative joint sensing feature map.
[0046] Specifically, in step S42, the set of phase-aligned feature pairs {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix} undergoes attention-based cross-modal sensing convergence to obtain the fire scene thermal distribution-three-dimensional spatial joint sensing feature map. Figure 6 This is a flowchart of sub-step S42 of the intelligent fire rescue method based on infrared thermal imaging according to an embodiment of this application. Figure 6 As shown, step S42 includes the following steps: S421, inputting each phase-aligned {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix} feature pair from the set of phase-aligned {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix} feature pairs into the cross-modal sensing focusing module to obtain a set of joint sensing attention weights for infrared thermal imaging and fire scene three-dimensional spatial features; S422, based on the set of joint sensing attention weights for infrared thermal imaging and fire scene three-dimensional spatial features, performing feature development under focusing modulation on the set of phase-aligned {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix} feature pairs to obtain the fire scene thermal distribution-three-dimensional spatial joint sensing feature map.
[0047] In a specific example of this application, step S421 can be expressed by the formula:
[0048] in, For cross-modal sensing and focusing modules, Let be the trace metric function of the matrix. Characteristic pairs indicating phase alignment For the first A joint sensing attention factor for phase-aligned feature pairs {infrared thermal imaging local feature encoding matrix, fire scene 3D spatial local feature encoding matrix} of infrared thermal imaging and fire scene 3D spatial features. It is the sigmoid activation function. express Infrared thermal imaging - joint perception attention weight of three-dimensional spatial features of fire scene.
[0049] In other words, the correlation and interaction relationships within feature pairs are captured by calculating the joint sensing encoding matrix of infrared thermal imaging and local 3D spatial features of the fire scene for each feature pair. The strength of this correlation and interaction relationship is quantified using matrix trace metric, thereby generating the corresponding joint sensing attention factor for infrared thermal imaging and 3D spatial features of the fire scene. Subsequently, the attention factor is mapped to a value between 0 and 1 using the sigmoid activation function, resulting in the joint sensing attention weight for infrared thermal imaging and 3D spatial features of the fire scene. That is, through the learning capability of the cross-modal sensing focusing module, the correlation between infrared thermal imaging and 3D spatial structural features—crucial for fire rescue personnel—is automatically mined and strengthened, thus providing precise weight allocation for subsequent feature aggregation.
[0050] Specifically, in a preferred embodiment of this application, step S421 includes: calculating the infrared thermal imaging-fire scene 3D spatial local feature encoding matrix joint sensing encoding matrix between the infrared thermal imaging local feature encoding matrix and the fire scene 3D spatial local feature encoding matrix in the phase-aligned {infrared thermal imaging local feature encoding matrix, fire scene 3D spatial local feature encoding matrix} feature pair; based on the stationary value of the feature distribution of the infrared thermal imaging local feature encoding matrix and the fire scene 3D spatial local feature encoding matrix in the phase-aligned {infrared thermal imaging local feature encoding matrix, fire scene 3D spatial local feature encoding matrix} feature pair, performing manifold regularization optimization on the matrix trace of the infrared thermal imaging-fire scene 3D spatial local feature joint sensing encoding matrix to obtain an optimized infrared thermal imaging-fire scene 3D spatial feature joint sensing attention factor; and inputting the optimized infrared thermal imaging-fire scene 3D spatial feature joint sensing attention factor into a sigmoid activation function to obtain the infrared thermal imaging-fire scene 3D spatial feature joint sensing attention weight.
[0051] Specifically, for each matrix trace distance operation... and ,make and ,and , By calculation, the corresponding and Between distance and distance The number of eigenvalues yields the matrix and Number of interconnecting components and That is, the stationary quantity of feature distribution between the local feature coding matrix of infrared thermal imaging for phase synchronization and the local feature coding matrix of three-dimensional space of the fire scene.
[0052] Then, in the discrete manifold representations of the infrared thermal imaging local feature encoding matrix and the fire scene 3D spatial local feature encoding matrix, manifold regularization optimization is performed on the trace metric operation of the infrared thermal imaging local feature encoding matrix and the fire scene 3D spatial local feature encoding matrix through manifold embedding compaction based on the number of interconnected components. This aims to improve the accuracy of the joint matrix trace metric calculation while enhancing robustness to far-affected correlations, as expressed by the formula:
[0053]
[0054] in, Dot product, and express and The number of interconnecting components between them This represents an exponential function with base e. Indicates optimization , Indicates optimization , This refers to the attention factor for optimizing the joint perception of infrared thermal imaging and three-dimensional spatial features of the fire scene.
[0055] That is, based on the initial and The calculated attention factor for joint perception of infrared thermal imaging and three-dimensional spatial features of the fire scene was obtained. Number of interconnect components and In the case of substitution , , ,initial and Optimize to obtain the optimized result. and, that is and And then according to Calculate the optimized trace metric, and then calculate the optimized matrix trace. As The optimized infrared thermal imaging-fire scene 3D spatial feature joint perception attention factor is obtained by normalization using the sigmoid activation function. Infrared thermal imaging - joint perception attention weight of three-dimensional spatial features of fire scene.
[0056] More specifically, step S422 is expressed by the formula:
[0057]
[0058] in, This indicates subtraction based on position. This indicates adding based on position points. and For different weight parameter matrices, This represents the feature matrix of local alignment and fusion in three-dimensional space between infrared thermal imaging and the fire scene. This represents a joint sensing feature map of heat distribution at a fire scene and three-dimensional space. , , and These respectively represent the 1st, 2nd, and 3rd features in the fire scene thermal distribution-3D spatial joint sensing feature map. The and the first A feature matrix of three-dimensional spatial local alignment and fusion of infrared thermal imaging and fire scene.
[0059] In other words, by dynamically guiding attention weights, the feature aggregation stage overcomes the limitations of traditional average pooling or stitching, achieving adaptive feature selection based on physical semantics. This involves assigning higher-weighted feature pairs a stronger contribution while suppressing the interference of lower-weighted feature pairs, thereby highlighting the thermal-structural coupling features of human targets in the joint sensing feature map of fire scene thermal distribution and 3D space. Specifically, through attention-driven significant aggregation, the local thermal radiation intensity distribution of infrared thermal imaging is cross-modal aligned with the spatial topological relationship of the 3D point cloud. Weight allocation is used to model the nonlinear dependency between the two, ensuring that the generated joint sensing feature map of fire scene thermal distribution and 3D space retains the temperature sensitivity of the infrared image while embedding geometric constraints in 3D space. This improves the signal-to-noise ratio of the target area, enhances the model's robustness to complex interference, and provides a multi-dimensional criterion for subsequent target detection that combines temperature anomaly detection and spatial localization.
[0060] In the aforementioned intelligent fire rescue method based on infrared thermal imaging, step S5 determines the human target detection result based on the fire scene thermal distribution-3D spatial joint sensing feature map. In a specific example of this application, step S5 includes: inputting the fire scene thermal distribution-3D spatial joint sensing feature map into a human target detection module based on the YOLOv5 model to obtain the human target detection result, which indicates whether a human target exists at the fire scene and the location of the human target. Specifically, to accurately determine whether a human target exists at the fire scene and to determine the specific location of the human target, this application uses the YOLOv5 model, which has excellent performance in the field of target detection, for human target detection. In this application, the YOLOv5 model adopts a single-stage target detection framework, transforming the target detection task into a regression problem. After inputting the fire scene thermal distribution-3D spatial joint perception feature map into the YOLOv5 model, the model extracts and transforms its features through a series of convolutional layers. It then uses an anchorbox mechanism to pre-define a series of anchor boxes of different sizes and aspect ratios, sliding these anchor boxes across the feature map. Regression is then used to predict the bounding box coordinates, confidence score, and class probability of each anchor box. Each bounding box contains the location information of a human target (such as center point coordinates, width, and height) and a confidence score, representing the probability of a human target being present within that bounding box. By setting an appropriate confidence threshold, high-confidence human target detection results can be filtered out, providing crucial information support for subsequent fire search and rescue operations and helping rescuers quickly locate trapped individuals.
[0061] Specifically, after receiving the joint sensing feature map of thermal distribution and 3D space at the fire scene, the YOLOv5 model extracts and transforms features from the image through a series of convolutional layers. In this process, the convolutional layers identify key features in the image, such as edges and textures, and convert them into higher-level abstract representations. These abstract features not only contain basic contour information of objects but also retain important details of temperature distribution and 3D spatial structure, which is crucial for distinguishing human targets in complex backgrounds. As the feature map is passed between network layers, the model gradually learns more refined feature representations, resulting in high accuracy and robustness in the final output.
[0062] In the YOLOv5 model, anchor boxes play a crucial role. A pre-defined series of anchor boxes of different sizes and aspect ratios slide across the feature map, covering as many possible target regions as possible. Each anchor box corresponds to a specific location and its surrounding environment. Regression is used to predict the bounding box coordinates, confidence score, and class probability of each anchor box. The location information of the bounding box includes the coordinates of its center point, width, and height; these parameters collectively define the specific location of a potential human target in the image. Meanwhile, the confidence score reflects the probability of a human target being present within the bounding box, providing a basis for filtering out regions that actually contain human targets.
[0063] Setting and optimizing the confidence score is a crucial step in ensuring the accuracy of human target detection results. In practice, the confidence threshold needs to be adjusted based on the specific fire scene conditions and the expected targets. Thresholds that are too high or too low will affect the quality of the final detection results: if the threshold is set too high, some actually present human targets may be missed; conversely, if the threshold is too low, it may lead to a large number of false alarms, that is, misidentifying non-human targets as human targets. Therefore, repeated trials are necessary in practice to find the optimal balance, ensuring that all trapped personnel are detected to the greatest extent possible while effectively reducing the occurrence of false alarms.
[0064] Furthermore, when using the YOLOv5 model for human target detection, it is also necessary to consider how to effectively combine the thermal distribution at the fire scene with information from three-dimensional space. Due to the unique characteristics of fire scenes, simple temperature or spatial structure information is often insufficient to fully describe the presence of a human target. For example, in some cases, high-temperature obstacles may generate thermal signals similar to those of a human body, while in other scenarios, objects with similar geometric shapes but not human-like may be mistaken for human targets.
[0065] Throughout the process, data preprocessing is equally crucial. High-quality input data is a prerequisite for ensuring the YOLOv5 model achieves maximum performance. For the joint sensing feature map of fire scene heat distribution and 3D space, it is necessary to ensure that the data undergoes appropriate normalization processing so that the model can better understand and process it. Furthermore, attention should be paid to the application of data augmentation techniques, such as rotation, scaling, and flipping, to generate more training samples, increase the model's generalization ability, and enable it to maintain stable performance in various fire environments.
[0066] In summary, the intelligent fire rescue method based on infrared thermal imaging, as described in this application, is explained. It utilizes an infrared thermal imager and a lidar to simultaneously acquire fire scene data, obtaining infrared thermal imaging data and three-dimensional point cloud data of the fire scene. Then, deep learning-based data processing technology is introduced to extract features from the infrared thermal imaging data and the three-dimensional point cloud data of the fire scene, capturing the infrared thermal radiation distribution characteristics and three-dimensional spatial structure characteristics of the fire scene. Furthermore, fine-grained feature alignment and joint perception of both are performed to capture the deep-level correlation between the thermal distribution and three-dimensional space of the fire scene, thereby achieving intelligent detection of human targets. This approach effectively overcomes the problem of the separation between thermal radiation information and spatial structure information in traditional methods, improving information utilization and enhancing the accuracy and efficiency of human target recognition.
[0067] Furthermore, an intelligent search and rescue device for fire personnel based on infrared thermal imaging is also provided.
[0068] Figure 7 This is a block diagram of an intelligent fire rescue device based on infrared thermal imaging, according to an embodiment of this application. Figure 7 As shown, the intelligent fire rescue device 100 based on infrared thermal imaging according to an embodiment of this application includes: a fire scene data acquisition module 110, used to simultaneously acquire fire scene data using an infrared thermal imager and a lidar to obtain infrared thermal imaging data and three-dimensional point cloud data of the fire scene; an infrared feature extraction module 120, used to extract infrared features from the infrared thermal imaging data to obtain an infrared thermal imaging feature map; a three-dimensional spatial feature extraction module 130, used to extract three-dimensional spatial features from the three-dimensional point cloud data of the fire scene to obtain a three-dimensional spatial feature map of the fire scene; a cross-modal joint perception module 140, used to perform cross-modal joint perception based on feature dynamic search alignment on the infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene to obtain a fire scene thermal distribution-three-dimensional spatial joint perception feature map; and a human target detection module 150, used to determine the human target detection result based on the fire scene thermal distribution-three-dimensional spatial joint perception feature map.
[0069] Here, those skilled in the art will understand that the specific operation of each module in the above-mentioned intelligent search and rescue device for fire personnel based on infrared thermal imaging has been referenced above. Figures 1 to 6 The method for intelligent search and rescue of people in fires based on infrared thermal imaging has been described in detail, and therefore, its repeated description will be omitted.
[0070] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details of the above embodiments are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the specific details described above.
[0071] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not detailed or described in a particular embodiment can be referred to in the relevant descriptions of other embodiments. In the several embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for example, the unit division is only a logical functional division, and other division methods may exist in actual implementation. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0072] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the present invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.
[0073] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units stated in a system claim may also be implemented by a single unit through software or hardware.
[0074] Finally, it should be noted that the above description has been given for illustrative and descriptive purposes. Furthermore, the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention.
Claims
1. A method for intelligent search and rescue of people in a fire based on infrared thermal imaging, characterized in that, include: Infrared thermal imager and lidar are used to simultaneously collect fire scene data to obtain infrared thermal imaging data and three-dimensional point cloud data of the fire scene; Infrared features are extracted from the infrared thermal imaging data to obtain an infrared thermal imaging feature map; Three-dimensional spatial features are extracted from the three-dimensional point cloud data of the fire scene to obtain a three-dimensional spatial feature map of the fire scene; The infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene are subjected to cross-modal joint sensing based on feature dynamic search alignment to obtain the fire scene thermal distribution-three-dimensional spatial joint sensing feature map. Based on the joint sensing feature map of the fire scene's thermal distribution and three-dimensional space, the detection results of human targets are determined.
2. The intelligent search and rescue method for personnel in a fire based on infrared thermal imaging according to claim 1, characterized in that, Extracting infrared features from the infrared thermal imaging data to obtain an infrared thermal imaging feature map includes: The infrared thermal imaging data is subjected to image denoising and edge detection processing to obtain enhanced infrared thermal imaging data; The enhanced infrared thermal imaging data is subjected to infrared feature extraction based on the MobileNet model to obtain the infrared thermal imaging feature map.
3. The intelligent search and rescue method for personnel in a fire based on infrared thermal imaging according to claim 2, characterized in that, Extracting three-dimensional spatial features from the three-dimensional point cloud data of the fire scene to obtain a three-dimensional spatial feature map of the fire scene includes: The three-dimensional spatial features of the fire scene are extracted based on the VoxelNet model to obtain the three-dimensional spatial feature map of the fire scene.
4. The intelligent search and rescue method for personnel in a fire based on infrared thermal imaging according to claim 3, characterized in that, The infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene are subjected to cross-modal joint sensing based on feature dynamic search alignment to obtain a joint sensing feature map of fire scene thermal distribution-three-dimensional space, including: The infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene are dynamically searched and aligned based on local decoupling features to obtain a set of phase-aligned feature pairs {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix}. The set of phase-aligned feature pairs {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix} is subjected to cross-modal sensing convergence based on an attention mechanism to obtain the joint sensing feature map of the fire scene thermal distribution-three-dimensional space.
5. The intelligent search and rescue method for personnel in a fire based on infrared thermal imaging according to claim 4, characterized in that, The infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene are dynamically searched and aligned based on local decoupling features to obtain a set of phase-aligned feature pairs {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix}, including: The infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene are subjected to feature orthogonal decomposition to obtain a set of local feature coding matrices of infrared thermal imaging and a set of local feature coding matrices of three-dimensional spatial feature map of the fire scene; Calculate the feature phase alignment between the i-th infrared thermal imaging local feature coding matrix in the set of infrared thermal imaging local feature coding matrices and each of the three-dimensional spatial local feature coding matrices of the fire scene in the set of three-dimensional spatial local feature coding matrices of the fire scene to obtain the set of infrared thermal imaging-fire scene three-dimensional spatial feature phase alignment. Extract the fire scene three-dimensional spatial local feature encoding matrix corresponding to the maximum value in the set of infrared thermal imaging-fire scene three-dimensional spatial feature phase alignment, and form a phase alignment feature pair with the i-th infrared thermal imaging local feature encoding matrix to obtain the phase alignment {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix} feature pair.
6. The intelligent search and rescue method for personnel in a fire based on infrared thermal imaging according to claim 5, characterized in that, Calculate the feature phase alignment between the i-th infrared thermal imaging local feature coding matrix in the set of infrared thermal imaging local feature coding matrices and each of the three-dimensional spatial local feature coding matrices of the fire scene in the set of three-dimensional spatial local feature coding matrices of the fire scene to obtain a set of infrared thermal imaging-fire scene three-dimensional spatial feature phase alignment, including: Calculate the product of the transpose of the i-th infrared thermal imaging local feature coding matrix and the three-dimensional spatial local feature coding matrix of the fire scene to obtain the joint sensing coding matrix of infrared thermal imaging-three-dimensional spatial local features of the fire scene; The Frobenius norm of the joint sensing coding matrix of the infrared thermal imaging-fire scene three-dimensional spatial local features is calculated to obtain the phase alignment of the infrared thermal imaging-fire scene three-dimensional spatial features.
7. The intelligent search and rescue method for personnel in a fire based on infrared thermal imaging according to claim 6, characterized in that, The set of phase-aligned feature pairs {infrared thermal imaging local feature encoding matrix, fire scene 3D spatial local feature encoding matrix} is subjected to attention-based cross-modal sensing convergence to obtain the fire scene thermal distribution-3D spatial joint sensing feature map, including: Each phase-aligned feature pair in the set of phase-aligned {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix} is input into the cross-modal sensing focusing module to obtain a set of joint sensing attention weights for infrared thermal imaging and fire scene three-dimensional spatial features. Based on the set of attention weights for the joint perception of infrared thermal imaging and three-dimensional spatial features of the fire scene, the set of phase-aligned feature pairs {infrared thermal imaging local feature encoding matrix, fire scene three-dimensional spatial local feature encoding matrix} is subjected to focused modulation feature development to obtain the joint perception feature map of the fire scene thermal distribution and three-dimensional space.
8. The intelligent search and rescue method for personnel in a fire based on infrared thermal imaging according to claim 7, characterized in that, Each phase-aligned feature pair in the set of phase-aligned {infrared thermal imaging local feature encoding matrix, fire scene 3D spatial local feature encoding matrix} is input into the cross-modal sensing focusing module to obtain a set of joint sensing attention weights for infrared thermal imaging and fire scene 3D spatial features, including: Calculate the infrared thermal imaging-fire scene three-dimensional spatial local feature joint sensing coding matrix between the infrared thermal imaging local feature coding matrix and the fire scene three-dimensional spatial local feature coding matrix in the phase-aligned feature pair {infrared thermal imaging local feature coding matrix, fire scene three-dimensional spatial local feature coding matrix}. Based on the stationary feature distribution of the infrared thermal imaging local feature coding matrix and the fire scene three-dimensional spatial local feature coding matrix in the phase-aligned feature pair {infrared thermal imaging local feature coding matrix, fire scene three-dimensional spatial local feature coding matrix}, the matrix trace of the infrared thermal imaging-fire scene three-dimensional spatial local feature joint sensing coding matrix is optimized by manifold regularization to obtain the optimized infrared thermal imaging-fire scene three-dimensional spatial feature joint sensing attention factor. The optimized infrared thermal imaging-fire scene 3D spatial feature joint perception attention factor is input into the sigmoid activation function to obtain the infrared thermal imaging-fire scene 3D spatial feature joint perception attention weight.
9. The intelligent search and rescue method for personnel in a fire based on infrared thermal imaging according to claim 8, characterized in that, Based on the fire scene thermal distribution-3D spatial joint sensing feature map, the human target detection results are determined, including: The fire scene heat distribution-3D spatial joint perception feature map is input into the human target detection module based on the YOLOv5 model to obtain the human target detection result. The human target detection result is used to indicate whether there is a human target at the fire scene and the location of the human target.
10. A fire rescue intelligent personnel search and rescue device based on infrared thermal imaging, characterized in that, include: The fire scene data acquisition module is used to simultaneously acquire fire scene data using an infrared thermal imager and a lidar to obtain infrared thermal imaging data and three-dimensional point cloud data of the fire scene. An infrared feature extraction module is used to extract infrared features from the infrared thermal imaging data to obtain an infrared thermal imaging feature map. The three-dimensional spatial feature extraction module is used to extract three-dimensional spatial features from the three-dimensional point cloud data of the fire scene to obtain a three-dimensional spatial feature map of the fire scene. A cross-modal joint sensing module is used to perform cross-modal joint sensing based on feature dynamic search alignment on the infrared thermal imaging feature map and the three-dimensional spatial feature map of the fire scene to obtain a fire scene thermal distribution-three-dimensional spatial joint sensing feature map. The human target detection module is used to determine the human target detection result based on the joint perception feature map of the fire scene's thermal distribution and three-dimensional space.
Citation Information
Cited By
Control system and method of percutaneous surgical robot
CN120241258A
Vital sign perception method and system based on time sequence infrared image
CN121964030A
Fire scene trapped person multi-modal recognition method and system based on YOLOv8
CN122618663A
