Image enhancement method and device of fire-fighting search and rescue infrared thermal imager

Abstract

The invention relates to the technical field of image enhancement, and particularly discloses an image enhancement method and device for a fire-fighting search and rescue infrared thermal imager, and the method comprises the steps: collecting original RAW infrared image data through the infrared thermal imager; after non-uniformity correction processing is carried out on the infrared image, a deep learning-based image processing algorithm is further introduced to carry out pixel-level feature learning and temperature partition identification so as to generate a temperature partition probability distribution diagram, and a flexible linear mapping mechanism is introduced to carry out linear mapping processing on the gray value of each pixel point of the infrared image so as to generate a temperature partition probability distribution diagram; therefore, the display effect of the infrared image is optimized, and enhanced infrared image data is obtained. According to the method, the gray distribution of different temperature regions can be adjusted, the contrast and detail visibility of the infrared image are improved, artifacts and splitting phenomena caused by a traditional fixed threshold partitioning method are effectively avoided, and the image quality and readability are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of image enhancement technology, and more specifically, to an image enhancement method and device for a firefighting and search and rescue infrared thermal imager. Background Art

[0002] In firefighting and search and rescue scenarios, rescuers must quickly and accurately locate the source of the fire and trapped individuals. Infrared thermal imagers, as key equipment, detect infrared radiation emitted by objects and convert it into visual images, helping rescuers penetrate harsh environments such as smoke and darkness to obtain vital information. However, due to the complexity of the environment and the limitations of the equipment itself, images captured by infrared thermal imagers often suffer from excessive noise, low contrast, and blurred details, which seriously affect image quality and readability, reducing the efficiency and accuracy of rescue operations.

[0003] Specifically, traditional infrared thermal imaging systems typically use 14-bit or 16-bit sensors to collect raw data (RAW data). The upper limit of its dynamic range is limited by the physical properties of the sensor. If the full temperature range (e.g., 0 to 16,383 grayscale levels) is directly linearly mapped to an 8-bit display space (0 to 255), the high radiation intensity in the high-temperature fire area will occupy most of the grayscale resources, causing the low-temperature range corresponding to the human target to be compressed into an extremely narrow grayscale band (e.g., covering only a few dozen grayscale levels). This compression weakens the temperature difference signal between the human body and the background, making detailed features (such as body contours and posture) difficult to identify, seriously reducing search and rescue efficiency.

[0004] In existing techniques, images are typically divided into high-temperature and low-temperature zones using a preset fixed temperature threshold, and linear stretching is performed on each zone to adjust grayscale resource utilization. For example, the grayscale values ​​corresponding to the human body temperature range (e.g., 30°C to 45°C) are linearly stretched to expand the grayscale distribution range and improve contrast. High-temperature zones (e.g., >200°C) in fire scenes are slope-compressed to reduce the grayscale level occupied and avoid overexposure. However, the temperature distribution in fire scenes is highly dynamic, and methods based on fixed temperature threshold image temperature partitioning struggle to adapt to real-time changes in fire intensity, resulting in poor image enhancement. Furthermore, the transition between high-temperature and low-temperature zones in actual infrared images often exhibits a continuous gradient (e.g., temperature decay at the edge of a flame, or heat conduction between a human body and the hot background), lacking a clear physical boundary. When forced temperature partitioning using a fixed threshold, adjacent pixels may be assigned to different mapping intervals due to slight temperature differences, resulting in sudden grayscale value changes (e.g., a sudden change from a slope of 1.5 to 0.5). This manifests as artifactual streaks or fragmented regions in the image, severely interfering with rescuers' understanding of the scene structure.

[0005] Therefore, an optimized image enhancement method and device for firefighting and search and rescue infrared thermal imagers are desired. Summary of the Invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an image enhancement method and device for a fire-fighting search and rescue infrared thermal imager, which uses an infrared thermal imager to collect original RAW infrared image data, performs non-uniformity correction processing on it, and further introduces an image processing algorithm based on deep learning to perform pixel-level feature learning and temperature partition recognition on the corrected infrared image data to generate a temperature partition probability distribution map. On this basis, a flexible linear mapping mechanism is introduced, and its linear mapping parameters are calculated based on the temperature partition probability distribution of each pixel point of the infrared image, and the grayscale value of each pixel point of the infrared image is linearly mapped, thereby dynamically allocating grayscale resources and optimizing the display effect of the infrared image, thereby obtaining enhanced infrared image data. This method can adaptively adjust the grayscale distribution of different temperature areas, while improving the contrast and detail visibility of the infrared image, effectively avoiding the artifacts and fragmentation caused by the traditional fixed threshold partitioning method, and significantly improving the image quality and readability.

[0007] According to one aspect of the present application, a method for enhancing an image of a firefighting search and rescue infrared thermal imager is provided, comprising: Obtain the original RAW infrared image data collected by the infrared thermal imager; Performing non-uniformity correction processing on the original RAW infrared image data to obtain corrected infrared image data; Performing temperature partition processing on the corrected infrared image data to obtain a temperature partition probability distribution map; Based on the infrared image temperature partition probability distribution map, local flexible linear mapping processing is performed on the corrected infrared image data to obtain enhanced infrared image data.

[0008] According to another aspect of the present application, an image enhancement device for a firefighting search and rescue infrared thermal imager is provided, comprising: Infrared image data acquisition module, used to obtain the original RAW infrared image data collected by the infrared thermal imager; a non-uniformity correction module, configured to perform non-uniformity correction processing on the original RAW infrared image data to obtain corrected infrared image data; A temperature partition processing module, configured to perform temperature partition processing on the corrected infrared image data to obtain a temperature partition probability distribution map; The local flexible linear mapping processing module is used to perform local flexible linear mapping processing on the corrected infrared image data based on the infrared image temperature partition probability distribution map to obtain enhanced infrared image data.

[0009] Compared with the existing technology, the image enhancement method and device for fire-fighting search and rescue infrared thermal imagers provided in this application use an infrared thermal imager to collect original RAW infrared image data, perform non-uniformity correction processing on it, and further introduce an image processing algorithm based on deep learning to perform pixel-level feature learning and temperature partition recognition on the corrected infrared image data to generate a temperature partition probability distribution map. On this basis, a flexible linear mapping mechanism is introduced to calculate its linear mapping parameters based on the temperature partition probability distribution of each pixel point of the infrared image, and linear mapping processing is performed on the grayscale value of each pixel point of the infrared image, thereby dynamically allocating grayscale resources and optimizing the display effect of the infrared image, thereby obtaining enhanced infrared image data. This method can adaptively adjust the grayscale distribution of different temperature zones. While improving the contrast and detail visibility of the infrared image, it effectively avoids the artifacts and fragmentation caused by the traditional fixed threshold partitioning method, significantly improving the image quality and readability. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 The figure is a flow chart of an image enhancement method for a fire-fighting search and rescue infrared thermal imager according to an embodiment of the present application.

[0012] Figure 2 Schematic diagram of data flow of an image enhancement method for a firefighting search and rescue infrared thermal imager according to an embodiment of the present application.

[0013] Figure 3 This is a flowchart of sub-step S3 of the image enhancement method of the fire-fighting search and rescue infrared thermal imager according to an embodiment of the present application.

[0014] Figure 4 This is a flowchart of sub-step S32 of the image enhancement method of the fire-fighting search and rescue infrared thermal imager according to an embodiment of the present application.

[0015] Figure 5 This is a flowchart of sub-step S322 of the image enhancement method of the fire-fighting search and rescue infrared thermal imager according to an embodiment of the present application.

[0016] Figure 6 This is a flowchart of sub-step S4 of the image enhancement method of the fire-fighting search and rescue infrared thermal imager according to an embodiment of the present application.

[0017] Figure 7This is a block diagram of an image enhancement device for a firefighting search and rescue infrared thermal imager according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0019] Although the present application makes various references to certain modules in the system according to embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are illustrative only, and different aspects of the system and method can use different modules.

[0020] Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the various steps may be processed in reverse order or simultaneously, as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0021] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0022] It is worth noting that in this application, all actions to obtain data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0023] In response to the technical problems described in the above background technology, this application proposes an image enhancement method for firefighting and rescue infrared thermal imagers, which uses an infrared thermal imager to collect original RAW infrared image data, performs non-uniformity correction processing on it, and further introduces an image processing algorithm based on deep learning to perform pixel-level feature learning and temperature partition recognition on the corrected infrared image data to generate a temperature partition probability distribution map. On this basis, a flexible linear mapping mechanism is introduced, and its linear mapping parameters are calculated based on the temperature partition probability distribution of each pixel point of the infrared image. The grayscale value of each pixel point of the infrared image is linearly mapped, thereby dynamically allocating grayscale resources and optimizing the display effect of the infrared image, thereby obtaining enhanced infrared image data. This method can adaptively adjust the grayscale distribution of different temperature areas, while improving the contrast and detail visibility of the infrared image, effectively avoiding the artifacts and fragmentation caused by the traditional fixed threshold partitioning method, and significantly improving the image quality and readability.

[0024] Figure 1 The figure is a flow chart of an image enhancement method for a fire-fighting search and rescue infrared thermal imager according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the image enhancement method of the fire rescue infrared thermal imager according to the embodiment of the present application. Figure 1 and Figure 2 As shown, the image enhancement method of the fire-fighting search and rescue infrared thermal imager includes the following steps: S1, acquiring original RAW infrared image data collected by the infrared thermal imager; S2, performing non-uniformity correction processing on the original RAW infrared image data to obtain corrected infrared image data; S3, performing temperature partition processing on the corrected infrared image data to obtain a temperature partition probability distribution map; S4, based on the infrared image temperature partition probability distribution map, performing local flexible linear mapping processing on the corrected infrared image data to obtain enhanced infrared image data.

[0025] In the above-mentioned image enhancement method for firefighting search and rescue infrared thermal imagers, step S1 obtains the raw RAW infrared image data captured by the infrared thermal imager. It should be understood that infrared thermal imagers are the primary device for acquiring infrared radiation information at a fire scene. The physical meaning of the RAW data directly outputted by them is the quantified result of the infrared radiation intensity of each sensor pixel, including the unprocessed raw radiation response value (DN value). As the most direct and original source of information, raw RAW infrared image data retains all infrared radiation information sensed by the detector without any lossy processing, avoiding compression or distortion of temperature information in preprocessing steps (such as the ISP curing algorithm), and can reflect the actual heat distribution at the fire scene. Specifically, the infrared thermal imager's detector is composed of numerous detection units, each of which can sense infrared radiation emitted by an object and convert it into an electrical signal. After analog-to-digital conversion, the signal is recorded as a digital signal and stored in RAW format.

[0026] In practice, raw RAW infrared image data provides the most direct and original source of information at the fire scene. This means that each RAW image is a true reflection of the actual ambient temperature distribution, without any lossy processing, ensuring that all captured infrared radiation information is fully preserved. The process of acquiring raw RAW infrared image data involves not only hardware operations but also a series of sophisticated software algorithms to ensure data validity and accuracy. For example, during the design phase of an infrared thermal imager, special attention is paid to key performance indicators such as detector sensitivity, resolution, and dynamic range. High sensitivity means that the detector can maintain a high signal-to-noise ratio even at extremely low light intensities, which is particularly important for detecting subtle temperature differences in harsh environments such as smoke and low light. High resolution ensures that the final image has sufficient detail, allowing rescuers to more clearly identify the target object. Dynamic range determines the temperature range within which the detector can effectively operate, directly affecting the instrument's ability to adapt to complex and changing fire conditions.

[0027] Furthermore, to ensure the quality of raw infrared image data, the impact of external factors must be considered during the acquisition process. For example, ambient temperature, humidity, and the temperature of the detector itself can all affect the accuracy of detection results. To this end, modern infrared thermal imagers are typically equipped with temperature compensation mechanisms that adjust the detector's operating state in real time to offset interference from these external factors. Furthermore, to prevent the adverse effects of electromagnetic interference on signal transmission, internal shielding measures are implemented and circuit design is optimized to enhance the system's anti-interference capabilities.

[0028] When acquiring raw infrared image data, it's important to note that while the detector can quantify the intensity of the received infrared radiation into specific numerical values ​​(DN values), these values ​​do not represent actual temperature values. Instead, they are relative measurements related to factors such as the object's emissivity and distance. Therefore, subsequent processing steps require inferring the corresponding temperature distribution based on specific application scenarios and requirements, combined with other auxiliary information (such as object material properties and environmental conditions).

[0029] In the above-mentioned image enhancement method for a firefighting and rescue infrared thermal imager, step S2 performs non-uniformity correction on the raw RAW infrared image data to obtain corrected infrared image data. It should be understood that due to manufacturing process limitations of infrared detectors, the response characteristics of each detection unit are inconsistent. Even under the same infrared radiation conditions, the electrical signals output by different detection units can vary. This variation causes the captured raw RAW infrared image to contain non-uniform noise, manifesting as grid-like artifacts or regional grayscale anomalies in the image, making the temperature distribution in the image inaccurately reflect the actual temperature of the object. Therefore, to remove or significantly reduce the non-uniform noise in the infrared image and ensure that the grayscale value of each pixel accurately reflects the actual temperature of the object, the present application further performs non-uniformity correction on the raw RAW infrared image data. Specifically, the present application uses a two-point correction method to correct the raw RAW infrared image data, which is based on the assumption that the detector's response characteristics are linear within a certain range. First, the detector acquires two blackbody radiation images of known temperatures, representing the lower and upper limits of the detector's operating temperature range. By measuring the detector's response at these two temperatures, a linear model is established for the response and temperature of each detection unit. For each pixel in the original RAW image, a corrected grayscale value is calculated based on the linear model of its corresponding detection unit. This mathematical model compensates for the response differences between detection units and improves the accuracy of temperature inversion.

[0030] In the above-mentioned image enhancement method of the fire-fighting search and rescue infrared thermal imager, the step S3 performs temperature zoning processing on the corrected infrared image data to obtain a temperature zoning probability distribution map. Specifically, since the traditional temperature zoning and linear mapping methods based on fixed thresholds are difficult to deal with the ambiguity of the transition zone of the fire scene temperature distribution (such as the heat conduction boundary between the human body and the high-temperature background), the image enhancement effect is poor and a fragmentation phenomenon occurs. To address this problem, the present application introduces a deep learning algorithm to perform temperature zoning processing on the corrected infrared image data, so as to perform probability modeling on the pixel-level temperature distribution in the corrected infrared image data through a deep learning model, thereby adaptively dividing the temperature intervals and quantifying the attribution weights of the transition zone, avoiding grayscale mutations caused by hard threshold segmentation. Among them, Figure 3FIG. 1 is a flowchart of sub-step S3 of the image enhancement method of the fire-fighting search and rescue infrared thermal imager according to an embodiment of the present application. Figure 3 As shown, the step S3 includes the following steps: S31, extracting the infrared features of the corrected infrared image data to obtain an infrared image feature map; S32, performing feature hierarchy enhancement processing on the infrared image feature map based on collaborative modeling of spatial topology and semantic structure to obtain an infrared image enhanced feature map; S33, inputting the infrared image enhanced feature map into a temperature partitioning module based on a softmax function to obtain the infrared image temperature partition probability distribution map.

[0031] Specifically, in one specific example of this application, step S31 includes performing infrared feature extraction on the corrected infrared image data using an FPN network to obtain the infrared image feature map. It should be understood that temperature distribution in firefighting scenarios exhibits significant multi-scale characteristics. For example, human targets (such as trapped individuals) typically exhibit localized moderate-temperature regions (30°C-45°C), while the core of a fire (>500°C) may encompass a large, high-temperature region, along with gradient temperature bands at the edges of the flames. Traditional single-scale convolutional networks struggle to simultaneously capture both local temperature differences and global temperature distribution patterns. Therefore, this application introduces a feature pyramid network (FPN) to perform multi-scale feature extraction and fusion on the corrected infrared image data. This method fully utilizes temperature feature information at different scales, taking into account both pixel-level temperature sensitivity and regional temperature correlation, resulting in an infrared image feature map that provides a hierarchical feature representation for subsequent temperature zoning. Specifically, the FPN network constructs a pyramidal structure consisting of bottom-up base convolutional layers, top-down upsampling layers, and lateral connections. The bottom-up path progressively downsamples the input rectified infrared image through convolutional layers, extracting low-level features at different scales and capturing detailed image information. The top-down path fuses high-level semantic features with features of the same scale from the bottom-up path through upsampling operations. Horizontal connections concatenate features of the same scale, allowing features at different scales to complement each other. This approach enables the learning of rich temperature distribution features at different scales, ensuring accurate identification of temperature targets of varying scales and improving the comprehensiveness and accuracy of temperature zoning.

[0032] Specifically, the step S32 performs feature-level enhancement processing on the infrared image feature map based on the collaborative modeling of spatial topology and semantic structure to obtain an infrared image enhanced feature map. It should be understood that since the noise and interference of firefighting infrared images have spatial heterogeneity, for example, high-temperature fire areas produce shot noise due to the violent fluctuation of thermal radiation, and low-temperature backgrounds (such as walls and ground) are affected by the dark current of the sensor and present fixed pattern noise. Therefore, in order to further improve the distinction between key targets (such as trapped persons, fire sources, etc.) and noise, the present application proposes a feature-level enhancement processing method based on the collaborative modeling of spatial topology and semantic structure, by explicitly modeling the spatial correlation (such as the continuity of temperature changes across pixels) and semantic context (such as the temperature difference between the target and the background) of the infrared image feature map, thereby enhancing the smoothness of the temperature continuous area, suppressing isolated noise points, and improving the accuracy of temperature partitioning. Among them, Figure 4 FIG3 is a flowchart of sub-step S32 of the image enhancement method of the fire-fighting search and rescue infrared thermal imager according to an embodiment of the present application. Figure 4 As shown, the step S32 includes the steps of: S321, extracting the channel feature vector of the (i, j)th pixel position from the infrared image feature map as the channel feature vector of the infrared image to be enhanced; S322, based on the infrared image feature map, performing information residual modeling based on spatial-semantic dual-domain context-driven on the channel feature vector of the infrared image to be enhanced to obtain the implicit enhancement component coding vector of the channel to be enhanced of the infrared image; S323, fusing the implicit enhancement component coding vector of the channel to be enhanced of the infrared image and the channel feature vector of the infrared image to be enhanced to obtain the enhanced infrared image channel feature vector, wherein the enhanced infrared image channel feature vector is the channel feature vector of the pixel position (i, j) of the infrared image enhancement feature map.

[0033] More specifically, step S321 is expressed as follows: ; ;

[0034] in, 、 and Respectively represent the height, width and number of channels of the infrared image feature map, represents the infrared image feature map, Represents the channel feature vector of the (i, j)th pixel position of the infrared image feature map, Represents the channel feature vector of the infrared image to be enhanced.

[0035] That is, in order to reconstruct the grayscale resource allocation strategy of infrared images through dynamic mapping at the feature level, the present application replaces the global threshold rule with the local feature expression capability to avoid grayscale mutations caused by continuous changes in temperature gradients. Here, the channel feature vector of the infrared image to be enhanced can be the channel feature representation at any pixel position in the infrared image feature map. By processing the channel feature vector at each pixel position in the infrared image feature map one by one, the feature-level refined enhancement is achieved.

[0036] Figure 5 FIG3 is a flowchart of sub-step S322 of the image enhancement method of the fire-fighting search and rescue infrared thermal imager according to an embodiment of the present application. Figure 5 As shown, the step S322 includes the steps of: S3221, performing n random traversal sampling on the infrared image feature map to obtain n channel feature vectors as a sparse set of infrared image context feature vectors; S3222, calculating the information residual feature vector between each infrared image context feature vector in the sparse set of infrared image context feature vectors and the infrared image channel feature vector to be enhanced to obtain a set of information residual feature vectors of the infrared image channel to be enhanced; S3223, performing spatial-semantic dual-domain coupling analysis on the infrared image channel feature vector to be enhanced and the sparse set of infrared image context feature vectors to obtain a set of spatial modulation matrix of infrared image channel feature to be enhanced and semantic association coding matrix of infrared image channel feature to be enhanced; S3224, based on the set of spatial modulation matrix of infrared image channel feature to be enhanced and semantic association coding matrix of infrared image channel feature to be enhanced, performing structured fusion coding on the set of information residual feature vectors of the infrared image channel to be enhanced to obtain the implicit enhancement component coding vector of the infrared image channel to be enhanced.

[0037] In a specific example of the present application, step S3221 is expressed as follows: ; ;

[0038] in, 、 、 and Respectively represent the first, second, and third sparse sets of infrared image context feature vectors and infrared image context feature vector, Represents a sparse set of context feature vectors of infrared images.

[0039] That is, the infrared image feature map is randomly sampled at multiple locations through n random traversal samplings to construct a sparse set of infrared image context feature vectors. By introducing spatial diversity and randomness, the selective attention mechanism of biological vision is simulated, which not only avoids local overfitting that may be caused by fixed sampling, but also significantly reduces the computational load while retaining key features through sparsity, allowing the model to focus on representative local patterns.

[0040] In a specific example of the present application, step S3222 is expressed as follows: ;

[0041] in, express Relative to The residual feature vector of the channel information to be enhanced of the infrared image.

[0042] That is, by calculating the difference between each infrared image context feature vector and the feature vector of the infrared image channel to be enhanced, the relative changes and feature differences of pixels at different positions in the infrared image feature map relative to the pixels to be enhanced are revealed, and the local context information in the infrared image feature map is captured, thereby providing an information compensation basis for subsequent feature level enhancement processing.

[0043] In a specific example of the present application, step S3223 includes: first, calculating the Poincare measure between the infrared image channel feature vector to be enhanced and each infrared image context feature vector in the sparse set of infrared image context feature vectors to obtain the infrared image channel feature space modulation matrix to be enhanced composed of multiple Poincare measures, which is expressed as follows: ; ;

[0044] in, represents the square of the norm of the vector, represents the inverse hyperbolic cosine function, express and The Poincare measure between express and The Poincare measure between Represents the spatial modulation matrix of the channel features to be enhanced in the infrared image.

[0045] Here, since the context enhancement compensation information of each infrared image context feature vector relative to the infrared image channel feature vector to be enhanced is not completely useful. Therefore, the present application hopes to further utilize the feature correlation relationship between each infrared image context feature vector and the infrared image channel feature vector to be enhanced to perform refined screening and weight distribution on the set of residual feature vectors of the infrared image channel to be enhanced. Here, the calculation of the Poincare measure is introduced to more accurately characterize the hierarchical relationship between different pixel areas through the hyperbolic space metric characteristics, and the continuously changing temperature field in the infrared image is regarded as a feature set with spatial correlation. By quantifying the similarity between the infrared image channel feature vector to be enhanced and the infrared image context feature vector in the hyperbolic space, a feature space modulation matrix of the infrared image channel to be enhanced that reflects the local temperature distribution law is generated to construct a dynamic spatial attention mechanism.

[0046] Then, the implicit semantic association matrix between the infrared image channel feature vector to be enhanced and each infrared image context feature vector in the sparse set of the infrared image context feature vector is calculated to obtain a set of semantic association coding matrices of the infrared image channel feature to be enhanced, which is expressed as follows: ;

[0047] in, represents the weight matrix, represents the transpose of a vector, represents vector multiplication, represents the normalized exponential function, express and Semantic correlation coding matrix of channel features to be enhanced in infrared images.

[0048] Specifically, by mining the implicit semantic associations between the infrared image channel feature vectors for enhancement and the infrared image context feature vectors, the limitations of simple temperature values ​​are overcome and an association model based on high-level semantic concepts is established. Specifically, by quantifying the high-level similarities between the infrared image channel feature vectors for enhancement and the infrared image context feature vectors, the complex physical phenomena in fire scenes are transformed into interpretable semantic labels. This semantic association-based weighting mechanism accurately models complex scenarios such as flame edge attenuation and heat conduction between the human body and the high-temperature background, without relying on a fixed temperature threshold. The resulting semantic association encoding matrix for the infrared image channel features for enhancement leverages the ability of implicit semantics to capture deep feature patterns. This matrix can distinguish targets with similar temperatures but different physical properties within a fire scene, thereby improving rescuers' accuracy in identifying key targets. Furthermore, by strengthening the semantic consistency of target areas, the structural interpretability of the image is significantly improved, ultimately providing more practical visual perception support for firefighting and search and rescue operations.

[0049] In a specific example of the present application, step S3224 includes: first, based on the set of the infrared image channel feature spatial modulation matrix and the infrared image channel feature semantic association coding matrix, each infrared image channel information residual feature vector in the set of the infrared image channel information residual feature vector to be enhanced is subjected to dual-constraint optimization iteration to obtain a set of optimized infrared image channel information residual feature vectors to be enhanced, which is expressed as follows: ; ;

[0050] in, express The corresponding semantic nested optimization infrared image channel information residual feature vector to be enhanced, express The corresponding optimized residual feature vector of the channel information to be enhanced in the infrared image.

[0051] Specifically, by defining the spatial modulation matrix of the infrared image channel features to be enhanced as the underlying physical constraint field, a non-Euclidean metric framework is used to constrain the differential continuity level of the local structure, ensuring the smooth topological invariance of the grayscale mapping in physical space. Furthermore, the semantic association encoding matrix of the infrared image channel features to be enhanced is used as the adaptive transformation field. Through the covariant derivative mechanism on curved space, the high-level semantic features are associated with the spatial deformation of the underlying temperature field, ensuring that semantic consistency is not destroyed during the grayscale enhancement process. Specifically, through this dual constraint approach, namely the synergy of the basic gradient operator and the adaptive gradient operator, a deep fusion of physical space geometric constraints and semantic space topological constraints is achieved. The underlying physical constraint field constrains the local geometric structure of the temperature field, enabling the flexible linear mapping to adapt to the dynamic nonlinear response of complex temperature distributions and suppressing the distortion effects of noise amplification and overexposed areas. The adaptive transformation field, through the covariant transformation characteristics of semantic associations, couples abstract semantic patterns with physical space deformation, ensuring the coherence of the deep features of key targets during grayscale enhancement. The nested constraints of the two interact through gradient operators in the form of partial derivatives, unifying geometric constraints and semantic constraints in the global optimization framework of information residual features, avoiding the limitations of single constraints and enhancing the adaptability of information encoding to multimodal data features. The generated optimized infrared image channel information residual feature vector can adaptively balance the local details and global distribution of the temperature field, optimize the dynamic scheduling efficiency of grayscale resources, thereby improving the contrast, detail visibility and semantic interpretability of infrared images in complex scenes, and providing more accurate visual perception support for subsequent rescue decisions.

[0052] Then, each infrared image channel feature semantic association coding matrix in the set of the infrared image channel feature semantic association coding matrices to be enhanced is used as a primary mask operator, and the infrared image channel feature spatial modulation matrix to be enhanced is used as a secondary mask operator. The set of the optimized infrared image channel information residual feature vectors to be enhanced is subjected to structured fusion coding to obtain the implicit enhancement component coding vector of the infrared image channel to be enhanced, which is expressed as follows: ;

[0053] in, Represents the implicit enhancement component encoding vector of the channel to be enhanced in the infrared image.

[0054] Specifically, the semantic association coding matrix of the infrared image channel features to be enhanced is used as a first-level mask. The semantic mask is used to constrain the feature components in the residual feature vector of the optimized infrared image channel information to be enhanced that are highly semantically related to the infrared image channel feature vector to be enhanced, thereby enhancing the expressiveness of these components in the final enhancement process. The spatial modulation matrix of the infrared image channel features to be enhanced is used as a second-level mask. Based on the spatial correlation of the pixel neighborhood, the feature components in the residual feature vector of the optimized infrared image channel information to be enhanced that are closely related to the pixel position of the infrared image channel feature vector to be enhanced are constrained to ensure that spatial continuity is not destroyed during the enhancement process. Through the synergistic effect of the first-level mask and the second-level mask, dual modulation at the semantic level and the spatial level is achieved, which not only ensures the effective enhancement of key semantic features, but also maintains the spatial structure information of the image.

[0055] More specifically, in a specific example of the present application, step S323 includes: calculating element-wise weighted fusion between the implicit enhancement component coding vector of the infrared image channel to be enhanced and the feature vector of the infrared image channel to be enhanced to obtain the enhanced infrared image channel feature vector, which is expressed as: ;

[0056] in, and Represents different fusion weight parameters, Represents the enhanced infrared image channel feature vector.

[0057] That is, by fusing the implicit enhancement component coding vector of the infrared image channel to be enhanced and the infrared image channel feature vector to be enhanced to generate an enhanced infrared image channel feature vector, the original infrared radiation intensity information is retained and the context reference information of the infrared image feature map is incorporated, so that the final generated enhanced infrared image channel feature vector can significantly improve the expression ability of temperature range characteristics in complex scenes while maintaining physical authenticity.

[0058] Specifically, in step S33, the infrared image enhancement feature map is input into a temperature partitioning module based on a softmax function to obtain the infrared image temperature partition probability distribution map. It should be understood that the softmax function, as a commonly used multi-classification function, can convert the input feature vector into a probability distribution, so that the probability value of each category is between 0 and 1, and the sum of the probability values ​​of all categories is 1. In the present application, the channel feature vector of each pixel point in the infrared image enhancement feature map is used as the input of the softmax function. After calculation by the softmax function, the probability distribution of each pixel point belonging to a different temperature partition (such as a low-temperature background area, a human body temperature area, a medium-temperature fire area, and a high-temperature fire area) is obtained, thereby generating an infrared image temperature partition probability distribution map. The infrared image temperature partition probability distribution map not only reflects the spatial distribution of the fire scene temperature, but also reflects the uncertainty of the temperature attribution of each pixel point, providing a more refined data basis for subsequent grayscale value mapping.

[0059] In the above-mentioned image enhancement method of the fire-fighting search and rescue infrared thermal imager, the step S4 performs local flexible linear mapping processing on the corrected infrared image data based on the infrared image temperature partition probability distribution map to obtain enhanced infrared image data. It should be understood that the traditional fixed threshold partitioning method causes mapping jumps due to the blurred temperature boundary, and the fire scene requires the grayscale resource allocation to have dynamic adaptability and continuity. The present application converts the temperature partition into a probability distribution problem through the Softmax function, so that each pixel can belong to multiple temperature intervals at the same time (such as a high-temperature edge pixel may belong to the high temperature zone with a probability of 60% and the medium temperature zone with a probability of 40%). On this basis, the present application proposes a local flexible linear mapping strategy, which dynamically adjusts the grayscale value mapping relationship of each pixel according to the temperature partition probability distribution of each pixel, thereby allowing a smoother transition in the grayscale mapping process. Among them, Figure 6 FIG4 is a flowchart of sub-step S4 of the image enhancement method of the fire-fighting search and rescue infrared thermal imager according to an embodiment of the present application. Figure 6 As shown, the step S4 includes the following steps: S41, setting the initial linear mapping parameters of the low-temperature background area, the human body temperature area, the medium-temperature fire area and the high-temperature fire area to obtain the first to fourth initial linear mapping parameters; S42, based on the temperature partition probability distribution of each pixel point in the temperature partition probability map, weightedly fusing the first to fourth initial linear mapping parameters to obtain the flexible adjustment linear mapping parameters of each pixel point; S43, linearly mapping the grayscale values ​​of each pixel point in the corrected infrared image data based on the flexible adjustment linear mapping parameters of each pixel point to obtain the enhanced infrared image data.

[0060] Specifically, the step S41 sets the initial linear mapping parameters of the low-temperature background area, the human body temperature area, the medium-temperature fire area, and the high-temperature fire area to obtain the first to fourth initial linear mapping parameters. Specifically, in order to perform reasonable grayscale value mapping on the pixels in different temperature areas, a preliminary grayscale mapping rule is first set for each temperature area, and the basic direction and degree of grayscale value adjustment of each area are determined as the starting point for subsequent dynamic adjustment. For example, the initial linear mapping parameter k=0.5 for the low-temperature background compresses the grayscale to suppress noise; the initial linear mapping parameter k=2.0 for the human body area expands the grayscale to enhance contrast; the initial linear mapping parameter k=1.2 for the medium-temperature fire area moderately stretches; and the initial linear mapping parameter k=0.3 for the high-temperature fire area prevents overexposure.

[0061] Specifically, in step S42, based on the temperature partition probability distribution of each pixel point in the temperature partition probability map, the first to fourth initial linear mapping parameters are weightedly fused to obtain the flexible adjustment linear mapping parameters of each pixel point. It should be understood that since the temperature distribution in the fire scene is continuous and dynamic, in order to avoid adjacent pixels being assigned to different mapping intervals due to slight temperature differences, resulting in grayscale value mutations, artifact stripes or regional splits. To this end, the present application further performs weighted calculations on the corresponding initial linear mapping parameters based on the probability that each pixel point belongs to different temperature areas in the temperature partition probability map. Assuming that the probability of a pixel point belonging to the low-temperature background area is p1, the probability of belonging to the human body temperature area is p2, the probability of belonging to the medium-temperature fire area is p3, and the probability of belonging to the high-temperature fire area is p4, the corresponding initial linear mapping parameters are k1, k2, k3, and k4 respectively, then the flexible adjustment linear mapping parameter k of the pixel point can be calculated by k=p1k1+p2k2+p3k3+p4k4. In this way, the mapping parameters of each pixel comprehensively consider its possibility in different temperature zones, realizing flexible adjustment of parameters.

[0062] Specifically, step S43 linearly maps the grayscale values ​​of each pixel in the corrected infrared image data based on the flexible adjustment linear mapping parameters of each pixel to obtain the enhanced infrared image data. In this way, the grayscale mapping of each pixel is dynamically adjusted based on its temperature partition probability distribution. Since multiple probabilities coexist in the transition region, the mapping parameters transition smoothly. Even in the case of blurred temperature boundaries or slight temperature differences, reasonable mapping of grayscale values ​​can be achieved, avoiding artifacts and loss of details caused by fixed threshold partitioning, thereby improving the accuracy and adaptability of image enhancement and making the optimized infrared image more visually continuous and natural.

[0063] Specifically, for each pixel, the grayscale value mapping relationship is not simply determined based on a fixed temperature threshold, but the probability that the pixel may belong to different temperature ranges is taken into account. For example, there may be a transition area between a high-temperature fire scene and a human target. The pixels here may have the probability of having both high and low temperatures due to the influence of the surrounding environment. By introducing a flexible adjustment mechanism, it can be ensured that the pixels in these transition areas will not undergo sudden changes in grayscale due to small temperature differences during grayscale mapping. Instead, the mapping parameters transition smoothly based on the coexistence of multiple probabilities. In this way, even in the face of blurred temperature boundaries or small temperature changes, reasonable mapping of grayscale values ​​can be achieved, avoiding artifacts and detail loss caused by forced division.

[0064] During implementation, for each frame of corrected infrared image data, the flexible adjustment linear mapping parameters corresponding to each pixel must be precisely calculated. This step relies on a previously generated temperature partition probability distribution map, which contains rich information about the temperature properties of each pixel. By analyzing this information, a unique grayscale mapping scheme can be tailored for each pixel, ensuring that the resulting enhanced infrared image data is more visually continuous and natural. The flexible adjustment mechanism plays a particularly important role when processing pixels near temperature boundaries. Since these pixels often represent transitions from one temperature range to another, special attention must be paid to maintaining coherence and consistency between grayscale values ​​to prevent unreasonable jumps or gaps.

[0065] In addition, this method also greatly improves the accuracy and adaptability of image enhancement. In the complex and ever-changing firefighting and search and rescue environment, the temperature distribution changes all the time, and a single fixed threshold partitioning method is difficult to meet the needs of real-time adjustment. In contrast, the flexible adjustment linear mapping based on the probability distribution of temperature partitions shows higher flexibility and response speed. Whether it is between high-temperature fire sources or low-temperature trapped people, or any other place with temperature differences, the grayscale value can be dynamically adjusted according to the actual situation to ensure that the image quality is not affected. Especially when dealing with extreme conditions such as smoke and flames, this flexible grayscale mapping strategy can significantly improve the rescuers' ability to understand the on-site situation and help quickly locate the target position.

[0066] In summary, the image enhancement method of the fire-fighting search and rescue infrared thermal imager based on the embodiment of the present application is explained. It uses an infrared thermal imager to collect original RAW infrared image data, performs non-uniformity correction processing on it, and further introduces an image processing algorithm based on deep learning to perform pixel-level feature learning and temperature partition recognition on the corrected infrared image data to generate a temperature partition probability distribution map. On this basis, a flexible linear mapping mechanism is introduced. Based on the temperature partition probability distribution of each pixel point of the infrared image, its linear mapping parameters are calculated, and the grayscale value of each pixel point of the infrared image is linearly mapped, thereby dynamically allocating grayscale resources and optimizing the display effect of the infrared image, thereby obtaining enhanced infrared image data. This method can adaptively adjust the grayscale distribution of different temperature areas. While improving the contrast and detail visibility of the infrared image, it effectively avoids the artifacts and fragmentation caused by the traditional fixed threshold partitioning method, significantly improving the image quality and readability.

[0067] Furthermore, an image enhancement device for a fire-fighting search and rescue infrared thermal imager is also provided.

[0068] Figure 7 FIG is a block diagram of an image enhancement device for a fire-fighting search and rescue infrared thermal imager according to an embodiment of the present application. Figure 7 As shown, the image enhancement device 100 of the fire-fighting and search and rescue infrared thermal imager according to the embodiment of the present application includes: an infrared image data acquisition module 110, which is used to acquire the original RAW infrared image data collected by the infrared thermal imager; a non-uniformity correction module 120, which is used to perform non-uniformity correction processing on the original RAW infrared image data to obtain corrected infrared image data; a temperature partition processing module 130, which is used to perform temperature partition processing on the corrected infrared image data to obtain a temperature partition probability distribution map; and a local flexible linear mapping processing module 140, which is used to perform local flexible linear mapping processing on the corrected infrared image data based on the infrared image temperature partition probability distribution map to obtain enhanced infrared image data.

[0069] Here, those skilled in the art will appreciate that the specific operations of each module in the image enhancement device of the fire rescue infrared thermal imager have been described above with reference to FIG. Figures 1 to 6 The image enhancement method of the fire-fighting search and rescue infrared thermal imager has been described in detail, and therefore, its repeated description will be omitted.

[0070] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in the present invention are merely illustrative and non-limiting, and should not be construed as necessarily possessed by each embodiment of the present invention. Furthermore, the specific details of the above embodiments are provided for illustrative purposes and to facilitate understanding, and are not intended to be limiting. These details do not necessarily limit the present invention to being implemented using these specific details.

[0071] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0072] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be encompassed therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0073] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0074] Finally, it should be noted that the above description has been provided for purposes of illustration and description. Furthermore, the above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to be limiting. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art will appreciate that the technical solutions of the present invention may be modified or replaced with equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An image enhancement method for a fire rescue infrared thermal imager, characterized in that: include: Obtain the original RAW infrared image data collected by the infrared thermal imager; Performing non-uniformity correction processing on the original RAW infrared image data to obtain corrected infrared image data; Performing temperature partition processing on the corrected infrared image data to obtain a temperature partition probability distribution map; Based on the infrared image temperature partition probability distribution map, local flexible linear mapping processing is performed on the corrected infrared image data to obtain enhanced infrared image data.

2. The image enhancement method of the fire-fighting search and rescue infrared thermal imager according to claim 1, characterized in that: Performing temperature partition processing on the corrected infrared image data to obtain an infrared image temperature partition probability distribution map, including: extracting infrared features of the corrected infrared image data to obtain an infrared image feature map; Performing feature hierarchical enhancement processing on the infrared image feature map based on collaborative modeling of spatial topology and semantic structure to obtain an infrared image enhanced feature map; The infrared image enhancement feature map is input into a temperature partition module based on a softmax function to obtain the infrared image temperature partition probability distribution map.

3. The image enhancement method of the fire-fighting search and rescue infrared thermal imager according to claim 2, characterized in that: Extracting infrared features of the corrected infrared image data to obtain an infrared image feature map includes: The infrared feature extraction based on the FPN network is performed on the corrected infrared image data to obtain the infrared image feature map.

4. The image enhancement method for a firefighting search and rescue infrared thermal imager according to claim 3, characterized in that: Performing feature hierarchical enhancement processing on the infrared image feature map based on collaborative modeling of spatial topology and semantic structure to obtain an infrared image enhanced feature map, including: Extracting the channel feature vector of the (i, j)th pixel position from the infrared image feature map as the channel feature vector to be enhanced of the infrared image; Based on the infrared image feature map, performing information residual modeling based on spatial-semantic dual-domain context-driven on the feature vector of the channel to be enhanced of the infrared image to obtain an implicit enhancement component coding vector of the channel to be enhanced of the infrared image; The implicit enhancement component coding vector of the infrared image channel to be enhanced and the feature vector of the infrared image channel to be enhanced are fused to obtain an enhanced infrared image channel feature vector, wherein the enhanced infrared image channel feature vector is the channel feature vector of the pixel position (i, j) of the infrared image enhancement feature map.

5. The image enhancement method for a firefighting search and rescue infrared thermal imager according to claim 4, characterized in that: Based on the infrared image feature map, performing information residual modeling based on spatial-semantic dual-domain context-driven on the feature vector of the channel to be enhanced of the infrared image to obtain an implicit enhancement component coding vector of the channel to be enhanced of the infrared image, including: Performing n random traversal sampling on the infrared image feature map to obtain n channel feature vectors as a sparse set of infrared image context feature vectors; Calculating information residual feature vectors between each infrared image context feature vector in the sparse set of infrared image context feature vectors and the feature vector of the infrared image channel to be enhanced to obtain a set of information residual feature vectors of the infrared image channel to be enhanced; Performing a spatial-semantic dual-domain coupling analysis on the sparse set of the infrared image channel feature vectors to be enhanced and the infrared image context feature vectors to obtain a set of infrared image channel feature spatial modulation matrices to be enhanced and infrared image channel feature semantic association coding matrices to be enhanced; Based on the set of the infrared image channel feature spatial modulation matrix and the infrared image channel feature semantic association coding matrix, the set of the infrared image channel information residual feature vectors to be enhanced is structured fusion coded to obtain the implicit enhancement component coding vector of the infrared image channel to be enhanced.

6. The image enhancement method for a firefighting search and rescue infrared thermal imager according to claim 5, characterized in that: Performing a spatial-semantic dual-domain coupling analysis on the sparse set of the infrared image channel feature vector to be enhanced and the infrared image context feature vector to obtain a set of the infrared image channel feature spatial modulation matrix to be enhanced and the infrared image channel feature semantic association coding matrix to be enhanced, including: Calculating the Poincare measure between the infrared image channel feature vector to be enhanced and each infrared image context feature vector in the sparse set of infrared image context feature vectors to obtain a feature space modulation matrix of the infrared image channel to be enhanced composed of multiple Poincare measures; The implicit semantic association matrix between the infrared image channel feature vector to be enhanced and each infrared image context feature vector in the sparse set of infrared image context feature vectors is calculated to obtain a set of semantic association coding matrices of the infrared image channel feature to be enhanced.

7. The image enhancement method for a firefighting search and rescue infrared thermal imager according to claim 6, characterized in that: Based on a set of the infrared image channel feature spatial modulation matrix and the infrared image channel feature semantic association coding matrix, a set of residual feature vectors of the infrared image channel to be enhanced is subjected to structured fusion coding to obtain an implicit enhancement component coding vector of the infrared image channel to be enhanced, including: Based on the set of the infrared image channel feature spatial modulation matrix and the infrared image channel feature semantic association coding matrix, performing a dual-constraint optimization iteration on each infrared image channel information residual feature vector in the set of infrared image channel information residual feature vectors to obtain a set of optimized infrared image channel information residual feature vectors; Using each semantic association coding matrix of the infrared image channel feature to be enhanced in the set of semantic association coding matrices of the infrared image channel feature to be enhanced as a primary mask operator, and using the spatial modulation matrix of the infrared image channel feature to be enhanced as a secondary mask operator, the set of residual feature vectors of the optimized infrared image channel information to be enhanced is structured fusion coded to obtain the implicit enhancement component coding vector of the infrared image channel to be enhanced.

8. The image enhancement method for a firefighting search and rescue infrared thermal imager according to claim 7, characterized in that: The method of fusing the implicit enhancement component coding vector of the infrared image channel to be enhanced and the feature vector of the infrared image channel to be enhanced to obtain the enhanced infrared image channel feature vector comprises: An element-wise weighted fusion between the implicit enhancement component coding vector of the infrared image channel to be enhanced and the feature vector of the infrared image channel to be enhanced is calculated to obtain the enhanced infrared image channel feature vector.

9. The image enhancement method for a firefighting search and rescue infrared thermal imager according to claim 1, characterized in that: Based on the infrared image temperature partition probability distribution map, performing local flexible linear mapping processing on the corrected infrared image data to obtain enhanced infrared image data, including: Setting initial linear mapping parameters for the low-temperature background region, the human body temperature region, the medium-temperature fire region, and the high-temperature fire region to obtain first to fourth initial linear mapping parameters; Based on the temperature partition probability distribution of each pixel point in the temperature partition probability map, weighted fusion is performed on the first to fourth initial linear mapping parameters to obtain the flexible adjustment linear mapping parameter of each pixel point; The enhanced infrared image data is obtained by performing linear mapping processing on the grayscale value of each pixel in the corrected infrared image data based on the flexible adjustment linear mapping parameter of each pixel.

10. An image enhancement device for a firefighting search and rescue infrared thermal imager, characterized in that: include: Infrared image data acquisition module, used to obtain the original RAW infrared image data collected by the infrared thermal imager; a non-uniformity correction module, configured to perform non-uniformity correction processing on the original RAW infrared image data to obtain corrected infrared image data; A temperature partition processing module, configured to perform temperature partition processing on the corrected infrared image data to obtain a temperature partition probability distribution map; The local flexible linear mapping processing module is used to perform local flexible linear mapping processing on the corrected infrared image data based on the infrared image temperature partition probability distribution map to obtain enhanced infrared image data.

Images