Electrical fire hazard perception monitoring method and system based on infrared image

Abstract

The application discloses an electrical fire hidden danger perception monitoring method and system based on an infrared image, relates to the technical field of image monitoring, and comprises the following steps: acquiring an initial region of interest of a target device, expanding a bounding box of the initial region of interest outward, and generating an expanded region of interest; acquiring an original temperature matrix in the expanded region of interest in a continuous video frame, and calculating a moving average temperature baseline and a spatial temperature gradient of each pixel point in the original temperature matrix of the current video frame; marking candidate abnormal points according to the moving average temperature baseline and the spatial temperature gradient, and outputting an abnormal temperature mask; inputting the abnormal temperature mask into a shallow feature extraction network for downsampling and upsampling recovery, outputting a high-heat edge contour and a corresponding high-heat area; acquiring a reference area of the initial region of interest, calculating a relative area threshold value according to the reference area of the initial region of interest, and outputting a warning signal and the high-heat edge contour according to the relative area threshold value.

Description

Technical Field

[0001] This invention relates to the field of image monitoring technology, and in particular to a method and system for sensing and monitoring electrical fire hazards based on infrared images. Background Technology

[0002] Target equipment is prone to abnormal temperature rise during long-term high-load operation, which can escalate into electrical fires. Infrared thermal imaging technology is widely used for equipment condition monitoring due to its non-contact temperature measurement characteristics. However, actual operating environments are complex and temperatures change dynamically, and infrared images are often severely interfered with by background thermal noise. Existing monitoring methods mostly use global static temperature thresholds or simple image thresholding for judgment, failing to fully combine the dynamic temperature evolution patterns in the spatiotemporal dimensions with the structural scale of the target equipment. This approach makes it difficult to accurately locate local abnormal heating areas, and even more difficult to effectively extract clear edge contours of heating objects. As a result, subtle early heating characteristics of fire hazards are easily overlooked or misjudged, making it difficult to meet the high-precision, adaptive early warning requirements in industrial scenarios.

[0003] Currently, Chinese invention patent application CN120412183A discloses a fire monitoring method, a fire monitoring server, and a fire monitoring system. This method first performs a preliminary inspection of real-time images of the monitored scene using a first identification method. If the preliminary inspection indicates the presence of smoke or fire targets in the real-time images, a large-scale visual model is then invoked for analysis. This allows the large-scale visual model to identify flame and smoke features in the images, accurately determining whether a fire has occurred and effectively reducing the false alarm rate. The large-scale visual model only needs to process images where smoke or fire targets are not detected during the preliminary inspection, reducing the amount of data processing required for model verification and improving its real-time performance. When the large-scale visual model determines that a fire has occurred in the monitored scene, a large-scale language model is invoked to generate fire description information based on the real-time images. This allows rescue personnel to quickly grasp the key points and efficiently carry out rescue operations, improving rescue efficiency. However, related technologies lack reliability in tracking anomalies and providing adaptive early warnings for early electrical fire hazards. Summary of the Invention

[0004] The technical problem solved by this invention is that related technologies have difficulty in effectively distinguishing between occasional interference and real temperature rise anomalies in complex environments, cannot accurately segment and extract high-frequency edge contour details, and lack the reliability of anomaly tracking and early warning.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: The method for detecting and monitoring electrical fire hazards based on infrared images includes the following steps: Step S1: Obtain the initial region of interest of the target device, and expand the bounding box of the initial region of interest outward to generate an extended region of interest; Step S2: Obtain the original temperature matrix of the extended region of interest in the continuous video frames, and calculate the moving average temperature baseline and spatial temperature gradient of each pixel in the original temperature matrix of the current video frame. Candidate outliers are identified based on the moving average temperature baseline and spatial temperature gradient, and an anomaly temperature mask is output. Step S3: Input the abnormal temperature mask into the shallow feature extraction network for downsampling and upsampling recovery, and output the high-temperature edge contour and the corresponding high-temperature area. The shallow feature extraction network includes a three-layer encoder structure and a three-layer decoder structure; Step S4: Obtain the reference area of ​​the initial region of interest, calculate the relative area threshold based on the reference area of ​​the initial region of interest, and output the warning signal and high-heat edge contour based on the relative area threshold.

[0006] Preferably, the processing logic for generating the extended region of interest is as follows: Keeping the center coordinates of the initial region of interest unchanged, the pixel width and pixel height of the initial region of interest are expanded respectively, and the calculation expression is as follows: ; ; in, and These represent the pixel width and pixel height of the expanded region of interest, respectively. For expansion coefficient, and These are the pixel width and pixel height of the initial region of interest, respectively.

[0007] Preferably, step S2 specifically includes: Step S21: Obtain the original temperature matrix of the extended region of interest in the continuous video frames, and calculate the moving average temperature baseline and spatial temperature gradient of each pixel in the original temperature matrix of the extended region of interest in the current video frame. Step S22: When the difference between the current temperature value of the target pixel and the moving average temperature baseline is greater than the first threshold and the spatial temperature gradient is greater than the second threshold, the corresponding target pixel is marked as a candidate anomaly. Step S23: If the candidate anomaly point persists in N consecutive video frames, the region including the candidate anomaly point is extracted and the anomaly temperature mask is output.

[0008] Preferably, the formula for calculating the moving average temperature baseline is: ; in, Extend the moving average temperature baseline of the pixels in the original temperature matrix within the region of interest of the current video frame. To expand the region of interest in the current video frame with the actual temperature values ​​of the pixels in the original temperature matrix, To extend the moving average temperature baseline of the pixels in the original temperature matrix within the region of interest from the previous video frame, This is the smoothing coefficient.

[0009] Preferably, the processing logic for calculating the spatial temperature gradient of each pixel is as follows: Extract the target pixel and its surrounding 3×3 original temperature matrix; The weighted temperature difference in the horizontal direction and the weighted temperature difference in the vertical direction of the original 3×3 temperature matrix are calculated using the first-order discrete difference operator, and are used as the horizontal gradient component and the vertical gradient component, respectively. The spatial temperature gradient of the target pixel is obtained by calculating the square root of the sum of the squares of the horizontal gradient components and the vertical gradient components. The target pixel is a pixel in the original temperature matrix within the extended region of interest of the current video frame.

[0010] Preferably, step S3 specifically includes: Step S31: Multiply the abnormal temperature mask with the original temperature matrix in the extended region of interest of the current video frame pixel by pixel to extract the local abnormal heat map, and use the local abnormal heat map as the initial input of the three-layer encoder structure. Step S32: In each layer of the three-layer encoder structure, a Haar wavelet downsampling operation is performed on the input feature map to decompose the input feature map into low-frequency components and high-frequency components; High-frequency components include horizontal components, vertical components, and diagonal components; The low-frequency components are input into the next layer encoder for feature extraction, the high-frequency components are subjected to convolutional dimensionality reduction, and the skip connection branches of the corresponding layer are retained.

[0011] Preferably, step S3 further includes: Step S33: In each layer of the three-layer decoder structure, an upsampling operation is performed on the feature map output by the previous layer decoder, and the upsampled feature map is channel-stitched with the high-frequency components retained by the same-level skip connection branch. The concatenated feature maps are fused by convolution to restore them to their initial resolution, generating the final decoded feature map. Step S34: The final decoded feature map is mapped to a binarized high-temperature segmentation mask through a classifier; The outer contours of each connected region of the binarized high-heat segmentation mask are extracted as high-heat edge contours, and the total number of pixels inside the high-heat edge contours is counted and quantized to obtain the corresponding high-heat region area.

[0012] Preferably, step S4 specifically includes: Step S41: Extract the horizontal and vertical pixel dimensions from the original temperature matrix within the initial region of interest of the current video frame, and update the product of the horizontal and vertical pixel dimensions to the reference area of ​​the initial region of interest.

[0013] Preferably, step S4 further includes: Step S42: Multiply the reference area of ​​the initial region of interest by the area ratio coefficient to calculate the relative area threshold; Step S43: Determine whether the area of ​​the high-heat region is greater than the relative area threshold; If so, a warning signal with associated target device coordinate information is generated, and the high-heat edge contour is output simultaneously.

[0014] An infrared image-based electrical fire hazard detection and monitoring system includes: The region expansion module is used to obtain the initial region of interest of the target device and expand the bounding box of the initial region of interest outward to generate an expanded region of interest. The anomaly detection module is used to obtain the original temperature matrix of the extended region of interest in consecutive video frames, and to calculate the moving average temperature baseline and spatial temperature gradient of each pixel in the original temperature matrix of the current video frame. Candidate outliers are identified based on the moving average temperature baseline and spatial temperature gradient, and an anomaly temperature mask is output. The edge recovery module is used to input the abnormal temperature mask into the shallow feature extraction network for downsampling and upsampling recovery, and output the high-temperature edge contour and the corresponding high-temperature area. The shallow feature extraction network includes a three-layer encoder structure and a three-layer decoder structure; The early warning output module is used to obtain the reference area of ​​the initial region of interest, calculate the relative area threshold based on the reference area of ​​the initial region of interest, and output an early warning signal and a high-heat edge contour based on the relative area threshold.

[0015] The beneficial effects of this invention are as follows: This invention integrates a spatiotemporal anomaly discrimination mechanism based on moving average temperature baseline and spatial temperature gradient with a shallow feature extraction network using Haar wavelets for multi-scale decomposition, thereby improving the accuracy and dynamic adaptability of electrical fire hazard perception. By generating an expanded region of interest and combining it with spatiotemporal temperature features to mark candidate anomalies, and using continuous judgment of continuous video frames to output an anomaly temperature mask, sporadic environmental thermal noise is effectively filtered out. The local anomaly heatmap is input into the shallow feature extraction network. In the encoding stage, Haar wavelet downsampling is used to extract low-frequency components while retaining high-frequency components to skip connections. In the decoding stage, channel splicing and convolution fusion are used to accurately recover the outer contours of each connected domain of the binarized high-temperature segmentation mask while maintaining computational lightweightness, outputting the high-temperature edge contour and high-temperature area area. Finally, by extracting the initial region of interest size to calculate the benchmark area and converting it into a relative area threshold for judgment, the early warning mechanism achieves adaptive matching to the target equipment scale, avoiding false alarms and missed alarms caused by traditional fixed thresholds. Attached Figure Description

[0016] Figure 1 A flowchart illustrating the steps of an electrical fire hazard sensing and monitoring method based on infrared images, provided in one embodiment of the present invention; Figure 2 This is a basic flowchart of an electrical fire hazard perception and monitoring system based on infrared images, provided as an embodiment of the present invention. Detailed Implementation

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0018] Example 1, referring to Figure 1 This paper provides a method for detecting and monitoring electrical fire hazards based on infrared images, including the following steps: Step S1: Obtain the initial region of interest (ROI) of the target device, and expand the bounding box of the initial ROI outward to generate an expanded ROI; Step S2: Obtain the original temperature matrix of the extended region of interest in the continuous video frames, and calculate the moving average temperature baseline and spatial temperature gradient of each pixel in the original temperature matrix of the current video frame. Candidate outliers are identified based on the moving average temperature baseline and spatial temperature gradient, and an anomaly temperature mask is output. Step S3: Input the abnormal temperature mask into the shallow feature extraction network for downsampling and upsampling recovery, and output the high-temperature edge contour and the corresponding high-temperature area area. Shallow feature extraction networks consist of a three-layer encoder structure and a three-layer decoder structure; Step S4: Obtain the reference area of ​​the initial region of interest, calculate the relative area threshold based on the reference area of ​​the initial region of interest, and output the warning signal and high-heat edge contour based on the relative area threshold.

[0019] Specifically, for target devices that are stationary or in a fixed position, the two-dimensional pixel coordinate boundaries of the target device in the current infrared camera's field of view are extracted; the polygonal region enclosed by these coordinate boundaries is locked as the initial region of interest for fixed mapping.

[0020] The shallow feature extraction network structure consists of an input layer, which receives a preprocessed local anomaly heatmap. This feature map is a pure heatmap generated by performing a pixel-by-pixel mapping Hadamard product operation on the original temperature matrix and the anomaly temperature mask. The feature encoding layer comprises three concatenated Haar wavelet downsampling blocks. Each layer uses a wavelet orthogonal basis filter to deconstruct the input feature map into low-frequency components representing a smooth thermal distribution background, and three high-frequency components representing the spatial thermal gradients in the horizontal, vertical, and diagonal directions. High-dimensional abstract thermal features are extracted through layer-by-layer downsampling. The feature decoding... The code layer consists of three cascaded upsampling fusion blocks. Within each layer, transposed convolutions are used to expand the spatial resolution of deep features, and high-frequency components that have undergone dimensionality reduction are introduced through a skip connection mechanism to achieve channel splicing and convolutional reconstruction of deep semantic features and shallow physical thermal gradient boundaries. The output layer uses a 1×1 convolutional layer and a sigmoid nonlinear activation function to map the fused feature map restored to the initial resolution into a pixel-wise confidence probability matrix, and outputs a binarized high-thermal segmentation mask through a step decision. The training parameters include a loss function, which is a joint loss function composed of a weighted average of binary cross-entropy loss for pixel classification and Dice loss for region contour overlap, to effectively overcome the extreme imbalance between small heat-generating targets and large background areas in infrared images; the optimizer is the AdamW optimizer, with an initial learning rate of 1×10⁻³ and a weight decay coefficient of 1×10⁻⁴; the batch size is 32; the number of training epochs is 300; the dataset constructed for infrared features in industrial sites is divided into training, validation, and test sets in a ratio of 80%, 10%, and 10%; a cosine annealing strategy is used for periodic learning rate decay to prevent the network from getting trapped in local optima on smooth thermal radiation feature surfaces in the later stages of training; In one training session, the initial learning rate is set to 1×10. -3With a batch size of 32, the weights of the binary cross-entropy loss and Dice loss in the joint loss function are assigned as 0.4:0.6 to enhance the network's fine-grained capture of high-heat diffusion edges. When the depth N of the encoding / decoding structure is set to 3, the average cross-union ratio (CUI) of the validation set reaches over 0.91 after approximately 250 training rounds.

[0021] In a specific embodiment, the processing logic for generating the extended region of interest is as follows: Keeping the center coordinates of the initial region of interest unchanged, the pixel width and pixel height of the initial region of interest are expanded respectively, and the calculation expression is as follows: ; ; in, and These represent the pixel width and pixel height of the expanded region of interest, respectively. For expansion coefficient, and These are the pixel width and pixel height of the initial region of interest, respectively.

[0022] Specifically, the initial temperature matrix corresponding to the initial region of interest in the current video frame is extracted; Count the total number of valid pixels contained in the central region of the initial temperature matrix; Traverse every pixel in the central region, extract the temperature gradient magnitude of each pixel and sum them up; divide the total gradient value obtained by summing by the total number of pixels to calculate the central thermal gradient.

[0023] The total number of valid pixels contained in the edge region of the initial temperature matrix is ​​counted; each pixel in the edge region is traversed, the temperature gradient magnitude of each pixel is extracted and summed; the total gradient value obtained by summing is divided by the total number of pixels to calculate the boundary thermal gradient.

[0024] Based on the intensity of thermal diffusion, an expansion coefficient is generated: ; in, For expansion coefficient, As a regulating factor, For the boundary thermal gradient, The thermal gradient is centered on the center.

[0025] When the edge gradient is extremely large (heat spreads rapidly outward), the system automatically expands the interception range; when the heat is concentrated in the center, the system shrinks the range to save computing power.

[0026] The preferred values ​​for the adjustment factor are shown in Table 1. Table 1 ; By constraining the adjustment factor within the range of [0.5, 0.8], the extended system exhibits optimal resilience when facing different gradient ratios. It can ensure that the false negative rate of edge thermal features is controlled within a safe level of less than 2%, while keeping the processing time of a single frame stable below 16ms, perfectly balancing the effectiveness of spatial interception and the real-time performance of the temporal dimension.

[0027] In a specific embodiment, step S2 specifically includes: Step S21: Obtain the original temperature matrix of the extended region of interest in the continuous video frames, and calculate the moving average temperature baseline and spatial temperature gradient of each pixel in the original temperature matrix of the extended region of interest in the current video frame. Step S22: When the difference between the current temperature value of the target pixel and the moving average temperature baseline is greater than the first threshold and the spatial temperature gradient is greater than the second threshold, the corresponding target pixel is marked as a candidate anomaly. Step S23: If the candidate anomaly point persists in N consecutive video frames, the region including the candidate anomaly point is extracted and the anomaly temperature mask is output.

[0028] Specifically, the system receives video frame data streams transmitted in real time from an infrared thermal imager; it parses the absolute temperature radiation value data layer attached to the video frame data stream, which is a RAW format data layer that directly records the absolute physical temperature value of each pixel without pseudo-color rendering; based on the generated two-dimensional coordinate boundary of the extended region of interest, the system performs spatial clipping on the absolute temperature radiation value data layer, extracts the set of discrete temperature values ​​in the corresponding region, converts it into a two-dimensional matrix format, and outputs it as the original temperature matrix of the current video frame.

[0029] The first threshold is used to determine whether the current temperature deviates from the historical smooth trend. Electrical fire hazards initially involve an extremely slow heat accumulation process with minimal transient temperature rise. If the threshold is set too high, a large number of early, slow-changing hazards will be missed; setting it between 0.5℃ and 1.5℃ is slightly higher than the system temperature fluctuation of the industrial infrared thermal imager itself, while still being able to sensitively capture extremely weak abnormal heat releases.

[0030] The core function of the second threshold is to eliminate false thermal reflections in the environment. Real electrical heating points exhibit a temperature field in physical space that is high at the center and smoothly decays outwards, always showing a significant spatial temperature gradient; while artifacts formed by direct sunlight or reflections from metal cabinet surfaces typically have a uniformly bright, dead white temperature within their area, with a gradient approaching zero. Setting the second threshold at... The left and right angles are just enough to cut off those environmentally reflected light spots that, although very hot, do not have a gradient at the heat center.

[0031] The preferred value range for N is 15-30 frames. Assuming the infrared camera sampling rate is 30fps, this corresponds to approximately 0.5 to 1 second of physical time. Transient interference frequently occurs in industrial settings. The requirement for anomalies to persist continuously for 15-30 frames is equivalent to requiring the anomaly to remain stable in the physical world for at least 0.5 seconds, thereby utilizing temporal continuity to thoroughly filter out high-frequency transient noise.

[0032] For the original temperature matrix of the current frame, a blank two-dimensional matrix with the same size as it is established as the initial mask matrix; the coordinate mapping relationship of the candidate anomaly points that continuously meet the judgment conditions in N consecutive video frames is traversed, the pixel value of the corresponding spatial coordinate position in the initial mask matrix is ​​set to 1, and the pixel value of the rest that does not meet the conditions is set to 0; then, morphological closing operation is performed on the region with a pixel value of 1 to eliminate possible discrete holes, and a binarized matrix containing the spatial topology of all anomaly points is output as the anomaly temperature mask.

[0033] In a specific embodiment, the calculation expression for the moving average temperature baseline is as follows: ; in, Extend the moving average temperature baseline of the pixels in the original temperature matrix within the region of interest of the current video frame. To expand the region of interest in the current video frame with the actual temperature values ​​of the pixels in the original temperature matrix, To extend the moving average temperature baseline of the pixels in the original temperature matrix within the region of interest from the previous video frame, This is the smoothing coefficient.

[0034] Specifically, the preferred values ​​for the smoothing coefficient are shown in Table 2; Table 2 ; As shown in Table 2 By setting the smoothing coefficient within this optimal range, the algorithm achieves a dynamic balance. The baseline update rate at this coefficient is both fast enough to perfectly smooth out the slow temperature rise caused by changes in illumination, and slow enough to not keep up with the endogenous continuous heating caused by electrical faults.

[0035] In a specific embodiment, the processing logic for calculating the spatial temperature gradient of each pixel is as follows: Extract the target pixel and its surrounding 3×3 original temperature matrix; The weighted temperature difference in the horizontal direction and the weighted temperature difference in the vertical direction of the original 3×3 temperature matrix are calculated using the first-order discrete difference operator, and are used as the horizontal gradient component and the vertical gradient component, respectively. The spatial temperature gradient of the target pixel is obtained by calculating the arithmetic square root of the sum of the squares of the horizontal and vertical gradient components. The target pixel is the pixel in the original temperature matrix within the extended region of interest of the current video frame.

[0036] Specifically, the original 3×3 temperature matrix of the target pixel and its surrounding area is calculated as follows: ; in, The physical temperature value of the target pixel, in the horizontal direction, with subscripts indicating the temperature. The pixel to the left of the target pixel; with The pixel to the right of the target pixel is represented on the vertical axis, with the subscript indicating the pixel. The pixel above the target pixel; with represents the pixel below the target pixel.

[0037] The neighborhood temperature matrix is ​​convolved with both the horizontal and vertical Sobel convolution kernels respectively. ; ; After obtaining the horizontal and vertical gradients, the final comprehensive spatial temperature gradient of the target pixel is obtained by calculating the magnitude of the vector sum: ; in, For the lateral gradient component, For the longitudinal gradient component, This represents the space temperature gradient.

[0038] The weighting matrix for the weighted temperature difference in the horizontal direction is as follows: ; The weighting matrix for the weighted temperature difference in the vertical direction is: ; Positions located directly above, below, to the left, and to the right of the target pixel. These are closest to the center pixel (physical distance of 1 unit), therefore they have the greatest impact on the temperature gradient at the center point and are given double the weight. Positions with a weight of 1: Located at the four diagonal corners of the target pixel. These are slightly farther from the center pixel (physical distance of...). Since they are only one unit, their influence is relatively weak, and they are only given a single weight.

[0039] It should be noted that this invention integrates the moving average temperature baseline and spatial temperature gradient into the anomaly candidate point marking logic, resulting in a corresponding anomaly temperature mask that can be determined based on the continuous state of consecutive video frames. The data in this anomaly temperature mask simultaneously characterizes the temporal dimension of temperature deviation from the historical smooth trend and the spatial dimension of the target pixel exhibiting a decay gradient from the center outwards. This solves the problems of easily missing early slow-change hazards in traditional static threshold determination and the inability to effectively eliminate artifacts formed by false thermal reflections from the environment.

[0040] In a specific embodiment, step S3 specifically includes: Step S31: Multiply the abnormal temperature mask with the original temperature matrix in the extended region of interest of the current video frame pixel by pixel to extract the local abnormal heat map, and use the local abnormal heat map as the initial input of the three-layer encoder structure. Specifically, the original temperature matrix and the anomalous temperature mask are extracted from the extended region of interest in the current frame. The consistency of their two-dimensional pixel resolution is verified to ensure strict alignment within the same local coordinate system. The original temperature matrix is ​​a floating-point data matrix containing absolute temperature values, while the anomalous temperature mask is a binary feature matrix containing only the values ​​0 or 1.

[0041] A blank two-dimensional matrix with the same resolution as the expanded region of interest is established as the initial matrix for the local anomaly heatmap. A local spatial Cartesian coordinate system is established with the top-left pixel of the expanded region of interest as the origin (0,0).

[0042] Traverse all pixel coordinates along the x-axis and y-axis of the local Cartesian coordinate system. Perform an element-wise Hadamard product operation on the physical temperature value T(x,y) at coordinates (x,y) in the original temperature matrix and the mask feature value M(x,y) at the same coordinate position in the anomalous temperature mask. The calculation expression is as follows: ; Where H(x,y) is the pixel feature value of the local anomaly heatmap at coordinates (x,y).

[0043] When the mask feature value M(x,y)=1, that is, the point is determined to be a real anomaly by the preceding physical model, the corresponding position of the heat map retains the real physical temperature value and its gradient information. When the mask feature value M(x,y)=0, that is, the point is a normal device background or environmental thermal reflection artifact, the pixel value of the corresponding position in the heat map is forcibly set to zero.

[0044] Step S32: In each layer of the three-layer encoder structure, perform Haar wavelet downsampling on the input feature map to decompose the input feature map into low-frequency components and high-frequency components; High-frequency components include horizontal components, vertical components, and diagonal components; The input feature map refers to the data block that enters a specific network layer in a three-layer encoder. In layer 1, the input feature map is equal to the initial input. In layer 2, the input feature map is no longer the original heatmap, but a deep feature tensor output from layer 1 after wavelet downsampling and convolution. In layer 3, the input feature map is an abstract feature vector with lower resolution but more channels, passed down from layer 2.

[0045] Define four orthogonal basis filters for the Haar wavelet transform, including a low-pass filter LL and three filters that incorporate high-pass features: a horizontal high-pass filter LH, a vertical high-pass filter HL, and a diagonal high-pass filter HH.

[0046] Obtain the input feature map of the current layer (assuming its spatial resolution is ). (The number of channels is C). A discrete wavelet convolution operation with a stride of 2 is used, and the above four filters are applied to perform a spatial sliding window operation on the input feature map. The input feature map is losslessly downsampled in the spatial dimension as follows: In the channel dimension, it is separated into low-frequency component LL and high-frequency component; Low-frequency components are input into the next layer encoder for feature extraction, high-frequency components are subjected to convolutional dimensionality reduction, and the skip connection branches of the corresponding layer are retained.

[0047] Specifically, the low-frequency components characterizing the overall thermal field distribution are used as the mainstream data and directly input into the standard convolutional layer of the next encoder stage to extract deeper abstract thermodynamic features. For the high-frequency components in the three directions, they are concatenated along the channel dimension. Since the number of channels increases exponentially after concatenation of high-frequency components, direct cross-layer transmission would cause an explosion of decoder parameters and introduce redundant noise; therefore, a size of [missing information] is used... The point cloud convolution kernel performs a cross-channel linear combination calculation on the concatenated high-frequency feature matrix.

[0048] In a specific embodiment, step S3 further includes: Step S33: In each layer of the three-layer decoder structure, perform an upsampling operation on the feature map output by the previous layer decoder, and perform channel splicing between the upsampled feature map and the high-frequency components retained by the same-level skip connection branch. The concatenated feature maps are fused by convolution to restore them to their initial resolution, generating the final decoded feature map. Step S34: The final decoded feature map is mapped to a binarized high-temperature segmentation mask through a classifier; The outer contours of each connected region of the binarized high-heat segmentation mask are extracted as high-heat edge contours, and the total number of pixels inside the high-heat edge contours is counted and quantized to obtain the corresponding high-heat region area.

[0049] Specifically, it receives a low-resolution, high-semantic feature map from the decoder of the previous layer. It then performs a transposed convolution operation with a stride of 2 to expand the spatial dimensions of the low-resolution feature map. By dynamically calculating and filling interpolated features between adjacent pixels, it doubles the horizontal and vertical pixel resolution of the feature map.

[0050] At the same network depth level, the upsampled and amplified deep feature maps are extracted, and the dimensionality-reduced high-frequency components with the same spatial resolution retained in the skip connection branches are extracted simultaneously. While ensuring strict alignment of the spatial dimensions of both, a matrix concatenation operation is performed along the depth direction of the feature maps.

[0051] The composite feature map formed by concatenating the channels is input into a continuous array of features. The feature fusion module consists of standard convolutional layers, batch normalization layers, and ReLU activation functions. Through multiple sliding window convolutions within the local receptive field, the checkerboard effect caused by transposed convolutions is eliminated, deeply fusing deep semantics with shallow gradient edge features. This cascaded operation is repeated until, after passing through a three-layer decoder structure, the spatial resolution of the feature map is progressively enlarged and strictly aligned to the resolution of the initial input local anomaly heatmap, outputting the final decoded feature map.

[0052] The final decoded feature map is then fed into a pixel-level classifier at the end. The classifier consists of a size of... The convolutional layer is composed of a sigmoid activation function.

[0053] use The convolutional layer maps the feature map of multiple channels into a continuous probability matrix of a single channel; then, the sigmoid activation function nonlinearly compresses the value of each pixel in the matrix to the (0,1) interval, which represents the confidence probability that each spatial location belongs to a real high-temperature hazard.

[0054] Based on a preset decision confidence threshold (usually set to 0.5), a step binarization decision is performed on the probability matrix. Pixels with a confidence probability greater than the threshold are set to 1 (determined as abnormal heat regions), while pixels with a confidence probability less than or equal to the threshold are set to 0 (determined as safe backgrounds). The output is a two-dimensional matrix composed of 0s and 1s, which is the binarized high-heat segmentation mask.

[0055] It should be noted that this invention fuses the abnormal temperature mask and the original temperature matrix of the current video frame through pixel-by-pixel mapping and multiplication, and inputs the resulting local abnormal heat map into a three-layer encoder for downsampling feature extraction. By adopting a feature map update and transfer method that performs Haar wavelet downsampling in each layer, the input feature map can be decomposed into low-frequency and high-frequency components. Low-pass features are further extracted, and the high-frequency components are convolved and dimensionality reduced to the skip connection branches of the same layer. This effectively solves the problem of the decoder lacking original spatial boundary guidance during upsampling recovery, and has the effect of accurately quantizing the boundaries of connected domains and outputting high-fidelity, high-thermal edge contours.

[0056] In a specific embodiment, step S4 specifically includes: Step S41: Extract the horizontal and vertical pixel dimensions from the original temperature matrix within the initial region of interest of the current video frame, and update the product of the horizontal and vertical pixel dimensions to the reference area of ​​the initial region of interest.

[0057] Specifically, the two-dimensional spatial boundary coordinates of the initial region of interest in the infrared image of the current video frame are obtained; the boundary coordinates include the spatial coordinates of the top-left pixel vertex of the region. And the spatial coordinates of the bottom right pixel vertex of the region. ; Subtract the x-coordinate of the top-left pixel from the x-coordinate of the bottom-right pixel vertex to calculate the horizontal pixel size W of the original temperature matrix. Subtract the y-axis coordinate of the top-left pixel from the y-axis coordinate of the bottom-right pixel vertex to calculate the vertical pixel size H of the original temperature matrix. .

[0058] In a specific embodiment, step S4 further includes: Step S42: Multiply the baseline area of ​​the initial region of interest by the area ratio coefficient to calculate the relative area threshold; Specifically, the expression for calculating the relative area threshold is: ; in, The relative area threshold. The baseline area of ​​the initial region of interest. This is the area ratio coefficient.

[0059] The preferred area ratio coefficient is [0.05, 0.15]. The typical physical form of an electrical fire hazard in its initial stage is point-source heating. In the early stages, the high-temperature area is highly concentrated, and the heat has not yet been extensively conducted to the entire surface of the equipment. If the area ratio coefficient is set too high, it means that the heating area must reach one-quarter of the total reference area of ​​the equipment to trigger an alarm. When 25% of the equipment area is abnormally hot, the insulation layer has often already melted and open flames have formed, defeating the original technical purpose of early hazard detection.

[0060] Step S43: Determine whether the area of ​​the high-heat region is greater than the relative area threshold; If so, a warning signal with associated target device coordinate information is generated, and the high-heat edge contour is output simultaneously.

[0061] If not, the device is determined to be in normal thermal state, the warning determination for the current video frame ends, and the process returns to step S2 to continue acquiring the next video frame for monitoring.

[0062] Example 2, refer to Figure 2 An infrared image-based electrical fire hazard detection and monitoring system is provided, including: The region expansion module is used to obtain the initial region of interest of the target device and expand the bounding box of the initial region of interest outward to generate an expanded region of interest. The anomaly detection module is used to obtain the original temperature matrix of the extended region of interest in consecutive video frames, and to calculate the moving average temperature baseline and spatial temperature gradient of each pixel in the original temperature matrix of the current video frame. Candidate outliers are identified based on the moving average temperature baseline and spatial temperature gradient, and an anomaly temperature mask is output. The edge restoration module is used to input the abnormal temperature mask into the shallow feature extraction network for downsampling and upsampling restoration, and output the high-temperature edge contour and the corresponding high-temperature area area; Shallow feature extraction networks consist of a three-layer encoder structure and a three-layer decoder structure; The early warning output module is used to obtain the reference area of ​​the initial region of interest, calculate the relative area threshold based on the reference area of ​​the initial region of interest, and output the early warning signal and the high-heat edge contour based on the relative area threshold.

[0063] Example 3: Assume that the physical surface area of ​​a certain electrical device in real three-dimensional space is... The actual physical area of ​​the high-heat zone on its surface where potential hazards may occur is... .

[0064] The infrared camera has a focal length of f and a physical distance of D from the lens to the device.

[0065] Based on the optical perspective principle of pinhole imaging, the physical area of ​​the pixel mapped onto the photosensitive element is related to... Proportional.

[0066] Therefore, in the infrared image matrix: The pixel area of ​​the initial region of interest can be expressed as ; The pixel area of ​​the hot region segmented by the shallow feature extraction network can be expressed as: ; The conditions for triggering a fire alarm include: If the area of ​​the segmented high-heat pixels is greater than the relative area threshold, then... ; Substituting the perspective mapping formula into both sides of the equation, the final judgment condition is equivalent to... ; in, This is the area ratio coefficient.

[0067] Regardless of how far the infrared temperature camera is installed from the target device, and regardless of the focal length of the lens used, because the optical scale factors on both sides of the determination formula are canceled out, the system ultimately monitors the true ratio of the physical lesion area to the total physical area of ​​the device.

[0068] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, and the software modules can be housed in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.

[0069] In this specification, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0070] It should be noted that the above description of the embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. It should be pointed out that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A method for detecting and monitoring electrical fire hazards based on infrared images, characterized in that, Includes the following steps: Step S1: Obtain the initial region of interest of the target device, and expand the bounding box of the initial region of interest outward to generate an extended region of interest; Step S2: Obtain the original temperature matrix of the extended region of interest in the continuous video frames, and calculate the moving average temperature baseline and spatial temperature gradient of each pixel in the original temperature matrix of the current video frame. Candidate outliers are identified based on the moving average temperature baseline and spatial temperature gradient, and an anomaly temperature mask is output. Step S3: Input the abnormal temperature mask into the shallow feature extraction network for downsampling and upsampling recovery, and output the high-temperature edge contour and the corresponding high-temperature area. The shallow feature extraction network includes a three-layer encoder structure and a three-layer decoder structure; Step S4: Obtain the reference area of ​​the initial region of interest, calculate the relative area threshold based on the reference area of ​​the initial region of interest, and output the warning signal and high-heat edge contour based on the relative area threshold.

2. The method for detecting and monitoring electrical fire hazards based on infrared images as described in claim 1, characterized in that, The processing logic for generating extended regions of interest is as follows: Keeping the center coordinates of the initial region of interest unchanged, the pixel width and pixel height of the initial region of interest are expanded respectively, and the calculation expression is as follows: ； ； in, and These represent the pixel width and pixel height of the expanded region of interest, respectively. For expansion coefficient, and These are the pixel width and pixel height of the initial region of interest, respectively.

3. The method for detecting and monitoring electrical fire hazards based on infrared images as described in claim 1, characterized in that, Step S2 specifically includes: Step S21: Obtain the original temperature matrix of the extended region of interest in the continuous video frames, and calculate the moving average temperature baseline and spatial temperature gradient of each pixel in the original temperature matrix of the extended region of interest in the current video frame. Step S22: When the difference between the current temperature value of the target pixel and the moving average temperature baseline is greater than the first threshold and the spatial temperature gradient is greater than the second threshold, the corresponding target pixel is marked as a candidate anomaly. Step S23: If the candidate anomaly point persists in N consecutive video frames, the region including the candidate anomaly point is extracted and the anomaly temperature mask is output.

4. The method for detecting and monitoring electrical fire hazards based on infrared images as described in claim 3, characterized in that, The formula for calculating the moving average temperature baseline is: ； in, Extend the moving average temperature baseline of the pixels in the original temperature matrix within the region of interest of the current video frame. To expand the region of interest in the current video frame with the actual temperature values ​​of the pixels in the original temperature matrix, To extend the moving average temperature baseline of the pixels in the original temperature matrix within the region of interest from the previous video frame, This is the smoothing coefficient.

5. The method for detecting and monitoring electrical fire hazards based on infrared images as described in claim 4, characterized in that, The processing logic for calculating the spatial temperature gradient of each pixel is as follows: Extract the target pixel and its surrounding 3×3 original temperature matrix; The weighted temperature difference in the horizontal direction and the weighted temperature difference in the vertical direction of the original 3×3 temperature matrix are calculated using the first-order discrete difference operator, and are used as the horizontal gradient component and the vertical gradient component, respectively. The spatial temperature gradient of the target pixel is obtained by calculating the square root of the sum of the squares of the horizontal gradient components and the vertical gradient components. The target pixel is a pixel in the original temperature matrix within the extended region of interest of the current video frame.

6. The method for detecting and monitoring electrical fire hazards based on infrared images as described in claim 1, characterized in that, Step S3 specifically includes: Step S31: Multiply the abnormal temperature mask with the original temperature matrix in the extended region of interest of the current video frame pixel by pixel to extract the local abnormal heat map, and use the local abnormal heat map as the initial input of the three-layer encoder structure. Step S32: In each layer of the three-layer encoder structure, a Haar wavelet downsampling operation is performed on the input feature map to decompose the input feature map into low-frequency components and high-frequency components; High-frequency components include horizontal components, vertical components, and diagonal components; The low-frequency components are input into the next layer encoder for feature extraction, the high-frequency components are subjected to convolutional dimensionality reduction, and the skip connection branches of the corresponding layer are retained.

7. The method for detecting and monitoring electrical fire hazards based on infrared images as described in claim 6, characterized in that, Step S3 also includes: Step S33: In each layer of the three-layer decoder structure, an upsampling operation is performed on the feature map output by the previous layer decoder, and the upsampled feature map is channel-stitched with the high-frequency components retained by the same-level skip connection branch. The concatenated feature maps are fused by convolution to restore them to their initial resolution, generating the final decoded feature map. Step S34: The final decoded feature map is mapped to a binarized high-temperature segmentation mask through a classifier; The outer contours of each connected region of the binarized high-heat segmentation mask are extracted as high-heat edge contours, and the total number of pixels inside the high-heat edge contours is counted and quantized to obtain the corresponding high-heat region area.

8. The method for detecting and monitoring electrical fire hazards based on infrared images as described in claim 1, characterized in that, Step S4 specifically includes: Step S41: Extract the horizontal and vertical pixel dimensions from the original temperature matrix within the initial region of interest of the current video frame, and update the product of the horizontal and vertical pixel dimensions to the reference area of ​​the initial region of interest.

9. The method for detecting and monitoring electrical fire hazards based on infrared images as described in claim 8, characterized in that, Step S4 also includes: Step S42: Multiply the reference area of ​​the initial region of interest by the area ratio coefficient to calculate the relative area threshold; Step S43: Determine whether the area of ​​the high-heat region is greater than the relative area threshold; If so, a warning signal with associated target device coordinate information is generated, and the high-heat edge contour is output simultaneously.

10. An infrared image-based electrical fire hazard detection and monitoring system, applied in the infrared image-based electrical fire hazard detection and monitoring method as described in any one of claims 1-9, characterized in that, include: The region expansion module is used to obtain the initial region of interest of the target device and expand the bounding box of the initial region of interest outward to generate an expanded region of interest. The anomaly detection module is used to obtain the original temperature matrix of the extended region of interest in consecutive video frames, and to calculate the moving average temperature baseline and spatial temperature gradient of each pixel in the original temperature matrix of the current video frame. Candidate outliers are identified based on the moving average temperature baseline and spatial temperature gradient, and an anomaly temperature mask is output. The edge recovery module is used to input the abnormal temperature mask into the shallow feature extraction network for downsampling and upsampling recovery, and output the high-temperature edge contour and the corresponding high-temperature area. The shallow feature extraction network includes a three-layer encoder structure and a three-layer decoder structure; The early warning output module is used to obtain the reference area of ​​the initial region of interest, calculate the relative area threshold based on the reference area of ​​the initial region of interest, and output an early warning signal and a high-heat edge contour based on the relative area threshold.

Images