Glass curtain wall panel spontaneous explosion early warning device based on visual identification

Abstract

The invention particularly relates to a glass curtain wall panel spontaneous explosion early warning device based on visual identification, and relates to the technical field of computer vision and building detection. An image preprocessing and registering module; a feature extraction and fusion module; and a decision-making and positioning module. According to the invention, through cooperation of visible light and thermal imaging multi-mode fusion and an active thermal excitation technology, inherent defects of a single sensor are effectively overcome; the visible light camera captures morphological characteristics such as butterfly rash and cracks on the glass surface, the thermal imaging camera captures temperature abnormity caused by stress unbalance in a spontaneous explosion area, and information of the visible light camera and the thermal imaging camera is complementary. The active thermal excitation unit actively amplifies the thermal behavior difference between a crack area and a normal area through controllable heating and cooling processes, and gets rid of dependence on the temperature difference of the natural environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and building detection technology, and particularly relates to a glass curtain wall panel self-explosion early warning device based on visual identification. BACKGROUND

[0002] Glass curtain walls have become the core enclosure structure of modern high-rise buildings due to their advantages such as beauty, transparency, energy saving, and are widely used in commercial complexes, office buildings, landmark buildings and other scenes.

[0003] However, glass panels are prone to self-explosion due to factors such as stress concentration, impurity defects, sudden temperature changes or long-term aging during production, installation and service, which not only may cause property loss, but also may pose a serious safety hazard to pedestrians and vehicles around the building, so early warning and accurate detection of glass curtain wall panel self-explosion have important engineering significance.

[0004] Existing glass curtain wall self-explosion detection methods mainly include manual inspection and single sensor automatic detection: manual inspection relies on the naked eye observation of operation and maintenance personnel or the use of simple tools for troubleshooting, which is low in efficiency, strong in subjectivity, and difficult to cover all areas of the outer facade of high-rise buildings, and there is a risk of missed detection; Single sensor automatic detection mainly uses visible light cameras or thermal imaging cameras to work alone, wherein the visible light camera is easily disturbed by environmental light, glass reflection, surface stains and other factors, has limited recognition ability for early small cracks and high false positive rate, and the thermal imaging camera relies on the temperature difference formed by the natural environment, and in the scene where the sunlight is unstable and the environmental temperature difference is small, the thermal anomaly signal is weak, resulting in high false negative rate, so both single modal detection technologies are difficult to meet the requirements of detection accuracy, reliability and environmental adaptability in engineering practice.

[0005] Therefore, a glass curtain wall panel self-explosion early warning device based on visual identification is proposed to solve the above-mentioned problems. SUMMARY

[0006] The purpose of the present application is to solve the above problems and propose a glass curtain wall panel self-explosion early warning device based on visual identification.

[0007] In order to achieve the above purpose, the present application adopts the following technical solutions: A glass curtain wall panel self-explosion early warning device based on visual identification, comprising: A multi-source data perception module configured to acquire complete and effective original visible light, thermal imaging data and thermal excitation process information through synchronous acquisition of visible light cameras and thermal imaging cameras and cooperative work of active thermal excitation units; An image preprocessing and registration module configured to optimize the quality of visible light and thermal imaging images, and to achieve spatial registration of multi-modal images through feature matching and transformation calculation, thereby ensuring the spatial consistency of fusion; A feature extraction and fusion module configured to extract visible light features and thermal imaging features through a double-branch deep learning network, and to achieve deep feature fusion through a cross-modal attention mechanism, thereby breaking through the limitations of single-modal information; A decision and positioning module configured to achieve self-explosion judgment through hard / soft decision logic based on the fused features, and to complete physical positioning and panel number mapping through coordinate conversion.

[0008] Preferably, the multi-source data perception module specifically includes: After determining the core parameters of the visible light camera, the installation and synchronization processing are performed according to the preset requirements to output corresponding image data: After determining the core parameters of the thermal imaging camera, the corresponding calibration is performed to output the required image data; The hardware composition of the active thermal excitation unit includes: an array type infrared heating lamp with a single lamp power of 50-100 W, a lamp array coverage range matched with the camera field of view, and a temperature control module and a timing controller; The working process is as follows: Preheating stage: start heating 10s before detection, and gradually increase the power to 50%; Excitation stage: full-power heating for 3-5s to ensure that the glass surface is heated by 2-5℃; Cooling monitoring stage: continue to collect for 30s after stopping heating, and record the heat diffusion process.

[0009] Preferably, the image preprocessing and registration module specifically includes: The visible light image is preprocessed including illumination normalization, reflection suppression, and denoising and sharpening; The thermal imaging image is preprocessed including radiation calibration, temperature scene normalization, and non-uniformity correction; Then, multi-modal image registration is performed, including: Feature extraction: Visible light image: SIFT algorithm is used to extract a preset number of feature points, and scale and rotation invariant features are retained; Thermal imaging image: first convert the temperature map into a grayscale image, and then use the ORB algorithm to extract feature points, and retain the corner points in the high temperature gradient area; Feature matching and transformation calculation: cross-modal feature matching is performed through the FLANN matcher, the RANSAC algorithm is used to remove false matches, and the perspective transformation matrix is calculated to make the thermal imaging image pixels and the visible light image pixels meet the preset requirements.

[0010] Preferably, the feature extraction and fusion module specifically comprises: A double-branch deep learning network is constructed, which is divided into: Visible light branch: ResNet50, remove the last 3 fully connected layers, keep the first 4 convolution stages, the output feature map size is 1 / 16 of the input image; Add attention gate after conv3 and conv4 layers, enhance the feature response of crack edge by learning weight, and suppress background interference; Thermal imaging branch: Lightweight network MobileNetV3, modify the input channel to 1, keep the first 3 convolution blocks, the output feature map size is consistent with the visible light branch, and the channel number is 512; Add temperature gradient perception layer after each convolution block to calculate the temperature gradient in a 3×3 neighborhood; The temperature gradient value is spliced as additional features with the convolution output to enhance the thermal anomaly features of the crack area.

[0011] Preferably, the feature layer fusion further comprises: Fusion level: select the visible light feature map output by the conv4 layer of the double-branch , thermal imaging feature map At this time, the features contain high-level semantics and retain spatial position information; The basic fusion operation includes: The visible light feature map , thermal imaging feature map are concatenated to obtain , which retains all features, and is compressed to 512 channels by 1×1 convolution subsequently; And the features with uniform channel number are respectively marked as 、 ; After the preset 、 weight factor, the 、 and its corresponding weight factor are weighted and summed to obtain .

[0012] Preferably, the cross-modal attention mechanism further comprises: Thermal guided visible light attention: the is reduced to 1 channel by 3×3 convolution, including generating a thermal attention map , which is multiplied pixel by pixel with to enhance the visible light features in the thermal anomaly area; Visible light guided thermal attention: the Down to 1 channel by 3x3 convolution, including generating visible light attention map , and Pixel by pixel multiplication, enhancing the thermal features of the morphological anomaly area; Output: fusion features , after 3x3 convolution and BatchNorm, into the subsequent classification and segmentation network.

[0013] Preferably, the decision and positioning module specifically includes: Output the crack probability map through the U-Net segmentation network. If the pixel probability of a certain area is ≥0.8 and the connected domain area is ≥50 pixels, it is determined as a suspected morphological anomaly; In the registered thermal map, the temperature difference between the corresponding area and the surrounding normal area is ≥0.5℃, and the temperature gradient is ≥0.1℃ / pixel, which is determined as a suspected thermal anomaly; Only when the overlap degree of suspected morphological anomaly and suspected thermal anomaly area is ≥70%, the final determination is self-explosion; Fusion features Output the self-explosion probability map through the U-Net++ segmentation network; Add a global average pooling layer at the last layer of the segmentation network to output the global probability P. Combined with the maximum connected domain probability of the pixel-level probability map, the comprehensive judgment is: if P≥0.7 and the average probability of the maximum connected domain is ≥0.65, it is determined as self-explosion.

[0014] Preferably, the still includes: Output the binary mask of the self-explosion area through the segmentation network, extract the minimum bounding rectangle, and record the top-left and bottom-right pixel coordinates ; Combine the camera intrinsic matrix , extrinsic matrix (rotation , translation ) and curtain three-dimensional model to convert pixel coordinates to world coordinates : Step 1: Get normalized image coordinates by camera distortion correction; Step 2: Calculate the world coordinates combined with the rotation , translation of the extrinsic parameters; Match the world coordinates with the curtain panel account to output the unique number of the self-explosion panel to match the positioning of the corresponding self-explosion panel; After obtaining the crack length , crack number , maximum temperature difference and thermal diffusion rate , determine the risk level.

[0015] Preferably, the still further comprises: The alarm and report generation module is configured to display detection information through multi-modal result visualization, realize hierarchical alarm based on risk levels, and generate a report containing detailed data and suggestions as an interactive interface of the system and the user.

[0016] To sum up, due to the adoption of the above technical solutions, the application has the following beneficial effects: 1. The application effectively solves the inherent defects of a single sensor through the synergy of visible light, thermal imaging multi-modal fusion and active thermal excitation technology; a visible light camera captures morphological features such as butterfly spots and cracks on the glass surface, a thermal imaging camera captures temperature anomalies generated by stress imbalance in the self-explosion area, and the information of the two is complementary; the active thermal excitation unit actively amplifies the thermal behavior difference between the crack area and the normal area through controllable heating and cooling processes, and breaks away from the dependence on natural environmental temperature difference.

[0017] 2. The application realizes deep interaction and fusion of visible light morphological features and thermal imaging thermal features through a double-branch deep learning network and a cross-modal attention mechanism, so that the two modal information guides and enhances each other in the decision-making process, significantly improves the feature expression ability and decision accuracy; at the same time, through hard / soft decision logic and multi-index risk grading, not only can the self-explosion event be accurately determined, but also the self-explosion risk can be divided into low, medium and high levels based on crack length, quantity, temperature difference, thermal diffusion rate and other quantitative indicators, providing a scientific decision basis for operation and maintenance personnel. BRIEF DESCRIPTION OF DRAWINGS

[0018] In the following description of exemplary embodiments in conjunction with the accompanying drawings, more details, features and advantages of the application are disclosed, in which: Figure 1 The figure is a system structure diagram of the application. DETAILED DESCRIPTION

[0019] Several embodiments of the application will be described in more detail below with reference to the accompanying drawings so as to enable those skilled in the art to implement the application. The application can be embodied in many different forms and purposes and should not be limited to the embodiments described herein. These embodiments are provided to make the application comprehensive and complete, and to fully convey the scope of the application to those skilled in the art. The embodiments do not limit the application.

[0020] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having a meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0021] Example 1 Its specific implementation method is combined with the appendix Figure 1 Please provide a detailed explanation.

[0022] Appendix Figure 1 This invention provides a structural block diagram of a glass curtain wall panel self-explosion early warning device based on visual recognition, which shows the connection relationship between the multi-source data sensing module and the decision and positioning module, and marks the main functional interaction flow of each module.

[0023] In this embodiment, it includes: The multi-source data sensing module is configured to acquire complete and effective raw visible light and thermal imaging data and thermal excitation process information through the synchronous acquisition of visible light cameras and thermal imaging cameras and the coordinated work of the active thermal excitation unit, providing a foundation for subsequent processing. Specifically, it includes: After determining the core parameters of the visible light camera, install and synchronize it according to preset requirements to output the corresponding image data: Key parameters: Resolution ≥ 4K (3840×2160), frame rate ≥ 30fps (supports dynamic scene capture), sensor size ≥ 1 inch (improves low-light performance), lens focal length 8-50mm (can zoom according to detection distance, covering a 10-50 meter curtain wall range), HDR mode (dynamic range ≥ 120dB) and autofocus function, supports polarized lens interface (optional, used to suppress reflections at specific angles).

[0024] Installation and Synchronization: Installed coaxially or with a fixed baseline (baseline distance ≤ 50cm to reduce parallax), and achieve frame-level synchronization with the thermal imaging camera through hardware trigger lines (such as GPIO synchronization signals) or timestamps (accuracy ≤ 1ms) to avoid registration errors caused by motion blur.

[0025] Data output: raw RAW format (retaining maximum information content) and JPEG preview image, including exposure parameters (shutter speed, ISO) and timestamp metadata.

[0026] After determining the core parameters of the thermal imaging camera, it is calibrated accordingly to output the required image data; Key parameters: The infrared detector type is an uncooled microbolometer, with a resolution of ≥640×512 (ensuring accuracy at long distances), a spectral response range of 8-14μm (atmospheric window, reducing attenuation), a NETD (noise equivalent temperature difference) of ≤20mK (capable of identifying minute temperature differences of 0.1℃), a frame rate of ≥30fps, a temperature measurement range of -20℃ to 150℃ (covering environments and heating scenarios), and supports radiation measurement mode (outputting temperature values ​​instead of relative grayscale).

[0027] Calibration requirements: Temperature calibration should be completed before shipment (error ≤ ±1℃), and blackbody calibration should be performed on-site every 3 months (using a 50℃ / 80℃ standard blackbody) to eliminate the influence of ambient temperature drift.

[0028] Data output: 16-bit temperature data (unit: °C, accuracy: 0.01 °C) and 8-bit pseudo-color preview image, including metadata such as detector temperature and ambient humidity.

[0029] The hardware components of the active thermal excitation unit include: an array of infrared heating lamps (wavelength 3-5μm, avoiding the 8-14μm band of thermal imaging cameras to prevent direct interference), a single lamp power of 50-100W, and the lamp array coverage area matched with the camera's field of view (e.g., a 2m×2m area), equipped with a temperature control module (to prevent overheating) and a timing controller (accuracy ±0.1s). The workflow is as follows: Preheating phase: Start heating 10 seconds before testing, gradually increasing the power to 50% (to avoid instantaneous thermal shock); Excitation phase: Full power heating for 3-5 seconds (adjust according to glass thickness, 4 seconds is recommended for 6mm glass), ensuring that the glass surface temperature rises by 2-5℃ (monitored in real time by thermal imager). Cooling monitoring phase: After heating is stopped, data is continuously collected for 30 seconds to record the heat diffusion process (the difference in heat dissipation rate between the cracked area and the normal area will gradually increase).

[0030] The active thermal excitation unit is linked with the dual cameras through the same trigger signal, and the start / end time of the excitation is synchronized with the camera frame to ensure complete recording of the thermal process.

[0031] The image preprocessing and registration module is configured to optimize the quality of visible light and thermal imaging images, and to achieve spatial registration of multimodal images through feature matching and transformation calculation, so as to ensure the spatial consistency of fusion. Specifically, it includes: Preprocessing of visible light images includes illumination normalization, reflection suppression, and noise reduction and sharpening; Illumination normalization: The CLAHE (Contrast Limit Adaptive Histogram Equalization) algorithm is adopted, with a block size of 8×8 and cliplimit=2.0, to solve the uneven brightness of the glass surface caused by the angle of illumination (such as dark edges and bright center).

[0032] Reflection suppression: If equipped with a polarization camera: acquire 0° / 90° dual-polarization images, separate the reflective area (high polarization degree) by calculating the degree of polarization, and perform adaptive smoothing on the reflective area; without polarization equipment: use multi-scale Gaussian filtering based on Retinex theory to decompose the image into reflection component (including texture) and illumination component (including reflection), and suppress the highlight area in the illumination component.

[0033] Denoising and sharpening: First, Gaussian noise (σ≤20) is removed using BM3D (block-matched 3D filter), and then the crack edges are enhanced using the Laplacian operator (kernel=3×3, weight 0.5) to preserve details while avoiding noise amplification.

[0034] The thermal imaging images undergo preprocessing including radiometric calibration, temperature scene normalization, and non-uniformity correction. Radiometric calibration involves converting the original grayscale values ​​to absolute temperature based on camera intrinsic parameters. , , The calibration coefficients are obtained through blackbody calibration, and the ambient temperature is compensated (for every ±5℃ difference in ambient temperature, the temperature measurement result is corrected by ±0.3℃).

[0035] Temperature scene normalization: Adaptive threshold stretching is used to normalize the current scene temperature range. Mapped to [0, 255] grayscale, where =Scene minimum temperature -0.5℃, =Highest scene temperature + 0.5℃, to ensure that minute temperature differences (such as 0.2℃) can be visually distinguished.

[0036] Non-uniformity correction: Combining spatial and temporal methods: Spatial domain: 3×3 median filtering is used to replace fixed noise (such as bad pixels); Temporal domain: Two-point correction method is used for slowly varying noise (such as detector drift), and the offset is calculated by using 5 consecutive frames of unexcited images to update the correction matrix in real time.

[0037] The above image processing procedures are all direct references to existing technologies and will not be elaborated upon here.

[0038] Then, multimodal image registration is performed, including: Feature extraction: Visible light images: Extract a predetermined number of feature points (at least 500 stable points) using the SIFT algorithm, preserving scale- and rotation-invariant features; Thermal imaging images: First, the temperature map is converted into a grayscale image (based on the normalization result), and feature points are extracted using the ORB algorithm (faster and more adaptable to real-time requirements), with a focus on preserving corner points in areas with high temperature gradients (such as window frames and glass edges); Feature matching and transformation calculation: Cross-modal feature matching is performed using the FLANN matcher, and the RANSAC algorithm is used to remove false matches (interior ratio ≥ 80%). The perspective transformation matrix is ​​then calculated. (3×3) to make the thermal imaging image pixels With visible light image pixels Meets preset requirements: ; Accuracy optimization: After registration, the reprojection error is evaluated (≤1.5 pixels). If the error exceeds the limit, dynamic registration compensation is enabled (based on the displacement of feature points in adjacent frames, updated in real time). (Matrix) to adapt to minor camera shakes (such as tripod swaying caused by wind).

[0039] The feature extraction and fusion module is configured to extract visible light morphological features and thermal imaging thermal features respectively using a dual-branch deep learning network, and combine a cross-modal attention mechanism to achieve deep feature fusion, thus breaking through the limitations of single-modal information. Specifically, it includes: Construct a two-branch deep learning network, which consists of: Visible light branching (morphological feature extraction): Backbone network: ResNet50 (pre-trained on ImageNet), the last 3 fully connected layers are removed, and the first 4 convolutional stages (conv1-conv4) are retained. The output feature map size is 1 / 16 of the input image (e.g., 256×144×1024 for a 4K image). Targeted optimization: Add attention gate after conv3 and conv4 layers to enhance the feature response of crack edges (high gradient regions) by learning weights and suppress background (such as sky, walls) interference; Thermal imaging branch (thermal feature extraction): Backbone network: Lightweight MobileNetV3 (adapted to the low resolution of heatmaps), with the input channel modified to 1 (single-channel temperature map), retaining the first 3 convolutional blocks, and the output feature map size consistent with the visible light branch (adjusted by upsampling / downsampling to ensure spatial dimension matching), with 512 channels; Targeted optimization: Add a temperature gradient sensing layer after each convolutional block to calculate the temperature gradient within a 3×3 neighborhood. , Representing the temperature gradient, it is a vector that describes the rate of temperature change in space and reflects the severity of temperature change. The larger the value, the more significant the temperature change in that region. It represents temperature, specifically the temperature value of each pixel in a thermal imaging image; It is temperature right The partial derivative of the value represents the temperature at which the temperature changes. The rate of change in the horizontal direction, that is, the amount of temperature change per unit length; It is temperature right The partial derivative of the value represents the temperature at which the temperature changes. The rate of change in the direction (vertical direction), that is, the amount of temperature change per unit length; In the thermal imaging branch of glass curtain wall spontaneous explosion detection, this formula is used to calculate the temperature gradient, highlighting the difference in thermal changes between cracked areas and normal areas (the temperature gradient is usually larger in cracked areas due to stress concentration), thereby enhancing the identifiability of thermal anomaly features. This formula is a direct reference to existing technology and will not be elaborated here.

[0040] Temperature gradient values ​​are concatenated with the convolution output as additional features to enhance the thermal anomaly characteristics of the crack region.

[0041] It also includes feature layer fusion: Fusion level: Select the dual-branch conv4 layer to output visible light feature maps. Thermal imaging feature map (Visible light characteristic map) , This indicates that each element of this feature map is a real number. , These represent the height and width of the feature map, respectively, and 1024 is the number of channels in the feature map. (Thermal imaging feature map) At this point, the features contain both high-level semantics (such as cracks and normal areas) and spatial location information (can be located down to the 10×10 pixel level). Basic fusion operations include: Visible light feature map Thermal imaging feature map By connecting them in series, we obtain All features are preserved, and then compressed to 512 channels using a 1×1 convolution. And the features after unifying the number of channels are respectively marked as , ; Preset , After assigning weighting factors, respectively , The weighted summation is performed on its corresponding weighting factors to obtain the result. : , This is the weighting factor.

[0042] It also includes cross-modal attention mechanisms: Thermally guided visible light attention: Dimensionality is reduced to 1 channel using 3×3 convolution, including the generation of thermal attention maps. , ,and Pixel-wise multiplication ( Enhance the visible light characteristics of thermal anomaly regions; Visible light guides thermal attention: Dimensionality is reduced to 1 channel using 3×3 convolution, including the generation of visible light attention maps. , ,and Pixel-wise multiplication ( Enhance the thermal characteristics of morphologically abnormal areas; Output: Fusion features After 3×3 convolution and BatchNorm, it enters the subsequent classification and segmentation network.

[0043] The decision-making and positioning module is configured to determine self-destruction based on fusion features through hard / soft decision logic, complete physical positioning and panel number mapping by coordinate transformation, and assess the risk level based on morphology and thermal indicators to solve the core decision-making problem. Specifically, it includes: Integration of decision-making logic: Hard decision rule (low false alarms take precedence): Visible light branch determination: The crack probability map is output by the U-Net segmentation network. If the pixel probability of a certain region is ≥0.8 and the area of ​​the connected region is ≥50 pixels (corresponding to the actual size ≥1 square meter), it is determined to be a suspected morphological abnormality. Thermal imaging branch determination: In the registered thermal image, if the temperature difference between the corresponding area and the surrounding normal area is ≥0.5℃ (cooling stage after excitation), and the temperature gradient is ≥0.1℃ / pixel (i.e., the temperature difference between adjacent pixels is significant), it is determined to be a suspected thermal anomaly; Joint determination: Only when the overlap between the suspected morphological anomaly and the suspected thermal anomaly area is ≥70%, is it ultimately determined to be a spontaneous explosion; Soft decision rules (adapting to complex scenarios): Input: Fusion features The U-Net++ segmentation network outputs a self-destruct probability map (each pixel corresponds to the self-destruct probability); Classification Head: Add a global average pooling layer to the last layer of the segmentation network to output a global probability P(0-1). Combine this with the maximum connected component probability of the pixel-level probability graph to make a comprehensive judgment: if P≥0.7 and the average probability of the maximum connected component ≥0.65, it is judged as self-destruction; Decision verification: Perform time consistency verification on the judgment result (confirmation is only made if three consecutive frames are judged as self-destruction to avoid noise interference from a single frame).

[0044] The binary mask of the self-destruction region is output by the segmentation network. The minimum bounding rectangle is extracted, and the pixel coordinates of the top left and bottom right corners are recorded. ; Combined with camera intrinsic matrix extrinsic parameter matrix (rotation) Translation ) and the 3D model of the curtain wall (pre-modeled), with pixel coordinates Convert to world coordinates : Step 1: Camera distortion correction (based on intrinsic parameters) The normalized image coordinates are obtained. Step 2: Combining the rotation of extrinsic parameters Translation Calculate world coordinates with an error controlled within ±5cm (to meet panel positioning requirements). Match the world coordinates with the curtain wall panel ledger (including the coordinate range of each panel) and output the unique number of the self-destructing panel (e.g., the 5th panel on the 3rd floor of the east facade) to match the location of the corresponding self-destructing panel. Extract the crack length corresponding to the self-destruction region from the segmentation results of the visible light image. The process includes: Deep learning segmentation networks (such as U-Net) output a binary mask of the self-destructing region. Contour detection is performed on the mask (such as OpenCV's findContours function). The pixel length of each crack contour is calculated. Then, combined with camera calibration parameters (intrinsic matrix and extrinsic matrix), the pixel distance is converted into the actual physical length. Next, connected component analysis is performed on the contour of the visible light segmentation mask. Each independent connected component corresponds to one crack, and the number of cracks is obtained by counting the number of connected components. ; Extracting the maximum temperature difference from time-series data of thermal imaging images ; During the cooling phase of active thermal excitation (within 30 seconds after heating stops), the average temperature of the registered thermal map is extracted from both the self-destruction area and the surrounding normal area. The peak temperature difference between the two is calculated, which represents the maximum temperature difference. ; Based on time-series temperature data from thermal imaging, the cooling rate (rate of temperature change over time, unit: ℃ / s) of the normal region and the cracked region is calculated separately. Then, the difference between the two rates is calculated and the absolute value is taken to obtain the thermal diffusion rate. ; Based on crack length Number of cracks Maximum temperature difference and thermal diffusion rate Determine the risk level: Low risk: ; Medium risk:

[0045] High risk: .

[0046] The alarm and report generation module is configured to visualize detection information through multimodal results, implement graded alarms based on risk levels, and generate reports containing detailed data and recommendations. This serves as the interface between the system and the user, specifically including: Visualization of multimodal fusion results: Core View: Fusion overlay: The visible light image serves as the base image, with the self-destruction area marked by a solid red frame. A pseudo-color heatmap (JET color scale, with warm colors indicating high-temperature anomalies) is overlaid inside, with 50% transparency. Feature comparison map: The original visible light, the pre-processed visible light (de-reflection), the original thermal map, and the thermal gradient map are displayed in 4 sub-windows to highlight the areas of difference; Time series heatmap: Displays temperature change curves before excitation, during excitation, and 10s / 20s / 30s of cooling (comparing the average temperature of the self-destruction area and the normal area).

[0047] Labeling information: Overlay physical coordinates, panel number, risk level (labeled in red / yellow / green), and detection time (accurate to the second) onto the image.

[0048] Alarm mechanism: Tiered alarm: High risk: Triggers audible and visual alarms (on-site equipment), SMS push (responsible person), and platform pop-up (management system). If no confirmation is received within 10 minutes, the responsible person will be automatically dialed. Medium risk: SMS push, platform pop-up; if not confirmed within 30 minutes, a second reminder will be sent. Low risk: Only platform records are compiled and pushed out daily.

[0049] Alarm information includes panel number, risk level, location coordinates, thumbnail link, and emergency contact person.

[0050] A report is generated, with the following structure: Basic information: testing time, equipment number, environmental parameters (temperature, humidity, light); Detection results: Details of the self-destructing panel (number, coordinates, risk level), multimodal image evidence (original image, fused image); Data analysis: crack size measurement table, temperature change curve, and quantitative values ​​of risk assessment indicators; Repair recommendations: Prioritize handling based on risk level (replace within 24 hours for high risk, inspect within 72 hours for medium risk), temporary protective measures (such as setting up a warning area), and spare parts model reference (matching panel size / model).

[0051] Output formats: PDF report (printable) and structured JSON data (for integration with management systems), with support for automatic archiving to cloud databases.

[0052] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0053] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0054] It should be noted that, in this document, the use of relational terms such as "first" and "second" is merely for distinguishing one entity or operation from another, and does not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0055] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0056] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0057] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0058] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0059] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0060] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0061] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A visual recognition-based early warning device for spontaneous explosion of glass curtain wall panels, characterized in that, include: The multi-source data sensing module is configured to acquire complete and effective raw visible light and thermal imaging data and thermal excitation process information through the synchronous acquisition of visible light camera and thermal imaging camera and the coordinated work of active thermal excitation unit. The image preprocessing and registration module is configured to optimize the quality of visible light and thermal imaging images, and to achieve spatial registration of multimodal images through feature matching and transformation calculation, so as to ensure the spatial consistency of fusion. The feature extraction and fusion module is configured to extract visible light morphological features and thermal imaging thermal features respectively using a dual-branch deep learning network, and combine a cross-modal attention mechanism to achieve deep feature fusion, thus breaking through the limitations of single-modal information. The decision-making and positioning module is configured to determine self-destruction based on fused features through hard / soft decision logic, and to complete physical positioning and panel number mapping by combining coordinate transformation.

2. The glass curtain wall panel self-explosion early warning device based on visual recognition according to claim 1, characterized in that, The multi-source data sensing module specifically includes: After determining the core parameters of the visible light camera, install and synchronize it according to preset requirements to output the corresponding image data: After determining the core parameters of the thermal imaging camera, it is calibrated accordingly to output the required image data; The hardware components of the active thermal excitation unit include: an array of infrared heating lamps, with a single lamp power of 50-100W, the lamp array coverage matching the camera's field of view, and equipped with a temperature control module and a timing controller; The workflow is as follows: Preheating phase: Heating begins 10 seconds before testing, with power gradually increasing to 50%. Excitation phase: Full power heating for 3-5 seconds to ensure the glass surface temperature rises by 2-5℃; Cooling monitoring phase: After heating is stopped, data is continuously collected for 30 seconds to record the heat diffusion process.

3. The glass curtain wall panel self-explosion early warning device based on visual recognition according to claim 1, characterized in that, The image preprocessing and registration module specifically includes: Preprocessing of visible light images includes illumination normalization, reflection suppression, and noise reduction and sharpening; The thermal imaging images undergo preprocessing including radiometric calibration, temperature scene normalization, and non-uniformity correction. Then, multimodal image registration is performed, including: Feature extraction: Visible light images: Extract a predetermined number of feature points using the SIFT algorithm, preserving scale- and rotation-invariant features; Thermal imaging images: First, convert the temperature map to a grayscale image, then use the ORB algorithm to extract feature points, retaining corner points in areas with high temperature gradients; Feature matching and transformation calculation: Cross-modal feature matching is performed using the FLANN matcher, RANSAC algorithm is used to eliminate mismatches, and perspective transformation matrix is ​​calculated to ensure that the pixels of thermal imaging image and visible light image meet the preset requirements.

4. The glass curtain wall panel self-explosion early warning device based on visual recognition according to claim 1, characterized in that, The feature extraction and fusion module specifically includes: Construct a two-branch deep learning network, which consists of: Visible light branch: ResNet50, remove the last 3 fully connected layers, retain the first 4 convolutional stages, and the output feature map size is 1 / 16 of the input image; An attention gate is added after the conv3 and conv4 layers to enhance the feature response of the crack edge by learning weights and suppressing background interference. Thermal imaging branch: The lightweight network MobileNetV3 has its input channel modified to 1, retains the first 3 convolutional blocks, and outputs a feature map with the same size as the visible light branch, with 512 channels. A temperature gradient sensing layer is added after each convolutional block to calculate the temperature gradient in a 3×3 neighborhood. Temperature gradient values ​​are concatenated with the convolution output as additional features to enhance the thermal anomaly characteristics of the crack region.

5. The glass curtain wall panel self-explosion early warning device based on visual recognition according to claim 4, characterized in that, It also includes feature layer fusion: Fusion level: Select the dual-branch conv4 layer to output visible light feature maps. Thermal imaging feature map At this point, the features contain both high-level semantics and retain spatial location information; Basic fusion operations include: Visible light feature map Thermal imaging feature map By connecting them in series, we obtain All features are preserved, and then compressed to 512 channels using a 1×1 convolution. And the features after unifying the number of channels are respectively marked as , ; Preset , After assigning weighting factors, respectively , The weighted summation is performed on its corresponding weighting factors to obtain the result. .

6. The glass curtain wall panel self-explosion early warning device based on visual recognition according to claim 5, characterized in that, It also includes cross-modal attention mechanisms: Thermally guided visible light attention: Dimensionality is reduced to 1 channel using 3×3 convolution, including the generation of thermal attention maps. ,and Pixel-by-pixel multiplication enhances the visible light characteristics of thermal anomaly regions; Visible light guides thermal attention: Dimensionality is reduced to 1 channel using 3×3 convolution, including the generation of visible light attention maps. ,and Pixel-by-pixel multiplication enhances the thermal characteristics of regions with abnormal shapes. Output: Fusion features After 3×3 convolution and BatchNorm, it enters the subsequent classification and segmentation network.

7. A glass curtain wall panel self-explosion early warning device based on visual recognition according to claim 6, characterized in that, The decision-making and positioning module specifically includes: The crack probability map is output by the U-Net segmentation network. If the pixel probability of a certain region is ≥0.8 and the area of ​​the connected region is ≥50 pixels, it is judged as a suspected morphological abnormality. In the registered thermal map, if the temperature difference between the corresponding area and the surrounding normal area is ≥0.5℃ and the temperature gradient is ≥0.1℃ / pixel, it is judged as a suspected thermal anomaly. Only when the overlap between the suspected morphological anomaly and the suspected thermal anomaly area is ≥70%, is it ultimately determined to be a spontaneous explosion; Fusion features The U-Net++ segmentation network outputs a self-destruct probability map; A global average pooling layer is added to the last layer of the segmentation network to output a global probability P. Combined with the maximum connected component probability of the pixel-level probability graph, a comprehensive judgment is made: if P ≥ 0.7 and the average probability of the maximum connected component ≥ 0.65, it is judged as self-destruction.

8. A glass curtain wall panel self-explosion early warning device based on visual recognition according to claim 7, characterized in that, Also includes: The binary mask of the self-destruction region is output by the segmentation network, the minimum bounding rectangle is extracted, and the pixel coordinates of the top left and bottom right corners are recorded. ; Combined with camera intrinsic matrix extrinsic parameter matrix (rotation) Translation ) and the 3D model of the curtain wall, including pixel coordinates Convert to world coordinates : Step 1: Obtain normalized image coordinates through camera distortion correction; Step 2: Combining the rotation of extrinsic parameters Translation Calculate world coordinates; Match the world coordinates with the curtain wall panel ledger and output the unique number of the self-destructing panel to match the location of the corresponding self-destructing panel; Obtain the crack length separately Number of cracks Maximum temperature difference and thermal diffusion rate Then, the risk level is determined.

9. A glass curtain wall panel self-explosion early warning device based on visual recognition according to claim 1, characterized in that, Also includes: The alarm and report generation module is configured to visualize detection information through multimodal results, implement graded alarms based on risk levels, and generate reports containing detailed data and suggestions, serving as an interface for interaction between the system and the user.

Images