Method, system, medium and program product for detecting absence of glass curtain wall tray

Abstract

The application discloses a kind of glass curtain wall tray missing detection method, system, medium and program product, the method of the application includes the infrared image of glass curtain wall tray is input into pre-trained target detection model to carry out target detection to obtain the detection result including crossbeam and tray;Determine the type of detected crossbeam is complete crossbeam, left half crossbeam or right half crossbeam;Carry out tray missing judgment, including according to the type of detected crossbeam (including complete crossbeam, left half crossbeam or right half crossbeam) and the number of tray contained in detection result, to determine that the detected crossbeam exists tray missing.The application aims to solve the problems of weak small target features, strong crossbeam area interference, easy to miss detection of tray and difficult to accurately determine the missing state in existing glass curtain wall tray missing detection method, to realize low-cost, efficient, high-precision glass curtain wall tray missing detection.

Description

Technical Field

[0001] This invention relates to the field of safety inspection and image processing technology for urban high-rise buildings, specifically to a method, system, medium, and program product for detecting missing glass curtain wall trays. Background Technology

[0002] With the continuous increase in the number of high-rise buildings in cities and the widespread application of glass curtain walls in building envelope systems, the long-term service safety, operational stability, and condition monitoring and assessment of curtain wall structures have gradually become a key focus in the field of engineering operation and maintenance. Trays are load-bearing components installed on the horizontal beams inside the glass curtain wall, typically attached to both sides of the internal horizontal beams, with two trays generally installed on each beam. Because the trays are located inside the curtain wall and are highly concealed, it is difficult to determine their integrity through external observation; therefore, indirect detection using methods such as infrared imaging is usually required. If a tray is missing, it will affect the support and connection reliability of the related structure, thus requiring effective identification of its absence. Existing methods for tray loss detection have the following problems: 1. Traditional tray condition detection mainly relies on manual inspection or auxiliary contact detection methods. Manual inspection has limited operating space, long observation and recording cycles, and data analysis relies on the subjective judgment of the inspectors; contact detection methods have shortcomings such as high deployment costs, complex implementation, and difficult maintenance, making it difficult to meet the needs of large-scale, continuous engineering inspection. 2. Since the trays are located inside the curtain wall, image detection methods mainly use infrared imaging. However, in infrared images, trays typically appear as small targets with limited effective imaging areas and feature information, making them prone to detail loss during multi-layer downsampling in networks. Furthermore, the blurred edges and low contrast of tray targets hinder the stable extraction of effective features by existing detection methods, leading to missed detections and insufficient recognition accuracy. 3. Tray absence is not a feature directly visible in the image. In infrared images, what is typically visible are beams and existing trays, while tray absence indicates the absence of a corresponding target at a location where a tray should have been. In other words, the "missing" element itself lacks a clear shape, boundary, or texture, making it undetectable like a regular target. Therefore, conventional target detection methods alone are insufficient to directly determine if a tray is missing. 4. Beams in infrared images typically appear as long, thin strips with significant length, narrow width, and uneven local response. While existing detection methods can roughly identify the beam's location, they often suffer from insufficient detection box coverage, detection of only the middle area, and incomplete detection at the ends. Unstable beam detection results further affect the correlation analysis between the tray and the beam, reducing the reliability of subsequent tray absence determinations. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method, system, medium and program product for detecting missing trays in glass curtain walls, in view of the above-mentioned problems in the existing tray missing detection methods. The present invention aims to solve the problems of weak small target features, strong interference in the beam area, easy missed detection of trays and difficulty in accurately determining the missing status in the existing tray missing detection methods, so as to achieve low cost, high efficiency and high precision tray missing detection.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for detecting missing glass curtain wall trays includes the following steps: S1, inputting an infrared image of the glass curtain wall tray into a pre-trained target detection model for target detection to obtain detection results including beams and trays; S2, classifying the detected beams in the detection results to determine whether the detected beam type is a complete beam, a left half beam, or a right half beam; S3, determining whether a tray is missing. If the detected beam type is a complete beam, it is determined whether the number of trays included in the detection result is 2. If the number of trays included in the detection result is not 2, it is determined that the detected beam has a missing tray. If the detected beam type is a left half beam or a right half beam, it is determined whether the number of trays included in the detection result is 1. If the number of trays included in the detection result is not 1, it is determined that the detected beam has a missing tray.

[0005] Optionally, after determining that a tray is missing in the detected beam in step S3, the missing tray labeling is also included: when the detected beam type is a complete beam, if the detection result includes a tray count of 1, then compare whether the tray associated with the current beam is located to the left or right of the center of the current beam. If it is located to the left, then label the missing tray box on the right side of the current beam; if it is located to the right, then label the missing tray box on the left side of the current beam; if the number of trays associated with the current beam is 0, then determine that the missing tray boxes are labeled on both sides of the current beam; when the detected beam type is a left half beam, if the number of trays associated with the current beam is 0, then label the missing tray box on the left side of the current beam; when the detected beam type is a right half beam, if the number of trays associated with the current beam is 0, then label the missing tray box on the right side of the current beam.

[0006] Optionally, the target detection model is an improved YOLOv11-OBB model. The improved YOLOv11-OBB model is based on the YOLOv11-OBB model consisting of a backbone feature extraction network, a neck feature fusion network, and a detection network. The improved YOLOv11-OBB model introduces a beam perception guidance module (BAGM) in the neck feature fusion network to output the fused features to the detection network for target detection to obtain detection results including beams and trays.

[0007] Optionally, the beam perception guidance module BAGM includes two parallel branches: the first branch passes the input features through two stacked background convolutions (BGConv) to obtain the direction-aware enhanced features beam_feat, and then passes them through a 1×1 convolution, batch normalization, and a Sigmoid activation function to generate a guidance weight map beam_map of the same scale as the input features. The guidance weight map beam_map is multiplied by the original input features and then added to the original input features to obtain the guidance feature guide_feat to achieve guidance enhancement of the input features; the second branch passes the input features through a 3×3 convolutional layer, batch normalization, and a SiLU activation function to obtain the local detail features local_feat; the guidance feature guide_feat obtained from the first branch and the local detail features local_feat obtained from the second branch are concatenated in the channel dimension, and then passed through a 1×1 convolutional layer, batch normalization, and a SiLU activation function to obtain the output features of the beam perception guidance module BAGM.

[0008] Optionally, the background convolution BGConv includes three parallel branches: the first branch sequentially processes the input features through a 1×1 convolution, batch normalization, SiLU activation function, 3×3 convolution, batch normalization, and SiLU activation function to generate a spatial attention mask, in which the effective area of ​​the beam and the background area are distinguished by values ​​of 1 and 0; the second branch sequentially processes the input features through a 1×7 convolution, batch normalization, and SiLU activation function to generate a horizontal enhancement feature map; the third branch sequentially processes the input features through a 3×3 convolution, batch normalization, and SiLU activation function to generate a local spatial feature map; the spatial attention mask and the horizontal enhancement feature map are multiplied element-wise and then concatenated with the local spatial feature map, and then processed through a 1×1 convolution layer, batch normalization, and SiLU activation function to obtain the final output features of the background convolution BGConv.

[0009] Optionally, the backbone feature extraction network outputs four backbone features of different scales, P1 to P4, to the neck feature fusion network. The processing of the four backbone features of different scales by the neck feature fusion network includes: upsampling backbone feature P1 and concatenating it with backbone feature P2, then extracting feature F1 using the first simple mode C3k2 feature extraction module; upsampling feature F1 and concatenating it with backbone feature P3, then extracting feature F2 using the second simple mode C3k2 feature extraction module; passing feature F2 through a convolutional layer, upsampling it, and concatenating it with backbone feature P4, then extracting feature F3 using the third simple mode C3k2 feature extraction module; passing feature F3 through the first beam perception guidance module BAGM to obtain the first output feature O1 and outputting it to the first rotated bounding box detection head of the detection network; simultaneously, the first... The output feature O1 is concatenated with feature F2 after passing through a convolutional layer, and then extracted by the fourth simple mode C3k2 feature extraction module to obtain feature F4. Feature F4 is then passed through the second beam perception guidance module BAGM to obtain the second output feature O2, which is then output to the second rotating bounding box detection head of the detection network. Simultaneously, the second output feature O2 is concatenated with feature F1 after passing through a convolutional layer, and then dynamically weighted and fused by the dynamic weighted feature fusion module DWFF. This is then extracted by the fifth simple mode C3k2 feature extraction module to obtain the third output feature O3, which is then output to the third rotating bounding box detection head of the detection network. At the same time, the third output feature O3 is concatenated with feature P1 after passing through a convolutional layer, and then extracted by a complex mode C3k2 feature extraction module to obtain the fourth output feature O4, which is then output to the fourth rotating bounding box detection head of the detection network.

[0010] Optionally, the Dynamic Weighted Feature Fusion Module (DWFF) includes two parallel branches. Each branch processes one input feature and multiplies the input feature by a learnable fusion weight after passing it through a 1×1 convolution, batch normalization, and SiLU activation function. The output features of the two parallel branches are then added together to obtain the fused features. The fused features are then passed through a 1×7 convolution, batch normalization, and SiLU activation function to expand the lateral receptive field and enhance the continuity of the beam structure, resulting in lateral continuity enhancement features. These lateral continuity enhancement features are then processed by a coordinate attention module for direction-aware modeling, followed by a 3×3 convolution, batch normalization, and SiLU activation function for refinement. Finally, the refined features are added to the fused features via residual connections to obtain the output features of the Dynamic Weighted Feature Fusion Module (DWFF).

[0011] The present invention also provides a missing glass curtain wall tray detection system, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the missing glass curtain wall tray detection method.

[0012] The present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the missing glass curtain wall tray method by a processor.

[0013] The present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the missing detection method for the glass curtain wall tray via a processor.

[0014] Compared with existing technologies, the present invention mainly achieves the following beneficial effects: Tray absence is not a feature directly visible in an image. In infrared images, what is typically visible are beams and existing trays, while tray absence indicates the absence of a corresponding target at a location where a tray should have been. Existing technologies usually only output the detection results for beam and tray targets individually, lacking a mechanism for further logical analysis and relational reasoning of the detection results. Therefore, it is difficult to directly conclude whether a tray is missing based solely on the detection result of a single target. To address the problem in existing technologies where missing trays cannot be directly identified as independent targets, and only the detection results for beams and trays are output separately, this invention provides a method that includes inputting infrared images of glass curtain wall trays into a pre-trained target detection model for target detection to obtain detection results including beams and trays; determining the type of detected beam as a complete beam, a left half beam, or a right half beam; and determining tray missingness, including judging whether a tray is missing based on the type of detected beam (complete beam, left half beam, or right half beam) and the number of trays included in the detection results. By analyzing and reasoning about the quantitative relationship, spatial relationship, and distribution logic of the beams and trays based on the obtained detection results, the method automatically determines the tray missingness. This solves the problems of weak small target features, strong interference in the beam area, easy missed detection of trays, and difficulty in accurately determining the missing status in existing glass curtain wall tray missing detection methods, achieving low-cost, high-efficiency, and high-precision glass curtain wall tray missing detection. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram of the network structure of the improved YOLOv11-OBB model in an embodiment of the present invention.

[0017] Figure 3 This is a schematic diagram of the network structure of the beam sensing and guidance module BAGM in an embodiment of the present invention.

[0018] Figure 4 This is a schematic diagram of the background convolutional network structure BGConv in an embodiment of the present invention.

[0019] Figure 5 This is a schematic diagram of the network structure of the Dynamic Weighted Feature Fusion (DWFF) module in an embodiment of the present invention.

[0020] Figure 6 This is a schematic diagram illustrating the working principle of the small target detection head part of the improved YOLOv11-OBB model in an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0022] like Figure 1 As shown, the glass curtain wall tray missing detection method in this embodiment includes the following steps: S1, inputting the infrared image of the glass curtain wall tray into a pre-trained target detection model for target detection to obtain detection results including beams and trays; S2, classifying the beams detected in the detection results to determine whether the detected beam type is a complete beam, a left half beam, or a right half beam; S3, performing tray missing judgment. If the detected beam type is a complete beam, then determine whether the number of trays included in the detection result is 2. If the number of trays included in the detection result is not 2, then determine that the detected beam has a missing tray. If the detected beam type is a left half beam or a right half beam, then determine whether the number of trays included in the detection result is 1. If the number of trays included in the detection result is not 1, then determine that the detected beam has a missing tray.

[0023] In step S3 of this embodiment, after determining that a tray is missing from the detected crossbeam, the missing tray is further marked: When the detected crossbeam type is a complete crossbeam, if the detection result includes a tray count of 1, then it is compared whether the tray associated with the current crossbeam is located to the left or right of the center of the current crossbeam. If it is located to the left, then the missing tray is marked on the right side of the current crossbeam; if it is located to the right, then the missing tray is marked on the left side of the current crossbeam; if the number of trays associated with the current crossbeam is 0, then it is determined that the missing tray is marked on both sides of the current crossbeam; when the detected crossbeam type is a left half crossbeam, if the number of trays associated with the current crossbeam is 0, then the missing tray is marked on the left side of the current crossbeam; when the detected crossbeam type is a right half crossbeam, if the number of trays associated with the current crossbeam is 0, then the missing tray is marked on the right side of the current crossbeam.

[0024] like Figure 2 As shown, the target detection model in this embodiment is an improved YOLOv11-OBB model. The improved YOLOv11-OBB model is based on the existing YOLOv11-OBB model, which consists of a backbone feature extraction network, a neck feature fusion network, and a detection network. The improved YOLOv11-OBB model introduces a Beam-Aware Guidance Module (BAGM) into the neck feature fusion network to output the fused features to the detection network for target detection, thereby obtaining detection results including beams and trays.

[0025] Because small tray targets are small in size and have limited effective texture information in infrared images, and their feature responses are easily affected by background thermal noise, imaging blur, and interference from nearby components, edge information, local details, and weak response features contained in small targets are easily lost after multiple downsampling and feature compression, resulting in insufficient representation capabilities for small tray targets. Beams in infrared images are typically distributed in a long, thin strip shape, characterized by large length, small width, and uneven local thermal response. Existing detection methods, in the process of multi-scale feature extraction and fusion, often focus more on the response of local salient regions, while failing to adequately model the continuity, span, and long-distance correlation information of slender structures. Since the feature distribution of beam targets along their length is not uniform, and the features of the end regions, weak response regions, or locally occluded regions are usually weak, existing methods tend to only produce strong responses to local salient parts, making it difficult to form a complete representation of the entire beam. To address the problem of insufficient recognition capability of small pallet targets in existing technologies due to the small feature size, easy loss of local details and weak response features, this invention proposes a beam perception guidance module (BAGM). This module utilizes the fixed attachment relationship between the pallet and the beam, using beam information to assist in pallet detection. This allows the model to more easily locate the pallet area and improves the recognition capability of pallet targets. Simultaneously, it enhances the image features of the pallet area, improving pallet recognition performance against complex infrared backgrounds; for example... Figure 3 As shown, the beam perception guidance module BAGM in this embodiment includes two parallel branches: (1) The first branch is the beam prior guidance branch, which processes the input features as follows: ① After two stacked background convolutions BGConv, the direction-aware enhanced feature beam_feat is obtained (the purpose of this process is to extract prior information related to the beam orientation, beam distribution area and structural context from the original features); ② After 1×1 convolution, batch normalization and Sigmoid activation function are applied sequentially to generate a guidance weight map beam_map of the same scale as the input features. Among them, 1×1 convolution is responsible for channel information integration, batch normalization is used to stabilize the training process, and Sigmoid constrains the output to 0. Within a certain range, it enables explicit weight modulation; ③ The guiding weight map beam_map is multiplied by the original input features and then added back to the original input features to obtain the guiding feature guide_feat, thereby enhancing the input features. This can be represented as: ; In the formula, This indicates that the input features are weighted element by element. This indicates that the relevant responses within the beam region are enhanced while retaining the original feature information. (2) The second branch is the local detail extraction branch, which processes the input features as follows: ① Local detail features local_feat are obtained through a 3×3 convolutional layer, batch normalization, and SiLU activation function. This aims to retain the local detail information corresponding to the tray target itself and avoid the problem of insufficient characterization of the local features of the tray itself caused by the module relying solely on the prior information of the beam region for guidance. Finally, after the feature extraction is completed in both branches, the guiding feature guide_feat obtained from the first branch and the local detail feature local_feat obtained from the second branch are concatenated in the channel dimension and then passed through a 1×1 convolutional layer, batch normalization, and SiLU activation function to obtain the output features of the beam perception guidance module BAGM, providing a more effective basis for recognition for the subsequent detection head. The above structure of the beam perception guidance module BAGM can make full use of the spatial prior information provided by the beam region to enhance the relevant features of the tray in a guided manner. Since trays are typically attached to beams and are often small in size, with weak texture and indistinct response in infrared images, simply relying on conventional convolutional extraction often leads to missed detections or inaccurate localization. To address this, the Beam Perception Guidance Module (BAGM) first extracts structural responses related to the beam region from the input features, then transforms these responses into guidance weights. This adaptively modulates the original features, causing the network to focus more on areas near the beam where trays might be present during feature learning, thereby improving the tray target recognition ability and localization accuracy. The BAGM uses a dual-branch structure to enhance tray target guidance. The beam prior guidance branch extracts the beam region prior and generates enhanced guidance features, while the local detail extraction branch extracts local detail features of the tray target. Finally, the enhanced result is output through a fusion operation. Compared to convolutional processing only on single-path features, this module more effectively utilizes the beam region as a reference structure, guiding the neural network to focus on areas where trays are more likely to appear, while preserving the tray's own detailed information, thus improving tray recognition accuracy.

[0026] To achieve efficient and accurate feature extraction from horizontal or slightly tilted beams (pallets), such as Figure 4As shown, in this embodiment, the background convolution BGConv (Beam-Guided Convolution) includes three parallel branches (basic local convolution branch, horizontal strip convolution branch, and guided map generation branch): The first branch sequentially processes the input features through 1×1 convolution, batch normalization, SiLU activation function, 3×3 convolution, batch normalization, and SiLU activation function to generate a spatial attention mask. The spatial attention mask distinguishes the effective beam region from the background region by taking values ​​of 1 and 0. The second branch sequentially processes the input features through 1×7 convolution, batch normalization, and SiLU activation function to generate a horizontal enhancement feature map. The third branch sequentially processes the input features through 3×3 convolution, batch normalization, and SiLU activation function to generate a local spatial feature map. The spatial attention mask and the horizontal enhancement feature map are multiplied element-wise and then concatenated with the local spatial feature map. After passing through a 1×1 convolution layer, batch normalization, and SiLU activation function, the final output feature of the background convolution BGConv is obtained. Of the three parallel branches mentioned above, the basic local feature extraction branch is responsible for capturing the basic visual information of the target. It extracts local spatial features of the image through a standard 3×3 two-dimensional convolution, batch normalization, and the SiLU activation function. For small tray targets, this branch can effectively extract and preserve the target's basic texture, edge contours, and local variations in depth. The final output is a basic feature map. The horizontal strip feature extraction branch is specifically designed for the geometry of beam targets. It uses a 1×7 convolution kernel with a height of 1 and a relatively large width. The horizontal receptive field of feature extraction is artificially lengthened so that its physical receptive area perfectly matches the structural prior of the beam's large aspect ratio. After the same batch normalization and SiLU activation, it outputs a horizontally enhanced feature map. Since the flattened convolution of the horizontal strip feature extraction branch inevitably introduces some background noise when expanding the horizontal receptive field, the spatial guided mask generation branch uses a guided map generation mechanism for "noise reduction and purification." Cross-channel information is integrated sequentially through 1×1 convolutions, local spatial context is perceived through 3×3 convolutions, and finally, the output is mapped to probability values ​​between 0 and 1 using the Sigmoid function. A spatial attention mask is generated to identify which spatial locations are the true effective areas of the beam and which are background interferences that need to be suppressed. After the three branches complete their respective information extraction, the module enters the final feature aggregation stage. First, the "spatial guidance mask" generated by the third branch is used as a gating switch to perform pixel-by-pixel weighted modulation on the "horizontal strip features" extracted by the second branch, thereby filtering out background noise while preserving the large horizontal receptive field and obtaining pure "directional guidance features".Finally, the guiding feature is concatenated with the "basic local features" of the first branch, and cross-channel information interaction and dimensionality reduction are performed through a fusion layer (1×1 convolution) to output the final enhanced feature map.

[0027] like Figure 2 As shown, in this embodiment, the backbone feature extraction network outputs four backbone features of different scales, P1 to P4, to the neck feature fusion network. The processing of the four backbone features of different scales by the neck feature fusion network includes: upsampling backbone feature P1 and concatenating it with backbone feature P2, then extracting feature F1 through the first simple mode C3k2 feature extraction module; upsampling feature F1 and concatenating it with backbone feature P3, then extracting feature F2 through the second simple mode C3k2 feature extraction module; passing feature F2 through a convolutional layer, upsampling it, and concatenating it with backbone feature P4, then extracting feature F3 through the third simple mode C3k2 feature extraction module; passing feature F3 through the first beam perception guidance module BAGM to obtain the first output feature O1 and outputting it to the first rotated bounding box detection head of the detection network; simultaneously, the first... One output feature O1 is concatenated with feature F2 after passing through a convolutional layer, and then extracted by the fourth simple mode C3k2 feature extraction module to obtain feature F4. Feature F4 is then passed through the second beam perception guidance module BAGM to obtain the second output feature O2, which is then output to the second rotating bounding box detection head of the detection network. Simultaneously, the second output feature O2 is concatenated with feature F1 after passing through a convolutional layer, and then dynamically weighted and fused by the dynamic weighted feature fusion module DWFF. This is then extracted by the fifth simple mode C3k2 feature extraction module to obtain the third output feature O3, which is then output to the third rotating bounding box detection head of the detection network. At the same time, the third output feature O3 is concatenated with feature P1 after passing through a convolutional layer, and then extracted by a complex mode C3k2 feature extraction module to obtain the fourth output feature O4, which is then output to the fourth rotating bounding box detection head of the detection network. This invention adds a detection branch for small target recognition to the existing detection network: a first rotated bounding box detection head (small target detection head). This enhances the expressive power of shallow high-resolution features and improves the localization accuracy and recognition rate of small targets. It can utilize higher-resolution feature maps for detection, especially considering the small size and weak texture of trays, to retain more detailed information. See also... Figure 6The input features from the previous layer are processed by a 1×1 convolution, which compresses channels and reorganizes information, reducing computational cost while preserving key semantic information. The output channel count is 128. The reduced-channel features are then upsampled by a factor of 2 to obtain a higher-resolution feature map. This upsampling operation restores the original low-resolution semantic features to a scale more suitable for small target detection, thereby enhancing the network's ability to recognize small pallet targets. After upsampling, the upsampled features are concatenated and fused with the corresponding backbone features P4 (high-resolution shallow features) in the backbone network. The backbone features P4 have high spatial resolution, preserving more edge, contour, and local texture information; while the upsampled features have stronger high-level semantic expression capabilities. By concatenating these two features, a joint expression of shallow detail information and deep semantic information can be achieved, enabling the detection branch to not only finely characterize the position and shape of small targets but also discriminate target categories. In this embodiment, the backbone feature extraction network constructs a high-resolution detection branch specifically for small-sized pallet targets through a processing flow of "channel compression - resolution restoration - shallow and deep feature fusion - feature extraction".

[0028] To address the issues of incomplete coverage, insufficient continuity, and local breaks in beam target detection in existing technologies, this invention proposes a Dynamic Weighted Feature Fusion (DWFF) module. This module enhances the model's ability to recognize the overall beam structure by fusing image features from different layers and automatically adjusting the weight of each feature. For beams in infrared images, which often exhibit incomplete recognition, detection of only the middle portion, and insufficient coverage at both ends, this module enhances the continuity and integrity of the beam region, thereby improving beam detection performance and providing a more reliable foundation for subsequent tray detection. Figure 5As shown, the Dynamic Weighted Feature Fusion Module (DWFF) in this embodiment includes two parallel branches. Each branch processes one input feature and multiplies the input feature by a learnable fusion weight after passing through a 1×1 convolution, batch normalization, and SiLU activation function. By introducing learnable fusion weights, the network can automatically adjust the contribution ratio of the branches according to the training process. The output features of the two parallel branches are added together to obtain the fusion feature. The fusion feature is then passed through a 1×7 convolution, batch normalization, and SiLU activation function to expand the lateral receptive field and enhance the continuity expression of the beam structure to obtain the lateral continuity enhancement feature. The lateral continuity enhancement feature is then passed through a coordinate attention module (CoordAtt) for direction-aware modeling, and then refined through a 3×3 convolution, batch normalization, and SiLU activation function. Finally, the refined feature is added to the fusion feature through residual connection to obtain the output feature of the Dynamic Weighted Feature Fusion Module (DWFF). The Dynamically Weighted Feature Fusion (DWFF) module achieves efficient fusion of multi-scale features and targeted enhancement of the slender structural features of beams through steps such as channel alignment, adaptive weighted fusion, lateral strip convolution enhancement, orientation-aware attention modeling, local edge correction, and residual output. This effectively improves the structural representation ability and detection stability of target detection models for beam-like targets, providing a more reliable feature foundation for subsequent tray recognition and missing item determination. It should be noted that the CoordAtt module is a known existing network module. Its core idea is to introduce positional information encoding during channel attention modeling, thereby simultaneously achieving channel feature recalibration and preservation of orientational positional information. The input to the CoordAtt module is a set of two-dimensional feature maps, which undergo one-dimensional global average pooling operations along both the horizontal and vertical directions. After obtaining the encoded features in these two directions, the CoordAtt module concatenates them spatially and inputs them into a shared transformation module. This transformation module consists of 1×1 convolutions, normalization layers, and nonlinear activation functions. Its function is to perform channel compression and information fusion on the concatenated orientation features, extracting a joint representation while reducing the number of parameters and computational cost. The intermediate features after the shared transformation are then separated according to the original two directions and fed into two independent branches. These two independent branches correspond to the horizontal attention branch and the vertical attention branch, respectively. Each branch maps the intermediate features back to a dimension consistent with the number of input channels through a separate 1×1 convolution, and generates weight coefficients ranging from 0 to 1 using the Sigmoid activation function. Finally, the coordinate attention module generates corresponding attention weights based on the two coordinate directions and performs weighted adjustments on the input feature map, enhancing important features and suppressing secondary features.

[0029] To verify the effectiveness and advancement of the missing glass curtain wall tray detection method in this embodiment, infrared images of the building's glass curtain wall were acquired using a DJI M350RTK drone equipped with an H20T camera. The images were then labeled using labelimg software to construct the dataset. To objectively evaluate the detection performance of different target detection models, the following metrics were used: 1) Precision (P), representing the proportion of samples correctly identified as positive by the model. 2) Recall (R), representing the ratio of correctly identified positive samples to the total number of positive samples. 3) Mean Average Precision (mAP), where mAP is the average of the average precision (AP) across all classes, and AP is the area under the precision and recall curves. Finally, the ablation experiment results of the improved YOLOv11-OBB model are shown in Table 1.

[0030] Table 1: Ablation experimental results of the improved YOLOv11-OBB model.

[0031]

[0032] In Table 1, Net-1 represents the introduction of a small target detection head into the YOLOv11-OBB model to enhance the expression of shallow detail features and improve the detection performance of small-scale targets such as trays. Compared with the original YOLOv11-OBB model, the precision (P) for beam detection is improved by 11.4 percentage points, and the recall (R) is improved by 9.4 percentage points; the precision (P) for tray detection is improved by 2.4 percentage points, and the recall (R) is improved by 18.1 percentage points; the overall mAP is improved by 10.4 percentage points. It can be seen that after introducing the small target detection head, the model's ability to detect beams and trays is significantly enhanced, especially the problem of missed detection of small targets such as trays is significantly improved. Net-2 represents the introduction of the DWFF module (Dynamic Weighted Feature Fusion Module) into the YOLOv11-OBB model to enhance the multi-scale feature fusion capability and improve the continuous expression of beam structural features. Compared to the original YOLOv11-OBB model, the beam detection precision (P) improved by 5 percentage points, and the recall (R) improved by 10 percentage points; the tray detection precision (P) improved by 3.2 percentage points, and the recall (R) improved by 11.6 percentage points; the overall mAP improved by 9.6 percentage points. This indicates that the DWFF module can enhance the model's ability to extract features of long, narrow structural targets and related regions to a certain extent, thereby improving the completeness and stability of the detection results. Net-3 represents the introduction of the BAGM module (Beam Perception Guidance Module) on the basis of the YOLOv11-OBB model, which is used to introduce prior information about beam-related structures to guide the tray target detection process. Compared to the original YOLOv11-OBB model, the beam detection precision (P) improved by 3.1 percentage points, and the recall (R) improved by 9.8 percentage points; the tray detection precision (P) improved by 3.7 percentage points, and the recall (R) improved by 14.7 percentage points; the overall mAP improved by 9.9 percentage points. It can be seen that after the introduction of the BAGM module, the beam recall capability is enhanced, but the improvement in beam precision is relatively limited, while the improvement in pallet detection precision is more significant. This indicates that the main function of this module is to improve the discrimination capability of pallet targets by utilizing the contextual information related to the beam, thereby enhancing the model's recognition effect and detection reliability of pallet targets. Net-4 is an improved YOLOv11-OBB model of the method in this embodiment. Compared with the original YOLOv11-OBB model, the beam detection precision P is improved by 17 percentage points, and the recall R is improved by 10.4 percentage points; the pallet detection precision P is improved by 3.4 percentage points, and the recall R is improved by 21.3 percentage points; the overall mAP is improved by 11.7 percentage points.Compared with the ablation experiment results of the other groups, Net-4 achieved the best results in terms of beam detection precision, beam recall, tray recall, and overall mAP. This indicates that the small target detection head, DWFF module, and BAGM module proposed in this embodiment have a good synergistic gain effect. The combination of the three can simultaneously improve the detection performance of beam and tray, thereby providing more accurate and reliable detection results for subsequent tray missing determination.

[0033] In summary, the glass curtain wall tray missing detection method of this embodiment can solve the problems of weak small target features, strong interference in the beam area, easy missed detection of trays, and difficulty in accurately determining the missing status of existing glass curtain wall tray missing detection methods. By improving the YOLOv11-OBB model, adding a small target detection head facing the tray, constructing a beam perception guidance module, and combining it with a missing determination program based on the positional relationship between the beam and the tray, a low-cost, high-efficiency, and high-precision solution for glass curtain wall tray missing detection can be provided.

[0034] Furthermore, this embodiment also provides a missing glass curtain wall tray detection system, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the missing glass curtain wall tray detection method. This embodiment also provides a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the missing glass curtain wall tray detection method via a processor. This embodiment also provides a computer program product, including a computer program or instructions programmed or configured to execute the missing glass curtain wall tray detection method via a processor.

[0035] Those skilled in the art will understand that the technical solutions provided by this invention may take the form of a method, system, or computer program product. Therefore, this invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce an implementation of the flowchart... Figure 1One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0036] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting missing glass curtain wall trays, characterized in that, The process includes the following steps: S1, inputting the infrared image of the glass curtain wall tray into a pre-trained target detection model for target detection to obtain detection results including beams and trays; S2, classifying the detected beams in the detection results to determine whether the detected beam type is a complete beam, a left half beam, or a right half beam; S3, determining if a tray is missing. If the detected beam type is a complete beam, it is determined whether the number of trays included in the detection result is 2. If the number of trays included in the detection result is not 2, it is determined that the detected beam has a missing tray. If the detected beam type is a left half beam or a right half beam, it is determined whether the number of trays included in the detection result is 1. If the number of trays included in the detection result is not 1, it is determined that the detected beam has a missing tray. The target detection model is an improved YOLOv11-OBB model. The improved YOLOv11-OBB model is based on the YOLOv11-OBB model, which consists of a backbone feature extraction network, a neck feature fusion network, and a detection network. The improved YOLOv11-OBB model introduces a beam perception guidance module (BAGM) in the neck feature fusion network to output the fused features to the detection network for target detection, thereby obtaining detection results including beams and trays. The beam perception guidance module BAGM comprises two parallel branches: the first branch passes the input features through two stacked background convolutions (BGConv) to obtain the direction-aware enhanced features beam_feat, which are then passed through a 1×1 convolution, batch normalization, and a Sigmoid activation function to generate a guidance weight map beam_map of the same scale as the input features. The guidance weight map beam_map is multiplied by the original input features and then added to the original input features to obtain the guidance feature guide_feat, thereby enhancing the input features. The second branch passes the input features through a 3×3 convolutional layer, batch normalization, and a SiLU activation function to obtain the local detail features local_feat. The guidance feature guide_feat obtained from the first branch and the local detail features local_feat obtained from the second branch are concatenated along the channel dimension and then passed through a 1×1 convolutional layer, batch normalization, and a SiLU activation function to obtain the output features of the beam perception guidance module BAGM. The background convolution BGConv comprises three parallel branches: the first branch sequentially processes the input features through a 1×1 convolution, batch normalization, SiLU activation function, 3×3 convolution, batch normalization, and SiLU activation function to generate a spatial attention mask, in which the effective area of ​​the beam and the background area are distinguished by values ​​of 1 and 0; the second branch sequentially processes the input features through a 1×7 convolution, batch normalization, and SiLU activation function to generate a horizontal enhancement feature map; the third branch sequentially processes the input features through a 3×3 convolution, batch normalization, and SiLU activation function to generate a local spatial feature map; the spatial attention mask and the horizontal enhancement feature map are multiplied element-wise and then concatenated with the local spatial feature map, and then processed through a 1×1 convolution layer, batch normalization, and SiLU activation function to obtain the final output feature of the background convolution BGConv.

2. The method for detecting missing glass curtain wall trays according to claim 1, characterized in that, Step S3, after determining that a detected crossbeam has a missing tray, also includes marking the missing tray: When the detected crossbeam type is a complete crossbeam, if the detection result contains 1 tray, then compare whether the tray associated with the current crossbeam is located to the left or right of the center of the current crossbeam. If it is located to the left, then mark the missing tray box on the right side of the current crossbeam; if it is located to the right, then mark the missing tray box on the left side of the current crossbeam; if the number of trays associated with the current crossbeam is 0, then determine that the missing tray boxes are marked on both sides of the current crossbeam; when the detected crossbeam type is a left half crossbeam, if the number of trays associated with the current crossbeam is 0, then mark the missing tray box on the left side of the current crossbeam; when the detected crossbeam type is a right half crossbeam, if the number of trays associated with the current crossbeam is 0, then mark the missing tray box on the right side of the current crossbeam.

3. The method for detecting missing glass curtain wall trays according to claim 1, characterized in that, The backbone feature extraction network outputs four backbone features of different scales, P1 to P4, to the neck feature fusion network; The neck feature fusion network processes four main features at different scales as follows: Main feature P1 is upsampled and concatenated with main feature P2, then processed by the first simple mode C3k2 feature extraction module to obtain feature F1; feature F1 is upsampled and concatenated with main feature P3, then processed by the second simple mode C3k2 feature extraction module to obtain feature F2; feature F2 is passed through a convolutional layer, upsampled, and concatenated with main feature P4, then processed by the third simple mode C3k2 feature extraction module to obtain feature F3; feature F3 is passed through the first beam perception guidance module BAGM to obtain the first output feature O1, which is then output to the first rotated bounding box detection head of the detection network; simultaneously, the first output feature O1 is passed through a convolutional layer and processed by feature F2. The features are concatenated and then extracted using the fourth simple mode C3k2 feature extraction module to obtain feature F4. Feature F4 is then passed through the second beam perception guidance module BAGM to obtain the second output feature O2, which is then output to the second rotating bounding box detection head of the detection network. Simultaneously, the second output feature O2 is passed through a convolutional layer and then dynamically weighted and fused with feature F1 using the dynamic weighted feature fusion module DWFF. This fusion is then passed through the fifth simple mode C3k2 feature extraction module to obtain the third output feature O3, which is then output to the third rotating bounding box detection head of the detection network. Finally, the third output feature O3 is passed through a convolutional layer and then concatenated with feature P1. This concatenation is then passed through a complex mode C3k2 feature extraction module to obtain the fourth output feature O4, which is then output to the fourth rotating bounding box detection head of the detection network.

4. The method for detecting missing glass curtain wall trays according to claim 3, characterized in that, The Dynamic Weighted Feature Fusion Module (DWFF) comprises two parallel branches. Each branch processes one input feature and multiplies the input feature by a learnable fusion weight after passing it through a 1×1 convolution, batch normalization, and SiLU activation function. The output features of the two parallel branches are then summed to obtain the fused features. These fused features are then passed through a 1×7 convolution, batch normalization, and SiLU activation function to expand the lateral receptive field and enhance the continuity of the beam structure, resulting in lateral continuity enhancement features. These lateral continuity enhancement features are then processed by a coordinate attention module for direction-aware modeling, followed by a 3×3 convolution, batch normalization, and SiLU activation function for refinement. Finally, the refined features are added to the fused features via residual connections to obtain the output features of the Dynamic Weighted Feature Fusion Module (DWFF).

5. A missing glass curtain wall tray detection system, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to perform the method for detecting the missing glass curtain wall tray as described in any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the method for detecting the missing glass curtain wall tray as described in any one of claims 1 to 4.

7. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the method for detecting the missing glass curtain wall tray as described in any one of claims 1 to 4.

Images