A multi-task autonomous intelligent agent for building defect detection and a working method thereof

Abstract

The application discloses a kind of multi-task autonomous intelligent agent and working method of building defect detection, it is related to wisdom building technology field.Multiple unlabeled building defect images are acquired and defect scene subset and artificial labeling benchmark set are constructed;Initial target detector library is constructed by multiple deep learning detection models, and the output format of each detector is unified to form a standardized detector library;Artificial labeling benchmark set is used to train dynamic gating network, and the routing weight matrix between scene and detector is generated;In the online application stage, the priority detector is selected according to the routing weight, and the long tail detector is randomly extracted for concurrent inference, the model prediction divergence is calculated, and the building defect detection result is fused and output;Based on the building defect detection result, unlabeled risk monitoring is performed on the current scene batch image set, and according to the unlabeled risk monitoring result, the detector local fine tuning, periodic full detector training, a small amount of manual correction and routing adaptive adjustment are performed.

Description

Technical Field

[0001] This invention relates to the field of smart building technology, and in particular to a multi-task autonomous intelligent agent and its working method for detecting building defects. Background Technology

[0002] In recent years, with the increasing demand for refined urban governance, building safety inspections, and public space supervision, the use of computer vision technology to identify building anomalies such as wall damage, window damage, corridor damage, public space occupation, and illegal construction has become an important direction in the field of smart buildings.

[0003] Existing methods often rely on single detection models or fixed detection processes. However, architectural scenes vary significantly in spatial structure, lighting conditions, shooting angles, and defect morphologies. A single model struggles to continuously adapt to multiple complex scenes, leading to issues such as missed detections, false detections, and insufficient generalization ability. Different target detection models differ in network architecture, bounding box representation, and confidence output format. While existing research has explored methods such as multi-model detection result fusion, scene-related model selection, and semi-supervised or active learning updates, a cohesive and integrated mechanism is still lacking for aligning unified results from multiple detectors, scene-level dynamic scheduling, and continuous closed-loop updates for complex architectural scenes. Furthermore, while existing low-annotation-cost update methods can reduce manual input through pseudo-labels and active sample screening, they still need to consider the reliability of pseudo-labels, anomaly risk correction, and update decision stability during long-term online evolution. On the other hand, as detection models continuously learn new scene knowledge, they may weaken the detection capabilities already established for existing scenes, leading to catastrophic forgetting problems. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention proposes a multi-task autonomous intelligent agent for building defect detection, which enables automatic segmentation of building scenes, dynamic scheduling of multiple detectors, online fusion detection, and closed-loop updates to prevent forgetting.

[0005] In a first aspect, the present invention provides a multi-task autonomous intelligent agent for building defect detection, comprising: A standardized detector library building module is used to obtain an initial detector library covering various deep learning detection models. ; Build and configure a heterogeneous result alignment parser The bounding box coordinates and confidence scores output by each detector are uniformly mapped to the standard probability tensor space. At the same time, the cross-union ratio (CUB) filtering threshold for non-maximum suppression is dynamically adjusted according to the spatial distribution density of the predicted boxes, thus forming a standardized detector library. ; The annotation benchmark set construction module is used to obtain an image set of unlabeled building defects. From the unlabeled building defect image set Extract global feature vectors The unlabeled building defect image set Divided into S defect scenario subsets From the S subsets of defective scenarios A predetermined number of representative images are manually annotated to obtain multiple annotated sample sets corresponding to each defect scenario. All annotated sample sets are then compiled to form a manual annotation benchmark set. ; The dynamic gating training and scene-level routing construction module is used to compute the standardized detector library. The Middle n The detector at the 1st s Comprehensive score under each defect scenario And convert them into scene-level detector probability soft labels. Extract scene feature representations corresponding to each defect scenario, and use these scene feature representations as input, along with the probability soft labels of the scene-level detector. As a supervisory signal, training the dynamic gating network G Output the routing weights of each detector in each defect scenario; summarize all S The routing weights of the detectors in each defect scenario are used to construct a routing weight matrix between the scenario and the detectors. ; An online image dynamic routing detection module, incorporating a random sampling mechanism, is used to continuously acquire newly added unlabeled building defect images and determine the corresponding defect scenes and the current scene batch image set corresponding to the defect scenes; based on the defect scenes in the routing weight matrix... The corresponding detector routing weights form the optimal detector group. and long-tail detector group The current scene batch image set is synchronously input into the optimal detector group. With the long-tailed detector group Perform concurrent inference and align the parser using the heterogeneous results. For the optimal detector group With long-tail detector group The output predicted bounding boxes are uniformly formatted, and spatial matching is performed on predicted bounding boxes of the same category to form a cluster of predicted boxes corresponding to the same candidate defect target. The model prediction divergence degree for each image is calculated based on the differences between predicted boxes of the same category. For predicted bounding boxes of the same category with a spatial intersection-union ratio not lower than a preset fusion threshold, weighted bounding box fusion is performed using the combination of detector routing weight and prediction confidence as the fusion weight to obtain the building defect detection results of each image in the current scene batch image set. The budget-constrained anti-forgetting closed-loop update module is used to generate pseudo-labels for the current scene batch image set based on the building defect detection results, and predict the divergence degree based on the model. The degree of change in detector ranking and the long-tailed detector group With the optimal detector group The system adapts to the image set in the current scene by performing unlabeled risk monitoring on the batch of images in the current scene. Based on the results of the unlabeled risk monitoring on the batch of images in the current scene, the system performs local fine-tuning of the detector, periodic full detector training, a small amount of manual correction, and adaptive route adjustment.

[0006] Optionally, the construction and configuration of the heterogeneous result alignment parser The bounding box coordinates and confidence scores output by each detector are uniformly mapped to the standard probability tensor space. At the same time, the cross-union ratio (CUB) filtering threshold for non-maximum suppression is dynamically adjusted according to the spatial distribution density of the predicted boxes, thus forming a standardized detector library. ,include: Construct a heterogeneous result alignment parser It receives the raw prediction results output by each detector in parallel, converts the bounding box coordinate representation of different detectors into a preset standard coordinate format, normalizes the confidence scores output by each detector, and maps them to a unified standard probability tensor space. In the heterogeneous result alignment parser Internally, the spatial distribution density features of predicted bounding boxes in the current image are statistically analyzed, and the cross-union ratio (CUP) filtering threshold for non-maximum suppression is dynamically determined based on these features. When the predicted bounding boxes are densely distributed, the CUP filtering threshold is increased to reduce the probability of over-suppressing adjacent defect targets. Based on the CUP filtering threshold, overlapping redundant boxes are removed, forming a standardized detector library with a unified calling interface and a unified result output format. .

[0007] Optionally, the process involves semantic segmentation, region-of-interest cropping, and feature encoding from the unlabeled building defect image set. Extract global feature vectors ,include: For the unlabeled building defect image set For each image, a pixel-level mask of the target building structural elements is generated using a preset semantic segmentation model, and a region of interest image containing the core building defect area is obtained by cropping based on the pixel-level mask. After performing size scaling and pixel normalization preprocessing on the region of interest image, a feature encoder is used. Extract global feature vectors representing the overall building structure, defect morphology, and scene semantics of the image. .

[0008] Optionally, the density clustering based on feature distance will Divided into S defect scenario subsets ,include: Calculate the unlabeled building defect image set The cosine similarity between the global feature vectors corresponding to each image is calculated, and the cosine similarity is converted into feature distance to construct a feature distance matrix; Density clustering is performed based on the feature distance matrix to obtain the unlabeled building defect image set. The image set of unlabeled building defects is categorized according to the defect scene category corresponding to each image. Divided into S defect scenario subsets It also filters outlier noise samples generated during density clustering.

[0009] Optionally, the calculation of the standardized detector library The Middle n The detector at the 1st s Comprehensive score under each defect scenario And convert them into scene-level detector probability soft labels. ,include: Regarding the first s The first defect scenario and the first n Individual detectors, overall score The calculation formula is: .

[0010] in, Indicates the first n The detector at the 1st s In each defect scenario, the mean intersection-over-union ratio (IoU) between the predicted bounding boxes and the manually labeled ground truth bounding boxes is used to characterize the localization accuracy. Indicates the first n The mean overlap and consistency between the predictions of each detector and those of other detectors is used to characterize the overall consistency. Indicates the first n The detector at the 1st s The expected calibration error in each scenario is used to characterize the deviation between the prediction confidence and the actual prediction reliability. , , Preset weighting coefficients; The comprehensive score is calculated using the Softmax function. Transform into scene-level detector probabilistic soft labels .

[0011] Optionally, the extraction of scene feature representations corresponding to each defect scene includes: Regarding the first A subset of defective scenarios Obtain the global feature vectors corresponding to each image in the defect scene subset. and the global feature vector Perform mean aggregation to obtain the first... Scene feature representation corresponding to each defect scenario : .

[0012] in, Indicates the first The first defect scenario subset Zhang Image Representing an image The corresponding global feature vector, Indicates the first The number of images in a subset of defective scenes.

[0013] Optionally, the calculation of model prediction divergence for each image is based on spatial location differences, confidence differences, and target miss detection differences among prediction boxes of the same category. ,include: For each cluster of predicted bounding boxes, calculate the spatial location difference of the predicted bounding boxes separately. Confidence differences at the frame level Difference between target and missed detection And calculate the model prediction divergence degree corresponding to the candidate defect target according to the following formula. : .

[0014] in, This indicates the degree of inconsistency in the positions of the predicted bounding boxes corresponding to the same candidate defect target. This indicates the degree of difference between the confidence levels corresponding to each prediction box. This indicates the percentage of activated detectors that did not output the predicted bounding box for the candidate defect. , and The corresponding preset weight coefficients, and satisfying ; When multiple candidate defect targets exist in an image, the model prediction divergence scores corresponding to each candidate defect target are averaged and aggregated to obtain the model prediction divergence score for the image. .

[0015] Optionally, the model-based prediction of divergence... The degree of change in detector ranking and the long-tailed detector group With the optimal detector group The adaptation performance between them includes performing unlabeled risk monitoring on the current scene batch image set, including: Statistical analysis of model prediction divergence in the current scene batch image set Not lower than the preset divergence threshold The proportion of high-dissipation images; a temporary detector ranking is formed based on the unlabeled adaptation scores of each activated detector on the current scene batch image set, and the temporary detector ranking is combined with the routing weight matrix. The original detector order was compared to determine the degree of change in the detector order; the long-tail detector group was calculated separately. With the optimal detector group The average unlabeled adaptation score; The unlabeled risk monitoring results of the current scene batch image set are determined to meet the anomaly triggering condition by judging whether at least one of the following conditions is met: In the current scene batch image set, the model prediction divergence degree... Not lower than the preset divergence threshold The proportion of highly divergent images reaches a preset threshold; the temporary detector sorting and routing weight matrix is ​​formed based on the current scene batch image set. In the middle, the degree of change between the original detector rankings reaches the preset ranking change threshold; long-tail detector group The average unlabeled adaptation score is higher than that of the optimal detector group. The average unlabeled adaptation score is higher than the preset adaptation difference threshold.

[0016] Optionally, the step of performing local fine-tuning of the detector, periodic full detector training, minor manual correction, and adaptive route adjustment based on the unlabeled risk monitoring results of the current scene batch image set includes: When the unlabeled risk monitoring results of the current scene batch image set do not meet the abnormal triggering conditions, old images matching the number of images in the current scene batch are extracted from the historical training library to jointly construct a hybrid training set. Based on the hybrid training set, the optimal detector group activated in this round is selected. With long-tail detector group Perform local fine-tuning, and update the detector ranking relationship and routing weight matrix corresponding to the current defect scenario based on the fine-tuned detector performance. The updated results will be used for subsequent online dynamic triage of building defect detection; When any defect scenario has completed a preset number of batch evaluations, all [items] will be activated. N Each detector, based on the mixed training data accumulated within the preset rounds for this defect scenario, performs a comprehensive analysis of all... NEach detector performs periodic incremental training. After completing the periodic incremental training, when a new batch of scene images of the defective scene is formed again, the new batch of scene images is used to analyze all detectors. N Each detector performs a label-free scenario adaptability assessment, redetermines the ranking relationship of each detector in the defective scenario, and updates the routing weight matrix. and dynamic gating network G Routing supervision information; When the unlabeled risk monitoring results of the current scene batch image set meet the anomaly triggering conditions, the screening model predicts the divergence degree. Greater than or equal to the preset divergence threshold Difficult images, based on model-predicted divergence. Sort by high to low, and within the limit of the manual correction budget. Under these conditions, the most difficult images in the ranking are selected for manual correction to obtain correction labels. Compare the pseudo-label with the correction label. The degree of deviation between them; When the degree of deviation does not reach the preset anomaly confirmation threshold, the correction label is used. Correct the relevant pseudo-labels and perform local compensation updates on the relevant detectors; when the deviation reaches a preset anomaly confirmation threshold, determine that the detector adaptation relationship in the defective scenario has changed significantly, and wake up all detectors. Each detector underwent a complete update, and the dynamic gating network was retrained. G After completing the full update, reset the cumulative batch evaluation rounds corresponding to the defective scenario to zero, and restart the cycle counting from the next new scenario batch; update the detector sorting relationship and routing weight matrix after partial compensation update or full update. and dynamic gating network G The routing supervision information is used for online dynamic routing of building defect detection in subsequent batches of scene images.

[0017] Secondly, the present invention provides a multi-task autonomous intelligent agent working method for building defect detection, based on the multi-task autonomous intelligent agent for building defect detection described in the first aspect, comprising: Acquire images of various types of unlabeled building defects; Unlabeled building defect images are divided into scenes and representative samples are labeled to form a defect scene subset and a manually labeled baseline set. An initial target detector library consisting of multiple deep learning detection models is constructed, and the output format of each detector is unified through a heterogeneous result alignment parser to form a standardized detector library. The adaptability of each detector under different defect scenarios is evaluated using a manually labeled benchmark set. A dynamic gating network is trained to generate a routing weight matrix between the scenario and the detector. During the online application phase, newly added unlabeled building defect images are continuously acquired and the defect scene to which each newly added unlabeled building defect image belongs is determined. When the number of newly added images of any defect scene reaches the preset batch threshold, priority detectors are selected according to the routing weight, and long-tail detectors are randomly selected for concurrent inference. The model prediction divergence degree is calculated and the building defect detection results are fused and output. Based on the building defect detection results, pseudo-labels are generated. Based on the model prediction divergence, the degree of change in detector ranking, and the adaptation performance between long-tail detectors and the optimal detector, unlabeled risk monitoring is performed on the current scene batch image set. Based on the unlabeled risk monitoring results of the current scene batch image set, local fine-tuning of detectors, periodic full detector training, a small amount of manual correction, and adaptive route adjustment are performed to form a continuously evolving closed-loop intelligent agent.

[0018] By adopting the above technical solution, the present invention has at least the following beneficial effects: (1) This invention uses a heterogeneous result alignment resolver to unify the predicted bounding box coordinates, category information, and confidence format of different detectors, and dynamically adjusts the non-maximum suppression threshold according to the spatial distribution density of the predicted boxes. This solves the problems of inconsistent output formats of different detectors and the tendency of dense defect targets to be over-suppressed, providing a unified basis for multi-detector collaborative detection and result fusion.

[0019] (2) This invention extracts features and performs density clustering on unlabeled building defect images to form multiple defect scene subsets, and uses a small number of manually labeled samples to train a dynamic gating network to construct a routing weight matrix between the scene and the detector. As a result, the system can adaptively select a detector with a high degree of fit based on the structural features, defect morphology and scene semantics of different building scenes, reducing the probability of missed detections, false detections and insufficient adaptability of fixed detection models in complex scenes.

[0020] (3) In selecting the optimal detector group, the present invention At the same time, a long-tailed detector group is randomly selected from the remaining detectors. Participate in concurrent inference. This approach avoids excessive computational overhead from calling all detectors in each round, while continuously discovering detectors that were originally ranked lower but are more suitable for new scenarios, preventing the detector ranking from becoming fixed in the long term.

[0021] (4) Based on the spatial position differences, box-level confidence differences, and target missed detection differences among the prediction boxes of multiple detectors, this invention calculates the model prediction divergence. The system generates building defect detection results through weighted bounding box fusion. For predictions with low support or isolated patterns, they are not directly used as reliable fusion results; instead, their divergence information is retained for risk monitoring and screening of difficult samples. This reduces the impact of false positives from a single detector on the final detection result and improves the stability of detection results in complex building scenarios.

[0022] (5) Before updating the detector using pseudo-labels, this invention performs unlabeled risk monitoring based on the model prediction divergence, the degree of change in detector ranking, and the adaptation performance between the long-tail detector group and the optimal detector group. When the risk is controllable, a mixed training set is constructed using the current pseudo-labeled image and historical images to perform local fine-tuning of the activated detector; when the risk is abnormal, a small number of difficult images are selected for manual correction under the constraint of manual budget, and local compensation update or full update is performed based on the correction results.

[0023] (6) This invention performs incremental training by periodically waking up all detectors and re-evaluates the scene adaptation capabilities of all detectors using new batches of scene images. As a result, the detector sorting, routing weight matrix and routing supervision information of the dynamic gating network can be adjusted in a timely manner, and the forgetting of existing scene detection capabilities during continuous learning can be reduced, enabling the agent to continuously update with new building scenes. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic diagram illustrating the working principle of a standardized detector library construction module for a multi-task autonomous intelligent agent for building defect detection provided by this invention; Figure 2 A schematic diagram illustrating the working principle of a labeling benchmark set construction module for a multi-task autonomous intelligent agent for building defect detection provided by this invention; Figure 3 This invention provides a working principle diagram of a dynamic gating training and scene-level routing construction module for a multi-task autonomous intelligent agent for building defect detection. Figure 4 This invention provides a working principle diagram of an online image dynamic diversion detection module for building defect detection, which combines a multi-task autonomous intelligent agent with a random sampling mechanism. Figure 5A schematic diagram of the working principle of a budget-constrained anti-forgetting closed-loop update module for a multi-task autonomous intelligent agent for building defect detection provided by the present invention; Figure 6 The present invention provides a flowchart of a multi-task autonomous intelligent agent for building defect detection. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] This invention provides a multi-task autonomous intelligent agent for building defect detection, comprising: (1) The working principle of the standardized detector library construction module is as follows: Figure 1 As shown, the following functions are implemented: Obtain an initial detector library covering various deep learning detection models. , initial detector library Each detector in the system is deployed independently on a local computing node and encapsulated as a detection service with an independent inference interface.

[0028] Construct a heterogeneous result alignment parser It receives the raw prediction results from each detector in parallel, converts the bounding box coordinate representations from different detectors into a preset standard coordinate format, normalizes the confidence scores from each detector, and maps them to a unified standard probability tensor space.

[0029] In heterogeneous result alignment parser Internally, the system statistically analyzes the spatial distribution density features of predicted bounding boxes in the current image and dynamically determines the Cross-Union Ratio (CURRO) filtering threshold for non-maximum suppression based on these features. When the predicted bounding boxes are densely distributed, the CURRO filtering threshold is increased to reduce the probability of over-suppressing adjacent defect targets. Overlapping redundant boxes are removed based on the CURRO filtering threshold, resulting in a standardized detector library with a unified calling interface and a unified result output format. .

[0030] (2) The working principle of the annotation benchmark set construction module is as follows: Figure 2 As shown, the following functions are implemented: Acquire unlabeled building defect image set For unlabeled building defect image sets For each image, a pixel-level mask of the target building structural elements is generated using a preset semantic segmentation model, and the region of interest image containing the core building defect area is obtained by cropping based on the pixel-level mask.

[0031] After performing size scaling and pixel normalization preprocessing on the region of interest image, a feature encoder is used. Extract global feature vectors representing the overall building structure, defect morphology, and scene semantics of the image. .

[0032] Calculate unlabeled building defect image set The cosine similarity between the global feature vectors corresponding to each image is calculated, and the cosine similarity is converted into feature distance to construct a feature distance matrix.

[0033] Density clustering was performed based on the feature distance matrix to obtain a set of unlabeled building defect images. The image set of unlabeled building defects is categorized according to the defect scene category, and the defect scene category is used to further categorize the images. Divided into S defect scenario subsets It also filters outlier noise samples generated during density clustering.

[0034] From S subsets of defective scenarios A predetermined number of representative images are manually annotated to obtain multiple annotated sample sets corresponding to each defect scenario. All annotated sample sets are then compiled to form a manual annotation benchmark set. .

[0035] (5) The working principle of the dynamic gating training and scenario-level routing construction module is as follows: Figure 3 As shown, the following functions are implemented: Based on manually annotated benchmark set Regarding the first s The first defect scenario and the first n Individual detectors, overall score The calculation formula is: .

[0036] in, Indicates the first n The detector at the 1st s In each defect scenario, the mean intersection-over-union ratio (IoU) between the predicted bounding boxes and the manually labeled ground truth bounding boxes is used to characterize the localization accuracy. Indicates the first n The mean overlap and consistency between the predictions of each detector and those of other detectors is used to characterize the overall consistency. Indicates the first n The detector at the 1st sThe expected calibration error in each scenario is used to characterize the deviation between the prediction confidence and the actual prediction reliability. , , These are preset weighting coefficients.

[0037] The overall score is calculated using the Softmax function. Transform into scene-level detector probabilistic soft labels .

[0038] Regarding the first A subset of defective scenarios Obtain the global feature vectors corresponding to each image in the defect scene subset. and the global feature vector Perform mean aggregation to obtain the first... Scene feature representation corresponding to each defect scenario : .

[0039] in, Indicates the first The first defect scenario subset Zhang Image Representing an image The corresponding global feature vector, Indicates the first The number of images in a subset of defective scenes.

[0040] Using scene feature representations as input and scene-level detector probabilistic soft labels as input. As a supervisory signal, training the dynamic gating network G Output the routing weights of each detector in each defect scenario; summarize all S The routing weights of the detectors in each defect scenario are used to construct a routing weight matrix between the scenario and the detectors. This is used to characterize the priority ranking relationship of each detector under different defect scenarios.

[0041] (6) The working principle of the online image dynamic diversion detection module combined with random sampling mechanism is as follows: Figure 4 As shown, the following functions are implemented: Continuously acquire newly added unlabeled building defect images, and use the annotation benchmark set construction module to determine the defect scene corresponding to each newly added unlabeled building defect image. When the number of newly added images in any defect scene reaches the preset batch threshold, construct the current scene batch image set corresponding to the defect scene.

[0042] Based on this defect scenario, in the routing weight matrix The corresponding detector routing weights in the standardized detector library The detectors in the array are sorted in descending order, and the top-ranked detectors are activated. K The detectors form the optimal detector group .

[0043] Those not included in the optimal detector group The detectors were used as long-tail candidate detectors, and the sampling ratio was determined according to the exploration sampling ratio. Random selection M The detectors form a long-tailed detector group. ,in, .

[0044] Synchronously input the current scene batch image set into the optimal detector group With long-tail detector group Perform concurrent inference and align the parser using heterogeneous results. For the optimal detector group With long-tail detector group The predicted bounding box coordinates, category information, and confidence scores output by the detector are uniformly formatted. Spatial matching is performed on predicted bounding boxes of the same category to form predicted box clusters corresponding to the same candidate defect target. For each predicted box cluster, the spatial position difference of the predicted boxes is calculated. Confidence differences at the frame level Difference between target and missed detection And calculate the model prediction divergence degree corresponding to the candidate defect target according to the following formula. : .

[0045] in, This indicates the degree of inconsistency in the positions of the predicted bounding boxes corresponding to the same candidate defect target. This indicates the degree of difference between the confidence levels corresponding to each prediction box. This indicates the percentage of activated detectors that did not output the predicted bounding box for the candidate defect. , and The corresponding preset weight coefficients, and satisfying ; When multiple candidate defect targets exist in an image, the model prediction divergence scores corresponding to each candidate defect target are averaged and aggregated to obtain the model prediction divergence score for the image. .

[0046] For predicted bounding boxes of the same category with a spatial intersection-union ratio not lower than a preset fusion threshold, weighted bounding box fusion is performed using the combination of detector routing weights and prediction confidence as fusion weights to obtain the building defect detection results of each image in the current scene batch image set, and the building defect detection results are used in the subsequent closed-loop update process.

[0047] (7) The working principle of the anti-forgetting closed-loop update module under budget constraints is as follows: Figure 5 As shown, the following functions are implemented: Statistical analysis of model prediction divergence in the current scene batch image set Not lower than the preset divergence threshold The proportion of high-dissipation images; a temporary detector ranking is formed based on the unlabeled adaptation scores of each activated detector on the current scene batch image set, and the temporary detector ranking is combined with the routing weight matrix. The original detector order was compared to determine the degree of change in the detector order; the long-tail detector group was calculated separately. With the optimal detector group The average unlabeled adaptation score; The unlabeled risk monitoring results of the current scene batch image set are determined to meet the anomaly triggering condition by judging whether at least one of the following conditions is met: In the current scene batch image set, the model prediction divergence degree... Not lower than the preset divergence threshold The proportion of highly divergent images reaches a preset threshold; the temporary detector sorting and routing weight matrix is ​​formed based on the current scene batch image set. In the middle, the degree of change between the original detector rankings reaches the preset ranking change threshold; long-tail detector group The average unlabeled adaptation score is higher than that of the optimal detector group. The average unlabeled adaptation score is higher than the preset adaptation difference threshold.

[0048] When the unlabeled risk monitoring results of the current scene batch image set do not meet the anomaly triggering conditions, old images matching the number of images in the current scene batch are extracted from the historical training library to jointly construct a hybrid training set. The optimal detector group activated in this round is then selected based on the hybrid training set. With long-tail detector group Perform local fine-tuning, and update the detector ranking relationship and routing weight matrix corresponding to the current defect scenario based on the fine-tuned detector performance. The updated results will be used for subsequent online dynamic triage building defect detection.

[0049] When any defect scenario has completed a preset number of batch evaluations, all [items] will be activated. N Each detector, based on the mixed training data accumulated within a preset number of rounds for this defect scenario, performs a comprehensive analysis of all... N Each detector performs periodic incremental training. After completing the periodic incremental training, when a new batch of scene images of the defective scene is generated again, the new batch of scene images is used to train all detectors. NEach detector performs a label-free scenario adaptability assessment, redetermines the ranking relationship of each detector in the defective scenario, and updates the routing weight matrix. and dynamic gating network G Routing supervision information.

[0050] When the unlabeled risk monitoring results of the current batch of images meet the anomaly triggering conditions, the model predicts the divergence degree. Greater than or equal to the preset divergence threshold Difficult images, based on model-predicted divergence. Sort by high to low, and within the limit of the manual correction budget. Under these conditions, the most difficult images in the ranking are selected for manual correction to obtain correction labels. Comparing pseudo-labels and correction labels The degree of deviation between them.

[0051] When the deviation does not reach the preset abnormality confirmation threshold, the calibration label is used. Correct relevant pseudo-labels and perform local compensation updates on relevant detectors; when the deviation reaches a preset anomaly confirmation threshold, determine that the detector adaptation relationship in the defective scenario has changed significantly, and wake up all detectors. Each detector underwent a complete update, and the dynamic gating network was retrained. G After completing the full update, reset the cumulative batch evaluation rounds corresponding to the defective scenario to zero, and restart the cycle counting from the next new scenario batch; update the detector sorting relationship and routing weight matrix after partial compensation update or full update. and dynamic gating network G The routing supervision information is used for online dynamic routing of building defect detection in subsequent batches of scene images.

[0052] like Figure 6 As shown, the operation process of the multi-task autonomous intelligent agent for building defect detection is as follows: Acquire images of various types of unlabeled building defects; Unlabeled building defect images are divided into scenes and representative samples are labeled to form a defect scene subset and a manually labeled baseline set. An initial target detector library consisting of multiple deep learning detection models is constructed, and the output format of each detector is unified through a heterogeneous result alignment parser to form a standardized detector library. The adaptability of each detector under different defect scenarios is evaluated using a manually labeled benchmark set. A dynamic gating network is trained to generate a routing weight matrix between the scenario and the detector. During the online application phase, newly added unlabeled building defect images are continuously acquired and the defect scene to which each newly added unlabeled building defect image belongs is determined. When the number of newly added images of any defect scene reaches the preset batch threshold, priority detectors are selected according to the routing weight, and long-tail detectors are randomly selected for concurrent inference. The model prediction divergence degree is calculated and the building defect detection results are fused and output. Based on the building defect detection results, pseudo-labels are generated. Based on the model prediction divergence, the degree of change in detector ranking, and the adaptation performance between long-tail detectors and the optimal detector, unlabeled risk monitoring is performed on the current scene batch image set. Based on the unlabeled risk monitoring results of the current scene batch image set, local fine-tuning of detectors, periodic full detector training, a small amount of manual correction, and adaptive route adjustment are performed to form a continuously evolving closed-loop intelligent agent.

[0053] To further illustrate the technical solution of this invention, the following three specific application examples are provided: Case 1: An inspection system for damage to the exterior facade of old residential buildings based on the present invention; The system is designed for images of the exterior facades of older residential communities, and performs tasks to detect architectural anomalies such as wall damage, peeling exterior walls, broken windows, illegal stacking on balconies, and unauthorized constructions on the facades.

[0054] Inspection personnel use handheld terminals, property inspection equipment, or fixed cameras to collect images of building facades. The system caches newly added facade images daily into a set of unlabeled building defect images. The intelligent agent completes scene clustering, detector dynamic scheduling, fusion detection, and closed-loop updates according to a preset process.

[0055] First, the system constructs an initial object detector library, deploying 18 object detectors, including lightweight YOLO series detectors, two-stage Faster R-CNN-like detectors, Transformer-based DETR-like detectors, and improved detectors suitable for small and dense object detection. All these detectors are deployed on a local edge computing server, each with an independent inference interface. The system further configures a heterogeneous result alignment parser to uniformly parse the bounding box coordinates, class labels, and confidence scores output by different detectors, mapping all detector outputs to a unified result format. It should be noted that heterogeneous result alignment only changes the way the detector outputs are expressed, not the number of detectors; therefore, the 18 original detectors remain 18 standardized detectors after standardization.

[0056] In the scene clustering stage, the system acquires 2000 images of the exterior facades of old residential areas to construct an unlabeled image set of building defects. For each facade image, the system uses a pre-set semantic segmentation model to extract pixel-level masks for the building's exterior walls, windows, balconies, air conditioner outdoor units, and suspected damaged areas. When multiple suspected defect areas exist in the same image, the system identifies multiple separate connected defect regions based on the masks and crops them to form multiple regions of interest (ROI) images. Subsequently, the system performs size scaling and pixel normalization preprocessing on the ROI images and inputs them into a lightweight feature encoder obtained through knowledge distillation based on the DINOv3 visual representation model. The encoder extracts regional features and aggregates them into a global feature vector corresponding to the original image through mean pooling and maximum response feature fusion. The system constructs a feature distance matrix based on the feature distances between the global feature vectors and performs density clustering, dividing the 2000 original facade images into 5 defect scene subsets, corresponding to wall damage scenes, exterior wall peeling scenes, window damage scenes, abnormal stacking on balconies scenes, and illegal construction scenes on facades. It should be noted that the region of interest image is only used for feature extraction and scene clustering. After clustering is completed, the system will assign the original uncropped image to the corresponding defect scene subset.

[0057] Subsequently, the system extracts 60 representative facade images from each defect scene subset for manual annotation, resulting in a total of 300 manually annotated images, serving as a manual annotation baseline set. The manual annotations include defect categories, defect bounding box locations, and necessary manual correction instructions. Using this manually annotated baseline set, the system performs an initial evaluation of the detection performance of 18 standardized detectors in five defect scenes, calculating localization accuracy, group consistency, and confidence calibration deviation, and generating a comprehensive score for each detector in each defect scene. The system converts the comprehensive scores of the 18 detectors in the same defect scene into scene-level detector probabilistic soft labels. Using scene feature representations as input and scene-level detector probabilistic soft labels as supervisory signals, a dynamic gating network is trained to generate a routing weight matrix between the five scenes and the 18 detectors.

[0058] During the online detection phase, the system caches newly added facade images on a per-defect-scene basis. When the number of newly added facade images in any defect-scene reaches 200, a round of online dynamic traffic splitting detection for that defect-scene is triggered. For example, when the number of newly cached facade images in a wall damage scene reaches 200, the system reads the detector routing weights corresponding to that defect-scene in the routing weight matrix, sorts them from high to low weights, and selects the top 3 detectors as the optimal detector group. Simultaneously, the system treats the remaining 15 detectors that did not enter the optimal detector group as long-tail candidate detectors, and randomly selects 5 long-tail detectors according to a 30% exploration sampling ratio to form a long-tail detector group. Therefore, a total of 8 detectors are activated for concurrent inference in this round of wall damage scene detection.

[0059] For the 200 images in the current batch of wall damage scenes, the system synchronously inputs each image into the aforementioned 8 activation detectors. Each detector outputs its predicted location bounding box, category label, and box-level confidence score. The system uses a heterogeneous result alignment parser to uniformly format the outputs of each detector, and for the same candidate wall damage area, compares the spatial location differences between the predicted boxes of different detectors, the confidence differences of the corresponding predicted boxes, and whether any detectors have missed detecting the target. Based on this, the system calculates the model prediction divergence for each image.

[0060] Weighted bounding box fusion is performed on predicted bounding boxes of the same category with a spatial intersection-union ratio (IU) not lower than a preset fusion threshold, outputting the final wall damage detection bounding box and its corresponding fusion confidence score. The fusion weight is jointly determined by the routing weight of the detector in the wall damage scenario and the box-level confidence score of the predicted bounding box output by the detector, so that the detector that is more reliable and has a higher prediction confidence score in the current scenario has a greater impact on the final fusion result.

[0061] During the closed-loop update phase, the system generates initial pseudo-labels for the current 200 images of wall damage scenes based on the wall damage detection results obtained from weighted bounding box fusion. It then performs unlabeled risk monitoring based on the model prediction divergence, detector ranking changes, and the fit difference between the long-tail detector group and the optimal detector group for the current batch of images. When the unlabeled risk monitoring results do not meet the preset anomaly triggering conditions, the system extracts 200 historical images of wall damage or similar facade scenes from the historical training library and combines them with the current 200 newly added images carrying the initial pseudo-labels to construct a mixed training set of 400 images. Subsequently, the system uses this mixed training set to perform local fine-tuning on the eight detectors participating in the inference round, including three optimal detectors and five long-tail detectors. After fine-tuning, the system locally adjusts the routing weights corresponding to the wall damage scenes based on the update results of the aforementioned detectors in the current scene. This ensures that the long-tail detectors, whose performance has improved after training, have a higher probability of entering the optimal detector group in subsequent batches, thus avoiding long-term solidification of detector rankings.

[0062] The system is designed to trigger a periodic full detector training cycle after each defect scenario completes five rounds of batch evaluation. Once five rounds of evaluation have been completed for the wall damage scenario, processing a total of 1000 new images, the system wakes up all 18 detectors and performs periodic incremental training on all detectors based on the accumulated mixed training data within this cycle. After training, the system does not directly reorder the old data already used for training. Instead, it waits for a new batch of 200 images for the next wall damage scenario and uses these new images to perform an unlabeled scene adaptability evaluation on all 18 detectors. This re-determines the ranking relationship of each detector in this scenario and updates the routing weight matrix and the routing supervision information of the dynamic gating network. Through this periodic full detector training and new batch reordering mechanism, the system can provide long-tail detectors with opportunities for regular training and re-entry into the priority detector group, reducing the risk of routing stagnation caused by the long-term dominance of the optimal detector group.

[0063] When the unlabeled risk monitoring results meet the preset anomaly triggering conditions, the system filters difficult images whose model prediction divergence reaches or exceeds the preset divergence threshold, sorts them from high to low according to the model prediction divergence, and selects the top-ranked difficult images for manual correction within the manual correction budget limit of no more than 40 images per round to obtain correction labels. Subsequently, the system compares the degree of deviation between the initial pseudo-label and the correction label; if the degree of deviation does not reach the preset anomaly confirmation threshold, the system uses the correction label to correct the relevant pseudo-label and performs local compensation updates on the relevant detectors; if the degree of deviation reaches the preset anomaly confirmation threshold, the system determines that the detector adaptation relationship in the wall damage scenario has changed significantly, wakes up all 18 detectors to perform a full update, and retrains the dynamic gating network to reconstruct the detector ranking and routing weights in this scenario; after completing the full update, the system resets the cumulative batch evaluation rounds corresponding to the wall damage scenario to zero and restarts the cycle counting from the subsequent new scenario batches.

[0064] The old residential building facade damage inspection system based on this invention can adaptively detect various building anomalies such as wall damage, window damage, exterior wall peeling, abnormal stacking on balconies, and illegal construction in the facade inspection task of old residential buildings. At the same time, through collaborative reasoning between the optimal detector group and the long-tail detector group, box-level divergence calculation, weighted bounding box fusion, initial pseudo-label risk monitoring, low-risk local fine-tuning, periodic full detector training, and risk-triggered minor manual correction, the system can continuously update the detector ranking and routing relationship with low manual intervention cost, thereby improving the long-term adaptability to complex facade scenarios.

[0065] Case 2: Indoor public space security inspection system based on the present invention.

[0066] The system is designed for images of the interior spaces of residential buildings, office buildings, shopping malls, and public buildings, and performs tasks to detect building anomalies such as damaged corridors, damaged stair treads, peeling walls, occupancy of public spaces, blocked fire exits, and illegal dumping of debris.

[0067] Inspection personnel use handheld terminals, property inspection equipment, or fixed cameras to collect images of indoor public spaces. The system caches the newly added indoor public space images daily into an unlabeled building defect image set. The intelligent agent completes scene clustering, detector dynamic scheduling, fusion detection, and closed-loop updates according to a preset process.

[0068] First, the system constructs an initial target detector library, deploying 16 target detectors. These include a lightweight single-stage detector suitable for fast detection, a two-stage detector suitable for complex occlusion scenarios, a Transformer-based global modeling detector, and a detector optimized for small targets and dense clutter detection. All detectors are deployed on a local edge computing server, each with an independent inference interface. The system further configures a heterogeneous result alignment parser to uniformly parse the bounding box coordinates, class labels, and confidence scores output by different detectors, mapping all detector outputs to a unified result format. It should be noted that heterogeneous result alignment only changes the way the detector outputs are expressed, not the number of detectors; therefore, the 16 original detectors remain 16 standardized detectors after standardization.

[0069] In the scene clustering stage, the system acquires 5,000 indoor public space images to construct an unlabeled building defect image set. For each indoor public space image, the system uses a pre-set semantic segmentation model to extract pixel-level masks for walls, floors, stair treads, corridor passage areas, fire escape areas, and suspected occupied object areas. When multiple suspected defect areas exist in the same image, the system identifies multiple separate defect connected regions based on the masks and crops them to form multiple regions of interest (ROI) images. Subsequently, the system performs size scaling and pixel normalization preprocessing on the ROI images and inputs them into a lightweight feature encoder obtained through knowledge distillation based on the DINOv3 visual representation model. The encoder extracts region features and forms the global feature vector corresponding to the original image through mean pooling, maximum response feature fusion, or attention-weighted aggregation. The system calculates feature distances based on the global feature vectors corresponding to each original image, constructs a feature distance matrix, and performs density clustering to divide 5000 original images of indoor public spaces into five defect scene subsets: damaged corridor walls, damaged stair treads, occupied public spaces, blocked fire exits, and peeling or soiled interior walls. It should be noted that the region of interest (ROI) images are only used for feature extraction and scene clustering. After clustering, the system assigns the original uncropped images to the corresponding defect scene subsets to ensure that subsequent detection, pseudo-label generation, and model training are all performed in the original image coordinate system.

[0070] Subsequently, the system extracts 100 representative original images from each defect scene subset for manual annotation, resulting in a total of 500 manually annotated images, serving as a manual annotation baseline set. The manual annotations include anomaly categories, target bounding box locations, and necessary review notes. Using this manually annotated baseline set, the system performs an initial evaluation of the detection performance of 16 standardized detectors in five defect scenes, calculating localization accuracy, group consistency, and confidence calibration bias, and generating a comprehensive score for each detector in each scene. The system converts the comprehensive scores of the 16 detectors in the same scene into scene-level detector probabilistic soft labels, and trains a dynamic gating network using scene feature representations as input and probabilistic soft labels as supervision signals, generating a routing weight matrix between the five indoor scenes and the 16 detectors.

[0071] During the online detection phase, the system caches newly added indoor public space images on a per-defect scenario basis. When the number of newly added indoor public space images in any defect scenario reaches 200, a round of online dynamic traffic splitting detection for that defect scenario is triggered. For example, when the number of newly cached indoor public space images in the public space occupancy scenario reaches 200, the system reads the detector routing weights corresponding to that defect scenario in the routing weight matrix, sorts them from high to low weights, and selects the top 3 detectors as the optimal detector group. Simultaneously, the system treats the remaining 13 detectors that did not enter the optimal detector group as long-tail candidate detectors, and randomly selects 4 long-tail detectors according to a 30% exploration sampling ratio to form a long-tail detector group. Therefore, a total of 7 detectors are activated for concurrent inference in this round of public space occupancy scenario.

[0072] For the 200 images in the current batch of public space occupancy scenarios, the system synchronously inputs each image into the aforementioned 7 activation detectors. Each detector outputs its bounding box, category label, and box-level confidence score indicating whether public space is occupied, cluttered, or blocked. The system uses a heterogeneous result alignment parser to uniformly format the outputs of each detector, and for the same candidate occupancy region, compares the spatial location differences between the predicted boxes of different detectors, the confidence differences of the corresponding predicted boxes, and whether any detectors have missed detecting the target. Based on this, the system calculates the model prediction divergence for each image.

[0073] Weighted bounding box fusion is performed on predicted bounding boxes of the same category with a spatial intersection-union ratio (IU) not lower than a preset fusion threshold, outputting the final common space occupancy detection bounding box and its corresponding fusion confidence score. The fusion weights are jointly determined by the routing weights of the detectors in the common space occupancy scenario and the box-level confidence scores of the predicted bounding boxes output by those detectors, ensuring that detectors with higher reliability and prediction confidence in the current scenario have a greater impact on the final fusion result.

[0074] During the closed-loop update phase, the system generates initial pseudo-labels for the current 200 public space occupancy scene images based on the public space occupancy detection results obtained from weighted bounding box fusion. It then performs unlabeled risk monitoring based on the model prediction divergence, detector ranking changes, and the fit difference between the long-tail detector group and the optimal detector group for the current batch of images. When the unlabeled risk monitoring results do not meet the preset anomaly triggering conditions, the system extracts 200 historical images of public space occupancy or similar indoor public area scenes from the historical training library and combines them with the current 200 newly added images carrying the initial pseudo-labels to construct a mixed training set of 400 images. Subsequently, the system uses this mixed training set to perform local fine-tuning on the 7 detectors participating in the inference round, including 3 optimal detectors and 4 long-tail detectors. After fine-tuning, the system locally adjusts the routing weights corresponding to the public space occupancy scene based on the update results of the aforementioned detectors in the current scene. This ensures that the long-tail detectors, whose performance has improved after training, have a higher probability of entering the optimal detector group in subsequent batches, thereby avoiding long-term solidification of detector rankings.

[0075] The system is designed to trigger a periodic full detector training cycle after each defect scenario completes five rounds of batch evaluation. Once the public space occupancy scenario has completed five rounds and processed 1000 new images, the system wakes up all 16 detectors and performs periodic incremental training on all detectors based on the accumulated mixed training data within that cycle. After training, the system does not directly reorder the old data already used for training. Instead, it waits for the arrival of a new batch of 200 images for the next public space occupancy scenario and uses these new images to perform an unlabeled scene adaptability evaluation on all 16 detectors. This re-determines the ranking relationship of each detector in that scenario and updates the routing weight matrix and the routing supervision information of the dynamic gating network. Through this periodic full detector training and new batch reordering mechanism, the system can alleviate the problem of long-term, frequent training of the optimal detector while long-tail detectors lack update opportunities, allowing the detector ranking to be dynamically adjusted according to changes in the distribution of indoor public space images.

[0076] When the unlabeled risk monitoring results meet the preset anomaly triggering conditions, the system filters difficult images whose model prediction divergence reaches or exceeds the preset divergence threshold, sorts them from high to low according to the model prediction divergence, and selects the top-ranked difficult images for manual correction within the manual correction budget limit of no more than 40 images per round to obtain correction labels. Subsequently, the system compares the degree of deviation between the initial pseudo-label and the correction label; if the degree of deviation does not reach the preset anomaly confirmation threshold, the system uses the correction label to correct the relevant pseudo-label and performs local compensation updates on the relevant detectors; if the degree of deviation reaches the preset anomaly confirmation threshold, the system determines that the detector adaptation relationship in the public space occupation scenario has changed significantly, wakes up all 16 detectors to perform a full update, and retrains the dynamic gating network to reconstruct the detector ranking and routing weights in this scenario; after completing the full update, the system resets the cumulative batch evaluation rounds corresponding to the public space occupation scenario to zero and restarts the cycle counting from the subsequent new scenario batches.

[0077] The indoor public space safety inspection system based on this invention can adaptively detect various building anomalies in indoor public space safety inspection tasks, such as damaged corridors, damaged stair treads, peeling walls, occupancy of public spaces, blocked fire exits, and illegal stacking of debris. At the same time, through scene-level routing, collaborative reasoning between the optimal detector group and the long-tail detector group, box-level divergence calculation, weighted bounding box fusion, initial pseudo-label risk monitoring, low-risk local fine-tuning, periodic full detector training, and risk-triggered minor manual correction, the system can continuously optimize the detector sorting and routing relationship in indoor public spaces with low manual intervention costs, thereby improving the long-term adaptability to complex indoor building anomaly scenarios.

[0078] Case 3: Park building facility inspection system based on the present invention.

[0079] The system is designed for images of buildings and facilities within university campuses, industrial parks, office parks, and large public institution parks. It performs tasks to detect abnormal building environments such as wall damage, window damage, staircase damage, floor damage, ceiling damage, public space occupation, blocked fire exits, illegal construction, damage to public facilities, and abnormal leakage and soiling.

[0080] Park security patrol personnel, property management mobile terminals, patrol robots and fixed camera equipment collect images of park buildings and facilities. The system caches the newly added park building and facility images to the unlabeled building defect image set every day. The intelligent agent completes scene clustering, detector dynamic scheduling, fusion detection and closed-loop update according to the preset process.

[0081] First, the system constructs an initial target detector library, deploying 20 target detectors. These include a lightweight single-stage detector suitable for real-time inspection, a two-stage detector for complex building detail recognition, a Transformer-based global context modeling detector, a high-resolution detector for small target detection, and an improved detector optimized for dense occlusion and complex backgrounds. All 20 detectors are deployed on the park's edge computing server or the property management center's computing nodes, each with an independent inference interface. The system configures a heterogeneous result alignment parser to uniformly parse the bounding box coordinates, class labels, and box-level confidence scores output by each detector, mapping them to a unified output format. It should be noted that heterogeneous result alignment only unifies the expression of detection results and does not change the number of detectors; therefore, the 20 original detectors remain 20 standardized detectors after standardization.

[0082] In the scene clustering stage, the system acquires 5,000 images of park buildings and facilities to construct an unlabeled image set of building defects. For each image of park buildings and facilities, the system uses a preset semantic segmentation model to extract pixel-level masks for building exterior walls, windows, corridors, ground, public facilities, passage areas, and suspected abnormal areas. When multiple abnormal areas such as wall damage, window damage, debris piled in corridors, or damaged public facilities exist simultaneously in the same image, the system identifies multiple separate defect connected regions based on the masks and crops them to form multiple regions of interest (ROI) images. Subsequently, the system performs size scaling and pixel normalization preprocessing on the ROI images and inputs them into a lightweight feature encoder obtained by knowledge distillation based on the DINOv3 visual representation model. The encoder extracts regional features and forms the global feature vector corresponding to the original image through mean pooling, maximum response feature fusion, or attention-weighted aggregation. The system calculates feature distances based on the global feature vectors corresponding to each original image, constructs a feature distance matrix, and performs density clustering to divide 5000 original images of building facilities into 10 defect scene subsets, corresponding to scenes such as wall damage, window damage, staircase damage, floor damage, ceiling damage, public space occupation, blocked fire exits, illegal construction, damage to public facilities, and leakage and soiling. It should be noted that the region of interest image is only used for feature extraction and scene clustering. After clustering, the system assigns the original uncropped images to the corresponding defect scene subsets to ensure that subsequent detection, pseudo-label generation, and model training are all performed in the original image coordinate system.

[0083] Subsequently, the system extracts 80 representative original images from each defect scene subset for manual annotation, forming a total of 800 manually annotated images as a manual annotation baseline set. The manual annotations include anomaly categories, target bounding box locations, and necessary review notes. Using this manually annotated baseline set, the system performs an initial evaluation of the detection performance of 20 standardized detectors in 10 defect scenes, calculating localization accuracy, group consistency, and confidence calibration bias, and generating a comprehensive score for each detector in each scene. The system converts the comprehensive scores of the 20 detectors in the same scene into scene-level detector probabilistic soft labels, and trains a dynamic gating network using scene feature representations as input and probabilistic soft labels as supervision signals, generating a routing weight matrix between the 10 park building scenes and the 20 detectors.

[0084] During the online detection phase, the system caches newly added park building and facility images on a per-defect scenario basis. When the number of newly added park building and facility images in any defect scenario reaches 200, a round of online dynamic traffic splitting detection for that defect scenario is triggered. For example, when the number of newly cached park building images in the public space occupancy scenario reaches 200, the system reads the detector routing weights corresponding to that defect scenario in the routing weight matrix, sorts them from high to low weights, and selects the top 4 detectors as the optimal detector group. Simultaneously, the system treats the remaining 16 detectors that did not enter the optimal detector group as long-tail candidate detectors, and randomly selects 5 long-tail detectors according to a 30% exploration sampling ratio to form a long-tail detector group. Therefore, a total of 9 detectors are activated for concurrent inference in this round of the public space occupancy scenario.

[0085] For the 200 images in the current batch of public space occupancy scenarios, the system synchronously inputs each image into the aforementioned 9 active detectors. Each detector outputs its bounding box, category label, and box-level confidence score indicating that it believes there is passageway occupancy, stairwell obstruction, or abnormal occupancy in public areas. The system uses a heterogeneous result alignment parser to uniformly format the output results of each detector, and for the same candidate occupancy region, compares the spatial location differences between the predicted boxes of different detectors, the confidence differences of the corresponding predicted boxes, and whether any detectors have missed detecting the target, thereby calculating the model prediction divergence degree for each image.

[0086] Weighted bounding box fusion is performed on predicted bounding boxes of the same category with a spatial intersection-union ratio (IU) not lower than a preset fusion threshold, outputting the final common space occupancy detection bounding box and its corresponding fusion confidence score. The fusion weights are jointly determined by the routing weights of the detectors in the common space occupancy scenario and the box-level confidence scores of the predicted bounding boxes output by those detectors, ensuring that detectors with higher reliability and prediction confidence in the current scenario have a greater impact on the final fusion result.

[0087] During the closed-loop update phase, the system generates initial pseudo-labels for the current 200 public space occupancy scene images based on the public space occupancy detection results obtained from weighted bounding box fusion. It then performs unlabeled risk monitoring based on the model prediction divergence, detector ranking changes, and the fit difference between the long-tail detector group and the optimal detector group for the current batch of images. When the unlabeled risk monitoring results do not meet the preset anomaly triggering conditions, the system extracts 200 historical old images of public area scenes from the historical training library and combines them with the current 200 newly added images carrying the initial pseudo-labels to construct a mixed training set of 400 images. Subsequently, the system uses this mixed training set to perform local fine-tuning on the 9 detectors participating in the inference round, including 4 optimal detectors and 5 long-tail detectors. After fine-tuning, the system locally adjusts the routing weights corresponding to the public space occupancy scene based on the update results of the aforementioned detectors in the current scene. This ensures that the long-tail detectors, whose performance has improved after training, have a higher probability of entering the optimal detector group in subsequent batches, thereby avoiding long-term solidification of detector rankings.

[0088] The system is designed to trigger a periodic full detector training cycle after each defect scenario completes five rounds of batch evaluation. Once the public space occupancy scenario has completed five rounds and processed 1000 new images, the system wakes up all 20 detectors and performs periodic incremental training on all detectors based on the accumulated mixed training data within that cycle. After training, the system does not directly reorder the old data already used for training. Instead, it waits for a new batch of 200 images in the public space occupancy scenario and uses these new images to perform an unlabeled scene adaptability evaluation on all 20 detectors. This re-determines the ranking relationship of each detector in that scenario and updates the routing weight matrix and the routing supervision information of the dynamic gating network. Through this periodic full detector training and new batch reordering mechanism, the system can alleviate the problem of long-term, frequent training of the optimal detector while long-tail detectors lack update opportunities, allowing the detector ranking to be dynamically adjusted according to changes in the distribution of images of buildings and facilities within the park.

[0089] When the unlabeled risk monitoring results meet the preset anomaly triggering conditions, the system filters difficult images whose model prediction divergence reaches or exceeds the preset divergence threshold, sorts them from high to low according to the model prediction divergence, and selects the top-ranked difficult images for manual correction within the manual correction budget limit of no more than 40 images per round to obtain correction labels. Subsequently, the system compares the degree of deviation between the initial pseudo-label and the correction label; if the degree of deviation does not reach the preset anomaly confirmation threshold, the system uses the correction label to correct the relevant pseudo-label and performs local compensation updates on the relevant detectors; if the degree of deviation reaches the preset anomaly confirmation threshold, the system determines that the detector adaptation relationship in the public space occupation scenario has changed significantly, wakes up all 20 detectors to perform a full update, and retrains the dynamic gating network to reconstruct the detector ranking and routing weights in this scenario; after completing the full update, the system resets the cumulative batch evaluation rounds corresponding to the public space occupation scenario to zero and restarts the cycle counting from the subsequent new scenario batches.

[0090] Based on the present invention, the park building facility inspection system can adaptively detect various building environment anomalies in park building facility inspection tasks, such as wall damage, window damage, stair damage, ground damage, ceiling damage, public space occupation, fire lane blockage, illegal construction, public facility damage, and leakage and dirt abnormalities. At the same time, through multi-detector scene-level routing, collaborative reasoning between the optimal detector group and the long-tail detector group, box-level divergence calculation, weighted bounding box fusion, initial pseudo-label risk monitoring, low-risk local fine-tuning, periodic full detector training, and risk-triggered minor manual correction, the system can continuously optimize the detector sorting and routing relationship in park building scenes with low manual intervention costs, and improve the adaptability to newly added abnormal images in long-term operation scenarios.

[0091] The present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A multi-task autonomous intelligent agent for building defect detection, characterized in that, include: A standardized detector library building module is used to obtain an initial detector library covering various deep learning detection models. ; Build and configure a heterogeneous result alignment parser To form a standardized detector library ; The annotation benchmark set construction module is used to obtain an image set of unlabeled building defects. From the unlabeled building defect image set Extract global feature vectors The unlabeled building defect image set Divided into S defect scenario subsets ; The dynamic gating training and scene-level routing construction module is used to compute the standardized detector library. The Middle n The detector at the 1st s Comprehensive score under each defect scenario And convert them into scene-level detector probability soft labels. ; Extract scene feature representations corresponding to each defect scenario, and use these scene feature representations as input, along with probabilistic soft labels from a scene-level detector. As a supervisory signal, training the dynamic gating network G Output the routing weights of each detector in each defect scenario; Summary of all S The routing weights of the detectors in each defect scenario are used to construct a routing weight matrix between the scenario and the detectors. ; An online image dynamic diversion detection module with random sampling mechanism is used to continuously acquire newly added unlabeled building defect images and determine the corresponding defect scene and the current scene batch image set corresponding to the defect scene; Based on the defect scenario in the routing weight matrix The corresponding detector routing weights form the optimal detector group. and long-tail detector group Synchronously input the current scene batch image set into the optimal detector group. With long-tailed detector group Perform concurrent inference and align the parser using heterogeneous results. For the optimal detector group With long-tailed detector group The output predicted bounding boxes are uniformly formatted, and spatial matching is performed on predicted bounding boxes of the same category to form a cluster of predicted boxes corresponding to the same candidate defect target. The model prediction divergence degree for each image is calculated based on the differences between predicted boxes of the same category. For predicted bounding boxes of the same category with a spatial intersection-union ratio not lower than a preset fusion threshold, weighted bounding box fusion is performed using the combination of detector routing weight and prediction confidence as the fusion weight to obtain the building defect detection results of each image in the current scene batch image set. The budget-constrained anti-forgetting closed-loop update module is used to generate pseudo-labels for the current scene batch image set based on the building defect detection results, and predict the divergence degree based on the model. The degree of change in detector ranking and the long-tailed detector group With the optimal detector group The system adapts to the image set in the current scene by performing unlabeled risk monitoring and obtaining monitoring results. Based on the monitoring results, it performs local fine-tuning of the detector, periodic full detector training, a small amount of manual correction, and adaptive route adjustment.

2. The multi-task autonomous intelligent agent for building defect detection according to claim 1, characterized in that, The construction and configuration of the heterogeneous result alignment parser To form a standardized detector library ,include: Construct a heterogeneous result alignment parser It receives the raw prediction results output by each detector in parallel, converts the bounding box coordinate representation of different detectors into a preset standard coordinate format, normalizes the confidence scores output by each detector, and maps them to a unified standard probability tensor space. In the heterogeneous result alignment parser Internally, the spatial distribution density features of predicted bounding boxes in the current image are statistically analyzed, and the cross-union ratio (CUP) filtering threshold for non-maximum suppression is dynamically determined based on these features. When the predicted bounding boxes are densely distributed, the CUP filtering threshold is increased to reduce the probability of over-suppressing adjacent defect targets. Based on the CUP filtering threshold, overlapping redundant boxes are removed, forming a standardized detector library with a unified calling interface and a unified result output format. .

3. The multi-task autonomous intelligent agent for building defect detection according to claim 1, characterized in that, From the unlabeled building defect image set Extract global feature vectors ,include: For the unlabeled building defect image set For each image, a pixel-level mask of the target building structural elements is generated using a preset semantic segmentation model, and a region of interest image containing the core building defect area is obtained by cropping based on the pixel-level mask. After performing size scaling and pixel normalization preprocessing on the region of interest image, a feature encoder is used. Extract global feature vectors representing the overall building structure, defect morphology, and scene semantics of the image. .

4. The multi-task autonomous intelligent agent for building defect detection according to claim 1, characterized in that, The unlabeled building defect image set Divided into S defect scenario subsets Then from S defective scene subsets A predetermined number of representative images are extracted and manually annotated to obtain multiple annotated sample sets corresponding to each defect scenario. All annotated sample sets are then compiled to form a manual annotation benchmark set. ; Among them, the unlabeled building defect image set Divided into S defect scenario subsets ,include: Calculate the unlabeled building defect image set The cosine similarity between the global feature vectors corresponding to each image is calculated, and the cosine similarity is converted into feature distance to construct a feature distance matrix; Density clustering is performed based on the feature distance matrix to obtain the unlabeled building defect image set. The image set of unlabeled building defects is categorized according to the defect scene category corresponding to each image. Divided into S defect scenario subsets It also filters outlier noise samples generated during density clustering.

5. The multi-task autonomous intelligent agent for building defect detection according to claim 1, characterized in that, Calculate the standardized detector library The Middle n The detector at the 1st s Comprehensive score under each defect scenario And convert them into scene-level detector probability soft labels. ,include: Regarding the first s The first defect scenario and the first n Individual detectors, overall score The calculation formula is: ； in, Indicates the first n The detector at the 1st s In each defect scenario, the mean intersection-union ratio (IUU) between the predicted bounding boxes and the manually labeled ground truth bounding boxes is used to characterize the localization accuracy. Indicates the first n The mean overlap and consistency between the predictions of each detector and those of other detectors is used to characterize the overall consistency. Indicates the first n The detector at the 1st s The expected calibration error in each scenario is used to characterize the deviation between the prediction confidence and the actual prediction reliability. , , Preset weighting coefficients; The comprehensive score is calculated using the Softmax function. Transform into scene-level detector probabilistic soft labels .

6. The multi-task autonomous intelligent agent for building defect detection according to claim 1, characterized in that, The extraction of scene feature representations corresponding to each defect scene includes: Regarding the first A subset of defective scenarios Obtain the global feature vectors corresponding to each image in the defect scene subset. and the global feature vector Perform mean aggregation to obtain the first... Scene feature representation for each defect scenario : ； in, Indicates the first The first defect scenario subset Zhang Image Representing an image The corresponding global feature vector, Indicates the first The number of images in a subset of defective scenes.

7. The multi-task autonomous intelligent agent for building defect detection according to claim 1, characterized in that, The model prediction divergence degree for each image is calculated based on the differences between prediction boxes of the same category. ,include: For each cluster of predicted bounding boxes, calculate the spatial location difference of the predicted bounding boxes separately. Confidence differences at the frame level Difference between target and missed detection And calculate the model prediction divergence degree corresponding to the candidate defect target according to the following formula. : ； in, This indicates the degree of inconsistency in the positions of the predicted bounding boxes corresponding to the same candidate defect target. This indicates the degree of difference between the confidence levels corresponding to each prediction box. This indicates the percentage of activated detectors that did not output the predicted bounding box for the candidate defect. , and The corresponding preset weight coefficients, and satisfying ; When multiple candidate defect targets exist in an image, the model prediction divergence scores corresponding to each candidate defect target are averaged and aggregated to obtain the model prediction divergence score for the image. .

8. The multi-task autonomous intelligent agent for building defect detection according to claim 1, characterized in that, The model-based prediction of divergence The degree of change in detector ranking and the long-tailed detector group With the optimal detector group The adaptation performance between them, and the unlabeled risk monitoring for the current batch of image sets, including: Statistical analysis of model prediction divergence in the current scene batch image set Not lower than the preset divergence threshold The proportion of high-dissipation images; a temporary detector ranking is formed based on the unlabeled adaptation scores of each activated detector on the current scene batch image set, and the temporary detector ranking is combined with the routing weight matrix. The original detector order was compared to determine the degree of change in the detector order; the long-tail detector group was calculated separately. With the optimal detector group The average unlabeled adaptation score; The unlabeled risk monitoring results of the current scene batch image set are determined to meet the anomaly triggering condition by judging whether at least one of the following conditions is met: In the current scene batch image set, the model prediction divergence degree... Not lower than the preset divergence threshold The proportion of highly divergent images reaches a preset threshold; the temporary detector sorting and routing weight matrix is ​​formed based on the current scene batch image set. In the middle, the degree of change between the original detector rankings reaches the preset ranking change threshold; long-tail detector group The average unlabeled adaptation score is higher than that of the optimal detector group. The average unlabeled adaptation score is higher than the preset adaptation difference threshold.

9. A multi-task autonomous intelligent agent for building defect detection according to claim 8, characterized in that, The process of performing local fine-tuning of the detector, periodic full detector training, minor manual correction, and adaptive route adjustment based on monitoring results includes: When the unlabeled risk monitoring results of the current scene batch image set do not meet the abnormal triggering conditions, old images matching the number of images in the current scene batch are extracted from the historical training library to jointly construct a hybrid training set. Based on the hybrid training set, the optimal detector group activated in this round is selected. With long-tailed detector group Perform local fine-tuning, and update the detector ranking relationship and routing weight matrix corresponding to the current defect scenario based on the fine-tuned detector performance. The updated results will be used for subsequent online dynamic triage of building defect detection; When any defect scenario has completed a preset number of batch evaluations, all [items] will be activated. N Each detector, based on the mixed training data accumulated within the preset rounds for this defect scenario, performs a comprehensive analysis of all... N Each detector performs periodic incremental training. After completing the periodic incremental training, when a new batch of scene images of the defective scene is formed again, the new batch of scene images is used to analyze all detectors. N Each detector performs a label-free scenario adaptability assessment, redetermines the ranking relationship of each detector in the defective scenario, and updates the routing weight matrix. and dynamic gating network G Routing supervision information; When the unlabeled risk monitoring results of the current scene batch image set meet the anomaly triggering conditions, the screening model predicts the divergence degree. Greater than or equal to the preset divergence threshold Difficult images, based on model-predicted divergence. Sort by high to low, and within the limit of the manual correction budget. Under these conditions, the most difficult images in the ranking are selected for manual correction to obtain correction labels. Compare the pseudo-label with the correction label. The degree of deviation between them; When the degree of deviation does not reach the preset anomaly confirmation threshold, the correction label is used. Correct the relevant pseudo-labels and perform local compensation updates on the relevant detectors; when the deviation reaches a preset anomaly confirmation threshold, determine that the detector adaptation relationship in the defective scenario has changed significantly, and wake up all detectors. Each detector underwent a complete update, and the dynamic gating network was retrained. G After completing the full update, reset the cumulative batch evaluation rounds corresponding to the defective scenario to zero, and restart the cycle counting from the next new scenario batch; update the detector sorting relationship and routing weight matrix after partial compensation update or full update. and dynamic gating network G The routing supervision information is used for online dynamic routing of building defect detection in subsequent batches of scene images.

10. A multi-task autonomous intelligent agent working method for building defect detection, based on the multi-task autonomous intelligent agent for building defect detection as described in any one of claims 1 to 9, characterized in that, include: Acquire images of various types of unlabeled building defects; Unlabeled building defect images are divided into scenes and representative samples are labeled to form a defect scene subset and a manually labeled baseline set. An initial target detector library consisting of multiple deep learning detection models is constructed, and the output format of each detector is unified through a heterogeneous result alignment parser to form a standardized detector library. The adaptability of each detector under different defect scenarios is evaluated using a manually labeled benchmark set. A dynamic gating network is trained to generate a routing weight matrix between the scenario and the detector. During the online application phase, newly added unlabeled building defect images are continuously acquired and the defect scene to which each newly added unlabeled building defect image belongs is determined. When the number of newly added images of any defect scene reaches the preset batch threshold, priority detectors are selected according to the routing weight, and long-tail detectors are randomly selected for concurrent inference. The model prediction divergence degree is calculated and the building defect detection results are fused and output. Based on the building defect detection results, pseudo-labels are generated. Based on the model prediction divergence, the degree of change in detector ranking, and the adaptation performance between long-tail detectors and the optimal detector, unlabeled risk monitoring is performed on the current scene batch image set. Based on the unlabeled risk monitoring results of the current scene batch image set, local fine-tuning of detectors, periodic full detector training, a small amount of manual correction, and adaptive route adjustment are performed to form a continuously evolving closed-loop intelligent agent.

Images