A house intelligent agent examination system and method based on multi-agent cooperation

Abstract

The application relates to a house intelligent agent detection system and method based on multi-agent cooperation, and relates to the technical fields of building engineering detection and artificial intelligence. The application generates a detection scheme dynamically by standardizing multi-channel tasks, schedules multi-device cooperation to collect data, combines multi-modal fusion and a disease detection model, constructs a structured disease record, and completes disease mapping and twin synchronization on a three-dimensional BIM model, and finally realizes continuous evolution through an evaluation learning module, realizes house detection full-process automation and intelligentization, solves the pain points of traditional house detection, such as dependence on manual experience, low efficiency, non-uniform standards, serious data island, difficulty in real-time cooperation and closed-loop optimization, improves detection efficiency and accuracy, reduces manual dependence and misjudgment risk, realizes dynamic optimization of a detection scheme and intelligent matching of resources, and provides visual and traceable three-dimensional disease display.

Description

Technical Field

[0001] This invention relates to the fields of building engineering inspection and artificial intelligence technology, and in particular to a building intelligent inspection system and method based on multi-agent collaboration. Background Technology

[0002] Currently, online building inspection platforms have emerged in the building safety inspection industry, enabling online management of the inspection process, such as online completion of inspection forms, online allocation of inspection tasks, and electronic delivery of inspection reports. On the equipment side, technologies such as drone aerial photography, infrared thermal imaging, and 3D laser scanning have also been applied in building inspection.

[0003] The typical workflow of existing technologies is as follows: manual receipt of testing requests → manual preparation of testing plans (based on fixed templates) → manual operation of equipment to collect data on-site → manual analysis of data and identification of defects → manual writing of testing reports → manual archiving. The platform mainly undertakes task flow and document management functions, with AI algorithms only providing assistance in certain stages (such as image recognition).

[0004] The main drawbacks of existing technologies:

[0005] Task rigidity and lack of personalized planning capabilities: The existing platform's detection solutions are generated based on fixed process templates and cannot dynamically generate the optimal detection solution according to the individual characteristics of the building (structural type, construction year, distribution of historical defects, and impact of the surrounding environment), resulting in detection omissions or waste of resources.

[0006] Data silos and insufficient fusion of multi-source data: There is a lack of deep fusion between heterogeneous data (visible light images, infrared thermal images, 3D point clouds) collected by different devices, and the correlation analysis between the data and existing building models and historical archive data on the platform relies on manual alignment and interpretation.

[0007] Static scheduling cannot respond to changes on-site in real time: the equipment routes and tasks are all pre-set statically, and the detection path and resource allocation cannot be automatically adjusted when new high-risk disease areas are discovered.

[0008] Lack of evolutionary capability: Current AI model updates rely on manual retraining and deployment, and the platform cannot automatically use each detection data to optimize itself and accumulate knowledge;

[0009] Therefore, there is an urgent need for a smart home health check system and method based on multi-agent collaboration to solve the above problems. Summary of the Invention

[0010] In view of the above-mentioned technical problems in related technologies, the present invention proposes a smart house health check system and method based on multi-agent collaboration.

[0011] In a first aspect, the present invention provides a smart home health check system based on multi-agent collaboration, comprising the following modules:

[0012] The task scheduling module is used to standardize multi-channel tasks into standard task work orders and add them to the task queue. Then, it constructs a complete task context based on the standard task work orders and outputs the context package.

[0013] The scheme planning module is used to sort the task queue in descending order according to the multi-dimensional weighted score of the task based on the context packet to obtain the currently executing task at the top, and then generate a dynamic detection scheme for the currently executing task based on the context packet.

[0014] The resource scheduling module is used to convert the dynamic detection scheme into an executable instruction set for multiple devices, match all available detection personnel profiles with the detection items in the detection item list to generate individual work orders and push them to mobile terminals.

[0015] The sensing and acquisition module is used to construct multimodal raw data packets by collaboratively acquiring data from multiple devices according to the executable instruction set of the multiple devices, output the execution logs of each device, and generate a real-time event stream when a device detects an anomaly.

[0016] The understanding and analysis module is used to align and fuse the multimodal raw data packets to obtain multimodal fused data. The multimodal fused data is then used to detect diseases through a disease detection model and enhanced with domain knowledge. The data is combined with the results of close-range macro photography, audio, and hollow drum analysis to construct a structured disease record set.

[0017] The mapping and twin module is used to perform three-dimensional mapping and digital twin synchronous updates of defects on the three-dimensional BIM model based on the structured defect record set, and to obtain a report on the differences between the fully annotated three-dimensional BIM model and the digital twin version.

[0018] The evaluation and learning module is used to evaluate all detection data, analysis results, and equipment logs of the currently executed task to obtain a detection quality management report. Based on all the data of the currently executed task, it updates the learning evolution model, expands the knowledge graph, adds new case library records, and feeds them back to the task scheduling module, solution planning module, and understanding and analysis module to form a system-level continuous evolution closed loop.

[0019] Specifically, the system also includes:

[0020] The path replanning module is used to generate corresponding incremental device instructions and send them to the corresponding devices based on the real-time event stream generated by the sensing and acquisition module or the discovery of high-risk diseases in the structured disease record set, and to store the path change log in the device execution log.

[0021] Specifically, the task scheduling module includes:

[0022] The task receiving and parsing agent continuously listens for tasks from multiple channels on the online platform frontend via a message queue, performs different standardized transformations on different types of channel tasks, obtains standard task work orders, and writes them into a task queue to be sorted. The standard task work order includes: task number, basic building information, detection type, deadline, and constraints. The basic building information includes: unique building identifier, building address, usage type, building area, and number of floors. The constraints include a time window and a budget limit.

[0023] A context-based intelligent agent is used to retrieve historical building inspection reports corresponding to the standard task work order from the time-series database, construct a time series of defects to predict historical trends of defects and mark key areas of concern, obtain the spatial partitioning and defect density heat map of the 3D BIM model of the standard task work order, load the itemized rule base corresponding to the standard task work order, then integrate the building's basic information, component information, and historical defect records into a building knowledge graph fragment, calculate the context integrity score to obtain a list of missing data markers, and construct a context package.

[0024] Specifically, the scheme planning module includes:

[0025] The priority sorting agent is used to combine the context packet and the task queue to be sorted in the current system to calculate a multi-dimensional weighted score for each task, sort them in descending order according to the multi-dimensional weighted score, and take the task at the top of the current queue as the current task to be executed.

[0026] The formula for calculating the multi-dimensional weighted score P is as follows:

[0027] P = 0.30×Sg + 0.25×Sf + 0.20×St + 0.15×Sc + 0.10×Sb;

[0028] Among them, the Sg urgency score is obtained based on the detection type mapping;

[0029] Sf represents the risk level score, Sf = min(100, historical severity of defects × building age coefficient × structural type risk weight); building age coefficient = min(2.0, 1.0 + floor((current year - construction year) / 10) × 0.1);

[0030] St represents the time constraint score, where St = min(100, max(0, 100 - (deadline - current date) / standard duration × 100)).

[0031] Sc is the resource availability score, Sc = equipment idle rate × 0.4 + personnel availability rate × 0.4 + (1 - traffic distance normalization) × 0.2; traffic distance normalization is to divide the traffic distance of the currently executing task by the maximum traffic distance of all tasks in the current pending task queue;

[0032] Sb is the batch efficiency score, which is obtained by encoding the coordinates of the house address of the task into a geohash string, calculating the longest common prefix length of the geohash strings of the current task and other tasks in the queue, and taking the maximum value and normalizing it to [0,100].

[0033] A dynamic scheme generation agent is used to generate a list of mandatory inspection items based on an itemized rule base, and to determine a list of recommended additional inspection items based on historical disease trend predictions and a list of missing data markers. A directed acyclic graph is established based on the technical dependencies between the inspection items in the mandatory and recommended additional inspection item lists. The optimal execution order of the inspection items is determined through topological sorting, and constraint solving is performed within the time window and budget limit to obtain the estimated resource requirements, thus generating an inspection scheme. The estimated resource requirements include the number of personnel, the number of devices, and the estimated time consumption.

[0034] The guidelines compliance verification intelligent agent is used to verify the compliance of the detection scheme through rules;

[0035] The detection technology selection intelligent agent is used to select detection technologies for the detection scheme based on the contextual constraint data of the currently executed task through the detection technology selection decision tree, and to obtain the recommended detection technologies and alternative schemes for each detection item in the detection scheme; the contextual constraint data includes building height, surrounding space conditions, access conditions, budget limit and time window;

[0036] The dynamic testing scheme includes a list of testing items, recommended testing technologies and alternative solutions for each item, execution order, estimated resource requirements, and a visualization of testing point markings on a 3D model; the list of testing items includes a list of mandatory testing items and a list of recommended additional testing items.

[0037] Specifically, the detection technology selection decision tree selects the appropriate detection technology based on the contextual constraints of the current task, resulting in recommended detection technologies and alternative solutions for each detection item within the detection plan.

[0038] Driven by real-time reading of current task context constraints using a detection technology selection decision tree, recommendations are output based on the following four-level progressive filtering of each detection item in the detection scheme:

[0039] The first layer is disease type filtering: querying the set of technologies that can detect a disease by disease type;

[0040] The second layer is on-site constraint filtering: technologies that do not meet the on-site conditions are excluded sequentially from the technology set;

[0041] The third layer is precision matching filtering: techniques with insufficient precision in the technology set are excluded according to the required measurement precision, resulting in a filtered technology set.

[0042] The fourth layer is cost-benefit ranking: Each technology in the set of technologies that have passed the screening of the first three layers is calculated and ranked. The two technologies with the highest comprehensive technical scores are selected as the optimal solution and the alternative solution, respectively, and the technology with the highest cost score is selected as the economic solution.

[0043] Specifically, the comprehensive technical score is obtained through the following steps:

[0044] The first step is to independently calculate the estimated total time required for the detection technology:

[0045] Estimated total testing time = Equipment deployment time + Equipment data acquisition time;

[0046] Equipment acquisition time = Effective coverage area of ​​the detection item ÷ Equipment area acquisition rate;

[0047] The second step is to calculate the detection area efficiency of the technology based on the estimated total detection time:

[0048] Detection area efficiency = Effective coverage area of ​​the detection item / Estimated total detection time;

[0049] The third step is to calculate the scheduling cost of computing technology.

[0050] Dispatch cost = Number of devices × Device labor cost × Estimated total testing time + Number of personnel × Personnel labor cost × Estimated total testing time + Transportation distance × Transportation mileage cost;

[0051] Step 4: Calculate the Comprehensive Skills Score (CBS).

[0052] CBS = w1 × normalized value of detection area efficiency - w2 × normalized value of estimated total detection time - w3 × normalized value of scheduling cost;

[0053] Where w1 is the detection area efficiency weight; w2 is the expected total detection time weight; and w3 is the scheduling cost weight.

[0054] Then, based on the Comprehensive Technical Score (CBS) in descending order, the technology with the highest CBS is the optimal solution, and the technology with the second highest CBS is the alternative solution; the technology with the highest score of the normalized value of w3×scheduling cost is the economic solution.

[0055] Specifically, the resource scheduling module includes:

[0056] A device instruction generation agent is used to match specific resources with the estimated number of personnel and equipment as upper limits, and to transform the dynamic detection scheme into a set of executable instructions for multiple devices. The set of executable instructions for multiple devices specifically includes:

[0057] Drone flight path instructions: Based on the normal vectors of each detection surface in the 3D BIM model, view planning is performed to generate a sequence of waypoints, thereby obtaining the drone flight path instructions;

[0058] The path instructions for the wall-climbing robot are as follows: Extract the building's outer surface from the 3D BIM model, mark the traversable and inaccessible areas, take key detection points as necessary nodes on the traversable map, solve the optimal traversal path through the traveling salesman problem, set the detection action instructions according to the optimal traversal path, and obtain the path instructions for the wall-climbing robot.

[0059] Laser scanner site instructions: Based on visibility analysis, the set coverage algorithm is used to solve for the minimum number of sites, and generate the location coordinates, scanning range and point density parameters of each scanner site;

[0060] The intelligent agent for dispatching work orders is used to match specific resources with the upper limit constraints of the number of personnel and equipment. It matches the profiles of all available testing personnel with the manual testing requirements in the list of testing items, generates individual work orders, and pushes them to mobile terminals.

[0061] The multiple devices include drones, wall-climbing robots, and laser scanners.

[0062] Specifically, the sensing and acquisition module includes:

[0063] The drone autonomous inspection intelligent agent is used to control the drone to perform inspections according to the drone's flight path instructions. It uses a 4K visible light camera and an infrared thermal imaging camera to simultaneously acquire visible light images and infrared thermal imaging images, and obtain visible light image sequences and infrared thermal imaging image sequences.

[0064] The wall-climbing robot control agent is used to control the wall-climbing robot to move according to the wall-climbing robot path instructions, and to perform close-range macro photography and robotic arm tapping at key detection points to obtain close-range macro photography images and tapping audio detection data, thereby obtaining close-range macro photography image sequence and tapping audio detection data sequence.

[0065] A 3D laser scanning intelligent agent is used to generate stitched 3D point cloud data by scanning station by station according to the instructions of the laser scanner station;

[0066] The multimodal raw data package includes visible light image sequences, infrared thermal imaging image sequences, three-dimensional point cloud data, close-range macro images, and tapping audio detection data sequences.

[0067] Specifically, the understanding and analysis module includes:

[0068] A multi-source data alignment agent is used to register infrared thermal imaging images in an infrared thermal imaging image sequence to the coordinate system of visible light images in a visible light image sequence to obtain registered image pairs. Based on the alignment quality score and image quality score of the registered image pairs, the agent performs image quality scene segmentation. Then, using the spatial coordinate system of the 3D point cloud data as a reference, the agent performs multi-channel fusion of the registered image pairs and the 3D point cloud data to obtain multimodal fused data. The image quality scenes include high-quality dual-modal scenes and low-quality single-modal scenes.

[0069] The disease detection intelligent agent is used to obtain a list of disease detection results by selecting mid-term fusion or late-term fusion based on the image quality and scene of the preprocessed registered image pairs through the disease detection model.

[0070] The domain knowledge-enhanced intelligent agent is used to load spatial and temporal contexts for each disease in the disease detection result list, query the knowledge graph subgraph with the house knowledge graph fragment reference ID as the key, obtain the historical disease association relationship of the same component, execute IF-THEN rule reasoning, and obtain the corrected confidence and reasoning link record.

[0071] A disease geometry quantification intelligent agent is used to extract the geometric attributes of each disease detection result in the disease detection result list; the geometric attributes include length, width, area, depth and their uncertainty;

[0072] An audio percussion hollow drum analysis agent is used to receive percussion audio detection data sequences from multimodal raw data packets, perform signal preprocessing, fast Fourier transform spectrum analysis, hollow drum determination, and spatial mapping to obtain an audio hollow drum detection result set.

[0073] Secondly, the present invention provides a method for intelligent house inspection based on multi-agent collaboration, which, based on the intelligent house inspection system based on multi-agent collaboration described in the first aspect, includes the following steps:

[0074] S1. Standardize multi-channel tasks into standard task work orders and add them to the task queue. Then, construct a complete task context based on the standard task work orders and output the context package.

[0075] S2. Based on the context packet, sort the task queue in descending order according to the multi-dimensional weighted score of the task to obtain the currently executing task at the top of the queue, and then generate a dynamic detection scheme for the currently executing task based on the context packet.

[0076] S3. Convert the dynamic detection scheme into an executable instruction set for multiple devices, match all available detection personnel profiles with the detection items in the detection item list to generate individual work orders and push them to mobile terminals;

[0077] S4. Based on the executable instruction set of the multiple devices, perform multi-device collaborative acquisition to construct multimodal raw data packets, output execution logs of each device, and generate real-time event streams when a device detects an anomaly.

[0078] S5. After data alignment and fusion of the original multimodal data packets, multimodal fused data is obtained. The multimodal fused data is then used to detect diseases through a disease detection model and enhanced with domain knowledge. A structured disease record set is constructed by combining the results of close-range macro photography, audio, and hollow drum analysis.

[0079] S6. Based on the structured disease record set, perform three-dimensional mapping of diseases and synchronous update of digital twin on the three-dimensional BIM model to obtain a report on the difference between the fully annotated three-dimensional BIM model and the digital twin version.

[0080] S7. Evaluate all detection data, analysis results and equipment logs of the currently executed task to obtain a detection quality management report. Based on all data of the currently executed task, update the learning evolution model, expand the knowledge graph, add new case library records and feed them back to the task scheduling module, solution planning module and understanding analysis module to form a system-level continuous evolution closed loop.

[0081] This invention provides a building intelligent inspection system and method based on multi-agent collaboration. It dynamically generates inspection plans by standardizing multi-channel tasks, schedules multiple devices to collaboratively collect data, combines multi-modal fusion and disease detection models to construct structured disease records, and completes disease mapping and twin synchronization on a 3D BIM model. Finally, through an evaluation and learning module, it achieves continuous evolution, realizing full automation and intelligence of the building inspection process. This solves the pain points of traditional building inspection, such as reliance on human experience, low efficiency, inconsistent standards, severe data silos, and difficulty in real-time collaboration and closed-loop optimization. It improves inspection efficiency and accuracy, reduces reliance on manual labor and the risk of misjudgment, and achieves dynamic optimization of inspection plans and intelligent resource matching. It also provides a visualized and traceable 3D display of diseases.

[0082] Furthermore, the intelligent health check system and method for houses based on multi-agent collaboration provided by the present invention also have the following technical advantages:

[0083] a) Systematic multi-layer intelligent agent collaborative architecture: Existing technologies only introduce AI assistance in a single link, and each link still relies on manual connection. This invention constructs a complete multi-layer architecture from task reception to knowledge accumulation. Each layer achieves real-time collaboration through message queues and event-driven mechanisms, realizing an end-to-end intelligent closed loop.

[0084] b) Dynamic personalized detection solutions replace fixed templates: Combining individual building characteristics (structural type, historical defects, environmental impact) and domain standards (itemized rule base), a personalized optimal detection solution is dynamically generated through a three-layer architecture of "mandatory rule layer + risk recommendation layer + efficiency optimization layer".

[0085] c) Real-time discovery-driven scheduling based on detection value: Through real-time interaction between the perception layer and the understanding layer, a real-time closed loop of "discovering high-risk diseases → calculating detection value → second-level dynamic replanning → optimal resource reallocation" is achieved, upgrading "execution according to plan" to "execution according to value".

[0086] d) Built-in continuous learning and evolution capabilities: Data flywheel, incremental learning and knowledge graph construction mechanisms enable the platform to automatically optimize AI models, accumulate domain knowledge and accumulate case experience, and the detection accuracy continues to improve with the number of uses;

[0087] e) Improved efficiency: End-to-end testing time is reduced by about 70% compared to a purely manual process, and the testing space coverage can reach over 98%. Attached Figure Description

[0088] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0089] Figure 1 A schematic diagram of a smart home health check system based on multi-agent collaboration is provided for an embodiment of the present invention;

[0090] Figure 2 This is a schematic diagram of a smart home inspection method based on multi-agent collaboration, provided as an embodiment of the present invention. Detailed Implementation

[0091] The present invention will be explained in detail through the following embodiments. The purpose of this invention is to protect all technical improvements within its scope. In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0092] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0093] Example 1

[0094] refer to Figure 1 This embodiment provides a smart home health check system based on multi-agent collaboration, including the following modules:

[0095] The task scheduling module is used to standardize multi-channel tasks into standard task work orders and add them to the task queue. Then, it constructs a complete task context based on the standard task work orders and outputs the context package.

[0096] The task scheduling module specifically includes: a task receiving and parsing agent and a context building agent;

[0097] The task receiving and parsing intelligent agent continuously listens for multi-channel tasks from the online platform frontend via a message queue, performs different standardized transformations on different types of channel tasks, obtains standard task work orders, and writes them into a task queue to be sorted. The multi-channel tasks include: standardized entrustment orders, work orders automatically dispatched by the municipal housing management system, manual reports, and emergency post-disaster testing instructions from the emergency channel. The standard task work order includes: task number, basic building information, testing type, deadline, and constraints. The basic building information includes: unique building identifier, building address, usage type, building area, and number of floors. The constraints include a time window and a budget limit.

[0098] The time window is a time range window consisting of the current time and the deadline. The value of the time window = the difference between the deadline and the current time - the buffer period. The buffer period is a buffer time period reserved due to weather / equipment unavailability. The unit of the time window value can be hours or minutes, selected according to specific needs.

[0099] The task receiving and parsing agent continuously listens for multiple input channels via a message queue, including standardized order forms (JSON format) from the online platform frontend, work orders automatically dispatched by the municipal housing management system (XML format / API push), manual reports (telephone / email / on-site inspection), and emergency post-disaster inspection instructions. Upon receiving an order, it assigns a globally unique identifier (UUID) as the task ID for full lifecycle tracking. Different standardization transformations are performed based on the input source type.

[0100] For standardized JSON-formatted authorization forms, field parsing and validation are performed directly based on a predefined JSON Schema; the deadline is a hard constraint filled in by the client when submitting the authorization form.

[0101] For work orders in XML format, XSLT mapping rules are used to convert them into a unified internal JSON representation;

[0102] For unstructured manual reports, we call the Large Language Model (LLM) to perform Named Entity Recognition (NER) to extract key entities such as unique property identifiers (e.g., property ownership certificate numbers), property addresses, contact persons, and detection requirements (detection types).

[0103] After standardization, extract key fields: unique building identifier, building address, usage type (residential / commercial / industrial / public building), building area, number of floors, type of inspection (routine inspection / specialized inspection / emergency assessment / re-inspection), and constraints (time window (submission date, deadline), budget limit, special requirements).

[0104] The system maintains a task queue. Each time a task receiving and parsing agent generates a standard task order, it adds it to the task queue.

[0105] The context-based intelligent agent is used to retrieve the historical health check reports of the houses corresponding to the standard task work order from the time-series database, construct the time series of defects to predict the historical trend of defects and mark key areas of concern, obtain the spatial partitioning and defect density heat map of the 3D BIM model of the standard task work order, load the itemized rule base corresponding to the standard task work order, then integrate the basic information of the house, component information, and historical defect records into a house knowledge graph fragment, calculate the context integrity score to obtain the missing data label list, and construct the context package;

[0106] The context includes housing knowledge graph fragment reference ID, historical disease trends, historical disease severity, three-dimensional spatial partition index, typical disease prediction, itemized rule base, context integrity score and missing data marker list;

[0107] (a) Basic building information:

[0108] Using a unique identifier for the building, retrieve basic building information such as building structure type, construction year, design drawings, and historical renovation records from a relational database, and attach it to the basic building information;

[0109] The final basic information of the house includes: unique house identifier, house address, type of use, building structure type, building area, number of floors, year of construction, and design drawings, totaling 8 fields;

[0110] The standard task work order contains only the unique identifier of the house as the index key. The context-based intelligent agent uses this key to query the archive relationship database and further obtains detailed archive fields such as building structure type (frame / brick-concrete / steel structure / wood structure), construction year, design drawing path, and historical renovation records.

[0111] (b) Historical medical examination records and prediction of historical disease trends:

[0112] Based on the range, retrieve the historical health check records (which may include multiple records) of the house from the InfluxDB time series database, extract historical disease records from the historical health check records, including the detection date, location, type, and severity of the disease in the historical disease records, and construct a disease time series dataset by grouping by disease location ID.

[0113] The specific method for constructing the disease time series includes: using the historical disease severity (0-100 points) of each disease type in each region as the vertical axis and the number of days from the first detection date as the horizontal axis to form a time series;

[0114] The rules for quantifying disease severity are detailed below. Each sequence requires at least two time points for trend fitting; sequences with only one time point are marked as "insufficient historical data," and it is recommended to increase the detection frequency.

[0115] For each disease time series, linear fitting model, exponential fitting model, and S-curve fitting model were used for fitting. The goodness of fit R² of each fitting model was calculated. The fitting model with the highest goodness of fit R² was selected and extrapolated to the next expected detection time j for prediction. The predicted value is the predicted severity. Then, based on the growth slope of the predicted severity, the disease trend is divided into 4 levels:

[0116] If the growth slope is less than 0.5 points per year, the historical trend of the disease is set to stable, indicating that the disease has no significant development and the current monitoring frequency is maintained.

[0117] If 0.5 ≤ growth slope < 2 points / year, the disease historical trend is set to slow deterioration, indicating that the disease is developing slowly and the normal re-inspection cycle should be maintained.

[0118] If 2 ≤ growth slope < 5 points / year, the disease historical trend is set to continuous deterioration, indicating that the disease is continuously aggravating. This area is marked as a key area of ​​concern, and it is recommended to shorten the re-inspection cycle.

[0119] If the growth slope is ≥ 5 points / year, the disease historical trend is set to accelerated deterioration, indicating that the disease is developing rapidly. This area is marked as an area of ​​urgent concern and triggers an increase in task priority.

[0120] Diseases with a historical trend of continuous or accelerated deterioration, or diseases with a severity score ≥60 points in subsequent testing, will be marked as key areas of concern if either of these conditions is met. This marking will subsequently drive an increase in testing density (number of testing points ≥ standard density × 1.5).

[0121] (c) Three-dimensional model, spatial zoning, and heat map of disease density:

[0122] The component information is obtained by extracting semantic information of building components from the lightweight 3D BIM model already established on the platform. The component information includes component type and component location (GeoHash encoding). Based on the component information, the model surface is divided into standard detection zones (default 5m×5m grid), and a GeoHash spatial index (precision level 8) is generated for each standard detection zone, which is a 3D spatial partition index. The lightweight 3D BIM model is pre-built based on the design drawings.

[0123] Disease density heatmap calculation method: Based on the spatial coordinates of diseases from each examination record in historical health check reports, the cumulative occurrence of diseases within each 5m×5m partition is counted, and a continuous heatmap is generated using kernel density estimation (KDE, an existing image processing method).

[0124] Centered on each defect point within the standard testing zone, using a Gaussian kernel function with a bandwidth h=10m and the heatmap value range normalized to [0,1], the heatmap values ​​of each standard testing zone are divided into 4 levels:

[0125] A value of 0 ≤ heatmap value ≤ 0.2 indicates a low-density zone.

[0126] A value of 0.2 < heat map value ≤ 0.5 indicates a medium-to-low density zone.

[0127] A value of 0.5 < heat map value ≤ 0.8 indicates a medium-to-high density zone.

[0128] A value of 0.8 < heat map value ≤ 1.0 indicates a high-density zone.

[0129] High-density zones are prioritized as encrypted detection areas during subsequent scheme generation, and the detection density is set to 1.5 times the standard density during scheme planning. The KDE method is existing technology; the innovation of this application lies in introducing it into the spatial analysis scenario of building health checks and integrating it with the task context construction process.

[0130] (d) Environmental factor assessment:

[0131] By querying external data sources to obtain the geological conditions of the building (soft soil foundation / karst area), climate characteristics (coastal salt spray / freeze-thaw cycle frequency), and the impact of surrounding construction (subway construction / deep foundation pit project and its distance from the building), and combining them with the existing three-dimensional mapping table of "construction year × structural type × geological conditions", the expected typical defects are assessed.

[0132] (e) Standard matching and rule engine loading:

[0133] Based on the building structure type, usage type, and inspection type of the building, the rule engine is used to match the applicable inspection guidelines from the standard library (taking the "Ningbo City Building Inspection Guidelines" as an example) and load its itemized rule set.

[0134] A rule engine is an existing software technology that separates business rules from program logic. This application uses the Drools rule engine (open source, provided by Red Hat). Its working principle is as follows: the specification guidelines text is pre-parsed offline into structured rules in IF-THEN format and stored in the rule repository; the rule engine receives house attributes as facts, derives applicable rules through Rete algorithm pattern matching, performs inference, and outputs a list of mandatory items. The logical relationship is: rule layer → contains rule engine → the rule engine contains a rule repository (itemized guide rule set).

[0135] The rule base is decoupled from the engine, allowing rules to be updated independently without affecting the program code. The innovation of this application lies in the construction of an industry rule base for the building inspection field.

[0136] (f) Multi-source data fusion into housing knowledge graph fragments:

[0137] This application uses RDF triples (subject-predicate-object) to fuse multi-source data such as basic building information, component information, and defect information into a unified knowledge representation, namely a building knowledge graph fragment. Each building knowledge graph fragment is assigned a reference ID, referred to as the building knowledge graph fragment reference ID; specifically, it includes:

[0138] The context-building agent writes the basic information, component information, and historical damage records of the house involved in this task into the Neo4j graph database in the form of RDF triples. This generates a set of knowledge subgraph nodes corresponding to the house in this task. Neo4j automatically returns the root node UUID of this subgraph as a reference ID. This ID is an output field automatically generated and written into the context package by the context-building step; it does not require external input. The subsequent domain knowledge enhancement agent uses this ID to perform subgraph queries.

[0139] The steps for multi-source data fusion are as follows:

[0140] Entity extraction: Directly map entity nodes from the above structured data, including building nodes (building address, construction year, structural type), component nodes (component type, location coordinates), and defect nodes (defect type, defect severity, timestamp).

[0141] Relationship construction: Construct triples based on spatial hierarchy (disease → located in → component, component → belongs to → house) and temporal relationship (disease → historical record → detection event);

[0142] Multi-source alignment: Using "unique building identifier + component location GeoHash code" as the composite key, entity alignment is performed across data sources (archive library, time series library, 3D BIM model) to solve the problem of heterogeneous representation of the same component in different data sources;

[0143] Input: Batch write the aligned triples into the Neo4j graph database to generate a knowledge graph fragment of the house;

[0144] It is understandable that the context building agent contains a global knowledge graph, which is a collection of knowledge graph fragments corresponding to all standard task work orders and houses. It is a knowledge base continuously accumulated by the platform. The knowledge graph fragment is a set of local RDF triples generated by the context building agent for a single house in a single task. It is a subgraph in the global knowledge graph related to the current house and can be retrieved from the global knowledge graph by referencing the house knowledge graph fragment ID.

[0145] The housing knowledge graph fragments are subsets of the global knowledge graph (read direction). After the task is completed, newly added housing knowledge graph fragments are reviewed and merged into the global knowledge graph (write direction).

[0146] Knowledge graph technology and graph database (Neo4j) are both existing technologies.

[0147] (g) Calculation of contextual completeness score:

[0148] Contextual completeness score = 20% × basic file acquisition rate + 30% × historical record acquisition rate + 20% × 3D model acquisition rate + 15% × environmental factor acquisition rate + 15% × standard and specification acquisition rate;

[0149] The context completeness score is scored between 0 and 100.

[0150] Data retrieval rate for each dimension = Number of fields retrieved for that dimension / Total number of fields that should exist for that dimension. The definitions and weights of each dimension are shown in Table 1.

[0151] Table 1. Contextual Completeness Dimension and Weight Table

[0152] Basic Files 20% 8 Number of fields retrieved / 8; (e.g., retrieving 5 fields → retrieval rate is 62.5%) Historical medical examination records 30% 4 If the house has at least two historical health check records with different recording times, a disease time series can be constructed for trend analysis. The historical data completeness is the highest, with an acquisition rate of 100%. If the house has only one historical record, historical disease references can be provided, but trend analysis cannot be performed. The data is partially usable, with an acquisition rate of 50%. If the house has no historical record, it means it is the first health check, and there is no historical data reference. The acquisition rate is 0%. 3D model 20% 3 Number of fields retrieved / 3; (BIM available, spatial zoning available, and heatmap available each occupy one field) Environmental factors 15% 3 Number of fields retrieved / 3; (Geological conditions, climate characteristics, and the impact of surrounding construction each occupy one field) Standards 15% 2 Number of fields retrieved / 2; (one field is available for guide matching, and one field is available for the rule base)

[0153] Context completeness score is used to trigger policy threshold judgment:

[0154] If the context integrity score is below 60, list the missing data tags for the missing fields and generate data supplementation suggestions based on the recommended acquisition channels for the missing fields;

[0155] The subsequent planning module will include the component types and inspection types corresponding to the missing data marker list in the list of suggested additional inspection items to avoid missed inspections due to insufficient information;

[0156] Recommended data acquisition channels are static mapping relationships pre-configured for each type of data during system initialization and stored in the platform's data source configuration library. When the missing data marker list is generated, the system automatically retrieves the corresponding recommended acquisition channels from the data source configuration library based on the missing field type and fills them into the output list. Recommended acquisition channels are read from the platform's pre-configured data source library, and data supplementation suggestions are automatically obtained based on the missing field type; for example: historical physical examination records → query the archival relationship database; historical infrared data, it is recommended to query the InfluxDB time series database; BIM models, it is recommended to query the platform's BIM model management module; environmental factors → call the PostGIS spatial database interface.

[0157] The scheme planning module is used to sort the task queue in descending order according to the multi-dimensional weighted score of the tasks based on the context packet to obtain the currently executing task at the top, and then generate a dynamic detection scheme for the currently executing task based on the context packet; the dynamic detection scheme includes a list of detection items, recommended detection technologies and alternative schemes for each item, execution order, estimated resource requirements, and a visualization of the detection point annotations on the 3D model; the list of detection items includes a list of mandatory detection items and a list of suggested additional detection items;

[0158] The scheme planning module specifically includes:

[0159] (1) Priority sorting agent, which combines the context packet and the task queue to be sorted in the current system to calculate a multi-dimensional weighted score for each task, sorts them in descending order according to the multi-dimensional weighted score, and takes the task at the top of the current queue as the current task to be executed.

[0160] The unordered task queue is an ordered set of all standard task orders generated but not yet executed in the system, implemented using a Redis Sorted Set. The task receiving and parsing agent writes each generated standard task order into this queue; the priority sorting agent recalculates the multi-dimensional weighted score of all orders in the queue and rearranges them in descending order, taking the first order as the currently executing task, which is then removed from the task queue. A standard task order is a data structure for a single task, while the task queue is a dynamically ordered container storing multiple task orders.

[0161] The formula for calculating the multi-dimensional weighted score P is as follows:

[0162] P = 0.30×Sg + 0.25×Sf + 0.20×St + 0.15×Sc + 0.10×Sb;

[0163] Wherein, Sg is the urgency score, which is obtained based on the detection type mapping;

[0164] The urgency score Sg is based on a fixed priority mapping of the test type: emergency assessment = 100 points, identification and re-examination = 75 points, special test = 50 points, routine physical examination = 25 points.

[0165] Sf is the risk level score, Sf = min(100, historical severity of defects × building age coefficient × structural type risk weight);

[0166] Building age coefficient = min(2.0, 1.0 + floor((current year - year of construction) / 10) × 0.1);

[0167] Example: A house built in 1985 will be 40 years old in 2025. House age coefficient = min(2.0, 1.0 +4×0.1) = 1.4.

[0168] Structural type risk weights: brick-concrete structure 1.2, wood structure 1.3, frame structure 1.0, steel structure 0.9.

[0169] St represents the time constraint score, where St = min(100, max(0, 100 - (deadline - current date) / standard duration × 100)).

[0170] The deadline is derived from the constraint field of the standard task work order. Users can explicitly fill in the deadline when submitting the work order; if not filled in, the system sets a default value according to the testing type: Emergency Assessment = 3 days after submission date, Identification Re-inspection = 30 days after submission date, Specialized Testing = 60 days after submission date, Routine Physical Examination = 90 days after submission date. The submission date is extracted from the timestamp at the time of submission (current date).

[0171] Standard project duration source: From the platform's historical task statistics database, using "same structure type + same inspection type + same area range" as the query criteria, the median completion time (P50) of historical tasks is calculated as the standard project duration estimate (unit: days); if no historical data is available, a preset baseline value is used (3 days for routine physical examination, 5 days for special inspection, 7 days for appraisal re-inspection, and 1 day for emergency assessment).

[0172] If outdoor testing is conducted and the weather forecast indicates rainfall during the testing window, an additional +20 points will be added to the above calculation results. After the addition, the min(100, result) truncation method will be executed to ensure that the final value of the time constraint score St does not exceed 100 points.

[0173] Sc is the resource availability score, Sc = equipment idle rate × 0.4 + personnel availability rate × 0.4 + (1 - traffic distance normalization) × 0.2;

[0174] Among them, the ratio of the number of currently available devices in the estimated resource requirements to the number of devices required by the plan affects the queuing priority of the task;

[0175] Equipment idle rate = min(currently available equipment number / number of equipment required by the plan, 1.0). If the current number of available equipment is greater than or equal to the number of equipment required by the plan, the equipment idle rate is 1.0; if it is less than 1.0, the idle rate will directly lower the Sc score and thus reduce the queuing priority of the task.

[0176] Personnel availability = min(currently available testing personnel / number of personnel required by the plan, 1.0),

[0177] The number of equipment and personnel required for the proposed solution were approximated using historical experience estimates (the average value of similar tasks retrieved from the historical task database based on building area and detection type).

[0178] During the sequencing phase, the number of equipment and personnel required for the proposed solutions are estimated using historical experience and then replaced with precise values ​​from the constraint solution after the task is confirmed.

[0179] The system queries the archive relationship database using the unique identifier of the property (such as the real estate ownership certificate number) and automatically links historical medical examination records; it also prompts for deduplication and merging of duplicate entrustments for the same property within a short period of time, and marks and merges scheduling opportunities for batch entrustments in the same community.

[0180] Traffic distance refers to the road travel distance between the testing agency's dispatch base (fixed coordinates) and the house address in the standardized work order. After converting the house address into coordinates through geocoding, the actual road distance (km) is obtained by calling the map route planning API (such as the Gaode Map route planning interface).

[0181] Traffic distance normalization is achieved by dividing the traffic distance of the currently executing task by the maximum traffic distance of all tasks in the current queue of pending tasks (Max normalization).

[0182] Traffic distance normalization = Traffic distance of the currently executing task / Maximum traffic distance of all tasks in the task queue; when there is only 1 task in the queue, the normalization value = 0 (considered as the closest).

[0183] Sb is the batch efficiency score, which is obtained according to the following steps:

[0184] Step ①: Encode the latitude and longitude coordinates of the house address in the task into a GeoHash string with a precision level of 9 (9 characters, precision approximately ±2.4m);

[0185] Step 2: For each pair of tasks in the queue (current task i and other tasks j), calculate the length of the longest common prefix L(i,j) of their GeoHash strings, with a value range of [0,9] (0 represents extremely far apart, 9 represents extremely close apart);

[0186] Step ③: Sb(i) = max(L(i,j)) / 9×100, normalize the matching length with the nearest neighbor task to [0,100]. Example: If the first 4 bits of the GeoHash of the two tasks are the same (L=4), then Sb(i) = 4 / 9×100 ≈ 44 points.

[0187] GeoHash is a spatial indexing technique that encodes latitude and longitude coordinates into strings. The longer the common prefix of the strings, the closer the two points are.

[0188] The system sorts the currently executing tasks in descending order based on a multi-dimensional weighted score P, and supports dynamic reordering (new task addition / task status change / equipment failure / weather warning). When an emergency task arrives, the system assesses the interruptibility of the currently executing task. If it can be interrupted, the system pauses the task and inserts the emergency task; otherwise, it allocates the reserved resources.

[0189] (2) Dynamic scheme generation agent, which is used to generate a list of mandatory inspection items based on the itemized rule base, and to determine a list of recommended additional inspection items based on the prediction of disease historical trends and the list of missing data. It establishes a directed acyclic graph based on the technical dependencies between the inspection items in the list of mandatory inspection items and the list of recommended additional inspection items, and generates an inspection scheme by determining the optimal execution order of the inspection items through topological sorting.

[0190] The dynamic scheme generates an intelligent agent comprising the following three layers:

[0191] Rule Layer (Mandatory Compliance): Loads an itemized rule base (e.g., the "Ningbo Housing Inspection Guidelines") (stored in the Drools rule engine). Each rule is in the form of: IF Building Structure Type = X AND Functional Use Type = Y AND Service Life Greater Than or Equal to Z AND Inspection Type = W THEN Must Execute Inspection Item Set {A, B, C, ...}; Substitutes the current housing attributes into the rule engine to automatically generate a list of mandatory inspection items, ensuring 100% coverage of the guidelines.

[0192] Risk Layer (Intelligent Recommendation): Risk Layer: Based on historical trend predictions of disease, key areas of concern are marked, and it is recommended to increase the detection density to 1.5 times the standard density; based on the expected typical diseases, and referring to the disease statistics of similar houses in the same area (queried from the knowledge graph), suggested additional inspection items are generated for disease types that are frequently occurring but not covered by mandatory inspection items; and the component types and inspection types corresponding to the missing data in the context package are included in the suggested additional inspection items; a list of suggested additional inspection items is generated.

[0193] Optimization layer (efficiency maximization): A directed acyclic graph of dependencies is established for all inspection items (in the list of mandatory inspection items and the list of suggested additional inspection items). The optimal execution order is determined through topological sorting. Cross-project equipment sharing analysis and spatial path shortestization optimization are performed. Constraint solving is performed within the time window and budget limit to obtain the estimated resource requirements and generate an inspection plan. The estimated resource requirements include the number of personnel, the number of equipment, and the estimated time.

[0194] The budget limit is filled in by the client when submitting the medical examination request form and is a sub-item of the constraint field. If the client leaves it blank, the system will automatically generate a reference value based on the house area, house structure type, and examination type by calling the platform's preset cost estimation model. The budget limit is written into the context package along with the standard task work order and is used as the right-hand value of the constraint "cumulative estimated cost does not exceed the budget limit" in the dynamic solution generation intelligent agent constraint solving step.

[0195] It is worth noting that when the dynamic scheme generation agent generates the inspection item list, each inspection item is bound to the corresponding BIM component ID set through the rule engine; each BIM component corresponds to one or more standard inspection partitions (5m×5m grid), and the mapping relationship between standard inspection partitions and components has been established and stored in the partition data structure during the context construction step; the surface area of ​​each standard inspection partition can be read from the BIM model geometry layer (the BIM model pre-calculates the surface area of ​​each component and stores it in 5m×5m grid partitions); the sum of the areas of all bound standard inspection partitions for each inspection item is used as...

[0196] Effective coverage area of ​​the test items.

[0197] The standard inspection zone area is calculated from the geometric data of the BIM model. The area of ​​the edge zone may be less than 25m², while the area of ​​the other standard inspection zones is 25m².

[0198] The intermediate calculation results corresponding to the optimization layer constraints in the process of solving the estimated resource demand are specifically generated after the 0-1 integer programming / greedy approximation solution is completed, and include the following three sub-fields:

[0199] ① Number of personnel: Take the union of the "human resource requirement type" of all testing items in the final plan (determined by the testing technology selection agent, with the required number of operators and qualifications for each technology noted in the specification library), and calculate the peak number of personnel required for each category, which is the minimum required personnel configuration;

[0200] ② Number of devices: Take the union of the "device requirement type" of all testing items in the final solution, and consider the overlap of time windows after topological sorting (the devices required for testing items executed in parallel within the same time window cannot be shared), and calculate the maximum concurrent requirement of each type of device.

[0201] ③ Estimated time = Sum of estimated total time for each step of the critical path after topology sorting ÷ Daily effective working hours, rounded up to the nearest day. This value is the source of the "Estimated Time" parameter when dispatching work orders; the critical path after topology sorting is the longest dependency chain from the starting detection item to the ending detection item; the daily effective working hours are 8 by default.

[0202] Topological sorting (using the Kahn algorithm) is a classic algorithm for DAGs in graph theory (existing technology). The innovation of this application lies in: modeling the technical dependencies between various detection items (such as "3D laser scanning should be performed before the UAV flight path fine planning, because the scanned point cloud can assist in the optimization of flight path altitude parameters") as DAG edges, and using the Kahn algorithm to output the optimal linear execution sequence that satisfies all dependency constraints, minimizing the device idle waiting time;

[0203] Constraint Solving: Under the constraints of the time window and budget limit, with "maximizing detection coverage" as the objective function, a 0-1 integer programming approach is used to solve for all combinations of executing or not executing detection items (exact solution when the number of items is ≤20; for >20, a greedy approximation is used, sorting items in descending order of "detection value / estimated time consumption", adding items one by one until the constraint is close to saturation). The constraint satisfaction criterion is: cumulative estimated working hours ≤ available time window value AND cumulative estimated cost ≤ budget limit; if both are satisfied, the constraint is satisfied. If the constraint cannot be satisfied, automatic degradation occurs: "Suggested additional detection items" are deleted first, all "mandatory detection items" are retained, a degradation plan is generated, and the impact on coverage is marked.

[0204] (3) The guideline compliance verification intelligent agent is used to verify the compliance of the detection scheme through rules;

[0205] The generated schemes are validated one by one using a rule engine, and a two-dimensional coverage matrix of "guideline item × scheme item" is established to verify the following key indicators:

[0206] • Equipment accuracy verification: The measurement error index of the equipment used in the scheme shall be less than or equal to the maximum permissible error specified in the guidelines;

[0207] • Inspection coverage rate: The percentage of building area covered by the plan / area to be inspected is greater than or equal to the percentage required by the guidelines;

[0208] • When multiple standards apply simultaneously, the stricter one shall be applied. Non-compliant items shall be marked with the specific clauses violated and the gaps, and rectification suggestions shall be attached.

[0209] (4) The detection technology selection agent is used to select detection technologies based on the context constraint data of the current task through the detection technology selection decision tree, and obtain the recommended detection technologies and alternative schemes for each detection item in the detection scheme; among them, the context constraint data refers to the subset extracted from the context package, specifically including: ① building height (affecting the flight permit of UAVs and the applicable floor range of wall-climbing robots); ② surrounding space conditions (building spacing, whether it is in a no-fly zone); ③ access conditions (road traffic control, site clearance); ④ budget limit (affecting the cost dimension score, high-cost technologies will have a lower ranking when the budget is tight); ⑤ time window (affecting the efficiency dimension weight, technologies with a large coverage area per unit time in a short construction period are given priority).

[0210] The building height, surrounding spatial conditions, and access conditions are derived from the environmental factors field in the context package. This field is obtained and extracted by the context-building agent through a query of the PostGIS spatial database. The budget limit and time window are extracted from the constraint conditions field.

[0211] Driven by real-time reading of current task context constraints using a detection technology selection decision tree, recommendations are output based on the following four-level progressive filtering of each detection item in the detection scheme:

[0212] The first layer is disease type filtering: querying the set of technologies that can detect a disease by disease type;

[0213] For example, facade cracks → {drone visible light aerial photography, wall-climbing robot close-up imaging, manual suspended platform close-up measurement};

[0214] The second layer is on-site constraint filtering: technologies that do not meet the on-site conditions are excluded sequentially from the technology set;

[0215] For example, no-fly zones exclude drones; irregular building exteriors exclude wall-climbing robots; and unlicensed high-altitude work permits exclude manual suspended platforms.

[0216] The third layer is precision matching and filtering: technologies with insufficient precision are excluded from the technology set according to the required measurement precision, resulting in a set of screened technologies; for example, if the crack width precision requirement is ±0.1mm, equipment with insufficient precision is excluded.

[0217] The fourth layer is cost-benefit ranking: Each technology in the set of technologies that have passed the screening of the first three layers is evaluated and ranked by calculating the comprehensive technical score. The two technologies with the highest comprehensive technical scores are selected as the optimal solution and the alternative solution, respectively, and the technology with the highest cost score is selected as the economic solution.

[0218] The comprehensive technical score is obtained through the following steps:

[0219] The first step is to independently calculate the estimated total time required for the detection technology:

[0220] Estimated total testing time = Equipment deployment time + Equipment data acquisition time;

[0221] Equipment acquisition time = Effective coverage area of ​​the detection item ÷ Equipment area acquisition rate;

[0222] Each detection technology corresponds to a specific device. The time parameters of each device are retrieved from the platform's device specification library (standard operating times stored by device model). The device deployment time is a fixed value, read from the device specification library. The effective coverage area of ​​the detection item and the device area acquisition rate are both known quantities.

[0223] The second step is to calculate the detection area efficiency of the technology based on the estimated total detection time:

[0224] Detection area efficiency = Effective coverage area of ​​the detection item / Estimated total detection time, unit: m² / hour;

[0225] The third step is to calculate the scheduling cost of computing technology.

[0226] Dispatch cost = Number of devices × Device labor cost × Estimated total testing time + Number of personnel × Personnel labor cost × Estimated total testing time + Transportation distance × Transportation mileage cost;

[0227] The equipment cost is calculated as: number of devices × equipment labor cost × estimated total testing time, with the equipment labor cost in yuan / hour; the labor cost is calculated as: number of personnel × personnel labor cost × estimated total testing time, with the personnel labor cost in yuan / hour; the travel cost is calculated as: travel distance × travel mileage, in yuan / kilometer.

[0228] Each rate parameter (equipment labor cost, personnel labor cost, transportation mileage cost) can be queried from the platform's cost parameter library (unit price stored according to equipment model / personnel qualification level);

[0229] Step 4: Calculate the final cost-benefit score (CBS) based on technical performance.

[0230] CBS = w1 × (normalized value of detection area efficiency) - w2 × (normalized value of estimated total detection time) - w3 × (normalized value of scheduling cost);

[0231] Where w1 is the detection area efficiency weight, with a value of 0.5; w2 is the expected total detection time weight, with a value of 0.3; and w3 is the scheduling cost weight, with a value of 0.2 (the weights are determined through regression analysis of historical project data). The normalized values ​​of each dimension are calculated using Min-Max normalization to [0,1]. The Top 2 solutions are output in descending order of score: the technology with the highest score is the optimal solution, and the technology with the second highest score is the alternative solution.

[0232] The economic solution is evaluated independently from the cost dimension and is unrelated to the overall ranking. The economic solution is selected independently from all candidate solutions based on the highest cost dimension score (w3 × normalized value of scheduling cost), regardless of its overall ranking; the three scores may overlap or be different, and the optimal solution is used as the recommended testing technology for this testing project.

[0233] For example, in the crack detection of the north facade of a building, after the first three layers of filtering, two candidate technologies remain: drone visible light aerial photography and close-range imaging by a wall-climbing robot.

[0234] Drones: Effective coverage area of ​​600m², estimated total detection time of 1.5 hours (30 minutes for deployment + 60 minutes for data collection), scheduling cost of 1800 yuan;

[0235] Wall-climbing robot: Effective coverage area of ​​200m², estimated total detection time of 3 hours (60min deployment + 120min data collection), scheduling cost of 900 yuan.

[0236] After normalization, for UAVs: the normalized value of detection area efficiency = 1.0 (highest), the normalized value of estimated total detection time = 0 (lowest time), and the normalized value of scheduling cost = 1.0 (most expensive); the overall technical score = 0.5×1.0 - 0.3×0 - 0.2×1.0 = 0.3;

[0237] Wall-climbing robot: Normalized value of detection area efficiency = 0.33, normalized value of estimated total detection time = 1.0 (most time-consuming), normalized value of scheduling cost = 0 (cheapest); Overall technical score = 0.5×0.33-0.3×1.0-0.2×0=-0.133.

[0238] Therefore, drones are the "optimal solution," while wall-climbing robots are the "economical solution" (with a higher cost score).

[0239] The normalization method used is Min-Max normalization.

[0240] Based on the above, a dynamic detection scheme is finally obtained. The dynamic detection scheme includes a list of detection items, recommended detection technologies and alternative schemes for each detection item (output by the decision tree of the detection technology selection agent), execution order of detection items (generated by topological sorting of the directed acyclic graph of dependencies), estimated resource requirements (output by constraint solving steps), and a visualization diagram of the three-dimensional model detection point annotations. The list of detection items includes a list of mandatory detection items (forcedly extracted from the guidelines by the rule engine) and a list of suggested additional detection items (generated based on the historical trend of diseases and the expected inference of typical diseases).

[0241] The visualization of the 3D model inspection point annotations is generated by mapping the GeoHash coordinates of each inspection item to the surface of the BIM model and then rendering it.

[0242] Each detection item is bound to a corresponding standard detection partition during scheme planning; each standard detection partition has a pre-generated GeoHash spatial index (precision level 8, encoding the latitude and longitude coordinates of the partition center point) generated during the context construction step; therefore, the GeoHash coordinates of a detection item are the GeoHash encoding of the center point of the partition to which that detection item is bound. If a detection item spans multiple partitions, multiple GeoHash coordinate points are marked in the 3D model visualization. The GeoHash coordinates of the 3D model detection point annotation visualization come from the GeoHash index of the center point of the standard detection partition bound to each detection item.

[0243] The resource scheduling module is used to convert the dynamic detection scheme into an executable instruction set for multiple devices, match all available detection personnel profiles with the detection items in the detection item list to generate individual work orders and push them to mobile terminals; the multiple devices include drones, wall-climbing robots and laser scanners.

[0244] The resource scheduling module includes:

[0245] (1) An intelligent agent for generating equipment instructions, used to match specific resources with the estimated number of personnel and equipment required in the dynamic detection scheme as an upper limit constraint, and to convert the dynamic detection scheme into an executable instruction set for multiple devices (drones, wall-climbing robots, and laser scanners), specifically including:

[0246] (a) UAV flight path instructions: Based on the normal vectors of each detection surface of the 3D BIM model, view planning is performed to generate a sequence of waypoints (latitude and longitude, flight altitude, heading angle), thereby obtaining the UAV flight path instructions;

[0247] Flight altitude was determined through Pareto multi-objective optimization (a trade-off between GSD resolution, coverage efficiency, and safe distance), with adjacent image overlap ≥70%.

[0248] (b) Path instructions for the wall-climbing robot: Extract the building's outer surface from the 3D BIM model, mark the traversable and inaccessible areas, take the key detection points as the necessary nodes on the traversable map, solve the optimal traversal path through the Traveling Salesman Problem (TSP), set the detection action instructions according to the optimal traversal path, and obtain the path instructions for the wall-climbing robot.

[0249] The key detection points are derived from: ① The scheme planning module divides the surface to be inspected into a 5m×5m grid, with one detection point set at the center of each partition; the high-density partitions (value greater than 0.8) of the disease density heatmap are densified into a 2.5m×2.5m grid, with 4 detection points generated per partition; the detection points are selected and assigned to the wall-climbing robot according to the technology selection results; ② The densified detection points added in real time due to the discovery of high-risk diseases during the dynamic path replanning stage are added to the set in real time.

[0250] The surfaces to be inspected are the building exterior surfaces in the 3D BIM model that need to be included in the inspection coverage calculation; specifically, they refer to the exterior wall facades (including the outer side of balcony railings), roof surfaces (including the top surface of parapet walls), and exposed structural beam and column surfaces (the parts in direct contact with outdoor air), excluding interior surfaces, basement exterior walls (the parts below ground level), and enclosed surfaces of interior courtyards. The surfaces are obtained by filtering the component type attributes of the BIM model (selecting the ExteriorWall / Roof / ExposedColumn / ExposedBeam component surface sets), and this set is discretized into a 1m×1m grid.

[0251] (c) Laser scanner site instructions: Based on visibility analysis, the minimum number of sites is determined by the set coverage algorithm (at least 2 sites cover each detection surface, and adjacent sites overlap by more than 30%), and the location coordinates, scanning range and point density parameters of each scanner site are generated.

[0252] Finally, the executable instruction sets (drone flight path instructions, wall-climbing robot path instructions, and laser scanner site instructions) are verified one by one through the security rule base;

[0253] The aforementioned safety rule base is a static rule set pre-set during the system initialization phase. It is not dynamically generated by the method of this invention and is a prerequisite for operation. The rule content comes from: ① UAV flight safety regulations (maximum flight altitude, no-fly zone range); ② Construction inspection industry operation safety specifications (personnel safe distance not less than 5 meters); ③ Equipment manufacturer specifications (80% of battery life is the maximum for a single mission); ④ High-altitude operation safety specifications (adhesion force threshold of wall-climbing robots, etc.).

[0254] The executable instruction set is in JSON format or a device-specific protocol format;

[0255] (2) The intelligent agent for dispatching work orders is used to match specific resources with the upper limit constraints of the number of personnel and equipment, match all available inspection personnel profiles with the manual inspection item requirements in the inspection item list, generate personal work orders and push them to mobile terminals;

[0256] The intelligent agent for dispatching work orders obtains profiles of all available testing personnel (set of qualification certificates, skill level, current GPS location, and working hours for the day) from the personnel management module. It constructs a bipartite graph by combining the manual testing requirements in the testing item list with the personnel profiles. Based on the formula constraints, it solves the global optimal matching using the Hungarian algorithm, generates individual work orders, and pushes them to the mobile terminal App.

[0257] The personnel management module is a fundamental supporting component of this system. It maintains the database of testing personnel files, which includes: a collection of qualification certificates (certificate items and validity periods), skill level scores, the personnel's current GPS location (reported in real time by the personnel's handheld terminal, once every 30 seconds), and the accumulated working hours for the day (automatically timed).

[0258] Before generating employee work orders, the resource scheduling module calls the personnel management module via an internal API to query all available personnel profiles, which serve as the set of left-hand nodes for bipartite graph matching.

[0259] Working hours constraints: A mandatory rest is required if a single person works continuously for less than 4 hours. High-risk / high-altitude operations must be carried out by two people. The total daily working hours must be less than 10 hours.

[0260] The sensing and acquisition module is used to construct multimodal raw data packets by collaboratively acquiring data from multiple devices according to the executable instruction set of the multiple devices, output the execution logs of each device, and generate real-time event streams when a device detects an anomaly.

[0261] The sensing and acquisition module specifically includes:

[0262] (1) The UAV autonomous inspection intelligent agent is used to control the UAV to perform inspection according to the UAV flight route instructions. Through the 4K visible light camera and infrared thermal imaging camera on board, it performs synchronous acquisition of visible light images and infrared thermal imaging images to obtain visible light image sequences and infrared thermal imaging image sequences.

[0263] The UAV performs a pre-flight self-check (GPS satellite count ≥12, battery level ≥95%, camera focus normal, IMU calibration passed), and performs inspections according to the UAV flight path instructions in the executable instruction set; the UAV is equipped with a 4K visible light camera and an infrared thermal imaging camera, and performs simultaneous acquisition of visible light images and infrared thermal images to obtain visible light image sequences and infrared thermal image sequences; the visible light image data includes RTK pose information, and RTK differential positioning achieves centimeter-level accuracy.

[0264] The optimal time to acquire infrared thermal images is 2 hours before sunrise or 2 hours after sunset (when the temperature difference is greatest and noise is least). Perform infrared camera calibration (set emissivity parameters, record ambient temperature and humidity for subsequent temperature correction).

[0265] Real-time temperature field analysis is performed: a baseline for the temperature distribution in the normal area is established, and abnormal areas with temperature deviations >2℃ are marked (excluding known interference sources such as windows, air conditioner outdoor units, and pipes). The acquisition density is automatically increased for abnormal areas to perform temperature correction, and temperature correction parameters are obtained. The infrared thermal imaging data includes these temperature correction parameters.

[0266] The drone autonomous inspection intelligent agent also includes an adaptive data acquisition strategy: the drone is also equipped with an edge computing module (NVIDIA Jetson) to run a lightweight disease pre-inspection model in real time. When a suspected disease area is detected, it automatically decelerates and increases the shooting density, performs multi-angle surround shooting, and transmits the key frames of the suspected disease area back in real time via 5G or WiFi. The full data of the suspected disease area is uploaded in batches after landing on the onboard SD card.

[0267] (2) The wall-climbing robot control agent is used to control the wall-climbing robot to move according to the wall-climbing robot path instructions, and to perform close-range macro photography and manipulator knocking at key detection points to obtain close-range macro photography images and knocking audio detection data, thereby obtaining close-range macro photography image sequence and knocking audio detection data sequence.

[0268] The wall-climbing robot moves according to path instructions and automatically performs detection actions at key detection points.

[0269] The wall-climbing robot is equipped with a camera. When it travels to a key detection point, it captures close-up macro images using the camera to obtain a sequence of close-up macro images for crack width measurement. The close-up macro images are then quantified into crack width using sub-pixel edge detection. The results are transmitted via 5G or WiFi to the understanding and analysis module and written into the geometric attribute field of the structured disease record, and used when automatically classifying the severity.

[0270] A robotic arm taps on building surfaces to identify hollow areas (the frequency characteristics of the tapping sound in hollow areas are significantly different from those in solid areas). Specifically, this involves: acquiring tapping audio signals recorded by a microphone (sampling rate greater than or equal to 44.1kHz, duration 100-200ms / tape); performing a Fast Fourier Transform (FFT, existing technology) on each tapping signal to extract the dominant frequency (the frequency component with the most concentrated energy) and energy spectral density distribution of the tapping audio; and determining the criteria: when the dominant frequency is lower than a preset threshold (default value 500Hz, determined through calibration experiments on similar structural materials; the dominant frequency of tapping solid brick-concrete walls is usually 900-1200Hz, while the dominant frequency in hollow areas drops to 300-600Hz), it is identified as a suspected hollow point. The GPS coordinates and hollow confidence level (calculated by normalizing the dominant frequency deviation) of the suspected hollow point are recorded in the tapping audio detection data.

[0271] The tapping audio detection data includes: audio file, acquisition timestamp, GPS coordinates of acquisition location, and corresponding building surface component ID;

[0272] FFT is an existing technology in the field of signal processing. The innovation of this application lies in its application to the scenario of identifying hollow areas in building exterior walls. It establishes a main frequency-hollow area judgment threshold system for brick-concrete / frame structure wall materials and structures the detection results into data records with three-dimensional spatial coordinates for further processing.

[0273] The fine inspection section can also be switched via remote control by the operator.

[0274] The wall-climbing robot can perform three types of inspection actions: close-range macro photography (crack width quantification), robotic arm tapping and audio collection (hollow drum identification), and optional powder or coating sampling (material chemical testing, performed as needed).

[0275] (3) A three-dimensional laser scanning intelligent agent, used to generate stitched three-dimensional point cloud data by scanning station by station according to the instructions of the laser scanner station;

[0276] Scanning is performed station by station according to the laser scanner station instructions. Coarse stitching between stations is completed on-site (based on the coordinates of pre-deployed reflective targets). Hole detection is performed, and additional stations are added if necessary. After scanning is completed, fine stitching is performed: ICP (Iterative Closest Point) algorithm → global optimization (uniform distribution of closure error to each station), finally obtaining the stitched 3D point cloud data (color).

[0277] A multimodal raw data package is constructed based on data collected from multiple devices. The multimodal raw data package includes visible light image sequences, infrared thermal imaging image sequences, three-dimensional point cloud data, close-range macro imaging image sequences, and tapping audio detection data sequences.

[0278] The data in the multimodal raw data packet is transmitted back to the platform data storage layer in real time / in batches, and the execution logs of each device are output at the same time.

[0279] The sensing and acquisition module also includes: when a device malfunction or environmental change is detected, immediately pushing the real-time abnormal event to the path replanning module via an event message, and recording the real-time abnormal event;

[0280] When a device anomaly (camera failure, battery level below 20% threshold, communication interruption for more than 10 seconds) or environmental change (wind speed exceeding 8 m / s, rain sensor triggering) is detected, the system immediately pushes the real-time event to the path replanning module through the Kafka message queue. The real-time event includes the device's unique identifier, anomaly type, the device's current GPS coordinates and timestamp. The current real-time time is recorded as the anomaly event, and the replanning module completes the response processing within 30 seconds.

[0281] The path replanning module is used to generate corresponding incremental device instructions and send them to the corresponding devices based on the real-time event stream generated by the sensing and acquisition module or the discovery of high-risk diseases in the structured disease record set, and to store the path change log in the device execution log; dynamic replanning only generates incremental instructions for devices related to the triggering event (the closest available similar device to the high-risk area, or the device that has failed), while other devices continue to execute the original instruction set.

[0282] The path replanning module is based on an event-driven architecture and responds in real time. The triggering conditions are divided into three categories:

[0283] (a) High-risk disease detection (high-confidence disease detection results pushed by AI detection);

[0284] The detection value of high-risk diseases is calculated according to the following formula. When the detection value exceeds the value threshold, the nearest available equipment is dispatched to perform a detailed investigation through a greedy algorithm, and an incremental flight path instruction is generated to expand the surrounding scan.

[0285] The available devices can be queried in real time from the device management module (a basic system support component that maintains device ledgers and real-time status). The filtering criteria are: type matching technology selection requirements, current status is standby, and remaining power is greater than 30%.

[0286] Detection value = Disease severity × Spread risk coefficient × Undetected area;

[0287] The diffusion risk coefficient is obtained from the knowledge graph by disease type (structural cracks = 2.0, leakage = 1.5, surface peeling = 0.5); the undetected area is calculated in real time from the 3D model coverage record. The value setting threshold is set according to actual needs;

[0288] The severity of the disease is obtained from the structured disease record set. The severity of the disease is a continuous value, which is calculated by the geometric attribute quantification step according to the disease size (crack width / length, hollow area, etc.) and a preset calculation rule. The range is 0~100, and the higher the value, the more severe the disease.

[0289] (b) Equipment failure replanning, which is used to mark the remaining task path segments of the failed equipment based on real-time events, query the standby equipment, reallocate the incomplete path to the standby equipment, and generate incremental instructions;

[0290] The backup device is a device of the same type in the device management module that is currently in standby mode and has not been assigned any tasks. In the event of a failure, it will take over the remaining path segment.

[0291] The specific steps for equipment fault replanning are as follows: ① Receive a real-time event containing the unique identifier of the faulty equipment; ② Query the currently executed instruction set for the incomplete path segments of the equipment (all waypoints after the last confirmed waypoint number); ③ Mark these path segments as pending reassignment; ④ Query the equipment management module for backup equipment of the same type (standby and with remaining power greater than 30%); ⑤ If a backup device exists, use its current position as the starting point to generate a connecting path using the ACO algorithm, merge it with the original remaining path segments to generate a complete incremental instruction and push it to the backup device; ⑥ If there is no backup device, mark the relevant detection items as incomplete—requiring supplementary inspection, and write them into the quality assessment report to generate a supplementary inspection plan suggestion.

[0292] (c) Environmental change replanning, used to determine the affected equipment based on meteorological information, record the equipment status and current location of the affected equipment, issue safety incremental instructions to the affected equipment to suspend the affected equipment, and generate a reconnection incremental instruction for the affected equipment while waiting for meteorological information to meet the requirements.

[0293] For environmental changes, the handling methods and incremental command generation include: ① Receiving real-time push events from the meteorological API (including change type, current value, and timestamp); ② Determining affected equipment: When the wind speed is 8m / s, all outdoor drones and wall-climbing robots are affected; during rainfall, only drones are affected (the wall-climbing robot is waterproof with IP65 and can continue to operate); ③ Issuing safety incremental commands to affected equipment: Drones select the nearest safe landing point (the nearest flat, unobstructed surface within a 50m radius in the 3D model) to land, and mark uncompleted detection items as suspended; ④ The meteorological API continues to push, and when the wind speed drops below 6m / s and stabilizes for 5 minutes or the rainfall stops for 15 minutes, the system resumes from the suspended state, generates a continuation command (the shortest path from the current location of the equipment to the uncompleted detection point), and continues execution.

[0294] Security verification and rollback: All incremental commands are verified through the security rule base before being issued. Replanning delay is controlled within 30 seconds. If replanning fails, a rollback mechanism is triggered and a manual intervention request is generated.

[0295] The understanding and analysis module is used to align and fuse the visible light image sequence, infrared thermal imaging image sequence and three-dimensional point cloud data in the multimodal raw data package to obtain multimodal fused data. The multimodal fused data is then used to detect diseases through a disease detection model and enhanced with domain knowledge. A structured disease record set is constructed by combining the audio percussion hollow drum analysis results of the multimodal raw data package.

[0296] The multimodal raw data package includes visible light image sequences, infrared thermal imaging image sequences, three-dimensional point cloud data, close-range macro imaging image sequences, and tapping audio detection data sequences.

[0297] The understanding and analysis module specifically includes:

[0298] (1) A multi-source data alignment agent is used to register infrared thermal imaging images in an infrared thermal imaging image sequence to the coordinate system of visible light images in a visible light image sequence to obtain registered image pairs. Based on the alignment quality score and image quality score of the registered image pairs, the agent performs image quality scene division on the registered image pairs. Then, using the spatial coordinate system of the three-dimensional point cloud data as a reference, the agent performs multi-channel fusion of the registered image pairs and the three-dimensional point cloud data to obtain multi-modal fusion data. The image quality scenes include high-quality dual-modal scenes and single-modal low-quality scenes.

[0299] Infrared thermal imaging image registration with visible light image: Direct pixel alignment with a dual-light camera that is coaxial with the hardware; for images acquired non-coaxially, SIFT or ORB feature points are extracted for matching, affine transformation matrix or homography matrix is ​​calculated, and the infrared thermal imaging image is registered to the visible light image coordinate system to obtain a registered image pair. The registration error is controlled within 3 pixels, and semantic anchor points such as building edges and window corners are used to improve the registration robustness.

[0300] It is understandable that the registered image pair after the visible light image and the infrared thermal image are registered to correspond to the image area of ​​a certain detection sub-region of the 3D BIM model;

[0301] Then, based on the alignment quality score and image quality score of the registered image pairs, the registered image pairs are classified into different image quality scenes:

[0302] The alignment quality of the registered image pairs (infrared thermal imaging image and visible light image) is evaluated using three metrics:

[0303] 1) Reprojection error = Average Euclidean pixel distance of corresponding feature point pairs after registration - Registration error threshold;

[0304] The registration error threshold is 3 pixels;

[0305] 2) Structural similarity: The structural similarity is calculated after the registered visible light and infrared images are grayscaled. When the structural similarity is less than 0.6, the registration quality is marked as low, which is used for real-time and rapid early warning. Any single indicator that seriously fails to meet the standard will trigger a re-sampling immediately.

[0306] The specific calculation process for the structural similarity includes:

[0307] ① Convert both images to grayscale (visible light: 0.299R + 0.587G + 0.114B weighted; infrared: single channel used directly).

[0308] ② Calculate window structure similarity window by window using an 11×11 pixel sliding window:

[0309] SSIM=(2·μx·μy+C1)(2·σxy+C2) / [(μx²+μy²+C1)(σx²+σy²+C2)],

[0310] Where SSIM represents the window structure similarity; μx represents the average pixel grayscale value within the visible light image window; and μy represents the average pixel grayscale value within the infrared thermal imaging image window.

[0311] Specifically, μx represents the average grayscale value of all pixels in the visible light image within the current 11×11 sliding window (local brightness average); μy represents the average grayscale value of all pixels in the infrared thermal image within the same window (local brightness average). Both reflect the average brightness level of their respective images in that local region and are the basis for calculating the brightness similarity component in the SSIM formula;

[0312] σx² represents the variance of pixel grayscale values ​​in the visible light image within the current sliding window, reflecting the local contrast of the visible light image within that window; σy² represents the variance of pixel grayscale values ​​in the infrared thermal image within the same window, reflecting the local contrast of the infrared image within that window; σxy is the covariance of pixel grayscale values ​​in the visible light image and the infrared thermal image within the same window, reflecting the degree of correlation between the local structures of the two images (the larger the covariance, the more similar the structures of the two images in that region, and the more accurate the alignment).

[0313] C1 and C2 are stability constants in the SSIM formula, used to prevent numerical instability when the denominator approaches zero. The first stability constant C1 is added to the denominator containing μx² + μy² to stabilize the luminance comparison component (μx and μy are the mean gray values ​​of pixels in the visible light image window and the mean gray values ​​of pixels in the infrared thermal image window, respectively; when their mean values ​​are extremely small, the denominator will approach zero). The second stability constant C2 is added to the denominator containing σx² + σy² to stabilize the contrast comparison component. C1 = (k1 × L)² = (0.01 × 255)² = 6.5025, C2 = (k2 × L)² = (0.03 × 255)² = 58.5225, where k1 = 0.01 and k2 = 0.03 are the SSIM standard stability coefficients, and L = 255 is the pixel gray-scale dynamic range of the 8-bit image.

[0314] ③ Take the arithmetic mean of the SSIM values ​​of all windows in the entire image to obtain the image-level SSIM mean as the structural similarity;

[0315] ④ When the structural similarity is less than 0.6, the registration result of this image region is marked as a low-quality registration and a re-sampling suggestion is triggered;

[0316] 3) Edge overlap rate: Extract the Canny edge map of the infrared thermal image and the visible light image to calculate the edge overlap rate IoU. When IoU < 0.5, mark it as low confidence.

[0317] The steps for calculating the edge overlap rate are as follows:

[0318] ① Perform Canny edge detection on the infrared thermal imaging image and the visible light image respectively (Gaussian blur kernel 5×5 / σ=1.0 for noise reduction, low threshold 100 / high threshold 200 for edge connection), and output the binary edge map of the infrared thermal imaging image and the visible light image respectively (edge ​​pixel value = 1, non-edge = 0).

[0319] ② Obtain the edge intersection by performing a pixel-by-pixel logical AND operation on the binary edge maps of the infrared thermal imaging image and the visible light image, and obtain the edge union by performing a pixel-by-pixel logical OR operation on the binary edge maps of the infrared thermal imaging image and the visible light image;

[0320] ③ Edge overlap ratio IoU = |Number of pixels in edge intersection| / |Number of pixels in edge union|, with a value of 0~1. The higher the value, the better the edge alignment after registration;

[0321] ④ When the edge overlap rate (IoU) is less than 0.5, the registration result of this image region in the 3D BIM model is marked as a low-confidence registration. The alignment quality score of the low-confidence registration region will be reduced in the subsequent data quality score calculation.

[0322] Both the Canny edge detection operator and IoU (Intersection over Union) calculation are existing technologies. The innovation of this invention lies in combining these two existing technologies for the registration quality assessment of cross-modal images (visible light and infrared thermal images), and specifying specific parameters (Gaussian kernel 5x5, dual thresholds 100 / 200, and IoU low confidence threshold 0.5) for the characteristics of building detection.

[0323] The alignment quality score is obtained by normalizing the weighted average of feature point reprojection error, structural similarity and edge overlap rate (0-1).

[0324] The alignment quality scoring includes the following steps:

[0325] After obtaining the normalized score of the reprojection error, the normalized score, structural similarity, and edge overlap rate are weighted and summed according to the reprojection error weight w1=0.5, structural similarity weight w2=0.3, and edge overlap rate weight w3=0.2 to obtain the alignment quality score (0~1); as shown in the following formula:

[0326] Alignment quality score = Normalized reprojection error score * w1 + Structural similarity score * w2 + Edge overlap rate (IoU) * w3;

[0327] ① Reprojection error normalization score (pixels, the smaller the better): Min-Max inverse normalization, reprojection error normalization score = 1 - (reprojection error / 10); 0 reprojection error gets 1 point, reprojection error exceeding 10 pixels is truncated, and the reprojection error normalization score is 0 points.

[0328] ② Structural similarity (0~1, the higher the better): Use the original value directly;

[0329] ③ Edge overlap rate IoU (0~1, the larger the better): Use the original value directly;

[0330] Meanwhile, the three indicators of reprojection error, structural similarity SSIM and edge overlap rate are calculated for the registration pairs of visible light images and infrared images, and each registration image pair outputs an alignment quality score.

[0331] The image quality score calculation method specifically includes: using Laplacian variance as the core indicator, calculating the Laplacian variance of the visible light image and the infrared thermal imaging image in the registered image independently, and then normalizing it to obtain their respective image quality scores; the Laplacian variance is normalized to 0~1, and the higher the variance, the clearer the image, and the higher the image quality score.

[0332] If the image quality score of the visible light image is >= 0.7, the image quality score of the infrared thermal image is >= 0.7, and the alignment quality score of the registered image pair is >= 0.7, then it is considered a high-quality bimodal scene.

[0333] If only one of the image quality scores of the visible light image and the infrared thermal image is >= 0.7, it is considered a low-quality scene in a single mode.

[0334] After completing the registration and alignment of infrared thermal imaging images and visible light images, and scene segmentation, multi-channel fusion data is obtained by using the spatial coordinate system of the 3D point cloud data as a reference and performing multi-channel fusion according to the following channels:

[0335] Visible light texture channel: The visible light image is back-projected through PnP to map the RGB pixel values ​​to the corresponding points in the 3D point cloud, forming a colored point cloud with R / G / B three-channel color values ​​for each 3D point;

[0336] Infrared temperature field channel: The registered infrared image temperature values ​​(absolute temperature after emissivity and ambient temperature correction, unit: ℃) are also mapped to the corresponding points of the 3D point cloud through PnP back projection results, forming a temperature point cloud with temperature value T attached to each 3D point.

[0337] Geometric feature channel: Computes the geometric features of each point from the original point, such as the normal vector, curvature, and point density;

[0338] The three-channel data are aligned according to three-dimensional spatial coordinates to form a multi-dimensional data structure with (X,Y,Z) as the spatial index and (three-channel color values ​​(R,G,B), T, normal vector, curvature, point density) as the attribute vector, which is multi-modal fusion data, and stored as .las format extended attribute fields or HDF5 format matrix.

[0339] (2) Disease detection intelligent agent, which is used to obtain a list of disease detection results by selecting mid-term fusion or late-term fusion based on the image quality scene after preprocessing the registered image pairs through the disease detection model;

[0340] Data preprocessing (normalization, block slicing, and data augmentation) is performed on the visible light image and infrared thermal image in the registered image pair, respectively. Then, a multi-task learning framework is used for mid-term and late-term fusion to finally obtain a list of disease detection results. The multi-task learning framework consists of a shared backbone network and three parallel branches, including a detection branch, a classification branch, and a segmentation branch. The outputs of the three branches are the image coordinate bounding box, disease type, and pixel-level segmentation mask, respectively.

[0341] Simultaneously, the three-dimensional coordinates of the defects are recorded. The image coordinate bounding box is a two-dimensional rectangular frame represented by the image pixel coordinate system, reflecting the position of the defect in the image. The three-dimensional position coordinates are the three-dimensional spatial coordinates obtained by transforming the center point of the image coordinate bounding box to the BIM model coordinate system through a "coordinate transformation matrix" (a 4×4 homogeneous transformation matrix generated by the PnP+ICP alignment step), reflecting the precise position of the defect in the building entity space (unit: meters). The three-dimensional position coordinates are a common field for all types of defects (cracks, leaks, hollow areas, insulation defects, etc.). Each defect record has three-dimensional position coordinates, which are used to accurately locate and label the spatial position of the defect on the 3D BIM model.

[0342] The shared backbone network uses ResNet-50 or EfficientNet-D4 as a feature extractor; the detection branch outputs the image coordinate bounding box of the lesion based on YOLOv8 or Faster R-CNN; the classification branch outputs the lesion type; and the segmentation branch outputs a precise pixel-level segmentation mask of the contour based on Mask R-CNN or SAM.

[0343] The image bounding box, also known as the target box, is the output of the target detection branch (Faster-RCNN detection head) in the EfficientDet-D4 network. It is represented in the image pixel coordinate system and has the format [x_min, y_min, x_max, y_max]. It marks the bounding box range of each candidate disease region in the image.

[0344] For high-quality bimodal scenes, mid-term fusion is adopted: the registered visible light image and infrared thermal imaging image are used to extract independent feature maps through a shared backbone network. Then, the two independent feature maps are fused at the feature level through an attention mechanism. The fused feature map is simultaneously input into three branches for joint decision-making to obtain the final detection result and initial confidence. Mid-term fusion makes full use of the complementary information of the two modalities and has the best effect when the data quality is high.

[0345] The initial confidence score is the original confidence score directly output by the fusion confidence score (the weighted fusion value of the confidence scores output by each branch) after the EfficientDet-D4 multi-task network makes joint decisions in the current input modality through three branches (object detection branch, segmentation branch, and classification branch), without any post-processing or manual intervention.

[0346] For scenarios with low quality in a single modality, late-stage fusion should be used:

[0347] The registered visible light image and infrared thermal image are each subjected to a complete and independent detection-classification-segmentation process to generate their respective detection results (boundary box + category + confidence score). Then, a weighted voting fusion is performed on the initial confidence scores of the two modalities to obtain the fused confidence score.

[0348] Fusion confidence score = High-quality mode confidence score × 0.7 + Low-quality mode confidence score × 0.3

[0349] The high-quality modality confidence score is the initial confidence score of the image corresponding to an image quality score greater than or equal to 0.7; the low-quality modality confidence score is the initial confidence score of the detection branch output of the image corresponding to an image quality score less than 0.7.

[0350] The result with a fusion confidence score greater than the fusion threshold is taken as the final detection result. The fusion object is the content of the three branches' output (box, category, segmentation mask), and the fusion confidence score is used as the initial confidence score of the final detection result.

[0351] The fusion threshold is 0.5, which is a standard threshold set based on the balance between precision and recall, and can be adjusted according to the actual deployment scenario;

[0352] The disease detection results include disease ID, image coordinate bounding box, disease type, pixel-level segmentation mask, and initial confidence level, ultimately resulting in a list of disease detection results.

[0353] Post-processing is performed after detection: Non-maximum suppression (NMS) is used to remove overlapping detection boxes, area or shape filtering is used to eliminate noise and false detections, and cross-frame temporal consistency correlation is performed (detection results of the same lesion in multiple frames of images are merged).

[0354] For each detected disease, a structured record is generated. High-confidence disease detection results are immediately pushed to the path replanning module as high-risk disease discoveries, realizing a real-time closed loop of detection and scheduling.

[0355] The high confidence level judgment criterion is as follows: after the confidence level output by the disease detection model is corrected by the IF-THEN rule of the domain knowledge enhancement agent, the corrected confidence level is greater than 0.9, which is considered high confidence level.

[0356] The 0.9 threshold was experimentally calibrated: on a test set of 1200 labeled images, the accuracy rate of diseases with a corrected confidence level greater than 0.9 was 96.2%, and the false alarm rate was 3.8%, which can ensure a low false trigger rate when pushing to the replanning module in real time; this threshold is configurable and can be adjusted according to the actual scenario after deployment.

[0357] (3) Domain knowledge-enhanced intelligent agent, which loads spatial context (type of the component and adjacent discovered defects) and temporal context (historical defect records) for each defect in the defect detection result list. It queries the Neo4j knowledge graph subgraph with the house knowledge graph fragment reference ID as the key to obtain the historical defect association relationship of the same component. It executes IF-THEN rule reasoning and obtains two main outputs: ① Corrected confidence (the corresponding rule is triggered according to the spatial context and temporal context to perform multiplication factor correction. For example, when there is infrared thermal anomaly at the same location and there is a hollow record in history, the hollow confidence is multiplied by 1.3); ② Reasoning link record (the rule triggered this time, the matching evidence, and the final conclusion are recorded in the "conclusion ← evidence ← rule" triple chain structure and written into the reasoning link field of the structured defect record for the purpose of providing a traceable basis for the conclusion when the report is generated, and also as knowledge material for the subsequent case library.

[0358] Example rules:

[0359] If infrared imaging at the same location shows abnormal low temperature AND historical reports show a hollow area, then the confidence level for the hollow area is += 30%;

[0360] If vertical cracks are located in the middle of a load-bearing wall and the building is more than 30 years old, the risk level of structural cracks will be upgraded by one level.

[0361] If there are abnormal infrared readings in the roof area and there is a record of rainfall in the past 7 days, then the probability of leakage is marked as "high".

[0362] If there are multiple parallel horizontal cracks and the foundation type is soft soil, then the possibility of uneven settlement is marked as "high".

[0363] The rule base for IF-THEN rule reasoning is derived from expert experience coding and industry standards;

[0364] Using the basic probabilities of each cause corresponding to the building structure type, usage type, and construction year as the prior, and the support of the observed combination of disease features to each cause as the likelihood, the posterior probability P(cause|feature) ∝ P(feature|cause)×P(cause) is calculated. Candidate causes are sorted according to their posterior probabilities and recorded.

[0365] Wherein, P(cause|feature) is the posterior probability, that is, the probability of each disease cause occurring given the current combination of features. This is the ultimate goal of inference—to determine which cause is most likely to lead to the currently observed disease phenomenon.

[0366] P(feature|cause) is the likelihood function, which represents the probability of observing the current combination of features given a known cause. For example, the historical probability of observing the feature "multiple parallel horizontal cracks" under the cause of "uniform settlement" is obtained from statistical analysis of knowledge graph cases.

[0367] P (causation) is the prior probability, which is the basic probability of a certain cause occurring when no specific features are observed. It is obtained from historical statistics of the knowledge graph based on the building structure type, usage type, and construction year.

[0368] ∝(Proportional To): Since the denominator P (feature) of Bayes' formula is the same constant for all causes, the denominator can be omitted when calculating the relative probability. The posterior probability of each cause can be obtained by directly calculating the numerator and normalizing it.

[0369] The "|" in "feature|cause" is a conditional probability symbol, read as "under the condition of..." or "given...", that is, the probability of the event (feature) occurring on the left when the right-hand side condition (cause) is known to be true.

[0370] Automatic severity rating rules:

[0371] If the crack width is less than 0.2 mm, the severity of the disease is 25 points, and the severity level is slight.

[0372] If the crack width is less than 0.5 mm and the crack thickness is 0.2 mm, the severity of the disease is 50 points, and the severity level is medium.

[0373] If the crack width is less than 1 mm and the crack width is less than 0.5 mm, the severity of the disease is 75 points, and the severity level is relatively severe.

[0374] If the crack width is ≥1mm, the severity of the disease is 100 points, and the severity level is severe.

[0375] The severity score of the damage to load-bearing components is multiplied by 1.5 (or 100 if the score exceeds 100) before being graded.

[0376] (4) A disease geometry quantification intelligent agent, used to extract the geometric attributes of each disease detection result in the disease detection result list; the geometric attributes include length, width, area, depth and uncertainty;

[0377] Each quantization value comes with uncertainty (based on factors such as GSD accuracy, calibration error, and detection distance);

[0378] For example: Load the original high-resolution data of the corresponding area for each disease:

[0379] Width: Crack width is measured using subpixel measurement (Canny edge detection → subpixel refinement → converted to actual physical size via GSD);

[0380] Length: Crack length is calculated by extracting the skeleton;

[0381] Depth: Deformation (depth) is calculated by least-squares fitting of measured point cloud and design model → deviation cloud map;

[0382] Area: The actual area of ​​the peeled-off region is calculated by multiplying the area of ​​the segmented mask by GSD²;

[0383] Where GSD = H × d / f, in mm / pixel, represents the actual size of the ground corresponding to each pixel in the image; GSD (Ground Sampling Distance) is the ground sampling distance; H is the flight altitude; f is the focal length; and d is the pixel size.

[0384] The uncertainty (±δ) is calculated using the error propagation formula: δ= ∂ represents the partial derivative calculation; δH is the flight altitude measurement error, and δf is the focal length calibration error.

[0385] Taking the partial derivative with respect to H, we get This indicates that for every unit increase in flight altitude, GSD increases accordingly; taking the partial derivative with respect to f yields... This means that for every unit increase in focal length, the GSD decreases. Substituting the partial derivatives of both terms into the standard error propagation formula, we get δ = This corresponds to the description of the uncertainty accompanying each quantified value.

[0386] (5) Audio knocking hollow drum analysis intelligent agent, used to receive the knocking audio detection data sequence in the multimodal raw data packet, perform signal preprocessing, fast Fourier transform spectrum analysis, hollow drum determination and spatial mapping to obtain the audio hollow drum detection result set;

[0387] Perform the following processing flow:

[0388] Each tapping audio detection data includes: audio file, acquisition timestamp, GPS coordinates of acquisition location, and corresponding building surface component ID;

[0389] Signal preprocessing: Denoising is performed on each tapping audio detection data (Hannai window is applied to reduce spectral leakage), and the effective tapping segment (the time interval with energy more than 3 times the ambient noise floor, usually 50-150ms after the tap) is extracted to obtain a standardized effective tapping signal segment;

[0390] FFT Spectral Analysis: Perform Fast Fourier Transform (FFT, existing technology) on the effective knock signal segment to calculate the frequency domain energy spectrum; extract two feature quantities: the dominant frequency f_peak (the frequency component with the highest energy, in Hz) and the spectral centroid f_centroid (energy-weighted average frequency);

[0391] Hollow wall detection: Based on the current building structure type and exterior wall material, query the corresponding hollow wall detection threshold from the threshold parameter library (e.g., the main frequency threshold for hollow wall detection in brick-concrete structure exterior walls f_threshold=500Hz).

[0392] Judgment rules: If f_peak is lower than f_threshold and f_centroid is lower than f_threshold × 1.2, it is judged as "confirmed hollow" (confidence = 1 - f_peak / f_threshold, value [0,1]); if only f_peak is lower than f_threshold, it is judged as "suspected hollow" (confidence = 0.6 × (1 - f_peak / f_threshold)); otherwise, it is judged as "normal".

[0393] Spatial mapping: The GPS coordinates of detection points identified as "confirmed hollow" or "suspected hollow" are transformed into the coordinate transformation matrix provided by the intelligent agent through multi-source data alignment in the steps, and then converted to the coordinate system of the 3D BIM model to generate a hollow point cloud (each point contains 3D coordinates, hollow confidence, and f_peak value); adjacent hollow points (3D distance less than 0.3m) are aggregated to estimate the hollow area (number of aggregated points × single detection area, the single detection area is determined by the size of the robot arm's tapping head, default 0.04m² / time), and a hollow area record is generated;

[0394] The coordinate transformation matrix is ​​a 4×4 homogeneous transformation matrix from the sensor coordinate system to the 3D BIM model coordinate system, containing the rotation matrix R and the translation vector t. It is generated in the multi-source data alignment step, including: ① using the PnP algorithm to solve the camera extrinsic parameters based on the correspondence between the 3D feature points of the laser point cloud and the 2D points of the image; ② using the ICP algorithm to finely align the laser point cloud and the pre-built BIM model on the platform in the BIM coordinate system to obtain a rigid body transformation from the point cloud coordinate system to the BIM coordinate system; ③ multiplying the two transformation matrices to obtain the final complete coordinate transformation matrix. In addition, this matrix is ​​persistently stored after the alignment step is completed, and can be directly called in subsequent steps such as hollow space mapping and 3D defect mapping without repeated calculation.

[0395] Output: Generate an audio hollow detection result set. Each audio hollow detection result record includes {hollow region ID, three-dimensional location coordinates, area estimate (m²), average confidence level, dominant frequency feature value f_peak (Hz), severity level};

[0396] The severity level is determined based on the estimated area.

[0397] If the estimated area is <0.1m², the severity of the disease is 25 points, and the severity level is mild.

[0398] If 0.1m² ≤ estimated area < 0.5m², then the severity of the disease is 50 points, and the severity level is medium.

[0399] If 0.5m² ≤ estimated area < 2m², then the severity of the disease is 75 points, and the severity level is relatively severe.

[0400] If the estimated area is ≥2m², the severity of the disease is 100 points, and the severity level is severe.

[0401] The result set is input into the domain knowledge-enhancing agent to trigger IF-THEN rule inference related to hollow sounds (e.g., IF audio detection hollow sound confidence greater than 0.7 AND historical reports have recorded hollow sounds THEN hollow sound confidence += 30%). At the same time, high-confidence hollow sounds (confidence greater than 0.7) are immediately pushed to the path replanning module to trigger encrypted detection and evaluation.

[0402] The final enhanced structured disease record set is obtained, where each structured disease record contains the following 13 fields: (1) Disease ID (unique identifier); (2) Disease type (including subtypes); (3) Image coordinate bounding box (the rectangular box coordinates output by the target detection branch [x_min, y_min, x_max, ...). (4) Pixel-level segmentation mask (binary mask output by Mask-RCNN segmentation branch); (5) Initial confidence (original output of detection model); (6) Corrected confidence (value corrected by domain knowledge enhancement agent IF-THEN rule); (7) Three-dimensional position coordinates (spatial coordinates converted to BIM coordinate system by coordinate transformation matrix); (8) Geometric attributes (length, width, area, depth and various uncertainties); (9) Severity level (mild / moderate / severe / serious); (10) Candidate causes and posterior probability; (11) Reasoning link (three-tuple chain record of conclusion ← evidence ← rule); (12) Hollow subtype (output of hollow special analysis, this field is empty for non-hollow diseases); (13) Development trend prediction (short, medium and long term).

[0403] The mapping and twin module is used to perform three-dimensional mapping and synchronous updating of disease and digital twin on the three-dimensional BIM model based on the structured disease record set, and to obtain a report on the difference between the disease-annotated three-dimensional BIM model and the digital twin version.

[0404] (1) The disease three-dimensional mapping intelligent agent is used to load the lightweight three-dimensional BIM model of the platform. The coordinate transformation matrix provided by the intelligent agent is used to align the multi-source data to transform the three-dimensional position coordinates of the structured disease record to the model coordinate system. The disease is accurately attached to the building surface of the three-dimensional BIM model for annotation by the ray projection algorithm, and then rendered according to the severity color coding and LOD level rendering to obtain the disease annotation three-dimensional BIM model.

[0405] ① Coordinate transformation: The 4×4 coordinate transformation matrix (R|t) stored in the multi-source data alignment agent is used to batch transform the 3D position coordinates of each defect record to the BIM coordinate system; ② Ray projection algorithm for fitting: Starting from the transformed 3D defect coordinates, a ray is emitted along the normal vector direction of the point to the building surface. The first intersection point of the ray and the geometric surface of the BIM model is taken as the precise fitting position of the defect (ensuring that the annotation point is located on the building surface rather than floating in space). The annotation update record of the 3D BIM model corresponding to each defect record is recorded; ③ LOD hierarchical rendering distance threshold: When the distance between the view camera and the model is greater than 50m, a defect density heat map is displayed; when it is 10~50m, a defect type icon with color code (green / yellow / orange / red / purple) is displayed; when it is less than 10m, a precise defect outline (segmentation mask projection) is displayed and a details card containing all fields pops up; and a time axis slider is supported to switch between different detection periods; finally, a 3D BIM model with defect annotations containing all defect records is displayed; based on WebGL, smooth browsing on the browser side is achieved, supporting millions of annotation points and mobile adaptation.

[0406] (2) Digital twin update agent: Incremental updates are performed on the hierarchical model architecture that includes geometry layer, component layer, state layer and history layer. Only the changed components and regions are updated. A record is generated for each change and saved as a new version of the model to form a digital twin version difference report.

[0407] The layered model architecture of geometry layer / component layer / state layer / history layer is a common concept in existing BIM digital twin technology. This invention implements an automated incremental update mechanism based on the results of defect identification on this architecture. Traditional BIM digital twins require manual input of defect information and updating of model status. This invention automatically drives the incremental writing of the state layer and history layer through a structured defect record set. After each inspection, a version snapshot and version difference comparison report are automatically generated without manual intervention.

[0408] The evaluation and learning module is used to evaluate all data of the currently executed task to obtain a detection quality management report. Based on all data of the currently executed task, it learns and evolves to update the model, expands the knowledge graph, adds new case library records, and feeds them back to the task scheduling module, scheme planning module, and understanding and analysis module to form a system-level continuous evolution closed loop. The "all data" includes multimodal raw data packets, analysis results, equipment operation logs, and manual review and correction records.

[0409] The multimodal raw data package includes visible light image sequences, infrared thermal imaging image sequences, three-dimensional point cloud data, close-range macro images, and tapping audio detection data sequences.

[0410] The analysis results include coordinate transformation matrix, structured disease record set, and 3D BIM model annotation update record;

[0411] The device operation log includes the complete travel trajectory corresponding to the executable instruction set of each device, the power consumption curve of each device, and abnormal real-time events;

[0412] The manual review and correction refers to the process by which testing experts manually review the AI-generated identification results through the platform's mobile app or management backend. This includes: ① supplementing missed detections (manually annotating and entering information such as disease ID, type, and location for diseases not identified by AI); ② deleting false positives (manually marking and deleting false positives identified by AI and recording the reasons); and ③ correcting classifications (manually correcting errors in disease type or severity level identified by AI and recording the comparison before and after correction). All manual review and correction records include a timestamp, operator ID, correction type, and correction content, and are automatically included in the entire dataset of the current task as high-quality labeled samples for incremental model training.

[0413] The detection coverage assessment agent, data quality check agent, and quality inspection report generation agent evaluate all detection data, analysis results, and equipment logs for the currently executing task to obtain a detection quality management report.

[0414] (1) The intelligent agent for detecting coverage evaluation is used to discretize the surfaces to be inspected in the 3D BIM model into 1m×1m grids, calculate the effective coverage status of each grid based on the actual trajectory and effective field of view parameters of each device, and then compare it with the requirements of the guidelines. The uncovered areas are marked on the 3D BIM model and a supplementary inspection plan suggestion is generated.

[0415] The effective field of view parameters are read from the platform's preset device specification library, including the field of view (FOV) (horizontal angle α, vertical angle β), the effective acquisition distance range, and the pixel size. The FOV includes horizontal angle α and vertical angle β; the effective acquisition distance range is 0.5m to 30m; and the pixel size is the sensor pixel size, in μm.

[0416] The effective field of view (FOV) parameters are static device specification parameters, pre-stored in the platform's device specification library, including: ① Camera horizontal / vertical field of view (FOV) (read from the device manufacturer's specifications, e.g., FOV = 84 degrees x 62.7 degrees for a UAV camera); ② Effective acquisition distance range (minimum 0.5m to maximum 30m, determined by the camera sensor resolution and minimum ground sampling distance (GSD) requirement); ③ Sensor pixel size (used to calculate GSD at different distances). During coverage calculation, for each acquisition frame, the frustum of the frame is calculated in real time using the position coordinates and attitude quaternions (extrinsic parameters) of that frame in the device trajectory log, combined with parameters such as FOV from the device specification library, to determine which grids are within the effective field of view.

[0417] The surfaces to be inspected refer to the building exterior surfaces in the 3D BIM model that need to be included in the inspection coverage calculation. Specifically, these include exterior wall facades (including the outer side of balcony railings), roof surfaces (including the top surface of parapet walls), and exposed structural beam and column surfaces (the parts that are in direct contact with outdoor air). They do not include interior surfaces, basement exterior walls (the parts below ground level), and enclosed surfaces of inner courtyards. The surfaces are obtained by filtering the component type attributes of the BIM model (selecting the ExteriorWall / Roof / ExposedColumn / ExposedBeam component surface sets), and this set is discretized into a 1m×1m grid.

[0418] The calculation logic for the effective coverage status includes: ① Reading the complete trajectory of each device (timestamp + position coordinates + attitude quaternion) from the device operation log; ② For each grid center point, traversing all acquisition frames in the trajectory, determining the camera frustum orientation by taking the current device position as the vertex and the field of view (FOV) as the opening angle, combined with the attitude quaternion, and then constructing the camera frustum, determining whether the camera frustum of the frame covers the grid center point, and that the distance from the grid center point to the device is within the effective acquisition distance range, and that the effective resolution at the current flight altitude meets the minimum resolution requirement, i.e., GSD≤5mm; GSD is the ground sampling distance: GSD = flight altitude × pixel size / focal length; ③ Further checking the image quality of the acquisition frames that enter the field of view (grid within the frustum): if the Laplacian variance of the corresponding image area is greater than 100 (sharpness qualified) and there is no motion blur, it is a valid acquisition frame; ④ If at least one frame of the grid meets the above conditions, then mark the effective coverage status as yes, otherwise mark the effective coverage status as no.

[0419] The surfaces to be inspected in the 3D BIM model are discretized into 1m×1m grids. The effective coverage status of each grid is calculated based on the actual trajectory and effective field of view parameters of each device, and the effective coverage rate of the surfaces to be inspected is obtained. Compared with the guidelines (the effective coverage rate of the facade should be ≥95%, the effective coverage rate of the roof should be ≥90%, and the effective coverage rate of the mandatory inspection items should be 100%), the uncovered areas are marked on the 3D model and supplementary inspection plan suggestions are generated.

[0420] Effective coverage of the surface to be inspected = (Number of grid cells in the effective coverage state of the surface to be inspected / Total number of grid cells on the surface to be inspected) × 100%;

[0421] (2) Data quality inspection agent, used to check the clarity, exposure and overlap rate of image data, and check the density uniformity, noise level and stitching accuracy of point cloud data. The data quality level is obtained by weighting and summing the scores of each indicator.

[0422] Image data: sharpness (Laplacian variance > 100 = sharp), exposure (histogram uniformity), overlap rate (number of feature matching points between adjacent images);

[0423] The data quality score (0-100 points) is obtained by weighting and normalizing the sharpness, exposure, and overlap rate. If the data quality score is greater than or equal to 90 points, the data quality level is A; if the data quality score is greater than or equal to 70 points but less than 90 points, the data quality level is B; if the data quality score is greater than or equal to 50 points but less than 70 points, the data quality level is C; and if the data quality score is less than 50 points, the data quality level is D.

[0424] Point cloud data: density uniformity (standard deviation of each region), noise level (plane fitting residual), stitching accuracy (registration error of common areas);

[0425] Data quality scores (0-100 points) are obtained by weighted normalization of density uniformity, noise level, and splicing accuracy. If the data quality score is greater than or equal to 90 points, the data quality level is A; if the data quality score is greater than or equal to 70 points but less than 90 points, the data quality level is B; if the data quality score is greater than or equal to 50 points but less than 70 points, the data quality level is C; and if the data quality score is less than 50 points, the data quality level is D.

[0426] If the data quality level is D, a supplementary data collection plan is triggered, which means that the grid area corresponding to the D-level data is marked as "needs supplementary collection" on the 3D BIM model, and a supplementary inspection suggestion is generated in the quality inspection report (including the equipment type, supplementary collection area range, and supplementary collection priority).

[0427] The system automatically marks the area as "needs additional sampling", marks the corresponding non-compliant grids on the 3D BIM model, and generates supplementary inspection suggestions in the quality inspection report (including the type of equipment to be supplemented, the scope of the supplementary sampling area, and the priority of the supplementary sampling).

[0428] (3) Quality inspection report generation intelligent agent, used to summarize the effective coverage rate of the surface to be inspected, data quality level, compliance verification results, load the guide standard report template to automatically fill, calculate the reliability level of the conclusion, generate PDF / Word format report, and support electronic signature;

[0429] The reliability level calculation for the conclusion includes:

[0430] First, calculate the overall conclusion score: Overall Conclusion Score = Coverage Completeness Score × 0.4 + Data Quality Level Score × 0.4 + Solution Compliance Score × 0.2;

[0431] Among them, the coverage integrity score = number of effective coverage grids / total number of surface grids to be inspected; the data quality level score is the weighted average of the quality ratings of all data packets (A=1.0 / B=0.8 / C=0.5 / D=0.0); the scheme compliance score = number of compliance verification guidelines passed / total number of applicable guidelines.

[0432] Based on the conclusions, a comprehensive mapping level is assigned:

[0433] If the overall score of the conclusion is greater than 0.85 and less than or equal to 1.0, the reliability level of the conclusion is high confidence.

[0434] If the overall score of the conclusion is greater than 0.65 and less than or equal to 0.85, the reliability level of the conclusion is medium confidence.

[0435] If the overall score of the conclusion is greater than 0.45 and less than or equal to 0.65, the reliability level of the conclusion is low.

[0436] If the overall score of the conclusion is less than or equal to 0.45, the reliability level of the conclusion is low.

[0437] Based on all data from the currently executing tasks, the system learns and evolves, updates the model, expands the knowledge graph, and adds new case records, feeding back these updates to the task scheduling module, solution planning module, and understanding and analysis module to form a system-level continuous evolution loop. This specifically includes:

[0438] (4) The model continuously optimizes the agent:

[0439] Construct a data flywheel closed loop: The incremental training dataset is generated by pairing the disease detection results generated by each detection of the disease detection model with the manual review and correction. The dataset is trained using an incremental learning framework that combines knowledge distillation, elastic weight consolidation EWC, and experience replay. After training, the dataset is evaluated on a fixed standard test set. If all indicators are better than the old model, the dataset is automatically released online. Otherwise, the reasons are analyzed and the model is retried. If the model fails repeatedly, the old version is rolled back.

[0440] (5) Knowledge graph updating agent:

[0441] Automatic knowledge extraction is performed from all data using a combination of NER and relation extraction to extract new triple candidates, which are then compared with the global knowledge graph (entity alignment and conflict detection). New knowledge with high confidence is automatically added to the database, while knowledge with medium and low confidence is marked as awaiting expert review. After being added to the database, graph reasoning is triggered to discover implicit relationships and knowledge completion opportunities.

[0442] NER is an abbreviation for Named Entity Recognition, which belongs to natural language processing technology. NER is used to automatically identify and extract entities with professional meaning (such as disease type names, building component names, material names, etc.) from structured disease record sets and inspection report texts. It works in conjunction with Relation Extraction (RE) to complete the automatic extraction of "entity → relation → entity" triples.

[0443] The global knowledge graph is a global domain knowledge base (Neo4j graph database) that is continuously accumulated at the platform level. It contains all the disease knowledge (disease type nodes / cause nodes / treatment plan relationship edges) accumulated from all historical detection tasks, building code knowledge (guideline clause nodes), and building component knowledge (component types and common disease associations). Industry data is pre-imported during initialization and then continuously expanded with each detection task.

[0444] The confidence score for the new knowledge is determined as follows: Confidence score = Normalized number of source tasks (0~1, maximum 5 tasks corresponding to 1.0) × 0.5 + Consistency score with existing knowledge (entity alignment semantic similarity) × 0.3 + Source data quality rating (A=1.0 / B=0.8 / C=0.5 / D=0) × 0.2; The grading rules are as follows: ① Greater than 0.85 (high confidence): The triple comes from at least 3 independent tasks for cross-verification and is consistent with existing knowledge, and is automatically added to the database; ② 0.6~0.85 (medium confidence): It comes from 1~2 tasks without clear conflict, is marked for expert review, and is added to the database after the review is passed; ③ Less than 0.6 (low confidence): It has semantic contradiction with existing knowledge or the source data quality is C / D, and is not automatically added to the database, triggering the conflict detection process;

[0445] (6) Disease case data accumulation intelligence body:

[0446] The case value score is calculated as follows: Rarity × 0.4 + Educational Value × 0.3 + Data Completeness × 0.3. High-value cases are automatically recorded in a structured format, linked to multimodal evidence, labeled with multidimensional tags, and automatically associated with similar cases in the knowledge graph.

[0447] The high-value case is defined as follows: The three components of the case value score are calculated as follows:

[0448] ① Rarity = 1 - (Number of times this disease type appears in the historical case database / Total number of cases in the case database), normalized to 0~1;

[0449] ② Educational value is mapped according to severity level:

[0450] If the severity level is severe, then the educational value = 1.0;

[0451] If the severity level is "severe", then the educational value is 0.8.

[0452] If the severity level is medium, then the educational value is 0.5.

[0453] If the severity level is mild, then the educational value is 0.2;

[0454] ③ Data completeness = Number of filled fields in structured disease records / Total number of required fields;

[0455] When the score is greater than or equal to 0.7, it is judged as a high-value case and the structured case record creation process is automatically triggered;

[0456] The updated disease detection model is fed back to the disease detection agent in the understanding and analysis module; the expanded domain knowledge graph is fed back to the context building agent in the task scheduling module; and the new records in the case library are fed back to the solution planning module, forming a system-level continuous evolution closed loop.

[0457] The updated disease detection model is pushed to the disease detection agent in the understanding and analysis module, and the old model weights are replaced by a hot update (without restarting the service). The new version of the model has targeted improvements on new disease types found in the current detection task.

[0458] The expanded domain knowledge graph is fed back to the context building agent of the task scheduling module. When constructing context packages for subsequent tasks, more complete historical disease associations can be retrieved, making the prediction of typical diseases and the marking of key areas of concern more accurate.

[0459] The newly added records in the case library are fed back to the solution planning module. When the solution planning agent generates a dynamic detection solution for a new task, it searches for similar cases, optimizes the generation of additional detection items, and provides experience references for technology selection recommendations. The three paths correspond to the continuous improvement of recognition capability, context quality, and planning quality, respectively.

[0460] This embodiment provides a multi-agent collaborative intelligent building inspection system. It dynamically generates inspection plans by standardizing multi-channel tasks, schedules multiple devices to collaboratively collect data, combines multi-modal fusion and disease detection models to construct structured disease records, and completes disease mapping and twin synchronization on a 3D BIM model. Finally, through an evaluation and learning module, it achieves continuous evolution, realizing full automation and intelligence of the building inspection process. This solves the pain points of traditional building inspection, such as reliance on human experience, low efficiency, inconsistent standards, severe data silos, and difficulty in real-time collaboration and closed-loop optimization. It improves inspection efficiency and accuracy, reduces reliance on manual labor and the risk of misjudgment, and achieves dynamic optimization of inspection plans and intelligent resource matching. It also provides a visualized and traceable 3D display of diseases.

[0461] Compared with existing building inspection platforms and automated building defect documentation systems (such as DataGrid's publicly available AI Agent construction defect documentation solution), this invention provides a building intelligent inspection system based on multi-agent collaboration, which also has the following technical advantages:

[0462] Systematic multi-layer intelligent agent collaborative architecture: Existing technologies only introduce AI assistance in a single link, and each link still relies on manual connection; the present invention constructs a complete multi-layer architecture from task reception to knowledge accumulation, and each layer achieves real-time collaboration through message queues and event-driven mechanisms to achieve an end-to-end intelligent closed loop;

[0463] b) Dynamic personalized detection solutions replace fixed templates: Combining individual building characteristics (structural type, historical defects, environmental impact) and domain standards (itemized rule base), a personalized optimal detection solution is dynamically generated through a three-layer architecture of "mandatory rule layer + risk recommendation layer + efficiency optimization layer".

[0464] c) Real-time discovery-based value-driven scheduling: Through real-time interaction between the perception layer and the understanding layer, a real-time closed loop of "discovering high-risk diseases → calculating detection value → second-level dynamic replanning → optimal resource reallocation" is achieved, upgrading "execution according to plan" to "execution according to value".

[0465] d) Built-in continuous learning and evolution capabilities: Data flywheel, incremental learning and knowledge graph construction mechanisms enable the platform to automatically optimize AI models, accumulate domain knowledge and accumulate case experience, and the detection accuracy continues to improve with the number of uses.

[0466] e) Improved efficiency: End-to-end testing time is reduced by about 70% compared to a purely manual process, and the testing space coverage can reach over 98%.

[0467] Example 2

[0468] refer to Figure 2 This embodiment provides a method for intelligent house inspection based on multi-agent collaboration, which is based on the intelligent house inspection system based on multi-agent collaboration described in Embodiment 1, and includes the following steps:

[0469] S1. Standardize multi-channel tasks into standard task work orders and add them to the task queue. Then, construct a complete task context based on the standard task work orders and output the context package.

[0470] S2. Based on the context packet, sort the task queue in descending order according to the multi-dimensional weighted score of the tasks to obtain the currently executing task at the top. Then, generate a dynamic detection plan for the currently executing task based on the context packet. The dynamic detection plan includes a list of detection items, recommended detection technologies and alternative solutions for each item, execution order, estimated resource requirements, and a visualization of the detection point annotations on the 3D model. The list of detection items includes a list of mandatory detection items and a list of suggested additional detection items. The estimated resource requirements include the number of personnel (the total number of personnel required to perform this detection), the number of devices (the number of devices to be scheduled, including drones / wall-climbing robots / laser scanners), and the estimated time (the number of hours required to complete all detection items).

[0471] S3. Convert the dynamic detection scheme into an executable instruction set for multiple devices, match all available detection personnel profiles with the detection items in the detection item list to generate individual work orders and push them to mobile terminals; the multiple devices include drones, wall-climbing robots and laser scanners.

[0472] S4. Based on the executable instruction set of the multiple devices, perform multi-device collaborative acquisition to construct multimodal raw data packets, output execution logs of each device, and generate real-time event streams when a device detects an anomaly.

[0473] S5. After aligning and fusing the visible light image sequence, infrared thermal imaging image sequence and three-dimensional point cloud data in the multimodal raw data package, multimodal fused data is obtained. The multimodal fused data is then used to detect diseases through a disease detection model and enhanced with domain knowledge. A structured disease record set is constructed by combining the audio percussion hollow drum analysis results of the multimodal raw data package.

[0474] S6. Based on the structured disease record set, perform three-dimensional mapping of diseases and synchronous update of digital twin on the three-dimensional BIM model to obtain a report on the difference between the fully annotated three-dimensional BIM model and the digital twin version.

[0475] S7. Evaluate all detection data, analysis results and equipment logs of the currently executed task to obtain a detection quality management report. Based on all data of the currently executed task, update the learning evolution model, expand the knowledge graph, add new case library records and feed them back to the task scheduling module, solution planning module and understanding analysis module to form a system-level continuous evolution closed loop.

[0476] The parts of this invention not described in detail are prior art. It will be apparent to those skilled in the art that this invention is not limited to the details of the above exemplary embodiments, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects, and are intended to encompass all changes falling within the meaning and scope of equivalents within this invention.

Claims

1. A smart home health check system based on multi-agent collaboration, characterized in that, Includes the following modules: The task scheduling module is used to standardize multi-channel tasks into standard task work orders and add them to the task queue. Then, it constructs a complete task context based on the standard task work orders and outputs the context package. The scheme planning module is used to sort the task queue in descending order according to the multi-dimensional weighted score of the task based on the context packet to obtain the currently executing task at the top, and then generate a dynamic detection scheme for the currently executing task based on the context packet. The resource scheduling module is used to convert the dynamic detection scheme into an executable instruction set for multiple devices, match the profiles of all available detection personnel with the dynamic detection scheme to generate individual work orders and push them to mobile terminals; The sensing and acquisition module is used to construct multimodal raw data packets by collaboratively acquiring data from multiple devices according to the executable instruction set of the multiple devices, output the execution logs of each device, and generate a real-time event stream when a device detects an anomaly. The understanding and analysis module is used to align and fuse the multimodal raw data packets to obtain multimodal fused data. The multimodal fused data is then used to detect diseases through a disease detection model and enhanced with domain knowledge. The data is combined with the results of close-range macro photography, audio, and hollow drum analysis to construct a structured disease record set. The mapping and twin module is used to perform three-dimensional mapping and digital twin synchronous updates of defects on the three-dimensional BIM model based on the structured defect record set, and to obtain a report on the differences between the fully annotated three-dimensional BIM model and the digital twin version. The evaluation and learning module is used to evaluate all the detection data, analysis results, and equipment logs of the currently executed task to obtain a detection quality management report.

2. The system according to claim 1, characterized in that, The system also includes: The evaluation and learning module also includes updating the learning evolution model, expanding the knowledge graph, and adding new case library records based on all data of the currently executed task, and feeding them back to the task scheduling module, solution planning module, and understanding and analysis module to form a system-level continuous evolution closed loop. The path replanning module is used to generate corresponding incremental device instructions and send them to the corresponding devices based on the real-time event stream generated by the sensing and acquisition module or the discovery of high-risk diseases in the structured disease record set, and to store the path change log in the device execution log.

3. The system according to claim 1, characterized in that, The task scheduling module includes: The task receiving and parsing agent continuously listens for tasks from multiple channels on the online platform frontend via a message queue, performs different standardized transformations on different types of channel tasks, obtains standard task work orders, and writes them into a task queue to be sorted. The standard task work order includes: task number, basic building information, detection type, deadline, and constraints. The basic building information includes: unique building identifier, building address, usage type, building area, and number of floors. The constraints include a time window and a budget limit. A context-based intelligent agent is used to retrieve historical building inspection reports corresponding to the standard task work order from the time-series database, construct a time series of defects to predict historical trends of defects and mark key areas of concern, obtain the spatial partitioning and defect density heat map of the 3D BIM model of the standard task work order, load the itemized rule base corresponding to the standard task work order, then integrate the building's basic information, component information, and historical defect records into a building knowledge graph fragment, calculate the context integrity score to obtain a list of missing data markers, and construct a context package.

4. The system according to claim 3, characterized in that, The scheme planning module specifically includes: The priority sorting agent is used to combine the context packet and the task queue to be sorted in the current system to calculate a multi-dimensional weighted score for each task, sort them in descending order according to the multi-dimensional weighted score, and take the task at the top of the current queue as the current task to be executed. The formula for calculating the multi-dimensional weighted score P is as follows: P = 0.30×Sg + 0.25×Sf + 0.20×St + 0.15×Sc + 0.10×Sb; Among them, the Sg urgency score is obtained based on the detection type mapping; Sf represents the risk level score, Sf = min(100, historical severity of defects × building age coefficient × structural type risk weight); building age coefficient = min(2.0, 1.0 + floor((current year - construction year) / 10) × 0.1); St represents the time constraint score, St = min(100, max(0, 100 - (deadline - current date) / standard duration × 100)); Sc is the resource availability score, Sc = equipment idle rate × 0.4 + personnel availability rate × 0.4 + (1 - traffic distance normalization) × 0.2; traffic distance normalization is to divide the traffic distance of the currently executing task by the maximum traffic distance of all tasks in the current pending task queue; Sb is the batch efficiency score, which is obtained by encoding the coordinates of the house address of the task into a geohash string, calculating the longest common prefix length of the geohash strings of the current task and other tasks in the queue, and taking the maximum value and normalizing it to [0,100]. A dynamic scheme generation agent is used to generate a list of mandatory inspection items based on an itemized rule base, and to determine a list of recommended additional inspection items based on historical disease trend predictions and a list of missing data markers. A directed acyclic graph is established based on the technical dependencies between the inspection items in the mandatory and recommended additional inspection item lists. The optimal execution order of the inspection items is determined through topological sorting, and constraint solving is performed within the time window and budget limit to obtain the estimated resource requirements, thus generating an inspection scheme. The estimated resource requirements include the number of personnel, the number of devices, and the estimated time consumption. The guidelines compliance verification intelligent agent is used to verify the compliance of the detection scheme through rules; The detection technology selection intelligent agent is used to select detection technologies for the detection scheme based on the contextual constraint data of the currently executed task through the detection technology selection decision tree, and to obtain the recommended detection technologies and alternative schemes for each detection item in the detection scheme; the contextual constraint data includes building height, surrounding space conditions, access conditions, budget limit and time window; The dynamic testing scheme includes a list of testing items, recommended testing technologies and alternative solutions for each item, execution order, estimated resource requirements, and a visualization of testing point markings on a 3D model; the list of testing items includes a list of mandatory testing items and a list of recommended additional testing items.

5. The system according to claim 4, characterized in that, The detection technology selection decision tree selects the appropriate detection technology based on the contextual constraints of the current task, resulting in recommended detection technologies and alternative solutions for each detection item within the detection plan. Driven by real-time reading of current task context constraints using a detection technology selection decision tree, recommendations are output based on the following four-level progressive filtering of each detection item in the detection scheme: The first layer is disease type filtering: querying the set of technologies that can detect a disease by disease type; The second layer is on-site constraint filtering: technologies that do not meet the on-site conditions are excluded sequentially from the technology set; The third layer is precision matching filtering: techniques with insufficient precision in the technology set are excluded according to the required measurement precision, resulting in a filtered technology set. The fourth layer is cost-benefit ranking: Each technology in the set of technologies that have passed the screening of the first three layers is calculated and ranked. The two technologies with the highest comprehensive technical scores are selected as the optimal solution and the alternative solution, respectively, and the technology with the highest cost score is selected as the economic solution.

6. The system according to claim 5, characterized in that, The comprehensive technical score is obtained through the following steps: The first step is to independently calculate the estimated total time required for the detection technology: Estimated total testing time = Equipment deployment time + Equipment data acquisition time; Equipment acquisition time = Effective coverage area of ​​the detection item ÷ Equipment area acquisition rate; The second step is to calculate the detection area efficiency of the technology based on the estimated total detection time: Detection area efficiency = Effective coverage area of ​​the detection item / Estimated total detection time; The third step is to calculate the scheduling cost of computing technology. Dispatch cost = Number of devices × Device labor cost × Estimated total testing time + Number of personnel × Personnel labor cost × Estimated total testing time + Transportation distance × Transportation mileage cost; Step 4: Calculate the Comprehensive Skills Score (CBS). CBS = w1 × normalized value of detection area efficiency - w2 × normalized value of estimated total detection time - w3 × normalized value of scheduling cost; Where w1 is the detection area efficiency weight; w2 is the expected total detection time weight; and w3 is the scheduling cost weight. Then, based on the Comprehensive Technical Score (CBS) in descending order, the technology with the highest CBS is the optimal solution, and the technology with the second highest CBS is the alternative solution; the technology with the highest score of the normalized value of w3×scheduling cost is the economic solution.

7. The system according to claim 6, characterized in that, The resource scheduling module includes: A device instruction generation agent is used to match specific resources with the estimated number of personnel and equipment as upper limits, and to transform the dynamic detection scheme into a set of executable instructions for multiple devices. The set of executable instructions for multiple devices specifically includes: Drone flight path instructions: Based on the normal vectors of each detection surface in the 3D BIM model, view planning is performed to generate a sequence of waypoints, thereby obtaining the drone flight path instructions; The path instructions for the wall-climbing robot are as follows: Extract the building's outer surface from the 3D BIM model, mark the traversable and inaccessible areas, take key detection points as necessary nodes on the traversable map, solve the optimal traversal path through the traveling salesman problem, set the detection action instructions according to the optimal traversal path, and obtain the path instructions for the wall-climbing robot. Laser scanner site instructions: Based on visibility analysis, the set coverage algorithm is used to solve for the minimum number of sites, and generate the location coordinates, scanning range and point density parameters of each scanner site; The intelligent agent for dispatching work orders is used to match specific resources with the upper limit constraints of the number of personnel and equipment. It matches the profiles of all available testing personnel with the manual testing requirements in the list of testing items, generates individual work orders, and pushes them to mobile terminals. The multiple devices include drones, wall-climbing robots, and laser scanners.

8. The system according to claim 7, characterized in that, The sensing and acquisition module specifically includes: The drone autonomous inspection intelligent agent is used to control the drone to perform inspections according to the drone's flight path instructions. It uses a 4K visible light camera and an infrared thermal imaging camera to simultaneously acquire visible light images and infrared thermal imaging images, and obtain visible light image sequences and infrared thermal imaging image sequences. The wall-climbing robot control agent is used to control the wall-climbing robot to move according to the wall-climbing robot path instructions, and to perform close-range macro photography and robotic arm tapping at key detection points to obtain close-range macro photography images and tapping audio detection data, thereby obtaining close-range macro photography image sequence and tapping audio detection data sequence. A 3D laser scanning intelligent agent is used to generate stitched 3D point cloud data by scanning station by station according to the instructions of the laser scanner station; The multimodal raw data package includes visible light image sequences, infrared thermal imaging image sequences, three-dimensional point cloud data, close-range macro images, and tapping audio detection data sequences.

9. The system according to claim 8, characterized in that, The understanding and analysis module specifically includes: A multi-source data alignment agent is used to register infrared thermal imaging images in an infrared thermal imaging image sequence to the coordinate system of visible light images in a visible light image sequence to obtain registered image pairs. Based on the alignment quality score and image quality score of the registered image pairs, the agent performs image quality scene segmentation. Then, using the spatial coordinate system of the 3D point cloud data as a reference, the agent performs multi-channel fusion of the registered image pairs and the 3D point cloud data to obtain multimodal fused data. The image quality scenes include high-quality dual-modal scenes and low-quality single-modal scenes. The disease detection intelligent agent is used to obtain a list of disease detection results by selecting mid-term fusion or late-term fusion based on the image quality and scene of the preprocessed registered image pairs through the disease detection model. The domain knowledge-enhanced intelligent agent is used to load spatial and temporal contexts for each disease in the disease detection result list, query the knowledge graph subgraph with the house knowledge graph fragment reference ID as the key, obtain the historical disease association relationship of the same component, execute IF-THEN rule reasoning, and obtain the corrected confidence and reasoning link record. A disease geometry quantification intelligent agent is used to extract the geometric attributes of each disease detection result in the disease detection result list; the geometric attributes include length, width, area, depth and their uncertainty; An audio percussion hollow drum analysis agent is used to receive percussion audio detection data sequences from multimodal raw data packets, perform signal preprocessing, fast Fourier transform spectrum analysis, hollow drum determination, and spatial mapping to obtain an audio hollow drum detection result set.

10. A method for intelligent health inspection of houses based on multi-agent collaboration, characterized in that, The intelligent home health check system based on multi-agent collaboration according to any one of claims 1-9 includes the following steps: S1. Standardize multi-channel tasks into standard task work orders and add them to the task queue. Then, construct a complete task context based on the standard task work orders and output the context package. S2. Based on the context packet, sort the task queue in descending order according to the multi-dimensional weighted score of the task to obtain the currently executing task at the top of the queue, and then generate a dynamic detection scheme for the currently executing task based on the context packet. S3. Convert the dynamic detection scheme into an executable instruction set for multiple devices, match all available detection personnel profiles with the detection items in the detection item list to generate individual work orders and push them to mobile terminals; S4. Based on the executable instruction set of the multiple devices, perform multi-device collaborative acquisition to construct multimodal raw data packets, output execution logs of each device, and generate real-time event streams when a device detects an anomaly. S5. After data alignment and fusion of the original multimodal data packets, multimodal fused data is obtained. The multimodal fused data is then used to detect diseases through a disease detection model and enhanced with domain knowledge. A structured disease record set is constructed by combining the results of close-range macro photography, audio, and hollow drum analysis. S6. Based on the structured disease record set, perform three-dimensional mapping of diseases and synchronous update of digital twin on the three-dimensional BIM model to obtain a report on the difference between the fully annotated three-dimensional BIM model and the digital twin version. S7. Evaluate all detection data, analysis results and equipment logs of the currently executed task to obtain a detection quality management report. Based on all data of the currently executed task, update the learning evolution model, expand the knowledge graph, add new case library records and feed them back to the task scheduling module, solution planning module and understanding analysis module to form a system-level continuous evolution closed loop.

Images