Fire engineering robot installation system based on multi-modal model

Abstract

The application relates to the field of firefighting technology, and provides a firefighting engineering robot installation system based on a multi-modal model, which comprises an installation mechanical arm, a task analysis module, a space modeling module, a layout planning module, an action control module and a quality verification module; the task analysis module is used for analyzing an installation task, fusing multi-modal inputs of drawings and BIM models, extracting key information and outputting structured task instructions; the space modeling module performs three-dimensional structure modeling on a construction site to form a space point model, identifies building components and a constructible area, completes high-precision registration of drawing structures and real scenes, and analyzes environmental constraint conditions; the layout planning module generates a firefighting equipment layout scheme on the basis of meeting specification requirements according to the task instructions and the space point model; the action control module converts the layout scheme and the type of installed firefighting equipment into a sequence of action instructions executable by the robot; and the quality verification module automatically detects an installation result.

Description

Technical Field

[0001] This invention relates to the field of fire protection technology, and in particular to a fire protection engineering robot installation system based on a multimodal model. Background Technology

[0002] During building construction, the installation of fire protection systems is a crucial step in ensuring the safe operation of a building. Current installation methods generally rely on manual labor, requiring workers to refer to blueprints for equipment placement, positioning, and fixation. However, due to the complexity of building structures, the inherent subjectivity of interpreting blueprints, and the constantly changing spatial environment, installation often results in insufficient accuracy, low efficiency, and even violations of regulations and repetitive construction.

[0003] For example, the prior art disclosed in Chinese patent CN114432632B is a height positioning device for installing fire boxes. Although it provides a device and method for positioning fire boxes, it neglects the analysis and evaluation of the placement of fire boxes in the entire passage, resulting in unreasonable placement and hindering rapid response.

[0004] In addition, the existing technology also has the following drawbacks:

[0005] 1. Building spaces often contain diverse structural features such as beams, recesses, ceilings, and embedded pipes. Traditional robots struggle to accurately identify workable surfaces, obstructions, and spatial forms in construction areas, and are unable to effectively align design drawings with the on-site structure.

[0006] 2. The placement of fire protection equipment has strict requirements on spacing, angle and accessibility. Currently, it still relies heavily on manual measurement and experience-based placement, lacking the ability to adapt and optimize globally based on spatial structure and equipment objectives. This can easily lead to uneven installation density, wasted space or redundant pipelines.

[0007] 3. Most existing installation robots are execution-type devices, lacking autonomous planning capabilities based on task understanding and spatial perception, as well as task collaboration mechanisms among multiple robots, which easily leads to problems such as action conflicts, path overlap, and decreased efficiency.

[0008] 4. Fire protection installation and construction have high requirements for positioning accuracy, firmness and wiring specifications. However, at present, the main method after construction is to rely on manual visual inspection or spot checks, which has low inspection efficiency, high error rate and lacks automatic correction mechanism. Once a deviation occurs, rework is often required, which greatly reduces construction efficiency and cost control capabilities.

[0009] This invention addresses the common problems in the field, such as the lack of methods for determining the location of fire protection engineering projects, reliance on manual positioning which easily leads to high labor intensity, lack of intelligent layout decision-making, poor collaborative scheduling capabilities, and low installation efficiency. Summary of the Invention

[0010] The purpose of this invention is to address the shortcomings of current systems by proposing a fire protection engineering robot installation system based on a multimodal model.

[0011] To overcome the shortcomings of the prior art, the present invention adopts the following technical solution:

[0012] A fire engineering robot installation system based on a multimodal model is disclosed. The fire engineering robot installation system includes a server and an installation robotic arm. The fire engineering robot installation system also includes a task parsing module, a spatial modeling module, a layout planning module, a motion control module, and a quality verification module. The server is connected to the task parsing module, the spatial modeling module, the layout planning module, the motion control module, and the quality verification module, respectively.

[0013] The task parsing module is used to parse installation tasks, integrate multimodal inputs from drawings and BIM models, extract key information such as equipment type, target area, and layout parameters, and output structured task instructions.

[0014] The spatial modeling module performs three-dimensional structural modeling of the construction site to form a spatial point model, identifies key building components and constructible areas, completes high-precision registration between the drawing structure and the actual structure, and analyzes environmental constraints.

[0015] The layout planning module generates a fire equipment placement plan based on task instructions and spatial point models, while meeting the requirements of the specifications.

[0016] The motion control module converts the layout plan corresponding to the installation location and the type of fire-fighting equipment installed into a sequence of motion instructions that the robot can execute, and avoids task coordination and path conflicts for the robot.

[0017] The quality verification module automatically detects the installation results. If an anomaly is detected, it can trigger automatic correction or replan the installation task.

[0018] Optionally, the task parsing module includes a drawing parsing unit, a specification extraction unit, and a multimodal fusion unit. The drawing parsing unit parses CAD drawings / BIM models to extract installation information of fire protection equipment; the specification extraction unit parses national / industry specification documents to extract standard parameters of fire protection equipment; and the multimodal fusion unit integrates drawings, specifications, operational requirements, and correction information to output task target information in a unified format.

[0019] Optionally, the spatial modeling module includes a 3D reconstruction unit, a spatial structure recognition unit, a drawing registration unit, and an environmental constraint analysis unit. The 3D reconstruction unit collects real-time data of the current space based on LiDAR and generates a spatial point model. The spatial structure recognition unit performs semantic segmentation on the spatial point model to identify ceiling, wall, and manhole structures. The drawing registration unit spatially aligns the drawing structure with the on-site structure and detects errors or misalignments. The environmental constraint analysis unit analyzes the constraints of construction obstruction and angle limitations in the on-site environment to form the boundary conditions for subsequent layout planning.

[0020] Optionally, the layout planning module includes a candidate point generation unit, an intent mapping unit, a constraint calculation unit, and an optimized layout unit. The candidate point generation unit generates installation candidate points that meet basic specifications within the constructable area. The intent mapping unit constructs the installation candidate points into a spatial intent field to guide the formation of a priority placement sequence. The constraint calculation unit filters the candidate points in the priority placement sequence according to the installation specifications. The optimized layout unit optimizes the total path length, resource utilization rate, and equipment balance index based on meeting the specifications to output the final layout scheme.

[0021] Optionally, the motion control module includes a path planning unit, a motion sequence generation unit, and a state feedback monitoring unit. The path planning unit generates obstacle avoidance paths for the robot from its current position to each installation point; the motion sequence generation unit breaks down each installation task into specific motion instructions; and the state feedback monitoring unit monitors the real-time status of the installation robotic arm.

[0022] Optionally, the quality verification module includes an image comparison unit, a mechanical detection unit, an error judgment unit, and a task reflow trigger. The image comparison unit captures images of the installed parts and compares them with drawings or BIM models. The mechanical detection unit uses a torque sensor to detect whether the fire-fighting equipment is installed securely. The error judgment unit analyzes the images and sensor data to determine whether the error threshold is exceeded. If the verification fails, the task reflow trigger sends a re-execution command to the layout planning module or the motion control module.

[0023] Optionally, the 3D reconstruction unit includes a laser scanning acquisition subunit, a coordinate transformation subunit, a SLAM stitching subunit, and a point cloud optimization processing subunit. The laser scanning acquisition subunit acquires spatial depth information of the construction site in real time based on a rotating lidar; the coordinate transformation subunit converts the acquired polar coordinate data into spatial Cartesian coordinate points; the SLAM stitching subunit performs attitude correction and spatial stitching of continuous frame point clouds based on inertial navigation and laser odometry; and the point cloud optimization processing subunit performs voxel filtering, ground removal, and outlier removal on the point cloud to form a spatial point cloud model that can be used for structure recognition and path planning.

[0024] Optionally, the parameters by which the image comparison unit compares the drawings with the BIM model include position, direction, and angle.

[0025] Optionally, the action commands include raising the boom, drilling, installing, and testing.

[0026] Optionally, the installation information of the fire-fighting equipment includes the equipment type, serial number, and location label.

[0027] The beneficial effects achieved by this invention are:

[0028] 1. By combining the spatial modeling module and the layout planning module, the installation points generated during the layout planning process are only laid out within the actual workable area, avoiding problems such as structural obstruction and spatial errors, and ensuring the feasibility and accuracy of the layout plan.

[0029] 2. Through the cooperation of the task parsing module and the layout planning module, the system can automatically construct a layout strategy based on the equipment type, installation requirements and layout parameters extracted from the task, avoiding manual configuration and repeated debugging, and ensuring that the planning results are automatically generated under the premise of meeting the task objectives;

[0030] 3. By combining the layout planning module and the motion control module, the planned installation points can be converted into executable motion paths in real time, avoiding construction failures due to unreachable locations or path conflicts, and ensuring the continuity and path safety of robot execution.

[0031] 4. Through the cooperation of the motion control module and the quality verification module, the system can perform automated quality inspection immediately after the installation task is completed, and trigger backflow correction operation based on the inspection results to ensure closed-loop control of installation effect and quality consistency.

[0032] 5. By combining the task parsing module and the spatial modeling module, the equipment intent in the drawing task can be accurately mapped to the spatial structure of the actual construction site, achieving spatial consistency between the task objective and the real structure, and ensuring the locationability of equipment deployment and adaptability to the construction environment.

[0033] 6. Through the coordinated operation of the task parsing module, spatial modeling module, layout planning module, motion control module, and quality verification module, the fire protection engineering installation task can achieve fully automated processing from multimodal task identification, spatial environment modeling, standard constraint placement, automatic motion control to quality self-inspection closed loop, ensuring that the system has intelligent scheduling capability and self-repair capability in the four-in-one system of task-space-control-verification.

[0034] 7. Through the layered collaboration of the task parsing module, spatial modeling module, layout planning module, motion control module and quality verification module, the installation task can be fully automated from task input to construction completion, thereby improving the system's adaptability, fault tolerance and deployment efficiency. Attached Figure Description

[0035] The invention will be further understood from the following description taken in conjunction with the accompanying drawings. The components in the drawings are not necessarily drawn to scale, but rather the emphasis is on illustrating the principles of the embodiments. In different views, the same reference numerals designate the same parts.

[0036] Figure 1 This is a schematic diagram of the overall block shape of the present invention.

[0037] Figure 2 This is a block diagram of the multimodal fusion unit of the present invention.

[0038] Figure 3 This is a schematic diagram of the spatial modeling module of the present invention.

[0039] Figure 4 This is a flowchart illustrating the layout planning module of the present invention.

[0040] Figure 5 This is a flowchart illustrating the motion control module of the present invention. Detailed Implementation

[0041] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can understand the advantages and effects of the present invention from the content disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the spirit of the present invention. Furthermore, the accompanying drawings of the present invention are for simple illustrative purposes only and are not depictions of actual dimensions; this is stated beforehand. The following embodiments will further describe the relevant technical content of the present invention in detail, but the disclosed content is not intended to limit the scope of protection of the present invention.

[0042] Example 1: According to Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5As shown, this embodiment provides a fire engineering robot installation system based on a multimodal model. The fire engineering robot installation system includes a server and an installation robotic arm. The fire engineering robot installation system also includes a task parsing module, a spatial modeling module, a layout planning module, a motion control module, and a quality verification module. The server is connected to the task parsing module, spatial modeling module, layout planning module, motion control module, and quality verification module respectively, and stores the intermediate data and process data of the task parsing module, spatial modeling module, layout planning module, motion control module, and quality verification module in the server's database for querying and retrieval.

[0043] The task parsing module is used to parse installation tasks, integrate multimodal inputs from drawings and BIM models, extract key information such as equipment type, target area, and layout parameters, and output structured task instructions.

[0044] The spatial modeling module performs three-dimensional structural modeling of the construction site to form a spatial point model, identifies key building components and constructible areas, completes high-precision registration between the drawing structure and the actual structure, and analyzes environmental constraints.

[0045] The layout planning module generates a fire equipment placement plan based on task instructions and spatial point models, while meeting the requirements of the specifications.

[0046] The motion control module converts the layout plan corresponding to the installation location and the type of fire-fighting equipment installed into a sequence of motion instructions that the robot can execute, and avoids task coordination and path conflicts for the robot.

[0047] The quality verification module automatically detects the installation results. If an anomaly is detected, it can trigger automatic correction or replan the installation task.

[0048] The system also includes a central processing unit (CPU), which is connected to the task parsing module, spatial modeling module, layout planning module, motion control module, and quality verification module. The CPU provides centralized control over these modules, and the CPU stores the control data in the database, thereby improving the reliability and accuracy of the fire protection system installation.

[0049] Optionally, the task parsing module includes a drawing parsing unit, a specification extraction unit, and a multimodal fusion unit. The drawing parsing unit parses CAD drawings / BIM models to extract installation information of fire protection equipment, including equipment type, number, and location label. The specification extraction unit parses national / industry specification documents to extract standard parameters of fire protection equipment, including installation spacing, dimensions, and height. The multimodal fusion unit integrates drawings, specifications, and operational requirements to output task target information in a unified format.

[0050] Specifically, the drawing parsing unit receives CAD drawings or BIM model files as input, classifies and identifies layers and component types in the drawings, and extracts the type identifier, number information, and spatial location label of the target fire-fighting equipment. During the attribute reading process of component information in the BIM model, it needs to obtain information including the equipment instance's ID, category, and placement location (such as XYZ coordinates or the name of the space it resides in), thereby generating the drawing extraction result dataset T. 图纸 .

[0051] The specification extraction unit imports national or industry fire protection design specification documents and retrieves technical parameter entries related to fire protection equipment types using natural language processing models or rule templates. It then extracts and standardizes key parameter values ​​for each type of equipment, such as recommended installation spacing, equipment dimensions, and installation height limits, and outputs a structured parameter set T. 规范 :

[0052] T 规范 ={fire type, spacing, dimensions, height};

[0053] The multimodal fusion unit matches the extracted drawing results with the specification parameter table for equipment type and number; if it finds that some parameter items are missing in the drawing information, the system will call the corresponding fields in the specification to complete them; if it receives operation requests from the operator (such as area priority, installation adjustment, etc.), the fusion unit merges this information with the drawing / specification data, identifies and updates the relevant equipment parameter values; if there is information conflict (such as the drawing height is 2.5m, the specification recommends 2.8m), the system selects according to the preset priority, with the default priority being: user command > drawing annotation > specification parameter;

[0054] The final output is a set of structured installation task target information:

[0055] T 任务信息 = {Component Number, Equipment Type, Region, Installation Parameters};

[0056] Simultaneously, based on the mapping relationship between the "Region" field and the regional space in the drawing, the system extracts the target deployment area of ​​each type of equipment into a set of three-dimensional center points, forming a target deployment area set:

[0057] R target ={r i =(x i ,y i ,z i )}

[0058] The set of target deployment areas will serve as the basis for intent mapping in the subsequent layout planning module, used to generate spatial priority deployment guidance information.

[0059] Optionally, the spatial modeling module includes a 3D reconstruction unit, a spatial structure recognition unit, a drawing registration unit, and an environmental constraint analysis unit. The 3D reconstruction unit collects real-time data of the current space based on LiDAR and generates a spatial point model. The spatial structure recognition unit performs semantic segmentation on the spatial point model to identify ceiling, wall, and manhole structures. The drawing registration unit spatially aligns the drawing structure with the on-site structure and detects errors or misalignments. The environmental constraint analysis unit analyzes the constraints of construction obstruction and angle limitations in the on-site environment to form the boundary conditions for subsequent layout planning.

[0060] Optionally, the 3D reconstruction unit includes a laser scanning acquisition subunit, a coordinate transformation subunit, a SLAM stitching subunit, and a point cloud optimization processing subunit. The laser scanning acquisition subunit acquires spatial depth information of the construction site in real time based on a rotating lidar; the coordinate transformation subunit converts the acquired polar coordinate data into spatial Cartesian coordinate points; the SLAM stitching subunit performs attitude correction and spatial stitching of continuous frame point clouds based on inertial navigation and laser odometry; and the point cloud optimization processing subunit performs voxel filtering, ground removal, and outlier removal on the point cloud to form a spatial point cloud model that can be used for structure recognition and path planning.

[0061] Specifically, the three-dimensional reconstruction unit performs the following steps to form a spatial point cloud model that can be used for structure recognition and path planning;

[0062] S1. Activate the laser scanning acquisition subunit to trigger the rotating lidar device and enter the static scanning mode; the lidar emits laser beams at a preset angle (e.g., 360° horizontal + 16 vertical lines). Each laser beam is reflected after encountering an object, returning the ranging value r and the reflection intensity I. At the same time, the horizontal angle θ, vertical angle φ, and timestamp t of each laser point are recorded.

[0063] The original laser point set obtained after collection is as follows:

[0064]

[0065] In the formula, N is the number of laser points, p i polar r represents the raw polar coordinate data item of a point.i θ is the straight-line distance from the i-th laser point to the center of the lidar sensor; it is obtained by calculating the laser time-of-flight. i Let φ be the scanning angle of the i-th laser beam in the horizontal direction (usually around the Z-axis). i Let I be the elevation angle of the i-th laser beam relative to the horizontal plane. i Let t be the intensity of the signal returned after the i-th laser beam is reflected. i The time when the i-th laser beam is collected.

[0066] S2, the coordinate transformation subunit will transform the polar coordinate point p i polar Mapping to Cartesian coordinates using the following function f: The formula is as follows:

[0067]

[0068] In the formula, r i θ is the straight-line distance from the i-th laser point to the center of the lidar sensor; it is obtained by calculating the laser time-of-flight. i Let φ be the scanning angle of the i-th laser beam in the horizontal direction (usually around the Z-axis). i Let be the elevation angle of the i-th laser beam relative to the horizontal plane.

[0069] After processing according to the above formula, the output point cloud set is:

[0070] P cartesian ={p i =(x i ,y i ,z i ,I i ,t i )|i=1,2,...,N};

[0071] S3, the SLAM stitching subunit preprocesses the electric cloud collection. The preprocessing operations include voxel filtering, outlier removal, and ground removal.

[0072] Specifically, voxel filtering is performed on the point cloud set, and the voxel resolution δ is set. x ,δ y ,δ z It is generally set to δ = 0.05 meters;

[0073] For each point pi, calculate its corresponding voxel coordinates:

[0074]

[0075] All points with the same voxel coordinates are grouped into one class, denoted as V. j ;

[0076] For the point set V within each voxel j Calculate its centroid:

[0077]

[0078] In the formula, V j Let V be the set of points in the j-th voxel, p∈V j For a point p to belong to this voxel, that is, each point p is a three-dimensional vector p = (x, y, z), c j Let j be the representative point of the j-th voxel.

[0079] The overall formula in the above equation means: for voxel V j For all points p, calculate their geometric center (i.e., the centroid) as the representative point c of the voxel. j .

[0080] The point cloud set after the above filtering is as follows:

[0081] P voxel ={c j |j=1,2,…,M};

[0082] After voxel filtering, outlier removal is performed, specifically:

[0083] For each point p i Find its k-neighborhood (e.g., k=20);

[0084] Calculate the average Euclidean distance d between the point and its neighbors. i :

[0085]

[0086] In the formula, p i =(x i ,y i ,z i ), p j =(x j ,y j ,z j ), where ||*||2 is the three-dimensional Euclidean distance;

[0087] For all d i Calculate the global mean μ and standard deviation σ;

[0088] Among them, all d i The global mean is calculated using the following formula:

[0089]

[0090] In the formula, N is the total number of points in the point cloud;

[0091] All d i The global standard deviation σ is calculated using the following formula:

[0092]

[0093] In the formula, N is the total number of points in the point cloud; μ is the total number of points in the d-values. i Calculate the global mean.

[0094] Set the rejection threshold parameter α, if the d of a certain point... i If the value is greater than μ+α·σ, then it is determined to be an outlier.

[0095] In this embodiment, the value range of the rejection threshold parameter α is 1.0 to 1.5; Furthermore, this embodiment provides an example of the value of the rejection threshold parameter α:

[0096] 1) In scenarios where fire protection engineering robots are working in suspended ceiling spaces (with relatively uniform point cloud density and a large number of planar areas (such as ceilings and air conditioning panels)), the threshold parameter α = 1.0 is discarded.

[0097] 2) In construction sites with complex scaffolding and piles of debris (the point cloud contains regions of various densities, and there are obvious local stray points, but some points may be the edge of the structure), the threshold parameter α = 1.5 is removed.

[0098] 3) In scenes with extremely dense points and rich details, the threshold parameter α = 0.8 is removed.

[0099] In summary, the specific rejection threshold parameter α needs to be set according to actual needs and input from the human-computer interaction interface. This is a technical method well known to those skilled in the art, and therefore will not be described in detail in this embodiment.

[0100] After outlier removal, the denoised point set is obtained:

[0101] P denoised ={p i ∈P voxel ∣d i ≤μ+α·σ};

[0102] Next, the denoised point set P denoised Set a minimum height threshold Z along the Z-axis. min In this embodiment, Z is set to... min =0.1 meters, delete all points: P filtered ={p i ∈P denoised ∣z i >z min};

[0103] The final point cloud set of spatial points is:

[0104] P clean =P filtered ;

[0105] S4. Obtain the final point cloud set. The point cloud optimization processing subunit performs planar structure extraction. Before extracting the planar structure, the point cloud optimization processing subunit first sets the initial parameters for planar fitting, including the minimum number of plane support points n. min That is, each effective plane fitted should contain at least n min The number of points is set to 50 in this embodiment; the maximum tolerance distance δ from the points to the fitting plane is set. plane The value is set to 0.02 meters to determine whether a point belongs to the plane; the maximum number of iterations T for the fitting algorithm is set to 1000 in this embodiment to control the number of RANSAC iterations. Next, the system uses the preprocessed point cloud data P obtained in the previous step... clean Initialize to the current set of points to be processed, P remain This is used to initiate the planar recognition process. Simultaneously, the planar region number variable k = 1 is initialized to record the sequential number of each extracted planar region.

[0106] Subsequently, the point cloud optimization processing subunit enters the iterative extraction process. In each round, from P... remain By randomly sampling three points, a plane equation π is fitted. k : ax + by + cz + d = 0, where a, b, c are the normal vector components of the fitted plane, and d is the offset constant of the plane in three-dimensional space;

[0107] Next, calculate P. remain The distance from all points to the plane is used to determine whether it falls within the tolerance range δ. plane Inside. If a point satisfies the following conditions:

[0108] Then it is assigned to the current set of points R in the plane. k .

[0109] If R k The number of points satisfies: |R k |≥n min If the plane is considered valid, it is taken as the kth identifiable region, and R is set as the identifiable region. k All points from P remain Remove from the list. Increment the region number k by 1 and continue to the next iteration. If the current fitted plane does not meet the point count requirement, discard the result of this round and do not retain it. When the remaining points are insufficient to fit a new plane, end the iteration process. For each successfully extracted region R... kThe point cloud optimization processing sub-unit calculates its corresponding normal vector n. k = (a, b, c), and perform semantic classification based on the normal direction: when |n k·z When |>0.9, it is determined to be a horizontal structure, such as a ceiling or floor slab; when |n k·z When |<0.1, it is judged as a vertical structure, such as a wall surface; other angle cases are judged as inclined surfaces or unrecognized areas, and further judgment or marking is performed depending on the scene: other areas.

[0110] Finally, for each region R k ={p=(x i ,y i ,z i ) |i=1,2,…,n k} Calculate its spatial bounding box, and encapsulate the point set, plane equation parameters, normal vector, semantic labels and bounding box information into a structured output to form the output result.

[0111] The output result is used as a spatial structure model. Where R k For point set, For the normal vector, bbox k This represents the bounding box.

[0112] In this embodiment, for each extracted planar region, the spatial bounding box of that region is calculated for subsequent point placement feasibility assessment. Specifically, the minimum and maximum values ​​of all points within the region along the x, y, and z axes are calculated to form the axis-aligned bounding box of that region (taking an axis-aligned bounding box as an example, meaning the construction structure (walls, ceilings) is inherently axis-aligned in the architectural design). The calculation method is as follows:

[0113]

[0114] In the formula, R k Let p be the set of points in the k-th planar region. i ∈R k Let i be the i-th point in the point set, (x i y i z i Let x be the spatial coordinate value of the i-th point. min x max For planar region R k Minimum and maximum values ​​in the X-axis direction, y min y max For planar region R k Minimum and maximum values ​​in the Y-axis direction, z min , z max For planar region R kMinimum and maximum values ​​in the Z-axis direction;

[0115] The bounding box constructed in this way is used to define the spatial occupancy of each structural area and to provide spatial boundary basis for subsequent installation tasks such as placement analysis and path obstacle avoidance planning.

[0116] The spatial structure identification unit calculates the normal vector n for each region. k The dot product with the vertical unit vector z = (0,0,1):

[0117]

[0118] If |n k,z If | > 0.9, then it is a horizontal plane, and also higher than the set height threshold Z. max K ;

[0119] If |n k,z If | < 0.1, then it is a vertical surface and is considered a wall surface;

[0120] If the boundary location is close to the area of ​​the structural shaft, it is considered to be the surface of the well.

[0121] Convert the above judgment result into a semantic tag field l k :

[0122]

[0123] Thus, the set of structured semantic regions is output as follows:

[0124] The drawing registration unit matches the corresponding installation surface area with the equipment type (such as sprinkler head, smoke detector) and semantic structure tag, and performs drawing registration according to the following steps:

[0125] S11. Type semantic matching and spatial range matching:

[0126] 1) Type semantic matching:

[0127] If the fire-fighting equipment in the drawing is a sprinkler head, the point cloud optimization processing subunit will only look for the corresponding mounting surface in the structural area with the semantic label of ceiling.

[0128] If the fire protection equipment is a smoke detector, it should also be installed in the ceiling area;

[0129] If the fire-fighting equipment is a fire hydrant or other wall-mounted equipment, then the structural area should match the wall type.

[0130] 2) Spatial range matching

[0131] Determine the location of fire-fighting equipment p i图纸 Is it in a certain area of ​​bbox? k Inside, or with p i 图纸 Center C k If the distance is within the threshold δ, then the location of the fire-fighting equipment and the structural surface are matched;

[0132] S12. Calculate the spatial alignment transformation:

[0133] Obtain the set of fire equipment deployment locations (geometry from drawings and models) and the set of structural surface points (point cloud on site), and construct an optimization objective function:

[0134]

[0135] And iteratively solve for the optimal transformation: T * =(R * ,t * );

[0136] In the formula, T * For optimal spatial rigid body transformation, R * For the optimal rotation matrix, t * The optimal translation vector;

[0137] In addition, the above also satisfies:

[0138] In the formula, q j Let p be the j-th point on the component model in the drawing. j In the point cloud of the structural surface and q j The nearest matching point;

[0139] The optimization objective is to calculate a set of spatial rotation matrices and translation vectors so that each 3D point on the surface where the fire equipment is located, after the transformation, is as close as possible to its corresponding point on the site structure. In other words, the goal is to make the transformed fire equipment locations fit snugly against the site structure surface, achieving optimal alignment in both shape and position. The even-numbered transformation proximity is measured by minimizing the sum of the squared Euclidean distances between all point pairs.

[0140] The environmental constraint analysis unit obtains the registered fire equipment installation list and the structural surface point cloud P of the matching area. k Normal vector n k The system also measures robot operating parameters (arm span, tool orientation, installation angle range) and determines whether there are other structures above or in front of the fire protection engineering installation location that may obstruct the construction operation.

[0141] Specifically, at the fire equipment installation point p i 对齐A cylindrical or cubic detection volume is constructed above (in this embodiment, it is set to a radius of 0.2m and a height of 0.5m), and all points within this volume are extracted from the point cloud to count the number or density of points; simultaneously, a threshold ρ is set. thresh If the number of points exceeds this value, it is judged as severe occlusion.

[0142] In this embodiment, the environmental constraint analysis unit also performs angle limitation judgment, that is, it judges whether the installation orientation of the component matches the construction angle range allowed by the robot.

[0143] Specifically, the environmental constraint analysis unit calculates the included angle θ according to the following formula. i Calculation:

[0144]

[0145] In the formula, This refers to the orientation vector of the fire-fighting equipment after transformation; that is, its orientation direction in the actual structural space after spatial registration is completed. The normal vector of the structural surface;

[0146] in,

[0147] In the formula, R* is the optimal rotation matrix. This refers to the factory installation orientation of fire protection equipment as marked on the drawings / BIM model.

[0148] If the included angle θ i If the angle exceeds the robot's working range (such as the 15° to 75° limit in this embodiment), it is considered to be angle-limited.

[0149] If the corresponding location of a component is marked as "not for construction" or "severely obstructed", the layout planning module will remove it from the candidate placement points.

[0150] If the angle exceeds the limit or the location is unreachable, the action module needs to regenerate the path or trigger a rollback.

[0151] By combining the task parsing module and the spatial modeling module, the equipment intent in the drawing task can be accurately mapped to the spatial structure of the actual construction site, achieving spatial consistency between the task objective and the real-world structure, and ensuring the locationability of equipment deployment and its adaptability to the construction environment.

[0152] Optionally, the layout planning module includes a candidate point generation unit, an intent mapping unit, a constraint calculation unit, and an optimized layout unit. The candidate point generation unit generates installation candidate points that meet basic specifications within the workable area. The intent mapping unit constructs the installation candidate points into a spatial intent field to guide the formation of a priority placement sequence. The constraint calculation unit filters candidate points in the priority placement sequence according to installation specifications. The optimized layout unit optimizes the total path length, resource utilization rate, and equipment balance index based on meeting the specifications to output the final layout scheme. The workable area is identified and determined by the spatial modeling module. The system first obtains a set of structural surfaces with semantic tags and spatial boundaries through the spatial structure identification unit and drawing registration unit. Then, combined with the environmental constraint analysis results, areas that are obstructed, have limited angles, or are insufficient in size are eliminated, ultimately forming a limited set of workable areas. The candidate point generation unit only performs standardized placement within the above-mentioned areas to ensure the executability and accuracy of the layout planning.

[0153] The candidate point generation unit creates a regular grid on the target structure surface (in this embodiment, the grid is divided into 0.5m grids) and filters grid points according to installation specifications, namely, the distance from the boundary ≥ the safety margin (e.g., 0.3m) and the distance between adjacent points ≥ the installation spacing (e.g., 2.0m). Simultaneously, the center of each grid meeting the conditions is designated as a candidate point c. j (x j y j , z j );

[0154] The intent mapping unit is based on the deployment area R target Construct the spatial intention potential function:

[0155]

[0156] In the formula, (x, y) are the two-dimensional coordinates of the candidate points, that is, the two-dimensional coordinates of the candidate deployment points on the structural surface, r i Let ||(x,y)-r be the coordinates of the center point of the target region. i || represents the spatial distance between the candidate point and the center point of the i-th target, and σ is the Gaussian diffusion coefficient. In this embodiment, the range is set to 1.0 to 2.0. When the spatial range is small and the deployment accuracy is high, σ = 1.0; when the space is large and the deployment freedom is high, σ = 2.0.

[0157] In this invention, the intent mapping unit first bases its work on the task target region set R. target Construct a spatial intention potential function V(x,y) to simulate the degree of placement preference at different locations in space;

[0158] Then, for the candidate deployment point set C = {c j Each point c in}j Perform function sampling to obtain the intent score:

[0159] v j =V(c j );

[0160] Finally, the intent mapping unit determines the intent value v. j Candidate points are sorted from high to low to form a priority point sequence, which guides the subsequent constraint calculation and layout optimization process.

[0161] The constraint calculation unit performs a calculation on each candidate point c. j Perform the following judgment:

[0162] 1) Whether it falls within an obstructed area;

[0163] 2) Whether the included angle exceeds the limit (angle with the structural normal);

[0164] 3) Whether it is close to existing equipment, violating the minimum distance requirement;

[0165] 4) Whether it falls within a non-construction area;

[0166] If any of the four conditions is not met, the candidate point is eliminated. Simultaneously, the remaining candidate points are grouped together to form a candidate point set C. valid ;

[0167] Optimize the layout unit for each candidate point c j Calculate the overall score s j :

[0168] s j =λ1·v j +λ2·a j ;

[0169] In the formula, a j For constructability rating, v j The score is the intention value, and the closer to the target area, the higher the score. λ1 and λ2 are weight coefficients, and λ1+λ2=1;

[0170] In this embodiment, the constructability score a j The value will be determined based on the following conditions:

[0171] Whether the unobstructed point cloud within 0.5m above the candidate point is unobstructed: Obstructed: 0, Unobstructed: 1;

[0172] Is the angle between the installation direction and the structural surface within the allowable range? Is there a restriction? Exceeding the limit: 0, Compliant: 1;

[0173] Is the robot reachable within its reach range? Unreachable: 0, Reachable: 1

[0174] In this embodiment, an example of the value of the weighting coefficient is provided, specifically:

[0175] 1) In a scenario where sprinkler heads are installed on the ceiling and the user designates two rooms as the core area, then λ1 = 0.7.

[0176] λ² = 0.3;

[0177] 2) In scenarios where a gas control valve is installed inside the pipeline well and the construction angle is limited, then λ1=0.3, λ2=0.7;

[0178] 3) If smoke detectors are installed in the corridor, and the target area is large and the space is open, then λ1 = 0.5 and λ2 = 0.5.

[0179] 4) In scenarios where alarm bells are installed in stairwells and are severely affected by floor obstruction, λ1 = 0.4 and λ2 = 0.6.

[0180] For any candidate point c j =(x j y j Its intended value is determined by the following formula:

[0181]

[0182] In the formula, c j =(x j ,y j ) represents the coordinates of the candidate point, whose value is output by the candidate point generation unit, r i =(x i ,y i ) represents the center coordinates of the target region, and its value is based on R output by the task parsing module. target σ is the Gaussian diffusion coefficient, which is set to 1.0 to 2.0 in this embodiment, and ||·|| is the Euclidean distance.

[0183] In this embodiment, if the task requires the deployment of 20 devices, they can be allocated equally among the regions or proportionally according to the area of ​​each region, such as: Z1: 8 devices, Z2: 6 devices, Z3: 6 devices. Based on this, the optimized layout unit will allocate the devices in each sub-region Z... k Within, from its corresponding set of valid deployment candidate points C valid ∩Z k In the middle, according to the comprehensive score s j Select N from high to low k There are N distribution points, of which N k The number of deployment tasks is pre-allocated to this area, and then the point sets selected from each sub-region are merged to obtain the final global deployment scheme point set C. final .

[0184] By combining the spatial modeling module and the layout planning module, the installation points generated during the layout planning process are only placed within the actual workable area, avoiding problems such as structural obstruction and spatial errors, and ensuring the feasibility and reliability of the layout plan.

[0185] In addition, through the cooperation of the task parsing module and the layout planning module, the system can automatically build a layout strategy based on the equipment type, installation requirements and layout parameters extracted from the task, avoiding manual configuration and repeated debugging, and ensuring that the planning results are automatically generated under the premise of meeting the task objectives.

[0186] Optionally, the motion control module includes a path planning unit, a motion sequence generation unit, and a status feedback monitoring unit. The path planning unit generates obstacle avoidance paths for the robot from its current position to each installation point based on the on-site point cloud data and deployment point information. The motion sequence generation unit breaks down each installation task into specific motion instructions. The status feedback monitoring unit monitors the real-time status of the installation robotic arm.

[0187] The real-time status includes: whether the action is in place, whether the action is completed, and whether an abnormality has occurred.

[0188] Optionally, the action commands include raising the boom, drilling, installing, and testing.

[0189] The path planning unit loads the on-site point cloud data and deployment point information, extracts the current robot position and the coordinates of the installation target point, establishes a simplified spatial grid model in the point cloud data, and uses it to determine the distribution of obstacles and passable areas.

[0190] In addition, a rule-based search method (such as grid pathfinding or linear interpolation) is used to calculate an obstacle avoidance path between the starting point and the target point. If there is an obstacle blocking the way, inflection points are automatically generated along the shortest feasible detour path; the path can be simplified into several movement-stop segments.

[0191] Specifically, when constructing a path between the starting point and the target point, the path planning unit generates path points by default using linear interpolation. The path function is:

[0192]

[0193] In the formula, P i Let P be the i-th path point. star Let P be the starting coordinate. goal The endpoint coordinates, the number of difference segments, and i represent the current difference index;

[0194] If the minimum distance between a path point and an obstacle is dist(P) i O j If the value is less than the system's default safety threshold (a known value), then a detour point P will be automatically inserted.via Construct a polyline path to form an obstacle avoidance path sequence consisting of movement and docking.

[0195] Among them, the minimum distance dist(P) between the path point and the obstacle i O j Calculate according to the following formula:

[0196]

[0197] In the formula, P i For the i-th path point, O j For the j-th obstacle point, (x i ,y i ,z i (x) represents the coordinates of the path point. j ,y j ,z j () represents the coordinates of the obstacle point;

[0198] The point corresponding to the j-th obstacle is O j The value of is derived from the obstacle point set O = {O j The data is derived from the on-site structural point cloud model, and obstacle points are extracted according to the following rules:

[0199] 1) Remove point cloud areas from the workable surfaces;

[0200] 2) Filter areas on the ground or at excessively low heights;

[0201] 3) Perform density clustering on the remaining point cloud to retain points of fixed structural objects in space;

[0202] The final obstacle point formed.

[0203] The action sequence generation unit queries the action template corresponding to the type of fire equipment, such as: sprinkler head → boom raising → positioning → drilling → fixing → testing, and inserts the action at the end of the path to form an action queue. Each action step is encoded as a specific control command (such as controlling joints, motors, grippers, etc.), ultimately forming an action quality sequence list A. j ={a1, a2, ... a n}

[0204] The action templates corresponding to different types of fire-fighting equipment are pre-set in the database and can be called when needed.

[0205] In this embodiment, the robotic arm is equipped with at least two joints and a control panel, and posture sensors are provided on at least two joints to obtain the posture data of the robotic arm; the control panel collects the motion data stream of the robotic arm to collect the motion number and feedback timestamp of the robotic arm.

[0206] In addition, the end effector of the installation robotic arm is equipped with corresponding installation tools and a torque sensor as needed. The installation tools include, but are not limited to, the following: a hole drill and a suction cup. The torque sensor collects the torque exerted when the end effector of the installation robotic arm contacts the fire-fighting equipment.

[0207] The drilling tool is used to drill holes in the mounting surface, and the suction plate is used to adsorb the fire-fighting equipment to be installed. The fire-fighting equipment is installed at the mounting position corresponding to the drilling position by the movement of at least two joints of the installation robotic arm.

[0208] Meanwhile, the installation robotic arm also includes a motion controller, which controls the joints according to control commands to achieve the installation of fire-fighting equipment. The control of the joints using control commands is a conventional control method, well-known and mastered by those skilled in the art, and therefore will not be described in detail in this embodiment.

[0209] The status feedback monitoring unit monitors the robot's task execution status in real time, detecting whether the task is completed, whether the action is finished, and whether any abnormalities occur.

[0210] Specifically, the status feedback monitoring unit determines whether the installation point has been reached, that is, it compares whether the distance between the current position and the target point is within the allowable range;

[0211] In addition, the status feedback monitoring unit performs action completion detection, that is, monitors the return of action completion signals (such as torque, time, limit switch status).

[0212] Meanwhile, the status feedback monitoring unit also detects abnormal indicators, including: whether it cannot reach the destination, timeout, drilling failure, and sensor overshoot. If an abnormal indicator is detected, the system will re-plan or issue an alarm.

[0213] By combining the layout planning module and the motion control module, the planned installation points can be converted into executable motion paths in real time, avoiding construction failures due to unreachable locations or path conflicts, and ensuring the continuity and path safety of robot execution.

[0214] Optionally, the quality verification module includes an image comparison unit, a mechanical detection unit, an error judgment unit, and a task reflow trigger. The image comparison unit captures images of the installed parts and compares them with drawings or BIM models. The mechanical detection unit uses a torque sensor to detect whether the fire-fighting equipment is installed securely. The error judgment unit analyzes the images and sensor data to determine whether the error threshold is exceeded. If the verification fails, the task reflow trigger sends a re-execution command to the layout planning module or the action module.

[0215] Optionally, the parameters for the image comparison unit to compare the drawing with the BIM model include position, orientation, and angle.

[0216] The image comparison unit includes an image collector, a data memory, and an image comparer. The image collector collects image data of the installed part. The data memory stores the image data of the installed part collected by the image collector. The image comparer analyzes the installation quality based on the image data.

[0217] Among them, the image comparer obtains the on-site image after installation, as well as the position and installation orientation of the fire protection equipment in the drawing model or BIM data, and calculates the installation position error Δp according to the following formula:

[0218]

[0219] In the formula, p actual is the actual installation position vector, p design is the designed installation position vector, (x a , y a , z a ) is the actual installation point coordinate, (x a , y a , z a ) is the designed installation point coordinate.

[0220] If Δp is less than δ p (in this embodiment, it is set that δ p = 20mm), then it is qualified;

[0221] If Δp is greater than or equal to δ p , then it is unqualified.

[0222] The image comparer calculates the installation angle error Δθ according to the following formula:

[0223]

[0224] In the formula, is the actual installation normal vector, and its value is the normal vector of the equipment installation surface (or the equipment itself) identified from the image or depth point cloud, is the actual installation normal vector, and its value is determined by the drawing or BIM model.

[0225] If Δθ is less than θmax (in this embodiment, θmax is set to 10°), then the installation angle is qualified;

[0226] If Δθ is greater than or equal to θmax, then the installation is unqualified. [[ID=:61]]

[0227] In other embodiments, the designed installation orientation of the target equipment can also be extracted from the drawing or model to generate a unit normal vector Simultaneously, the image comparison unit extracts the local point cloud of the installed area using RGB-D images or structured light reconstruction results, and estimates the normal vector of the installation surface using principal component analysis. The angle between the two is used to determine attitude deviation.

[0228] The mechanical detection unit acquires the slight reverse pulling force applied to the target equipment by the installation robot arm after installation, and reads the feedback force and displacement values ​​from the torque sensor. Simultaneously, it calculates the loosening measurement factor I based on the feedback force and displacement values. loose Loosening measurement factor I loose Calculate according to the following formula:

[0229]

[0230] In the formula, △x max x is the actual maximum displacement. ref For reference displacement, F ref For reference force value, F avg The actual average applied force is given by ε, which is a dimensionless smoothing factor with a value of 0.001.

[0231] The actual average applied force is calculated according to the following formula:

[0232]

[0233] In the formula, F i Let F be the force value of the i-th sample, which is the instantaneous force value read by the sensor at the end of the robotic arm during the i-th sample. n is the number of sampling points. avg This represents the average reverse force.

[0234] The error judgment unit acquires the image comparison error and uses the mechanical judgment result (if it is a loosening measurement factor I) to determine the image comparison error. loose If the looseness threshold is exceeded, the installation is considered unqualified; otherwise, it is considered unqualified. If any condition is not met, the installation is marked as failed.

[0235] The task backflow trigger calls the judgment result of the error judgment unit in real time. If the error judgment is: unqualified, then execute:

[0236] 1) Notify the motion control module to reinstall;

[0237] 2) Notify the layout planning module to reassign deployment points (e.g., the current point cannot be used for construction);

[0238] 3) The alarm will be entered into the manual confirmation queue.

[0239] By combining the motion control module and the quality verification module, the system can perform automated quality inspection immediately after the installation task is completed, and trigger a backflow correction operation based on the inspection results, ensuring closed-loop control and quality consistency of the installation effect.

[0240] Through the coordinated operation of the task parsing module, spatial modeling module, layout planning module, motion control module, and quality verification module, the fire protection engineering installation task can achieve fully automated processing from multimodal task identification, spatial environment modeling, standard constraint placement, automatic motion control to quality self-inspection closed loop, ensuring that the system has intelligent scheduling capability and self-repair capability in the four-in-one system of task-space-control-verification.

[0241] Meanwhile, through the hierarchical collaboration of the task parsing module, spatial modeling module, layout planning module, action control module, and quality verification module, the installation task can achieve intelligent automation throughout the entire process from task input to construction completion. Specifically, this includes the joint driving of task parsing and spatial modeling data, the seamless connection between planning and control actions, and the feedback correction of result verification and scheduling strategies, forming a closed-loop control link, which improves the system's adaptability, fault tolerance, and deployment efficiency.

[0242] Example 2: This example should be understood as including all the features of any of the foregoing examples, and further improving upon them, according to... Figure 1 , Figure 2 , Figure 3 , Figure 4 ,as well as Figure 5 As shown, the verification module also includes a global evaluation unit, which assesses the quality verification results of the current installation area after the installation task is completed, and calculates the area's hotspot density H.

[0243]

[0244] In the formula, m is the total number of installed devices, M(i) is the neighbor set of the i-th device, that is, the neighbor set must satisfy the condition that the set of neighboring devices is less than or equal to the spatial safety distance R, where R is set to 0.5m in this embodiment; δ j To install the qualified status label (whose value is determined by the quality verification module), if equipment j is unqualified, then δ j =1; if qualified, it is 0.

[0245] The regional quality hot spot density H is used to reflect whether the non-conforming points have spatial clustering. If the density of non-conforming points in a certain area is concentrated and they are close to each other, the H value will increase significantly, which is considered to be a local installation degradation trend.

[0246] In this embodiment, an abnormal monitoring threshold UNHealthy is set if: H > UNHealthy;

[0247] This will trigger the following task reflow mechanism:

[0248] Send a re-execution instruction for the non-conforming point to the motion control module;

[0249] If multiple points in the neighborhood fail consecutively, request the layout planning module to redistribute points in the local area.

[0250] The area was marked with quality hotspots and included in the priority re-inspection areas for subsequent construction.

[0251] The anomaly monitoring threshold UNHealthy is set by the system based on the on-site deployment density and the distribution of installed equipment. Taking a typical office building corridor as an example, if each device has an average of 4 spatial neighbors, it is recommended to set UNHealthy = 1.5, which means that when more than 1.5 non-compliant points appear in the neighborhood of each device, it is considered that there is an installation anomaly trend in the area.

[0252] In summary, the abnormal monitoring threshold UNHealthy is set by the system according to the actual situation and input through the human-computer interaction interface. This is a technical means well known to those skilled in the art, and therefore will not be described in detail in this embodiment.

[0253] By coordinating the global evaluation unit with the image comparison unit, mechanical detection unit, and error judgment unit, the system can not only perform item-by-item inspection of the quality of a single installation point, but also perform hot spot trend analysis and quality aggregation judgment on the construction quality of the entire installation area after the task is completed, ensuring that the system has the closed-loop control capability of construction quality from local judgment to global early warning.

[0254] The content disclosed above is only a preferred and feasible embodiment of the present invention, and is not intended to limit the scope of protection of the present invention. Therefore, all equivalent technical changes made based on the content of the present invention specification and drawings are included within the scope of protection of the present invention. Furthermore, the elements therein can be updated as technology develops.

Claims

1. A fire engineering robot installation system based on a multi-modal model, the fire engineering robot installation system comprising a server, and an installation robot arm, characterized in that, The fire-fighting engineering robot installation system further comprises a task analysis module, a space modeling module, a layout planning module, an action control module and a quality verification module, and the server is connected with the task analysis module, the space modeling module, the layout planning module, the action control module and the quality verification module respectively; The task analysis module is used for analyzing an installation task, fusing multi-modal inputs of drawings and BIM models, extracting key information of equipment types, target areas and layout parameters, and outputting structured task instructions; The space modeling module performs three-dimensional structure modeling on a construction site to form a space point model, identifies key building components and constructible areas, completes high-precision registration of drawing structures and real scene structures, and analyzes environmental constraint conditions; The layout planning module generates a fire-fighting equipment layout scheme based on the task instructions and the space point model, on the basis of meeting specification requirements; The action control module converts the layout scheme corresponding to the installation position and the installed fire-fighting equipment type into a sequence of action instructions executable by the robot, and avoids task coordination and path conflicts of the robot; The quality verification module automatically detects the installation result, and triggers automatic correction actions or re-plans the installation task if an abnormality is found; The task analysis module comprises a drawing analysis unit, a specification extraction unit and a multi-modal fusion unit, the drawing analysis unit analyzes CAD drawings and BIM models to extract installation information of fire-fighting equipment, the specification extraction unit analyzes national / industry specification documents to extract standard parameters of fire-fighting equipment, and the multi-modal fusion unit fuses drawings, specifications and operation requirements, and correction information to output task target information in a unified format; The space modeling module comprises a three-dimensional reconstruction unit, a space structure identification unit, a drawing registration unit and an environmental constraint analysis unit, the three-dimensional reconstruction unit collects real-time data of the current space based on a laser radar in real time and generates a space point model, the space structure identification unit performs semantic segmentation on the space point model to identify ceiling, wall and pipe structure, the drawing registration unit spatially aligns drawing structures and field structures, detects errors or misalignments, and forms a registered fire-fighting equipment list, and the environmental constraint analysis unit obtains the registered fire-fighting equipment installation list, analyzes construction obstructions and angle limitations in the field environment, and forms boundary conditions for subsequent layout planning; The layout planning module comprises a candidate point generation unit, an intention mapping unit, a constraint calculation unit and an optimized layout unit, the candidate point generation unit generates installation candidate points meeting basic specifications in a constructible area, The intention mapping unit constructs the installation candidate points into a space intention field to guide the formation of a priority layout sequence, the constraint calculation unit filters the candidate points in the priority layout sequence according to installation specifications, and the optimized layout unit optimizes path total length, resource utilization rate and equipment balance indicators on the basis of meeting specifications to output a final layout scheme. The action control module comprises a path planning unit, an action sequence generation unit, and a state feedback monitoring unit. The path planning unit generates an obstacle-avoiding path for the robot from the current position to each installation point. The action sequence generation unit decomposes each installation task into specific action instructions. The state feedback monitoring unit detects the real-time state of the installation mechanical arm in real time. The quality verification module comprises an image comparison unit, a mechanical detection unit, an error judgment unit, and a task backflow trigger. The image comparison unit photographs the installed part and compares it with the drawing or BIM model. The mechanical detection unit uses a torque sensor to detect whether the fire-fighting equipment is installed firmly. The error judgment unit analyzes the image and sensor data to determine whether the error threshold is exceeded. The task backflow trigger initiates a re-execution instruction to the layout planning module or the action control module if the verification fails.

2. The multi-modal model based firefighting engineering robot mounting system of claim 1, wherein, The three-dimensional reconstruction unit comprises a laser scanning acquisition subunit, a coordinate conversion subunit, a SLAM splicing subunit, and a point cloud optimization processing subunit. The laser scanning acquisition subunit obtains the spatial depth information of the construction site in real time based on a rotating laser radar. The coordinate conversion subunit converts the collected polar coordinate data into spatial Cartesian coordinate points. The SLAM splicing subunit performs attitude correction and spatial splicing on continuous frame point clouds based on inertial navigation and laser odometry. The point cloud optimization processing subunit performs voxel filtering, ground removal, and outlier point rejection on the point cloud to form a spatial point cloud model that can be used for structure identification and path planning.

3. The multi-modal model-based fire engineering robot mounting system of claim 2, wherein, The parameters for comparing the drawing with the BIM model by the image comparison unit include position, direction, and angle.

4. The multi-modal model-based fire engineering robot mounting system of claim 3, wherein, The action instructions include lifting the arm, punching, installing, and testing.

5. The multi-modal model-based fire engineering robot mounting system of claim 4, wherein, The installation information of the fire-fighting equipment includes the equipment type, number, and location label.

Images