Fire-fighting engineering robot installation system based on multi-modal model

Abstract

The invention relates to the technical field of fire fighting, and provides a fire engineering robot installation system based on a multi-modal model, which comprises an installation mechanical arm, a task analysis module, a spatial modeling module, a layout planning module, an action control module and a quality verification module, drawing and BIM model multi-modal input are fused, key information is extracted, and a structured task instruction is output; the spatial modeling module performs three-dimensional structure modeling on a construction site to form a spatial point model, identifies building components and constructable areas, completes high-precision registration of a drawing structure and a real scene, and analyzes environmental constraint conditions; the layout planning module generates a fire-fighting equipment distribution scheme on the basis of meeting specification requirements according to the task instruction and the spatial point model; the action control module converts the point distribution scheme and the type of the installed fire fighting equipment into an action instruction sequence which can be executed by the robot; and the quality verification module performs automatic detection on the installation result.

Description

Technical Field

[0001] The present invention relates to the field of fire protection technology, and in particular to a fire protection engineering robot installation system based on a multimodal model. Background Art

[0002] During construction, the installation of fire protection systems is a critical step in ensuring safe building operation. Existing installation methods generally rely on manual labor, requiring construction workers to refer to blueprints for equipment placement, positioning, and securing. However, due to complex building structures, subjective interpretation of blueprints, and the ever-changing spatial environment, this often leads to installation inaccuracies, low efficiency, and even regulatory violations and duplicated work.

[0003] For example, the prior art disclosed in Chinese patent CN114432632B discloses a height positioning device for installing a fire hood. Although it provides a device and method for installing and positioning the fire hood, it ignores the analysis and evaluation of the layout position of the fire hood in the entire channel, resulting in an unreasonable layout position, which is not conducive to rapid response.

[0004] In addition, the prior art also has the following defects:

[0005] 1. Architectural spaces often contain diverse structural features such as beams, grooves, suspended ceilings, and embedded pipes. Traditional robots struggle to accurately identify the buildable surfaces, obstructions, and spatial forms within a construction area, and are even less able to effectively align design drawings with on-site structures.

[0006] 2. The layout of fire-fighting equipment has strict requirements on spacing, angles, and accessibility. Currently, it still relies heavily on manual measurement and empirical layout, lacking adaptive and global optimization capabilities based on spatial structure and equipment objectives. This can easily lead to uneven installation density, wasted space, or redundant pipelines.

[0007] 3. Most of the currently installed robots are execution-type devices, lacking the autonomous planning capabilities based on task understanding and spatial perception, and also lacking a task coordination mechanism among multiple robots, which makes it easy for problems such as action conflicts, path overlap, and reduced efficiency to occur.

[0008] 4. Fire protection installation construction has high requirements for positioning accuracy, firmness and wiring specifications. However, currently, after construction, it mainly relies on manual visual inspection or random inspection, which has low inspection efficiency, high error rate, and lacks automatic correction mechanism. Once deviation occurs, rework is often required, which greatly reduces construction efficiency and cost control capabilities.

[0009] The present invention is made in order to solve the common problems in this field, such as lack of means for arranging the location of fire protection engineering, reliance on manual positioning which easily leads to high labor intensity, lack of intelligent layout decision-making, poor collaborative scheduling capabilities, and poor installation efficiency. Summary of the Invention

[0010] The purpose of the present invention is to address the current deficiencies and propose a fire engineering robot installation system based on a multimodal model.

[0011] In order to overcome the deficiencies of the prior art, the present invention adopts the following technical solutions:

[0012] A firefighting engineering robot installation system based on a multimodal model, comprising a server and an installation robotic arm, the firefighting engineering robot installation system also comprising a task parsing module, a space modeling module, a layout planning module, a motion control module, and a quality verification module, wherein the server is connected to the task parsing module, the space modeling module, the layout planning module, the motion control module, and the quality verification module, respectively;

[0013] The task parsing module is used to parse the installation task and integrate the multimodal input of drawings and BIM models to extract key information such as equipment type, target area, and layout parameters, and output structured task instructions;

[0014] The spatial modeling module performs three-dimensional structural modeling of the construction site to form a spatial point model, identifies key building components and constructible areas, completes high-precision registration of the drawing structure with the actual scene structure, and analyzes environmental constraints;

[0015] The layout planning module generates a fire equipment layout plan based on the task instructions and the spatial point model, while meeting the regulatory requirements;

[0016] The motion control module converts the layout plan corresponding to the installation location and the type of fire-fighting equipment installed into a sequence of motion instructions executable by the robot, and avoids the robot's task coordination and path conflicts;

[0017] The quality verification module automatically detects the installation results and can trigger automatic correction actions or re-plan the installation task if an anomaly is found.

[0018] Optionally, the task parsing module includes a drawing parsing unit, a specification extraction unit, and a multimodal fusion unit. The drawing parsing unit parses CAD drawings / BIM models to extract installation information of fire-fighting equipment; the specification extraction unit parses national / industry specification documents to extract standard parameters of fire-fighting equipment; the multimodal fusion unit fuses drawings, specifications and operating requirements, as well as correction information, and outputs task target information in a unified format.

[0019] Optionally, the spatial modeling module includes a three-dimensional reconstruction unit, a spatial structure recognition unit, a drawing registration unit, and an environmental constraint analysis unit. The three-dimensional reconstruction unit collects real-time data of the current space based on a lidar and generates a spatial point model; the spatial structure recognition unit performs semantic segmentation on the spatial point model to identify ceilings, walls, and pipe well structures; the drawing registration unit spatially aligns the drawing structure with the on-site structure to detect errors or misalignments; the environmental constraint analysis unit analyzes the constraints of construction occlusion and angle restrictions in the on-site environment to form boundary conditions for subsequent layout planning.

[0020] Optionally, the layout planning module includes a candidate point generation unit, an intention mapping unit, a constraint calculation unit, and an optimized layout unit. The candidate point generation unit generates installation candidate points that meet basic specifications within the constructible area. The intention mapping unit constructs the installation candidate points into a spatial intention field to guide the formation of a priority point sequence; the constraint calculation unit filters the candidate points in the priority point sequence according to the installation specifications; the optimized layout unit optimizes the total path length, resource utilization, and equipment balance indicators based on meeting the specifications to output the final layout plan.

[0021] Optionally, the motion control module includes a path planning unit, an action sequence generation unit, and a state feedback monitoring unit. The path planning unit generates an obstacle avoidance path for the robot from the current position to each installation point; the action sequence generation unit breaks down each installation task into specific action instructions; and the state feedback monitoring unit detects the real-time status of the installation robotic arm in real time.

[0022] Optionally, the quality verification module includes an image comparison unit, a mechanical detection unit, an error judgment unit, and a task reflow trigger. The image comparison unit takes a picture of the installed part and compares it with the drawing or BIM model; the mechanical detection unit uses a torque sensor to detect whether the fire-fighting equipment is firmly installed; the error judgment unit analyzes the image and sensor data to determine whether the error threshold is exceeded; if the task reflow trigger fails to verify, it initiates a re-execution instruction to the layout planning module or the motion control module.

[0023] Optionally, the three-dimensional reconstruction unit includes a laser scanning acquisition subunit, a coordinate conversion subunit, a SLAM stitching subunit and a point cloud optimization processing subunit; wherein, the laser scanning acquisition subunit acquires the spatial depth information of the construction site in real time based on the rotating laser radar, the coordinate conversion subunit converts the collected polar coordinate data into spatial Cartesian coordinate points, the SLAM stitching subunit performs posture correction and spatial stitching on continuous frame point clouds based on inertial navigation and laser odometry, and the point cloud optimization processing subunit performs voxel filtering, ground removal and outlier elimination on the point cloud to form a spatial point cloud model that can be used for structure recognition and path planning.

[0024] Optionally, the parameters used by the image comparison unit to compare the drawing with the BIM model include position, direction, and angle.

[0025] Optionally, the action instructions include raising the arm, punching, installing, and testing.

[0026] Optionally, the installation information of the fire-fighting equipment includes equipment type, number, and location label.

[0027] The beneficial effects achieved by the present invention are:

[0028] 1. Through the collaboration between the spatial modeling module and the layout planning module, the installation points generated during the layout planning process are only placed within the actual constructible area, avoiding problems such as structural obstruction and spatial errors, and ensuring the feasibility and accuracy of the layout plan;

[0029] 2. Through the collaboration between the task analysis module and the layout planning module, the system can automatically build a layout strategy based on the equipment type, installation requirements, and layout parameters extracted from the task, eliminating manual configuration and repeated debugging, and ensuring that the planning results are automatically generated while meeting the task objectives;

[0030] 3. Through the cooperation of the layout planning module and the motion control module, the planned installation points can be converted into executable motion paths in real time, avoiding construction failures due to layout inaccessibility or path conflicts, and ensuring the consistency of robot execution and path safety.

[0031] 4. Through the cooperation of the motion control module and the quality verification module, the system can immediately perform automated quality inspection after the installation task is completed, and trigger the reflow correction operation based on the inspection results, ensuring closed-loop control and quality consistency of the installation effect;

[0032] 5. Through the collaboration of the task parsing module and the spatial modeling module, the equipment intent in the drawing task can be accurately mapped to the spatial structure of the actual construction site, achieving spatial consistency between the task objectives and the actual scene structure, ensuring the positionability of the equipment layout and adaptability to the construction environment;

[0033] 6. Through the coordinated cooperation of the task analysis module, spatial modeling module, layout planning module, motion control module, and quality verification module, fire protection engineering installation tasks can be fully automated, from multimodal task identification, spatial environment modeling, standardized constraint layout, automatic motion control, to a closed-loop quality self-inspection. This ensures that the system has the intelligent scheduling and self-repair capabilities of the four-in-one task-space-control-verification system.

[0034] 7. Through the layered collaboration of the task analysis module, space modeling module, layout planning module, motion control module and quality verification module, the entire installation task process from task input to construction completion is intelligently automated, improving the system's adaptability, fault tolerance and deployment efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily drawn to scale, but rather the emphasis is placed on illustrating the principles of the embodiments. In different views, the same reference numerals designate the same parts.

[0036] Figure 1 It is an overall block diagram of the present invention.

[0037] Figure 2 Schematic diagram of the multimodal fusion unit of the present invention.

[0038] Figure 3 It is a block diagram of the spatial modeling module of the present invention.

[0039] Figure 4 It is a flow chart of the layout planning module of the present invention.

[0040] Figure 5 Schematic diagram of the flow of the action control module of the present invention. DETAILED DESCRIPTION

[0041] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted in actual size. It is stated in advance. The following embodiments will further explain the relevant technical content of the present invention in detail, but the disclosed content is not intended to limit the scope of protection of the present invention.

[0042] Example 1: According to Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5As shown, this embodiment provides a firefighting engineering robot installation system based on a multimodal model, the firefighting engineering robot installation system includes a server and an installation robot arm, the firefighting engineering robot installation system also includes a task analysis module, a space modeling module, a layout planning module, a motion control module and a quality verification module, the server is connected to the task analysis module, the space modeling module, the layout planning module, the motion control module and the quality verification module respectively, and stores the intermediate data and process data of the task analysis module, the space modeling module, the layout planning module, the motion control module and the quality verification module in a database of the server for query and call;

[0043] The task parsing module is used to parse the installation task and integrate the multimodal input of drawings and BIM models to extract key information such as equipment type, target area, and layout parameters, and output structured task instructions;

[0044] The spatial modeling module performs three-dimensional structural modeling of the construction site to form a spatial point model, identifies key building components and constructible areas, completes high-precision registration of the drawing structure with the actual scene structure, and analyzes environmental constraints;

[0045] The layout planning module generates a fire equipment layout plan based on the task instructions and the spatial point model, while meeting the regulatory requirements;

[0046] The motion control module converts the layout plan corresponding to the installation location and the type of fire-fighting equipment installed into a sequence of motion instructions executable by the robot, and avoids the robot's task coordination and path conflicts;

[0047] The quality verification module automatically detects the installation results and can trigger automatic correction actions or re-plan the installation task if an anomaly is found.

[0048] Among them, it also includes a central processing unit, which is respectively controlled and connected to the task analysis module, the space modeling module, the layout planning module, the motion control module and the quality verification module, and centrally controls the task analysis module, the space modeling module, the layout planning module, the motion control module and the quality verification module based on the central processing unit. The control data of the central processing unit is stored in the database, thereby improving the reliability and accuracy of the entire system for the installation of the fire protection system.

[0049] Optionally, the task parsing module includes a drawing parsing unit, a specification extraction unit, and a multimodal fusion unit. The drawing parsing unit parses the CAD drawings / BIM models to extract the installation information of the fire-fighting equipment, wherein the installation information includes: equipment type, number, and location label; the specification extraction unit parses the national / industry specification documents to extract the standard parameters of the fire-fighting equipment, wherein the standard parameters include: installation spacing, size, and height; the multimodal fusion unit fuses the drawings, specifications, and operating requirements to output the task target information in a unified format.

[0050] Specifically, the drawing parsing unit receives CAD drawings or BIM model files as input, and classifies and identifies the layers and component types in the drawings, extracting the type identification, number information and spatial location label of the target fire-fighting equipment; in the process of reading the attribute information of the component information in the BIM model, it is necessary to obtain the ID, category, and placement location (such as XYZ coordinates or the name of the space where it is located) of the equipment instance, thereby generating a drawing extraction result dataset T 图纸 .

[0051] The specification extraction unit imports national or industry fire protection design specification documents and retrieves technical parameter entries associated with fire protection equipment types through natural language processing models or rule templates. It extracts and standardizes key parameter values for each type of equipment, such as recommended installation spacing, equipment size, and installation height limit, and outputs a structured parameter set T. 规范 :

[0052] T 规范 ={fire protection type, spacing, size, height};

[0053] The multimodal fusion unit matches the equipment type and number between the drawing extraction results and the specification parameter table. If the drawing information is missing certain parameter items, the system will call the corresponding fields in the specification to complete them. If the operator's operation requirements (such as area priority and installation adjustment) are received, the fusion unit will merge this information with the drawing / specification data, identify and update the relevant equipment item parameter values. If there is an information conflict (such as the drawing height of 2.5m, the specification recommends 2.8m), the system will select according to the preset priority. The default priority is: user instruction > drawing annotation > specification parameter.

[0054] The final output is a structured installation task target information set:

[0055] T 任务信息 ={component number, equipment type, region, installation parameters};

[0056] At the same time, based on the mapping relationship between the "Area" field and the area space in the drawing, the system extracts the target layout area of each type of equipment as a set of three-dimensional center points to form a target layout area set:

[0057] R target ={r i =(x i ,y i ,z i )}

[0058] The target layout area set will serve as the basis for intent mapping in the subsequent layout planning module to generate space priority layout guidance information.

[0059] Optionally, the spatial modeling module includes a three-dimensional reconstruction unit, a spatial structure recognition unit, a drawing registration unit, and an environmental constraint analysis unit. The three-dimensional reconstruction unit collects real-time data of the current space based on a lidar and generates a spatial point model; the spatial structure recognition unit performs semantic segmentation on the spatial point model to identify ceilings, walls, and pipe well structures; the drawing registration unit spatially aligns the drawing structure with the on-site structure to detect errors or misalignments; the environmental constraint analysis unit analyzes the constraints of construction occlusion and angle restrictions in the on-site environment to form boundary conditions for subsequent layout planning.

[0060] Optionally, the three-dimensional reconstruction unit includes a laser scanning acquisition subunit, a coordinate conversion subunit, a SLAM stitching subunit and a point cloud optimization processing subunit; wherein, the laser scanning acquisition subunit acquires the spatial depth information of the construction site in real time based on the rotating laser radar, the coordinate conversion subunit converts the collected polar coordinate data into spatial Cartesian coordinate points, the SLAM stitching subunit performs posture correction and spatial stitching on continuous frame point clouds based on inertial navigation and laser odometry, and the point cloud optimization processing subunit performs voxel filtering, ground removal and outlier elimination on the point cloud to form a spatial point cloud model that can be used for structure recognition and path planning.

[0061] Specifically, the three-dimensional reconstruction unit is executed according to the following steps to form a spatial point cloud model that can be used for structure recognition and path planning;

[0062] S1. Activate the laser scanning acquisition subunit to trigger the rotating laser radar device to enter the static scanning mode. The laser radar emits a laser beam at a preset angle (e.g., 360° horizontal + 16 vertical lines). Each laser beam is reflected after encountering an object, returning a distance value r and a reflection intensity I. At the same time, the horizontal angle θ, vertical angle φ, and timestamp t of each laser point are recorded.

[0063] Among them, the following original laser point set records are obtained after collection:

[0064]

[0065] Where N is the number of laser points, p i polar Represents the original polar coordinate data item of a point, ri is the straight-line distance from the i-th laser point to the center of the lidar sensor; obtained by calculating the laser flight time, θ i is the scanning angle of the i-th laser beam in the horizontal direction (usually around the Z axis), φ i is the pitch angle of the i-th laser beam relative to the horizontal plane, I i is the intensity of the return signal after the i-th laser beam is reflected, t i is the time when the i-th laser beam is collected.

[0066] S2, the coordinate conversion subunit converts the polar coordinate point p i polar Mapped to Cartesian coordinate points through the following function f: The processing formula is as follows:

[0067]

[0068] Where r i is the straight-line distance from the i-th laser point to the center of the lidar sensor; obtained by calculating the laser flight time, θ i is the scanning angle of the i-th laser beam in the horizontal direction (usually around the Z axis), φ i is the pitch angle of the i-th laser beam relative to the horizontal plane

[0069] After processing according to the above formula, the output point cloud set is:

[0070] P cartesian ={p i =(x i ,y i ,z i ,I i ,t i )|i=1,2,...,N};

[0071] S3, the SLAM stitching subunit pre-processes the electronic cloud set, and the pre-processing operations include voxel filtering, outlier removal, and ground removal;

[0072] Among them, voxel filtering is performed on the point cloud set, and the voxel resolution δ is set x ,δ y ,δ z , generally uniformly set to δ = 0.05 meters;

[0073] For each point pi, calculate the voxel coordinates to which it belongs:

[0074]

[0075] All points with the same voxel coordinates are grouped into one category, denoted as V j ;

[0076] For each point set V in a voxel j , calculate its center of mass:

[0077]

[0078] Where V j is the set of points in the jth voxel, p∈V j For point p to belong to the voxel, that is, each point p is a three-dimensional vector p = (x, y, z), c j is the representative point of the j-th voxel.

[0079] The meaning of the overall formula in the above formula is: for voxel V j For all points p in the voxel, calculate their geometric center (i.e., center of mass) as the representative point c of the voxel j .

[0080] The point cloud set after the above filtering is:

[0081] P voxel ={c j |j=1,2,…,M};

[0082] After voxel filtering, outlier removal is performed, specifically:

[0083] For each point p i , find its k-neighborhood (e.g. k=20);

[0084] Calculate the average Euclidean distance d between the point and its neighbors i :

[0085]

[0086] Where p i =(x i ,y i ,z i ), p j =(x j ,y j ,z j ), ||*||2 is the three-dimensional Euclidean distance;

[0087] For all d i Calculate the global mean μ and standard deviation σ;

[0088] Among them, all d i The global mean is calculated as follows:

[0089]

[0090] Where N is the total number of points in the point cloud;

[0091] All d i The global standard deviation σ is calculated according to the following formula:

[0092]

[0093] Where N is the total number of points in the point cloud; μ is the total number of all d i The global mean of the values.

[0094] Set the rejection threshold parameter α, if the d of a point i >μ+α·σ, it is determined to be an outlier.

[0095] In this embodiment, the value range of the rejection threshold parameter α is: 1.0 to 1.5; at the same time, in this embodiment, an example of the value of the rejection threshold parameter α is provided:

[0096] 1) In the scenario where the firefighting robot is working in a suspended ceiling space (the point cloud density is relatively uniform and there are a large number of flat areas (such as ceilings and air conditioning panels)), the rejection threshold parameter α = 1.0;

[0097] 2) In scenes with complex scaffolding and debris accumulation at the construction site (the point cloud contains areas of various densities, local stray points are obvious, but some points may be structure edges), the rejection threshold parameter α = 1.5;

[0098] 3) In scenes with extremely dense points and rich details, the rejection threshold parameter α = 0.8.

[0099] In short, the specific rejection threshold parameter α needs to be set in combination with actual needs and input through the human-computer interaction interface. This is a technical means well known to those skilled in the art, and therefore will not be described in detail in this embodiment.

[0100] After the outlier removal operation, the denoised point set is obtained:

[0101] P denoised ={p i ∈P voxel ∣d i ≤μ+α·σ};

[0102] Then the denoised point set P denoised , set a Z axis minimum height threshold Z min , as in this embodiment, set to Z min = 0.1m, delete all points: P filtered ={p i ∈P denoised ∣z i >z min};

[0103] The final point cloud set of spatial points is:

[0104] P clean =P filtered ;

[0105] S4. Obtain the final point cloud set, and the point cloud optimization processing subunit performs plane structure extraction. Before performing plane structure extraction, the point cloud optimization processing subunit first sets the initial parameters of plane fitting: including the minimum number of plane support points n min , that is, each effective plane fitted should contain at least n min points, which is set to 50 in this embodiment; the maximum tolerance distance δ from the set point to the fitting plane plane , used to determine whether a point belongs to the plane, which is set to 0.02 meters in this embodiment; set the maximum number of iterations T of the fitting algorithm, which is set to 1000 times in this embodiment to control the number of RANSAC iterations. Next, the system will obtain the pre-processed point cloud data P in the previous step. clean , initialized to the current set of points to be processed P remain , and is used to start the plane recognition process. At the same time, the plane region number variable k is initialized to 1, which is used to record the sequence number of each extracted plane region.

[0106] Then, the point cloud optimization processing subunit enters the iterative extraction process. In each round, remain By randomly sampling three points, a plane equation π is fitted k : ax+by+cz+d=0, where a, b, c are the normal vector components of the fitted plane, and d is the offset constant of the plane in three-dimensional space;

[0107] Next, calculate P remain The distance from all points to the plane is determined to be within the tolerance range δ plane If a point satisfies the following conditions:

[0108] Then it is classified into the point set R in the current plane k .

[0109] If R k The number of points satisfies: |R k ∣≥n min , then the plane is considered to be a valid plane, and it is used as the kth identifiable area, and R k All points from P remain Remove. The region number k increases by 1 and continues to the next round of iteration. If the current fitting plane does not meet the point number condition, the result of this round is discarded and not retained. When the remaining points are not enough to fit the new plane, the iteration process ends. For each successfully extracted region R k, the point cloud optimization processing subunit calculates its corresponding normal vector n k =(a,b,c), and classify them semantically according to the normal direction: when |n k·z When ∣>0.9, it is judged to be a horizontal structure, such as a ceiling or floor; when ∣n k·z When |<0.1, it is judged to be a vertical structure, such as a wall surface; other angles are judged to be inclined surfaces or unrecognized areas, and further judgment or marking as "other areas" is performed according to the scene.

[0110] Finally, for each region R k ={p=(x i ,y i ,z i )|i=1,2,…,n k}Calculate its spatial bounding box and encapsulate the point set, plane equation parameters, normal vector, semantic label and bounding box information into a structured output to form the output result.

[0111] The output result is used as a spatial structure model where R k is a point set, is the normal vector, bbox k is the bounding box.

[0112] In this embodiment, for each extracted plane area, the spatial bounding box of the area is calculated for subsequent point placement feasibility assessment. The specific method is: the minimum and maximum values of all points in the area in the x, y, and z axes are calculated respectively to form the axis-aligned bounding box of the area (taking the axis-aligned bounding box as an example, that is, the construction structure (walls, ceilings) is axis-aligned in architectural design). The calculation method is:

[0113]

[0114] Where R k is the point set of the k-th plane area, p i ∈R k is the i-th point in the point set, (x i 、y i 、z i ) is the spatial coordinate value of the i-th point, x min , x max is a plane area R k The minimum and maximum values in the X-axis direction, y min ,y max is a plane area R k The minimum and maximum values in the Y-axis direction, z min , z max is a plane area R kThe minimum and maximum values in the Z-axis direction;

[0115] The bounding box constructed in this way is used to define the spatial occupancy range of each structural area and provide a spatial boundary basis for the possible point analysis and path obstacle avoidance planning in subsequent installation tasks.

[0116] The spatial structure recognition unit calculates the normal vector n of each region k The dot product with the vertical unit vector z=(0,0,1):

[0117]

[0118] If|n k,z |>0.9, it is a horizontal plane and is higher than the set height threshold Z max K ;

[0119] If|n k,z |<0.1, it is a vertical surface and is considered a wall;

[0120] If the boundary position is close to the structural shaft range, it is considered to be the shaft surface.

[0121] Convert the above judgment results into semantic label field l k :

[0122]

[0123] The output structured semantic region set is:

[0124] The drawing registration unit matches the corresponding installation surface area according to the device type (such as sprinkler head, smoke detector) and the semantic structure label, and performs drawing registration according to the following steps:

[0125] S11. Type semantic matching and spatial range matching:

[0126] 1) Type semantic matching:

[0127] If the firefighting equipment in the drawing is a sprinkler head, the point cloud optimization processing subunit will only search for the corresponding installation surface in the structural area with the semantic label of ceiling;

[0128] If the fire protection equipment is a smoke detector, it should also be installed in the ceiling area;

[0129] If the fire protection equipment is a fire hydrant or other wall-mounted equipment, the structural area matches the wall type;

[0130] 2) Spatial range matching

[0131] Determine the location of fire fighting equipment i图纸 Is it in a certain area bbox k within, or with p i 图纸 Center c k If the distance is within the threshold δ, the fire-fighting equipment position and the structural surface are paired;

[0132] S12. Calculate the spatial alignment transformation:

[0133] Obtain the fire equipment layout location point set (drawing model geometry), structural surface point set (on-site point cloud), and construct an optimization objective function:

[0134]

[0135] And iteratively solve the optimal transformation: T * =(R * ,t * );

[0136] Where, T * is the optimal spatial rigid body transformation, R * is the optimal rotation matrix, t * is the optimal translation vector;

[0137] In addition, the above also satisfies:

[0138] Where q j is the jth point on the component model in the drawing, p j is the structure surface point cloud with q j The closest point of the match;

[0139] The optimization goal is to calculate a set of spatial rotation matrices and translation vectors so that each 3D point on the surface of the firefighting equipment placement, after undergoing these rotation and translation transformations, is as close as possible to the corresponding point on the on-site structural surface. In other words, the goal is to ensure that the firefighting equipment placement, after transformation, fits the entire surface onto the on-site structural surface, achieving optimal alignment in shape and position. The transformation proximity of pairs is measured by minimizing the sum of the squared Euclidean distances between all pairs of points.

[0140] The environmental constraint analysis unit obtains the registered fire equipment installation list and the structural surface point cloud P of the matching area. k , normal vector n k , as well as the robot's operating parameters (arm span, tool orientation, installation angle range), and determine whether there are other structures above or in front of the fire protection project installation location that affect the construction operation.

[0141] Specifically, at the fire equipment installation point p i 对齐A cylindrical or cubic detection volume is constructed above (set as: radius 0.2m, height 0.5m in this embodiment), and all points in the volume are extracted from the point cloud, and the number or density of points is counted; at the same time, a threshold ρ is set. thresh If the number of points exceeds this value, it is judged to be severely occluded.

[0142] In this embodiment, the environmental constraint analysis unit further performs angle restriction judgment, that is, judging whether the component installation orientation matches the construction angle range allowed by the robot.

[0143] Specifically, the environmental constraint analysis unit performs the angle θ according to the following formula: i Calculation:

[0144]

[0145] Where, is the transformed orientation vector of the fire-fighting equipment, that is, its orientation in the actual structural space after completing the spatial registration. is the normal vector of the structural surface;

[0146] in,

[0147] Where R* is the optimal rotation matrix, The factory installation direction of the fire-fighting equipment marked in the drawings / BIM model.

[0148] If the angle θ i If the angle exceeds the robot's workable range (such as 15° to 75° as defined in this embodiment), it is considered to be angle-restricted.

[0149] If the corresponding position of the component is marked as: unconstructable or severely obstructed, the layout planning module will remove it from the candidate layout points;

[0150] If the angle exceeds the limit or the position is unreachable, the action module needs to regenerate the path or trigger a fallback.

[0151] Through the cooperation of the task parsing module and the spatial modeling module, the equipment intent in the drawing task can be accurately mapped to the spatial structure of the actual construction site, achieving spatial consistency between the task goal and the real-scene structure, and ensuring the locatability of the equipment layout and the adaptability to the construction environment.

[0152] Optionally, the layout planning module includes a candidate point generation unit, an intention mapping unit, a constraint calculation unit, and an optimization layout unit. The candidate point generation unit generates installation candidate points that meet basic specifications within the constructible area. The intention mapping unit constructs the installation candidate points into a spatial intention field to guide the formation of a priority layout sequence. The constraint calculation unit filters the candidate points in the priority layout sequence according to the installation specifications. The optimization layout unit optimizes the total path length, resource utilization, and equipment balance indicators based on the specifications to output the final layout plan. The constructible area is identified and determined by the spatial modeling module. The system first obtains a set of structural surfaces with semantic labels and spatial boundaries through the spatial structure identification unit and the drawing registration unit. Then, combined with the environmental constraint analysis results, it eliminates areas that are blocked, have limited angles, or are insufficient in size, and finally forms a limited set of constructible areas. The candidate point generation unit only performs standardized layout within the above-mentioned area to ensure the feasibility and accuracy of the layout plan.

[0153] The candidate point generation unit creates a regular grid on the target structure surface (divided into 0.5m grids in this embodiment) and filters the grid points according to the installation specifications, that is, the distance to the boundary ≥ the safety margin (such as 0.3m), the distance between adjacent points ≥ the installation distance (such as 2.0m), and at the same time, each grid center that meets the conditions is used as a candidate point c j (x j ,y j , z j );

[0154] The intention mapping unit is based on the layout area R target , construct the spatial intention potential energy function:

[0155]

[0156] Where (x, y) is the two-dimensional coordinate of the candidate point, that is, the two-dimensional coordinate of the candidate layout point on the structural surface, r i is the coordinate of the center point of the target area, ||(x,y)-r i || is the spatial distance between the candidate point and the i-th target center point, σ is the Gaussian diffusion coefficient. In this embodiment, the setting range is 1.0 to 2.0. When the spatial range is small and the layout accuracy is high, σ = 1.0; when the space is large and the layout freedom is high, σ = 2.0.

[0157] In the present invention, the intention mapping unit firstly calculates the target area set R based on the task target area set R. target Construct a spatial intention potential energy function V(x,y) to simulate the layout preference of different locations in the space;

[0158] Then, for the candidate deployment point set C = {c j Each point c inj Perform function sampling to obtain intent scores:

[0159] v j =V(c j );

[0160] Finally, the intent mapping unit calculates the intent value v j Sort the candidate points from high to low to form a priority point sequence to guide the subsequent constraint calculation and layout optimization process.

[0161] The constraint calculation unit calculates each candidate point c j , perform the following judgment:

[0162] 1) Whether it falls into the occlusion area;

[0163] 2) Whether the angle exceeds the limit (angle with the structure normal);

[0164] 3) Is it close to existing equipment, violating the minimum spacing?

[0165] 4) Whether it falls into the non-construction area;

[0166] If any of the four conditions is not met, it will be eliminated. At the same time, the candidate points that are not eliminated are gathered together to form the candidate point set C valid ;

[0167] Optimize the layout unit for each candidate point c j , calculate the comprehensive score s j :

[0168] s j =λ1·v j +λ2·a j ;

[0169] Where a j is the constructability score, v j is the intention value, the closer to the target area, the higher the score, λ1 and λ2 are weight coefficients, and λ1+λ2=1;

[0170] In this embodiment, the constructability score a j The value is determined according to the following conditions:

[0171] Is there any obstruction in the unobstructed point cloud within 0.5m above the candidate point: with obstruction: 0, without obstruction: 1;

[0172] Whether the angle between the installation direction and the structural surface is within the allowable range: Exceed limit: 0, Compliant: 1;

[0173] Whether the robot can reach the area within the robot's arm span: Unreachable: 0, Reachable: 1

[0174] In this embodiment, an example of a value of a weight coefficient is provided, specifically:

[0175] 1) In a scenario where sprinklers are installed on the ceiling and the user specifies two rooms as the core area, λ1 = 0.7.

[0176] λ2=0.3;

[0177] 2) When a gas control valve is installed in the pipeline well and the construction angle is limited, λ1 = 0.3 and λ2 = 0.7;

[0178] 3) When smoke detectors are installed in corridors, the target area is large and the space is open, then λ1 = 0.5 and λ2 = 0.5;

[0179] 4) In the scenario where an alarm is installed in the stairwell and is severely obstructed by the floor, λ1 = 0.4 and λ2 = 0.6;

[0180] For any candidate point c j =(x j ,y j ), whose intent value is determined by the following formula:

[0181]

[0182] Where c j =(x j ,y j ) is the coordinate of the candidate point, and its value is output by the candidate point generation unit, r i =(x i ,y i ) is the center coordinate of the target area, and its value is based on the R output by the task analysis module target , σ is the Gaussian diffusion coefficient, which is set to 1.0-2.0 in this embodiment, and ||·|| is the Euclidean distance.

[0183] In this embodiment, if the task requires the layout of 20 devices, they can be divided equally by area or by area ratio, such as: Z1: 8, Z2: 6, Z3: 6; on this basis, the optimized layout unit will be in each sub-area Z k From the corresponding valid candidate point set C valid ∩Z k According to the comprehensive score j Select N from high to low k points, where N k The number of layout tasks pre-allocated to the area, and then the point sets selected in each sub-area are merged to obtain the final global layout solution point set C final .

[0184] Through the coordination of the spatial modeling module and the layout planning module, the installation points generated during the layout planning process are only placed within the actual constructible area, avoiding problems such as structural obstruction and spatial errors, and ensuring the feasibility and accuracy of the layout plan.

[0185] In addition, through the cooperation of the task analysis module and the layout planning module, the system can automatically build a layout strategy based on the equipment type, installation requirements, and layout parameters extracted from the task, avoiding manual configuration and repeated debugging, and ensuring that the planning results are automatically generated under the premise of meeting the task objectives.

[0186] Optionally, the motion control module includes a path planning unit, an action sequence generation unit, and a state feedback monitoring unit. The path planning unit generates an obstacle avoidance path for the robot from the current position to each installation point based on the on-site point cloud data and the layout point information; the action sequence generation unit breaks down each installation task into specific action instructions; and the state feedback monitoring unit detects the real-time status of the installation robotic arm in real time.

[0187] The real-time status includes whether the device is in place, whether the action is completed, and whether an abnormality occurs.

[0188] Optionally, the action instructions include raising the arm, punching, installing, and testing.

[0189] The path planning unit loads the on-site point cloud data and layout point information, extracts the current robot position and the coordinates of the installation target point, establishes a simplified spatial grid model in the point cloud data, and uses it to determine the obstacle distribution and passable area.

[0190] In addition, a regularized search method (such as grid pathfinding and linear interpolation) is used to calculate an obstacle-avoiding path between the starting point and the target point. If there is an obstacle blocking the way, turning points are automatically generated along the shortest feasible detour path; the path can be simplified into several moving-stopping segments.

[0191] Specifically, when the path planning unit constructs a path between the starting point and the target point, it generates path points by linear interpolation by default. The path function is:

[0192]

[0193] Where, P i is the i-th path point, P star is the starting point coordinate, P goal is the end point coordinate, the difference segment number, and i is the current difference number;

[0194] If the minimum distance between a path point and an obstacle is dist(P i ,O j ) is less than the system default safety threshold (known value), then the detour point P is automatically inserted.via , construct a broken line path, and form an obstacle avoidance path sequence consisting of: moving---docking.

[0195] Among them, the minimum distance between the path point and the obstacle is dist(P i ,O j ) is calculated according to the following formula:

[0196]

[0197] Where, P i is the i-th path point, O j is the jth obstacle point, (x i ,y i ,z i ) is the coordinate of the path point, (x j ,y j ,z j ) are the coordinates of the obstacle point;

[0198] O corresponding to the j-th obstacle point j The value of is obtained from the obstacle point set O = {O j It is derived from the on-site structure point cloud model, and obstacle points are extracted using the following rules:

[0199] 1) Eliminate the point cloud area of the constructible surface;

[0200] 2) Filter the ground or low-altitude areas;

[0201] 3) Perform density clustering on the remaining point clouds to retain the points of fixed structure objects in space;

[0202] The final obstacle point.

[0203] The action sequence generation unit queries the action template corresponding to the fire-fighting equipment type: such as sprinkler head → arm lifting → positioning → drilling → fixing → testing, and inserts the action at the end of the path to form an action queue. Each step of the action is encoded as a specific control instruction (such as controlling joints, motors, grippers, etc.), and finally forms an action quality sequence list A. j ={a1, a2, ... a n}.

[0204] Among them, the action templates corresponding to different types of fire-fighting equipment are preset in the database and can be called when used.

[0205] In this embodiment, the mounting robot arm is provided with at least two joints and a control panel, and posture sensors are provided on at least two joints to obtain the posture data of the manipulator; the control panel collects the motion data stream of the mounting robot arm, thereby collecting the motion number and feedback timestamp of the mounting robot arm.

[0206] In addition, the actuator end of the installation robot arm is equipped with corresponding installation tools and torque sensors according to actual needs. The installation tools include but are not limited to the following: a hole drill and a suction plate. The torque sensor collects the torque generated by the contact between the actuator end of the installation robot arm and the firefighting equipment.

[0207] Among them, the hole drill is used to drill holes on the installation surface, and the adsorption plate is used to adsorb the installed fire-fighting equipment, and move through at least two joints of the installation robotic arm to install the fire-fighting equipment at the installation position corresponding to the drilling position.

[0208] The installation robot arm also includes a motion controller that controls the joints according to control instructions to achieve installation of the firefighting equipment. Controlling the joints with control instructions is a conventional control method that is well known and mastered by those skilled in the art and will not be described in detail in this embodiment.

[0209] The status feedback monitoring unit monitors the robot's task execution status in real time, detecting whether it is in place, action completed, and any abnormalities have occurred.

[0210] Specifically, the state feedback monitoring unit determines whether the installation point has been reached, that is, compares the distance between the current position and the target point to see whether it is within an allowable range;

[0211] In addition, the state feedback monitoring unit performs action execution completion detection, that is, monitoring the action completion signal feedback (such as torque, time, limiter status);

[0212] At the same time, the state feedback monitoring unit also detects abnormal indicators, including: whether it is unreachable, timeout, drilling failure, and sensor exceeding the standard. If an abnormal indicator occurs, re-planning or alarming is performed.

[0213] Through the cooperation of the layout planning module and the motion control module, the planned installation points can be converted into executable motion paths in real time, avoiding construction failures due to unreachable layout or path conflicts, and ensuring the consistency of robot execution and path safety.

[0214] Optionally, the quality verification module includes an image comparison unit, a mechanical detection unit, an error judgment unit, and a task reflow trigger. The image comparison unit takes a picture of the installed part and compares it with the drawing or BIM model; the mechanical detection unit uses a torque sensor to detect whether the fire-fighting equipment is installed firmly; the error judgment unit analyzes the image and sensor data to determine whether the error threshold is exceeded; if the task reflow trigger fails to verify, it initiates a re-execution instruction to the layout planning module or the action module.

[0215] Optionally, the parameters used by the image comparison unit to compare the drawing with the BIM model include position, direction, and angle.

[0216] The image comparison unit includes an image collector, a data storage device, and an image comparator. The image collector collects image data of the installation part, the data storage device stores the image data of the installation part collected by the image collector, and the image comparator analyzes the quality of the installation based on the image data.

[0217] Among them, the on-site image after the image comparator is installed, as well as the position and installation direction of the fire-fighting equipment in the drawing model or BIM data, and the installation position error Δp is calculated according to the following formula:

[0218]

[0219] Where p actual is the actual installation position vector, p design is the design installation position vector, (x a ,y a , z a ) is the actual installation point coordinate, (x a ,y a , z a ) are the coordinates of the designed installation point.

[0220] If Δp is less than δ p (In this embodiment, it is set to δ p =20mm), it is qualified;

[0221] If Δp is greater than or equal to δ p , it is unqualified.

[0222] The image comparator calculates the installation angle error Δθ according to the following formula:

[0223]

[0224] Where, is the actual installation normal vector, whose value is the normal vector of the device installation surface (or the device itself) identified from the image or depth point cloud. It is the actual installation normal vector, and its value is determined by the drawing or BIM model.

[0225] If Δθ is less than θmax (in this embodiment, θmax is set to 10°), the installation angle is qualified;

[0226] If Δθ is greater than or equal to θmax, the installation is unqualified.

[0227] In other embodiments, the design installation direction of the target device can also be extracted from the drawing or model to generate the unit normal vector At the same time, the image comparison unit uses RGB-D images or structured light reconstruction results to extract the local point cloud of the installed area and estimates the normal vector of the installation surface through the principal component analysis method. The angle between the two is used to determine the attitude deviation.

[0228] The mechanical detection unit obtains the slight reverse pulling force applied by the installation robot to the target device after the installation is completed, and reads the feedback force value and displacement value of the torque sensor, and calculates the looseness measurement factor I according to the feedback force value and displacement value. loose , looseness measurement factor I loose Calculate according to the following formula:

[0229]

[0230] Where △x max is the actual maximum displacement, x ref is the reference displacement, F ref is the reference force value, F avg is the actual average force, ε is the dimensionless smoothing factor, and its value is equal to 0.001

[0231] The actual average applied force is calculated according to the following formula:

[0232]

[0233] Where, F i is the force value of the i-th sampling, which is the instantaneous force value read by the sensor at the end of the robotic arm at the i-th sampling time, n is the number of sampling points, F avg is the average opposing force.

[0234] The error judgment unit obtains the image comparison error and the mechanical judgment result (if the looseness measurement factor I loose If the looseness threshold is exceeded, it is unqualified, and vice versa). If any condition is not met, it is marked as installation failure.

[0235] The task reflow trigger calls the judgment result of the error judgment unit in real time. If the error judgment is: unqualified, it executes:

[0236] 1) Notify the action control module to reinstall;

[0237] 2) Notify the layout planning module to reallocate the layout point (for example, the current point is not constructible);

[0238] 3) The alarm enters the manual confirmation queue.

[0239] Through the cooperation of the motion control module and the quality verification module, the system can perform automated quality inspection immediately after executing the installation task, and trigger the reflow correction operation based on the inspection results to ensure closed-loop control and quality consistency of the installation effect.

[0240] Through the coordinated cooperation of the task analysis module, space modeling module, layout planning module, motion control module and quality verification module, the fire protection engineering installation task can realize the full process automation processing from multimodal task identification, space environment modeling, standard constraint layout, automatic motion control to quality self-inspection closed loop, ensuring that the system has the four-in-one intelligent scheduling capability and self-repair capability of task-space-control-verification.

[0241] At the same time, through the layered collaboration of the task analysis module, spatial modeling module, layout planning module, motion control module and quality verification module, the entire installation task from task input to construction completion is intelligently automated, including the joint drive of task analysis and spatial modeling data, the seamless connection of planning and control actions, and the feedback and correction of result verification and scheduling strategies, forming a closed-loop control link, which improves the system's adaptability, fault tolerance and deployment efficiency.

[0242] Embodiment 2: This embodiment should be understood to include all the features of any of the above embodiments and further improve upon them. Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 ,as well as Figure 5 As shown, the verification module also includes a global evaluation unit, which obtains the judgment result of the quality verification of the current installation area after the installation task is completed, and calculates the regional quality hot spot density H:

[0243]

[0244] Where m is the total number of installed devices, M(i) is the neighbor set of the i-th device, that is, the neighbor set must satisfy the set of adjacent devices that are less than or equal to the spatial safety distance R, where R is set to 0.5m in this embodiment; δ j is the qualified status tag of the installation (its value is determined by the judgment result of the quality verification module). If the device j fails to meet the requirements, then δ j =1; if qualified, it is 0.

[0245] The regional quality hot spot density H is used to reflect whether the unqualified points are spatially clustered. If the density of unqualified points in a certain area is concentrated and the adjacent points are close to each other, the H value will increase significantly, which is considered to be a trend of local installation degradation.

[0246] In this embodiment, an abnormal monitoring threshold UNHealthy is set if: H>UNHealthy;

[0247] The following task reflux mechanism is triggered:

[0248] Send re-execution instructions for unqualified points to the action control module;

[0249] If multiple points in the neighborhood fail consecutively, a request is made to the layout planning module to redistribute points in the local area;

[0250] The area will be marked as a quality hotspot and included in the priority re-inspection area for subsequent construction.

[0251] The system sets the abnormal monitoring threshold, UNHealthy, based on the density of devices deployed and the distribution of installed equipment. For example, in a typical office building corridor, if each device has an average of four spatial neighbors, we recommend setting UNHealthy to 1.5. This means that if there are more than 1.5 unqualified points in the neighborhood of each device, the area is considered to have an abnormal installation trend.

[0252] In short, the abnormal monitoring threshold UNHealthy is set by the system according to actual conditions and input through the human-computer interaction interface. This is a technical means well known to technicians in this field, so it will not be described in detail in this embodiment.

[0253] Through the coordination of the global evaluation unit with the image comparison unit, mechanical detection unit, and error judgment unit, the system can not only inspect the quality of a single installation point item by item, but also conduct hot spot trend analysis and quality aggregation judgment on the construction quality of the entire installation area after the task is completed, ensuring that the system has the ability to close the loop of construction quality from local judgment to global early warning.

[0254] The contents disclosed above are only preferred feasible embodiments of the present invention and do not limit the scope of protection of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention description and drawings are included in the scope of protection of the present invention. In addition, the elements therein can be updated as technology develops.

Claims

1. A fire engineering robot installation system based on a multimodal model, the fire engineering robot installation system comprising a server and an installation robot arm, characterized in that: The fire engineering robot installation system further includes a task analysis module, a space modeling module, a layout planning module, a motion control module and a quality verification module, and the server is connected to the task analysis module, the space modeling module, the layout planning module, the motion control module and the quality verification module respectively; The task parsing module is used to parse the installation task and integrate the multimodal input of drawings and BIM models to extract key information such as equipment type, target area, and layout parameters, and output structured task instructions; The spatial modeling module performs three-dimensional structural modeling of the construction site to form a spatial point model, identifies key building components and constructible areas, completes high-precision registration of the drawing structure with the actual scene structure, and analyzes environmental constraints; The layout planning module generates a fire equipment layout plan based on the task instructions and the spatial point model, while meeting the regulatory requirements; The motion control module converts the layout plan corresponding to the installation location and the type of fire-fighting equipment installed into a sequence of motion instructions executable by the robot, and avoids the robot's task coordination and path conflicts; The quality verification module automatically detects the installation results and can trigger automatic correction actions or re-plan the installation task if an anomaly is found.

2. The fire engineering robot installation system based on a multimodal model according to claim 1 is characterized in that: The task parsing module includes a drawing parsing unit, a specification extraction unit, and a multimodal fusion unit. The drawing parsing unit parses CAD drawings and BIM models to extract the installation information of fire-fighting equipment; the specification extraction unit parses national / industry specification documents to extract standard parameters of fire-fighting equipment; the multimodal fusion unit fuses drawings, specifications and operating requirements, as well as correction information, and outputs task target information in a unified format.

3. The fire engineering robot installation system based on a multimodal model according to claim 2, characterized in that: The spatial modeling module includes a three-dimensional reconstruction unit, a spatial structure recognition unit, a drawing registration unit, and an environmental constraint analysis unit. The three-dimensional reconstruction unit collects real-time data of the current space based on a lidar and generates a spatial point model; the spatial structure recognition unit performs semantic segmentation on the spatial point model to identify ceilings, walls, and pipe shaft structures; the drawing registration unit spatially aligns the drawing structure with the on-site structure, detects errors or misalignments, and forms a registered fire-fighting equipment list; the environmental constraint analysis unit obtains the registered fire-fighting equipment installation list and analyzes the constraints of construction occlusion and angle restrictions in the on-site environment to form boundary conditions for subsequent layout planning.

4. The fire engineering robot installation system based on a multimodal model according to claim 3 is characterized in that: The layout planning module includes a candidate point generation unit, an intention mapping unit, a constraint calculation unit, and an optimization layout unit. The candidate point generation unit generates installation candidate points that meet basic specifications within the constructible area. The intention mapping unit constructs the installation candidate points into a spatial intention field to guide the formation of a priority layout sequence; the constraint calculation unit filters the candidate points in the priority layout sequence according to the installation specifications; the optimization layout unit optimizes the total path length, resource utilization, and equipment balance indicators based on meeting the specifications to output the final layout plan.

5. The fire engineering robot installation system based on a multimodal model according to claim 4 is characterized in that: The motion control module includes a path planning unit, an action sequence generation unit, and a state feedback monitoring unit. The path planning unit generates an obstacle avoidance path for the robot from its current position to each installation point; the action sequence generation unit breaks down each installation task into specific action instructions; and the state feedback monitoring unit detects the real-time status of the installation robotic arm in real time.

6. The fire engineering robot installation system based on a multimodal model according to claim 5, characterized in that: The quality verification module includes an image comparison unit, a mechanical detection unit, an error judgment unit, and a task reflow trigger. The image comparison unit takes a picture of the installed part and compares it with the drawing or BIM model; the mechanical detection unit uses a torque sensor to detect whether the fire-fighting equipment is firmly installed. The error judgment unit analyzes the image and sensor data to determine whether an error threshold is exceeded; If the task reflow trigger fails to verify, it will send a re-execution instruction to the layout planning module or the action control module.

7. The fire engineering robot installation system based on a multimodal model according to claim 6, characterized in that: The three-dimensional reconstruction unit includes a laser scanning acquisition subunit, a coordinate conversion subunit, a SLAM stitching subunit and a point cloud optimization processing subunit; wherein, the laser scanning acquisition subunit obtains the spatial depth information of the construction site in real time based on the rotating laser radar, the coordinate conversion subunit converts the collected polar coordinate data into spatial Cartesian coordinate points, the SLAM stitching subunit performs posture correction and spatial stitching on continuous frame point clouds based on inertial navigation and laser odometry, and the point cloud optimization processing subunit performs voxel filtering, ground removal and outlier elimination on the point cloud to form a spatial point cloud model that can be used for structure recognition and path planning.

8. The fire engineering robot installation system based on a multimodal model according to claim 7, characterized in that: The image comparison unit compares the drawing with the BIM model using parameters including position, direction, and angle.

9. The fire engineering robot installation system based on a multimodal model according to claim 8, characterized in that: The action instructions include lifting the arm, punching, installing, and testing.

10. The fire engineering robot installation system based on a multimodal model according to any one of claims 2 or 9, characterized in that: The installation information of the fire-fighting equipment includes equipment type, number, and location label.

Images