Intelligent engineering detection robot system for cooperative detection of multi-modal sensor

Abstract

The invention relates to the technical field of engineering structure detection, and discloses an intelligent engineering detection robot system for cooperative detection of a multi-modal sensor, which comprises a mobile navigation module, a multi-modal detection tool module, a mechanical arm execution module, a sensor fusion and force application control module and a self-inspection and calibration module. High-precision navigation is realized by fusing laser SLAM and BIM models, multiple detection tools are integrated, modular quick change is supported, detection actions are accurately executed through a mechanical arm pose control and trajectory tracking algorithm, detection stability and accuracy are ensured by combining multi-sensor data fusion and dynamic force application adjustment, and automatic calibration and fault early warning capabilities are achieved. And the automation and intelligence level of engineering structure detection is obviously improved.

Description

Technical Field

[0001] This invention relates to the field of engineering structure inspection technology, and discloses an intelligent inspection robot system based on multimodal sensor collaborative control, which is particularly suitable for high-precision non-destructive testing and intelligent diagnosis of engineering entities such as concrete structures and steel reinforcement layouts in complex building environments. Background Technology

[0002] Currently, the quality inspection of engineering structures mainly relies on manual operation, with commonly used testing tools including rebound hammers, rebar measuring instruments, and core drilling equipment. Due to the dispersed nature of these tools and the complexity of their operation, manual inspection often suffers from low efficiency, poor data consistency, and significant human error. In multi-story buildings, confined structural areas, or hazardous work environments, the operational difficulty and safety risks for inspectors further increase, severely impacting the comprehensiveness and reliability of the inspection work.

[0003] To improve inspection efficiency and safety, some research and engineering practices have introduced robot-assisted inspection technology, using mobile platforms or robotic arms to perform single inspection tasks. However, existing systems mostly focus on a single type of inspection method, lacking integration and coordination mechanisms for multiple inspection tools, and are unable to flexibly select appropriate inspection methods based on the complexity of the engineering structure. Furthermore, path planning often relies on preset trajectories or simple obstacle avoidance algorithms, making it difficult to combine BIM and structural risk factors for intelligent judgment and efficient path generation.

[0004] Furthermore, inspection robots generally lack the ability to fuse and process sensor data and precisely control the force applied at the end effector during inspection. Due to the significant differences in the surface of building structures, fixed force application strategies can easily lead to decreased inspection accuracy or even damage to the structure itself. Existing systems lack automatic calibration, sensor error monitoring, and self-correction mechanisms, making it impossible to achieve high reliability and long-term continuous operation. Therefore, there is an urgent need to develop an intelligent engineering inspection robot system with rapid switching between multimodal tools, BIM-based intelligent path planning, multi-sensor fusion, and adaptive force control capabilities to meet the pressing needs of modern construction engineering for efficient, safe, and accurate inspection. Summary of the Invention

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solution: an intelligent engineering inspection robot system for multimodal sensor collaborative detection, comprising the following modules:

[0006] The mobile navigation module is used to build a 3D environment map of the site based on BIM building information model and laser SLAM technology, and automatically complete path planning and navigation execution;

[0007] The multimodal testing tool module integrates various engineering structure testing tools, including rebound hammers, drill bits, and rebar measuring instruments, and achieves rapid switching and automatic identification of testing tools through a modular quick-change mechanism;

[0008] The robotic arm execution module is used to accurately perform detection actions at the target detection position, and achieves precise positioning and stable operation of the tool through pose feedback and trajectory tracking algorithms;

[0009] The sensor fusion and force control module is used to collect and fuse feedback data from pressure sensors, gyroscopes, and current sensors in real time, and dynamically adjust the direction and force of the robotic arm's end effector.

[0010] The self-testing and calibration module is used to automatically calibrate the geometric and mechanical properties of the testing tool during the testing cycle, and to trigger fault warnings and control system intervention when the sensor deviation exceeds the limit.

[0011] Preferably, the mobile navigation module includes:

[0012] The environmental perception unit is equipped with a 360° lidar, an inertial measurement unit, and a vision-assisted camera. It is used to collect spatial data of the building's internal structure, build a local map in real time through lidar point clouds, and achieve high-precision positioning by combining inertial measurement data.

[0013] The path planning unit integrates a BIM model parsing interface and a graph optimization processor. It constructs a building structure risk weight map and uses a risk-priority-based path planning algorithm to calculate the path. Its scoring function is:

[0014] F(x) = α·R(x) - β·D(x);

[0015] Where: F(x) is the target path score, R(x) is the target structural risk level, D(x) is the path length, and α and β are empirical weighting factors. The planning can prioritize the coverage of high-risk structural nodes and automatically avoid obstacles.

[0016] Preferably, the multimodal detection tool module includes:

[0017] The tool mounting unit, installed at the end of the robotic arm, includes an electromagnetic locking device and a pneumatic pin mechanism. It supports the installation and unlocking of standardized interface inspection tools. Under the command of the central control system, it can automatically switch inspection tools within 3 seconds and has anti-vibration locking and mis-insertion protection functions.

[0018] The tool identification unit, with its built-in RFID identification chip and image recognition sensor, can accurately identify the type of currently loaded detection tool and configure the corresponding detection mode, force parameters, and data acquisition channel according to the tool's attributes.

[0019] Preferably, the robotic arm execution module includes:

[0020] The posture control unit, based on a six-degree-of-freedom servo control robotic arm, integrates an angle encoder and a posture sensor. It uses a PID closed-loop posture control algorithm to adjust the motion state of the robotic arm in real time, ensuring that the tool maintains perpendicularity in the normal direction of the detection surface with a control error of less than ±0.5°.

[0021] The trajectory tracking unit receives path points and target location information generated by the mobile navigation module. It uses an algorithm based on inverse kinematics and trajectory feedforward-feedback fusion to enable the robotic arm to run stably on the specified path trajectory and corrects displacement deviation and error accumulation in real time.

[0022] Preferably, the sensor fusion and force control module includes:

[0023] The multi-source data fusion unit collects in real time the output values ​​of the pressure sensor, the gyroscope angular velocity, and the current feedback signal from the robotic arm installed on the detection tool, and then uses a weighted data fusion algorithm to perform the data fusion.

[0024] P(t)=ω1·p1(t)+ω2·p2(t)+ω3·p3(t);

[0025] Where: P(t) is the force control signal of the fusion output, p1(t), p2(t), and p3(t) represent the data feedback of the three channels: pressure sensor output value, gyroscope angular velocity, and robotic arm current, respectively; ω1, ω2, and ω3 are weighting factors dynamically adjusted according to the working conditions to achieve precise control of the applied force during the detection process;

[0026] The adaptive force adjustment unit calculates the changes in the contact state between the detection tool and the detection surface in real time based on the fused data, and performs fine-grained adjustment of the motor output torque at the end of the robotic arm to ensure that the detection action does not damage the structural surface, while also ensuring the tool's response sensitivity and data validity.

[0027] Preferably, the self-testing and calibration module includes:

[0028] The calibration platform unit, located in the robot body tool library area, includes a high-precision reference plane, a standard stiffness block, and a pressure reference surface, and is used to calibrate the geometric positioning accuracy and force application error of the rebound hammer and drill bit.

[0029] The error monitoring and early warning unit calculates the difference between the sensor calibration results and the factory standard template data. When the deviation of the current output value of the detection tool exceeds the preset threshold, it automatically sends a control signal to suspend the detection action and outputs an early warning prompt to the central processing system, triggering the recalibration process.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0031] Improved testing efficiency and flexibility: By integrating modular, quick-change multimodal testing tools (including rebound hammers, rebar testers, drill bits, etc.) and combining a quick-change device with electromagnetic locks and pneumatic pins, tool switching can be completed within 3 seconds, significantly improving the response speed and tool adaptability of testing tasks, and making it suitable for different types of engineering structure testing tasks.

[0032] Achieving high-precision intelligent operations: By integrating BIM (Building Information Modeling) with laser SLAM technology, high-precision perception and navigation of the building structure environment are achieved. Simultaneously, a risk-priority path planning algorithm is used to intelligently prioritize the detection of high-risk areas such as beam-column joints, avoiding human oversights and improving the systematic and scientific nature of the inspection work.

[0033] Enhanced detection accuracy and system reliability: The system employs a combination of gyroscope, pressure sensor, and current sensor to control the direction and magnitude of force applied at the end of the robotic arm. Based on a weighted fusion algorithm and adaptive adjustment strategy, the system corrects the detection status in real time. Combined with a built-in calibration platform, the system automatically calibrates sensor errors, effectively reducing detection errors and ensuring long-term stable operation of the system. Attached Figure Description

[0034] Figure 1 This application provides a schematic diagram of the system modules.

[0035] Figure 2 A schematic diagram of the mobile navigation module provided in this application;

[0036] Figure 3 This is a schematic diagram of the multimodal detection tool module provided in this application;

[0037] Figure 4 A schematic diagram of the robotic arm execution module provided in this application;

[0038] Figure 5 This is a schematic diagram of the sensor fusion and force application control module provided in this application;

[0039] Figure 6 This is a schematic diagram of the self-testing and calibration module provided in this application. Detailed Implementation

[0040] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0041] refer to Figures 1-6 This invention provides an intelligent engineering inspection robot system for multimodal sensor collaborative detection, comprising the following modules:

[0042] The mobile navigation module is used to build a 3D environment map of the site based on BIM building information model and laser SLAM technology, and automatically complete path planning and navigation execution.

[0043] In the mobile navigation module, the environmental perception unit uses a 360° LiDAR with a sampling frequency of 10Hz, which, together with the inertial measurement unit and binocular vision cameras, forms a multi-sensor perception system. After the robot starts, the system first calls the LiDAR module to collect spatial point cloud data in real time and construct a local dense point cloud map; at the same time, the inertial measurement unit provides six-axis attitude estimation and performs data fusion with the visual SLAM algorithm to improve the localization robustness in narrow or occluded environments.

[0044] The path planning unit calls the BIM model file corresponding to the building structure and uses the constructed risk assessment function F(x) = α·R(x) - β·D(x), where α = 0.6 and β = 0.4. In the graph optimization module, beam-column connections and load-bearing wall areas are set as high-risk nodes (R = 5), and passage areas are set as low-risk nodes (R = 1). The planning module uses an improved A* algorithm to solve the path. The final generated path takes into account both detection priority and driving distance, and the navigation system controls the mobile chassis to automatically travel along this path.

[0045] The multimodal testing tool module integrates various engineering structure testing tools, including rebound hammers, drill bits, and rebar measuring instruments, and enables rapid switching and automatic identification of testing tools through a modular quick-change mechanism.

[0046] In the multimodal testing tool module, the tool mounting unit is installed at the end flange interface of the six-degree-of-freedom robotic arm. The mounting mechanism adopts a custom-designed quick-change connector, which contains a set of programmable electromagnets and a pneumatic pin system. Taking drill bits and rebar measuring instruments as examples, when the system receives a replacement command from the central control unit, the electromagnetic lock controller releases the magnetic lock of the current tool, and at the same time, the pneumatic device initiates the pin retraction, completing the removal of the current tool within 3 seconds. Subsequently, the robotic arm moves to the tool storage position, aligns with the new tool insertion interface, the electromagnetic lock is re-energized and locked, and the pin automatically pops out and inserts to complete the connection. The entire process requires no manual intervention, and the anti-vibration mis-insertion design ensures that even under slight impact, it will not accidentally fall off.

[0047] The tool identification unit employs an RFID + visual redundancy identification mechanism. Each tool has an embedded RFID chip at its bottom, providing a unique 64-bit identification code. The image recognition sensor uses a lightweight model trained with visual algorithms to assist in comparing the tool's appearance pattern at the identification location. After identification, the system automatically binds the detected tool type with its corresponding force range and data acquisition channel configuration.

[0048] The robotic arm execution module is used to accurately perform detection actions at the target detection position, and achieves precise positioning and stable operation of the tool through pose feedback and trajectory tracking algorithms.

[0049] The robotic arm execution module employs an industrial-grade six-degree-of-freedom electric servo robotic arm. Each joint of the arm is equipped with a high-precision angle encoder and attitude sensor, and a PID closed-loop attitude control algorithm is implemented in the central controller. After receiving a detection task, the robotic arm moves according to the set attitude trajectory. Its attitude control system reads sensor data in real time and adjusts the output to ensure that the normal angle deviation of the tool when it contacts the detection surface does not exceed ±0.5°. For example, when the angle deviation exceeds the threshold, the system will pause the impact and reposition itself for adjustment, ensuring the standardization and comparability of the detection data.

[0050] At the path execution layer, the trajectory tracking unit receives the path point sequence and target structure position coordinates output by the mobile navigation module, and executes a trajectory feedforward-feedback control algorithm based on inverse kinematics calculation. The feedforward part pre-calculates the angle change trend of each joint based on the path points, while the feedback part corrects the position offset caused by external disturbances in real time. To prevent collisions with the robotic arm due to protrusions or irregular surfaces of structures, the system also integrates an obstacle approach warning mechanism. Once the tool end approaches an obstacle area, the system automatically adjusts its detection posture or plans an avoidance path, thereby ensuring safety and stability.

[0051] The sensor fusion and force control module is used to collect and fuse feedback data from pressure sensors, gyroscopes, and current sensors in real time, and dynamically adjust the direction and force of the robotic arm's end effector.

[0052] In the sensor fusion and force control module, three signal channels are used as multi-source inputs: the pressure sensor on the detection tool, the angular velocity output from the gyroscope, and the current feedback value from the robotic arm's servo motor. The system uses a weighted data fusion algorithm to integrate the three-channel signals in real time, with the formula: P(t)=ω1·p1(t)+ω2·p2(t)+ω3·p3(t), where p1(t) is the pressure sensor output, reflecting the contact force between the tool and the structural surface; p2(t) is the gyroscope angular velocity, used to determine tool stability and external disturbances; and p3(t) is the current signal, used to estimate the robotic arm's force output. The weighting factors ω1, ω2, and ω3 are automatically adjusted according to different tasks, improving the accuracy and real-time performance of force estimation.

[0053] In terms of control strategy, the adaptive force adjustment unit compares the fused signal P(t) with the set target force value in real time, and analyzes the contact state by combining the historical torque output change trend. If the material of the detection surface undergoes slight elastic deformation or the tool posture changes slightly, causing fluctuations in the force signal, the system can adjust the torque output of the end motor at the millisecond level to achieve fine-grained force control on the order of 0.01N.

[0054] The self-testing and calibration module is used to automatically calibrate the geometric and mechanical properties of the testing tool during the testing cycle, and to trigger fault warnings and control system intervention when the sensor deviation exceeds the limit.

[0055] In the self-testing and calibration module, which is integrated into the tool library area of ​​the robot body, the core component is the calibration platform unit. This calibration platform consists of a high-precision grinding reference plane, multiple standard blocks of different materials and stiffnesses, and an integrated pressure reference surface. Before each round of inspection or after switching inspection tools, the robot automatically moves its robotic arm to the calibration area and executes a set of standard action sequences: including probe contacting the positioning point, rebound spring impacting the stiffness reference block, and drill bit contacting the pressure plate at low speed. The system collects the current tool's positioning accuracy, response time, rebound strength, or pressure feedback and compares it with the factory reference data.

[0056] To enable anomaly detection and risk warning of calibration data, the error monitoring and early warning unit employs difference calculation and a dynamic threshold model. For example, if the impact strength of the testing tool on the standard stiffness block deviates from the historical average by more than ±5%, or if the drill bit pressure instability index exceeds a set threshold, the system will automatically determine that the tool may be faulty or worn. At this time, the control system issues a pause command, suspends the subsequent testing process, and outputs an early warning prompt to the main control interface, guiding the operator to perform manual re-inspection or trigger the automatic recalibration process.

[0057] In addition, to improve the system's operational continuity and reliability, this module also supports a periodic calibration scheduling function. Users can set a detection frequency threshold, and the system automatically queues and inserts it into the calibration process, ensuring that all types of testing tools are always in a state of controllable performance, significantly reducing the risk of data deviation caused by tool errors, and ensuring the credibility and standard compliance of test results in engineering safety assessments.

[0058] It should be noted that, unless otherwise specified, the embodiments and features and technical solutions in the present invention can be combined with each other.

[0059] Obviously, the embodiments described above are merely some embodiments of the present invention, not all embodiments. The accompanying drawings show preferred embodiments of the present invention, but do not limit the patent scope of the present invention. The present invention can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the patent protection scope of this invention.

Claims

1. A smart engineering inspection robot system for multimodal sensor collaborative detection, characterized in that, Includes the following modules: The mobile navigation module is used to build a 3D environment map of the site based on BIM building information model and laser SLAM technology, and automatically complete path planning and navigation execution; The multimodal testing tool module integrates various engineering structure testing tools, including rebound hammers, drill bits, and rebar measuring instruments, and achieves rapid switching and automatic identification of testing tools through a modular quick-change mechanism; The robotic arm execution module is used to accurately perform detection actions at the target detection position, and achieves precise positioning and stable operation of the tool through pose feedback and trajectory tracking algorithms; The sensor fusion and force control module is used to collect and fuse feedback data from pressure sensors, gyroscopes, and current sensors in real time, and dynamically adjust the direction and force of the robotic arm's end effector. The self-testing and calibration module is used to automatically calibrate the geometric and mechanical properties of the testing tool during the testing cycle, and to trigger fault warnings and control system intervention when the sensor deviation exceeds the limit.

2. The intelligent engineering inspection robot system for multimodal sensor collaborative detection according to claim 1, characterized in that, The mobile navigation module includes: The environmental perception unit is equipped with a 360° lidar, an inertial measurement unit, and a vision-assisted camera. It is used to collect spatial data of the building's internal structure, build a local map in real time through lidar point clouds, and achieve high-precision positioning by combining inertial measurement data. The path planning unit integrates a BIM model parsing interface and a graph optimization processor. It constructs a building structure risk weight map and uses a risk-priority-based path planning algorithm to calculate the path. Its scoring function is: F(x) = α·R(x) - β·D(x); Where: F(x) is the target path score, R(x) is the target structural risk level, D(x) is the path length, and α and β are empirical weighting factors. The planning can prioritize the coverage of high-risk structural nodes and automatically avoid obstacles.

3. The intelligent engineering inspection robot system for multimodal sensor collaborative detection according to claim 1, characterized in that, The multimodal detection tool module includes: The tool mounting unit, installed at the end of the robotic arm, includes an electromagnetic locking device and a pneumatic pin mechanism. It supports the installation and unlocking of standardized interface inspection tools. Under the command of the central control system, it can automatically switch inspection tools within 3 seconds and has anti-vibration locking and mis-insertion protection functions. The tool identification unit, with its built-in RFID identification chip and image recognition sensor, can accurately identify the type of currently loaded detection tool and configure the corresponding detection mode, force parameters, and data acquisition channel according to the tool's attributes.

4. The intelligent engineering inspection robot system for multimodal sensor collaborative detection according to claim 1, characterized in that, The robotic arm execution module includes: The posture control unit, based on a six-degree-of-freedom servo control robotic arm, integrates an angle encoder and a posture sensor. It uses a PID closed-loop posture control algorithm to adjust the motion state of the robotic arm in real time, ensuring that the tool maintains perpendicularity in the normal direction of the detection surface with a control error of less than ±0.5°. The trajectory tracking unit receives path points and target location information generated by the mobile navigation module. It uses an algorithm based on inverse kinematics and trajectory feedforward-feedback fusion to enable the robotic arm to run stably on the specified path trajectory and corrects displacement deviation and error accumulation in real time.

5. The intelligent engineering inspection robot system for multimodal sensor collaborative detection according to claim 1, characterized in that, The sensor fusion and force control module includes: The multi-source data fusion unit collects in real time the output values ​​of the pressure sensor, the gyroscope angular velocity, and the current feedback signal from the robotic arm installed on the detection tool, and then uses a weighted data fusion algorithm to perform the data fusion. P(t)=ω1·p1(t)+ω2·p2(t)+ω3·p3(t); Where: P(t) is the force control signal of the fusion output, p1(t), p2(t), and p3(t) represent the data feedback of the three channels: pressure sensor output value, gyroscope angular velocity, and robotic arm current, respectively; ω1, ω2, and ω3 are weighting factors dynamically adjusted according to the working conditions to achieve precise control of the applied force during the detection process; The adaptive force adjustment unit calculates the changes in the contact state between the detection tool and the detection surface in real time based on the fused data, and performs fine-grained adjustment of the motor output torque at the end of the robotic arm to ensure that the detection action does not damage the structural surface, while also ensuring the tool's response sensitivity and data validity.

6. The intelligent engineering inspection robot system for multimodal sensor collaborative detection according to claim 1, characterized in that, The self-testing and calibration module includes: The calibration platform unit, located in the robot body tool library area, includes a high-precision reference plane, a standard stiffness block, and a pressure reference surface, and is used to calibrate the geometric positioning accuracy and force application error of the rebound hammer and drill bit. The error monitoring and early warning unit calculates the difference between the sensor calibration results and the factory standard template data. When the deviation of the current output value of the detection tool exceeds the preset threshold, it automatically sends a control signal to suspend the detection action and outputs an early warning prompt to the central processing system, triggering the recalibration process.

Images