A method and system for integrated safety protection of robots using multi-sensor fusion

By using a multi-sensor fusion architecture and a dual-core redundant controller, the problems of blind spots of single sensors, insufficient adaptability to fixed areas, and single point of failure in robot safety protection systems are solved, achieving highly flexible and highly reliable safety protection.

CN123103777APending Publication Date: 2026-11-17JINING KELI PHOTOELECTRIC IND CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611582481.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-13
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing robot safety protection systems suffer from problems such as blind spots in single sensor perception, insufficient adaptability of fixed protection areas, risk of single point of failure, and difficulty in coordinating multiple devices, resulting in limitations on safety and flexibility.

Method used

A multi-sensor fusion architecture is adopted, integrating 3D LiDAR, depth camera and IMU sensor. An adaptive weighted fusion algorithm and software-defined 3D protection area are designed. A dual-core redundant safety controller and multi-device coordination mechanism are used to realize real-time reliability adjustment and fault detection of sensor data.

Benefits of technology

It achieves highly flexible and reliable security protection, improves sensor fusion accuracy, reduces failure risk, and enhances the coordination efficiency of multiple devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123103777A_ABST
    Figure CN123103777A_ABST
Patent Text Reader

Abstract

This invention relates to the field of robot safety protection technology, and in particular provides a method and system for integrated safety protection of robots through multi-sensor fusion. The method includes constructing a multi-source sensor fusion architecture that integrates multiple types of sensors; designing an adaptive weighted fusion algorithm that dynamically adjusts the fusion weights based on the real-time reliability of multiple sensor types and outputs obstacle information; designing a software-defined three-dimensional protection region to support various shapes and parameter configurations; designing a dual-core redundant safety controller, with the master and slave controllers operating independently to perform safety level voting and fault detection; and designing a multi-device coordination mechanism to support overlapping detection of safety areas and task priority coordination among multiple LiDARs. Through the adaptive weighted fusion algorithm and the dual-core redundant safety controller, this method achieves highly flexible and reliable safety protection while ensuring a high safety level.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of robot safety protection technology, and in particular to a method and system for integrated safety protection of robots through multi-sensor fusion, applicable to the safety protection of all types of robots, including industrial robots, humanoid robots, special robots, and service robots. Background Technology

[0002] With the rapid development of robotics technology, robots are increasingly widely used in industrial manufacturing, domestic services, and specialized operations. The increasingly complex operating scenarios for robots place higher demands on the reliability of safety protection. Existing safety protection systems still face many technical challenges in practical engineering implementation: First, the limitations of single-sensor perception: a single sensor has blind spots and cannot achieve all-around safety protection. Industry test data shows that the failure rate of a single LiDAR under complex operating conditions is approximately 10%. -4 / h, which fails to meet the SIL2 safety level requirements (requiring a failure rate of <10). -7 / h). Single vision sensors are limited by lighting conditions, and their performance degrades significantly in strong or low light environments. Single ultrasonic sensors have low resolution (approximately 3cm) and cannot detect small obstacles.

[0003] Secondly, fixed protection zones lack adaptability: they cannot meet the diverse operational needs of robots. Existing technologies typically use protection zones of fixed sizes, which cannot be dynamically adjusted according to the robot's operational status. Industry research data shows that 80% of robot accidents occur near the boundaries of the protection zone. Industrial scenarios require strict safety boundaries (accuracy ±5mm), while service scenarios require flexible human-robot interaction boundaries.

[0004] Furthermore, there is the risk of single point of failure: traditional safety controllers lack redundancy, and a single point of failure can lead to safety malfunctions. Existing technologies typically employ a single processor architecture, and processor failure can cause the entire safety system to fail. Industry incident analysis shows that 30% of robot safety system failures originate from single point of failure in the controller.

[0005] Finally, multi-device coordination is challenging: the lack of a unified standard makes cluster security protection impossible. In multi-robot collaboration scenarios, achieving safe area overlap detection, task priority coordination, and emergency stop synchronization among multiple devices are technical difficulties. Traditional solutions typically employ a master-slave architecture, which carries the risk of single points of failure; distributed architectures lack a unified time synchronization and coordination mechanism.

[0006] Current technologies generally assume that multi-sensor fusion can only achieve simple data-level fusion, failing to realize deep fusion at the feature and decision levels. The industry also generally believes that safety controllers must use dedicated hardware (such as FPGAs or ASICs), making it impossible to achieve high safety using general-purpose processors. This technological bias limits the flexibility and reliability of robot safety systems. Summary of the Invention

[0007] In view of this, the present invention provides a method and system for integrated safety protection of robots through multi-sensor fusion, so as to achieve highly flexible and highly reliable safety protection.

[0008] In a first aspect, the present invention provides a method for integrated safety protection of a robot through multi-sensor fusion, the method comprising: Step 1: Construct a multi-source sensor fusion architecture that integrates multiple types of sensors; Step 2: Design an adaptive weighted fusion algorithm to dynamically adjust the fusion weights based on the real-time reliability of multiple sensor types and output obstacle information; Step 3: Design a software-defined 3D protected area to support various shapes and parameter configurations; Step 4: Design a dual-core redundant safety controller, with the master controller and slave controller operating independently to perform safety level voting and fault detection; Step 5: Design a multi-device coordination mechanism to support safe area overlap detection and task priority coordination of multiple lidars.

[0009] Optionally, step 1 includes: Step 11: Sensor selection and configuration. Various types of sensors include 3D LiDAR for three-dimensional spatial perception, depth cameras and ultrasonic sensors for near-range obstacle detection, and inertial measurement units (IMU) sensors for robot posture perception. Step 12: Preprocess the data from each sensor. For the 3D LiDAR data, filter out noise points, transform the coordinate system, and extract point cloud features. For the depth camera data, filter out background interference, transform the coordinate system, and extract obstacle information. For the ultrasonic sensor data, filter out multiple reflection interference, transform the coordinate system, and calculate obstacle distances. For the IMU sensor data, filter out drift and calculate robot posture. Step 13: Timestamp Synchronization. To ensure time consistency across multiple data sources, a unified time base is adopted. IMU sensor data is used as the time base, and other sensor data is synchronized to the IMU timestamp through an interpolation algorithm. The time synchronization accuracy is less than 1ms, meeting the real-time fusion requirements. The expression for time synchronization is: P_sync(t)=P_raw(t_k)+(P_raw(t_{k+1})-P_raw(t_k))×(t-t_k) / (t_{k+1}-t_k); Where P_sync(t) is the synchronized data, P_raw(t_k) and P_raw(t_{k+1}) are the original data, t is the IMU timestamp, and t_k and t_{k+1} are the original sensor timestamps; Step 14: Unify the coordinate system. Transform all sensor data to a unified robot coordinate system. The expression for coordinate transformation is: P_robot = T•P_sensor; Where T is a 4×4 transformation matrix from the sensor coordinate system to the robot coordinate system, which includes rotation and translation, and is obtained through calibration experiments with a calibration accuracy of less than 1 mm; P_sensor is the point coordinate in the sensor coordinate system; P_robot is the point coordinate in the robot coordinate system. The calibration method uses a checkerboard calibration plate and solves the transformation matrix T using the least squares method, with a calibration error of less than 1 mm.

[0010] Optionally, step 2 includes: Step 21: Sensor reliability assessment. Calculate the reliability of each sensor using the following expression: C_i = exp(-‖P_i(t)-P_fused(t-1)‖ 2 / (2σ_i 2 )); Where C_i is the reliability of sensor i, P_i(t) is the measurement value of sensor i at time t, P_fused(t-1) is the fusion result at time t-1, and σ_i is the historical deviation standard deviation of sensor i. The expression for σ_i is: σ_i=sqrt(Σ(P_i(t)-P_fused(t)) 2 / N); Where N is the size of the history window, and we take N=100; Step 22: Adaptive weight calculation, calculate the fusion weights based on credibility: w_i = C_i / ΣC_j; Where w_i is the fusion weight of sensor i, C_i is the confidence level of sensor i, and N is the number of sensors; the weight range is 0-1, and the sum of the weights is 1. Step 23: Weighted fusion calculation, performing weighted fusion of the measurements from all sensors: P_fused(t) = Σ(w_i·P_i(t)); Where P_fused(t) is the fusion result at time t, P_i(t) is the measurement value of sensor i at time t, and w_i is the corresponding weight; the fusion result is the weighted average of the measurement values ​​of all sensors. Step 24: Fusion accuracy evaluation. Assuming that the measurements from each sensor are independent, calculate the covariance matrix of the fusion result to evaluate the fusion accuracy. The expression for the covariance matrix is: Σσ_fused=Σ(w_i 2 ·Σσ_i); Where Σσ_fused is the covariance matrix of the fusion result, w_i is the fusion weight, and Σσ_i is the covariance matrix of sensor i; The expression for fusion accuracy is: σ_fused = sqrt(w_1 2 σ_1 2 + w_2 2 σ_2 2 + w_3 2 σ_3 2 +w_4 2 σ_4 2 ).

[0011] Optionally, step 3 includes: Step 31: Protect the region type, defined as a cube region (x_min, y_min, z_min, x_max, y_max, z_max); a cylinder region (x_c, y_c, z_min, z_max, r); a sphere region (x_c, y_c, z_c, r); or a custom polyhedron region defined by a list of vertex coordinates. Step 32: Configuration parameters for each protected area: ① Area type is cube, cylinder, sphere, or custom polyhedron; ② Define the specific dimensions of the area according to the type; ③ Safety level is SIL1, SIL2, or SIL3, used to determine detection accuracy and response time; ④ Determine the response time (10ms-100ms) according to the safety level; ⑤ Determine the trigger action according to the safety level, which includes stop, deceleration, alarm, or voice prompt. Step 33: The protected area is adjusted in real time with an adjustment delay of less than 10ms. The adjustment methods include manually modifying the protected area parameters through the host computer interface; automatically adjusting the size of the protected area according to the robot's operating status; and automatically switching the protected area configuration according to the task type. Step 34: Collision Detection. For each point cloud data point, determine whether it falls within the protected area: the criteria for a cube region are x_min≤x≤x_max and y_min≤y≤y_max and z_min≤z≤z_max; the criteria for a cylinder region is (x-xc).2 +(y-yc) 2 ≤r 2 And z_min≤z≤z_max; the condition for determining the spherical region is (x-xc). 2 +(y-yc) 2 +(z-zc) 2 ≤r 2 Where x, y, z are the coordinates of the point cloud data points, xc, yc, zc are the coordinates of the center of the protected area, and r is the radius of the cylindrical area or the radius of the spherical area. Step 35: Supports multi-level protection zone configuration for progressive safety response: Warning zone, 1.5m away from the robot, triggers deceleration and voice prompts; Danger zone, 0.5m away from the robot, triggers emergency stop.

[0012] Optionally, step 4 includes: Step 41: Dual-core architecture design. The safety controller includes two independent processor cores: a master controller and a slave controller. The master controller runs the main safety logic, monitors sensor data in real time, determines the safety status, and outputs control commands. The slave controller runs redundant safety logic, monitors sensor data in real time, independently determines the safety status, and performs safety voting with the master controller. Dual-core communication is achieved through shared memory and interrupt mechanisms, with a communication latency of less than 100μs. Step 42: Safety voting mechanism. The master controller and slave controller independently calculate the safety status and then vote on it. The voting rules are as follows: if both the master controller and slave controller determine that the system is safe, the system outputs a safe status; if both the master controller and slave controller determine that the system is dangerous, the system outputs a dangerous status; if the master controller and slave controller have different determinations, the system outputs a dangerous status. The voting delay time is less than 1ms to meet the real-time requirements. Step 43: Fault detection mechanism. The master controller and slave controller monitor each other to detect the following fault types: Processor failures: watchdog timeout, memory check error, program crash; Sensor failures: abnormal sensor data, sensor communication interruption; Communication failures: internal communication timeout, external communication interruption. Step 44: Based on the safety voting results and fault detection results, output the safety status; Safety status: The robot is running normally and outputs a safety signal; Warning status: The robot decelerates and issues a warning signal; Danger status: The robot stops and issues an alarm signal; The output delay time of the safety status is less than 5ms to meet the real-time requirements; Step 45: A dynamic pulse watchdog protection mechanism is adopted. The main controller periodically sends pulses to the slave controller. The slave controller detects the pulses. If it does not receive the pulse within a timeout, it determines that the main controller is faulty. The slave controller takes over the safety control and outputs a dangerous status. The expression for watchdog timeout calculation is: T_timeout = T_pulse × N_miss; Where T_pulse is the pulse period and N_miss is the number of pulses that can be missed.

[0013] Optionally, step 5 includes: Step 51: After each LiDAR device is started, it registers with the coordination server via the EtherCAT bus; the registration information includes device ID, device type, protection zone parameters, and task priority; the coordination server maintains the device registry and updates the device status in real time. Step 52: Design a security area overlap detection algorithm to coordinate the server to periodically check whether the security areas of all devices overlap. The overlap detection algorithm is as follows: For each pair of devices i and j, calculate the intersection of their protected areas; if the intersection is not empty, it is determined that there is an overlap; prioritize the obstacles in the overlapping area, the overlap detection cycle is 10ms; the overlap detection delay time is less than 5ms. Step 53: Task priority coordination. Priority rules: Emergency stop tasks have the highest priority, with a priority value of 1; obstacle avoidance tasks have the second highest priority, with a priority value of 2; inspection tasks have the lowest priority, with a priority value of 3; if the priorities are the same, the coordination server will designate the responsible equipment. Step 54: When any device triggers an emergency stop, the coordination server broadcasts an emergency stop command to all devices; the emergency stop synchronization delay time is less than 10ms to ensure that all devices stop at the same time. Step 55: Use the IEEE 1588 precision time protocol for multi-device time synchronization with a synchronization accuracy of less than 1μs to meet the requirements of EtherCAT communication. The time synchronization process is as follows: ① The coordination server sends a synchronization request, ② Each device records its local time and responds, ③ The coordination server calculates the clock offset, and ④ Each device adjusts its local clock.

[0014] Secondly, the present invention provides a system for integrated safety protection of robot multi-sensor fusion, the system being used to implement the method for integrated safety protection of robot multi-sensor fusion as described in the first aspect, the system comprising a multi-source sensor module, an adaptive weighted fusion module, a three-dimensional protection area module, a dual-core redundant safety controller module, and a multi-device coordination module; The multi-source sensor module is used to build a multi-source sensor fusion architecture and integrate multiple types of sensors; The adaptive weighted fusion module is used to design an adaptive weighted fusion algorithm, which dynamically adjusts the fusion weights based on the real-time reliability of multiple types of sensors and outputs obstacle information. The 3D protected area module is used to design software-defined 3D protected areas to support various shapes and parameter configurations; The dual-core redundant safety controller module is used to design a dual-core redundant safety controller, in which the master controller and slave controller operate independently to perform safety level voting and fault detection. The multi-device coordination module is used to design a multi-device coordination mechanism, supporting safe area overlap detection and task priority coordination of multiple lidars.

[0015] The technical solution provided by this invention includes a method that constructs a multi-source sensor fusion architecture, integrating multiple types of sensors; designs an adaptive weighted fusion algorithm to dynamically adjust the fusion weights based on the real-time reliability of multiple types of sensors, and outputs obstacle information; designs a software-defined three-dimensional protection area to support various shapes and parameter configurations; designs a dual-core redundant safety controller, with the master controller and slave controller operating independently to perform safety level voting and fault detection; and designs a multi-device coordination mechanism to support overlapping detection of safety areas and task priority coordination among multiple lidars. This method, through the adaptive weighted fusion algorithm and the dual-core redundant safety controller, achieves highly flexible and reliable safety protection while ensuring safety levels. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating the method for integrated safety protection of robots through multi-sensor fusion provided in this embodiment of the invention; Figure 2 This is a schematic diagram of multi-source sensor data fusion provided in an embodiment of the present invention; Figure 3 A diagram showing the configuration of the protection area provided in an embodiment of the present invention; Figure 4 This is a diagram of a dual-core redundant safety controller architecture provided in an embodiment of the present invention; Figure 5 This is a timing diagram for multi-device coordination provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0020] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0021] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0022] This invention provides a method for integrated safety protection of robots through multi-sensor fusion, such as... Figures 1 to 5 As shown, the method includes: Step 1: Construct a multi-source sensor fusion architecture that integrates multiple types of sensors; In this embodiment of the invention, step 1 includes: Step 11: Sensor selection and configuration. Multiple types of sensors are used, including 3D LiDAR for three-dimensional spatial perception, depth cameras and ultrasonic sensors for near-range obstacle detection, and inertial measurement units (IMU) sensors for robot posture perception, to achieve all-round, blind-spot-free environmental perception. In this embodiment of the invention, one 3D LiDAR is installed on the top of the robot, with a detection range of 0.1 m-100 m, a resolution of 2 cm, a horizontal field of view of 360°, a vertical field of view of 30°, and a refresh rate of 20 Hz; two depth cameras are installed in front of and behind the robot, with a detection range of 0.2 m-10 m, a resolution of 1 cm, a field of view of 90°×70°, and a refresh rate of 30 fps; eight ultrasonic sensors are distributed around the robot, with a detection range of 0.02 m-5 m, a resolution of 3 cm, and a refresh rate of 10 Hz; and one IMU sensor is installed on the robot body, with a sampling frequency of 100 Hz, an accelerometer range of ±16 g, and a gyroscope range of ±2000° / s.

[0023] Step 12: Preprocess the data from each sensor. For the 3D LiDAR data, filter out noise points (points with a distance >100m or <0.1m), transform the coordinate system, and extract point cloud features (normal vector, curvature). For the depth camera data, filter out background interference (points with a depth >10m), transform the coordinate system, and extract obstacle information (contour, center point). For the ultrasonic sensor data, filter out multiple reflection interference (take the median of 3 consecutive measurements), transform the coordinate system, and calculate obstacle distances. For the IMU sensor data, filter out drift (Kalman filtering) and calculate the robot's attitude (pitch angle, roll angle, yaw angle). Step 13: Timestamp Synchronization. To ensure time consistency across multiple data sources, a unified time base is adopted. IMU sensor data is used as the time base, and other sensor data is synchronized to the IMU timestamp through an interpolation algorithm. The time synchronization accuracy is less than 1ms, meeting the real-time fusion requirements. The expression for time synchronization is: P_sync(t)=P_raw(t_k)+(P_raw(t_{k+1})-P_raw(t_k))×(t-t_k) / (t_{k+1}-t_k); Where P_sync(t) is the synchronized data, P_raw(t_k) and P_raw(t_{k+1}) are the original data, t is the IMU timestamp, and t_k and t_{k+1} are the original sensor timestamps; Step 14: Unify the coordinate system. Transform all sensor data to a unified robot coordinate system. The expression for coordinate transformation is: P_robot = T•P_sensor; Where T is a 4×4 transformation matrix from the sensor coordinate system to the robot coordinate system, which includes rotation and translation, and is obtained through calibration experiments with a calibration accuracy of less than 1 mm; P_sensor is the point coordinate in the sensor coordinate system; P_robot is the point coordinate in the robot coordinate system. The calibration method uses a checkerboard calibration plate and solves the transformation matrix T using the least squares method, with a calibration error of less than 1 mm.

[0024] Step 2: Design an adaptive weighted fusion algorithm to dynamically adjust the fusion weights based on the real-time reliability of multiple sensor types and output obstacle information; In this embodiment of the invention, step 2 includes: Step 21: Sensor reliability assessment. Calculate the reliability of each sensor using the following expression: C_i = exp(-‖P_i(t)-P_fused(t-1)‖ 2 / (2σ_i 2 )); Where C_i is the reliability of sensor i, P_i(t) is the measurement value of sensor i at time t, P_fused(t-1) is the fusion result at time t-1, and σ_i is the historical deviation standard deviation of sensor i. The expression for σ_i is: σ_i=sqrt(Σ(P_i(t)-P_fused(t)) 2 / N); Where N is the size of the history window, and we take N=100; Step 22: Adaptive weight calculation, calculate the fusion weights based on credibility: w_i = C_i / ΣC_j; Where w_i is the fusion weight of sensor i, C_i is the confidence level of sensor i, and N is the number of sensors; the weight range is 0-1, and the sum of the weights is 1. Step 23: Weighted fusion calculation, performing weighted fusion of the measurements from all sensors: P_fused(t) = Σ(w_i·P_i(t)); Where P_fused(t) is the fusion result at time t, P_i(t) is the measurement value of sensor i at time t, and w_i is the corresponding weight; the fusion result is the weighted average of the measurement values ​​of all sensors. Step 24: Fusion accuracy evaluation. Assuming that the measurements from each sensor are independent, calculate the covariance matrix of the fusion result to evaluate the fusion accuracy. The expression for the covariance matrix is: Σσ_fused=Σ(w_i 2 ·Σσ_i); Where Σσ_fused is the covariance matrix of the fusion result, w_i is the fusion weight, and Σσ_i is the covariance matrix of sensor i; The expression for fusion accuracy is: σ_fused = sqrt(w_1 2 σ_12 + w_2 2 σ_2 2 + w_3 2 σ_3 2 +w_4 2 σ_4 2 ).

[0025] In this embodiment of the invention, typical values ​​are: σ_1=2cm (LiDAR), σ_2=1cm (depth camera), σ_3=3cm (ultrasound), σ_4=0.5cm (IMU); after fusion: σ_fused≈0.8cm, an improvement of about 2.5 times.

[0026] Traditional methods employ fixed-weight fusion, which cannot adapt to changes in sensor performance. This invention assesses reliability based on the consistency between the current sensor measurement and the fusion result from the previous moment. The reliability range is 0-1; higher reliability indicates greater consistency between the sensor measurement and historical fusion results. Higher reliability corresponds to higher weights, and lower reliability to lower weights, thus addressing the performance degradation problem of a single sensor under complex operating conditions.

[0027] Step 3: Design a software-defined 3D protected area to support various shapes and parameter configurations; In this embodiment of the invention, step 3 includes: Step 31: Protect the region type, defined as a cube region (x_min, y_min, z_min, x_max, y_max, z_max); a cylinder region (x_c, y_c, z_min, z_max, r); a sphere region (x_c, y_c, z_c, r); or a custom polyhedron region defined by a list of vertex coordinates. Step 32: Configuration parameters for each protected area: ① Area type is cube, cylinder, sphere, or custom polyhedron; ② Define the specific dimensions of the area according to the type; ③ Safety level is SIL1, SIL2, or SIL3, as shown in Table 1, used to determine detection accuracy and response time; ④ Determine the response time (10ms-100ms) according to the safety level; ⑤ Determine the trigger action according to the safety level, which includes stop, deceleration, alarm, or voice prompt. Table 1 Technical Specifications for SIL Level ; Step 33: The protected area is adjusted in real time with an adjustment delay of less than 10ms. The adjustment methods include manually modifying the protected area parameters through the host computer interface; automatically adjusting the size of the protected area according to the robot's operating status; and automatically switching the protected area configuration according to the task type. Step 34: Collision Detection. For each point cloud data point, determine whether it falls within the protected area: the criteria for a cube region are x_min≤x≤x_max and y_min≤y≤y_max and z_min≤z≤z_max; the criteria for a cylinder region is (x-xc). 2 +(y-yc) 2 ≤r 2 And z_min≤z≤z_max; the condition for determining the spherical region is (x-xc). 2 +(y-yc) 2 +(z-zc) 2 ≤r 2 Where x, y, z are the coordinates of the point cloud data points, xc, yc, zc are the coordinates of the center of the protected area, and r is the radius of the cylindrical area or the radius of the spherical area. Step 35: Supports multi-level protection zone configuration for progressive safety response: Warning zone (yellow), 1.5m away from the robot, triggers deceleration and voice prompts; Danger zone (red), 0.5m away from the robot, triggers emergency stop.

[0028] Traditional solutions use fixed hardware protection areas, which require hardware replacement for adjustments, resulting in high costs and long lead times. This invention designs a software-defined three-dimensional protection area that supports various shapes such as cubes, cylinders, and spheres, solving the problem that traditional fixed protection areas cannot adapt to diverse operational needs.

[0029] Step 4: Design a dual-core redundant safety controller, with the master controller and slave controller operating independently to perform safety level voting and fault detection; In this embodiment of the invention, step 4 includes: Step 41: Dual-core architecture design. The safety controller includes two independent processor cores: a master controller (Cortex-R52) and a slave controller (Cortex-R52). The master controller runs the main safety logic, monitors sensor data in real time, determines the safety status, and outputs control commands. The slave controller runs redundant safety logic, monitors sensor data in real time, independently determines the safety status, and performs safety voting with the master controller. Dual-core communication is achieved through shared memory and interrupt mechanisms, with a communication latency of less than 100μs. Step 42: Safety voting mechanism. The master controller and slave controller independently calculate the safety state and then vote on it. The voting rules are as follows: if both the master controller and slave controller determine that the system is safe, the system outputs a safe state; if both the master controller and slave controller determine that the system is dangerous, the system outputs a dangerous state; if the master controller and slave controller determine that the system is inconsistent, the system outputs a dangerous state (conservative strategy). The voting delay time is less than 1ms to meet the real-time requirements. Step 43: Fault detection mechanism. The master controller and slave controller monitor each other to detect the following fault types: Processor failure: Watchdog timeout (100ms), memory check error (CRC32 check), program crash; Sensor failure: Abnormal sensor data (out of range), sensor communication interruption (50ms); Communication failure: Internal communication timeout (10ms), external communication interruption (100ms). Fault detection coverage is >99%, ensuring that single points of failure are detected in a timely manner. Fault handling proceeds as follows: fault detection, fault log recording, switching to safe mode, notifying the host computer, and waiting for manual reset.

[0030] Step 44: Based on the safety voting results and fault detection results, output the safety status; Safety status: The robot is running normally and outputs a safety signal; Warning status: The robot decelerates (speed drops to 50%) and issues a warning signal (yellow indicator light + buzzer); Danger status: The robot stops (stopping time < 100ms) and issues an alarm signal (red indicator light + siren); The output delay time of the safety status is less than 5ms to meet the real-time requirements; Step 45: A dynamic pulse watchdog protection mechanism is adopted. The main controller periodically sends pulses to the slave controller (period 1ms). The slave controller detects the pulses. If no pulse is received within 100ms, the main controller is determined to be faulty. The slave controller takes over the safety control and outputs a dangerous status. The expression for watchdog timeout calculation is: T_timeout=T_pulse×N_miss =1ms×100=100ms; Where T_pulse is the pulse period and N_miss is the number of pulses that can be missed.

[0031] Traditional solutions employ a single-processor architecture, where processor failure can cause the entire security system to fail. This invention designs a dual-core redundant security controller, with the master and slave controllers operating independently. A security voting mechanism ensures that a single point of failure will not lead to security failure, thus solving the problem of single-point failure causing security failure in traditional solutions.

[0032] Step 5: Design a multi-device coordination mechanism to support safe area overlap detection and task priority coordination of multiple lidars.

[0033] In this embodiment of the invention, step 5 includes: Step 51: After each LiDAR device is started, it registers with the coordination server via the EtherCAT bus; the registration information includes device ID, device type, protection zone parameters, and task priority; the coordination server maintains the device registry and updates the device status in real time. Step 52: Design a security area overlap detection algorithm to coordinate the server to periodically check whether the security areas of all devices overlap. The overlap detection algorithm is as follows: For each pair of devices i and j, calculate the intersection of their protected areas; if the intersection is not empty, it is determined that there is an overlap; prioritize the obstacles in the overlapping area, the overlap detection cycle is 10ms; the overlap detection delay time is less than 5ms. Step 53: Task priority coordination. For obstacles in overlapping areas, the task priority determines which device is responsible for handling them. Priority rules: Emergency stop tasks have the highest priority, with a priority value of 1; obstacle avoidance tasks have the second highest priority, with a priority value of 2; inspection tasks have the lowest priority, with a priority value of 3. If the priorities are the same, the coordination server will assign the responsible device (usually the device with the smaller ID). Step 54: When any device triggers an emergency stop, the coordination server broadcasts an emergency stop command to all devices; the emergency stop synchronization delay time is less than 10ms to ensure that all devices stop at the same time. Step 55: Use the IEEE 1588 precision time protocol for multi-device time synchronization with a synchronization accuracy of less than 1μs to meet the requirements of EtherCAT communication. The time synchronization process is as follows: ① The coordination server sends a synchronization request, ② Each device records its local time and responds, ③ The coordination server calculates the clock offset, and ④ Each device adjusts its local clock.

[0034] This invention designs a multi-device coordination mechanism to support the detection of overlapping safe areas and task priority coordination among multiple LiDARs, addressing safety protection issues in multi-robot collaborative scenarios. Traditional solutions lack multi-device coordination mechanisms and cannot handle overlapping safe areas. This invention designs a safe area overlap detection algorithm to detect whether the safe areas of multiple devices overlap in real time, solving the problem of difficult multi-device coordination in traditional solutions.

[0035] This invention provides a system for integrated safety protection of robots through multi-sensor fusion. The system is used to implement the above-mentioned method for integrated safety protection of robots through multi-sensor fusion. The system includes a multi-source sensor module, an adaptive weighted fusion module, a three-dimensional protection area module, a dual-core redundant safety controller module, and a multi-device coordination module. The multi-source sensor module is used to build a multi-source sensor fusion architecture, integrating multiple types of sensors; the adaptive weighted fusion module is used to design an adaptive weighted fusion algorithm, dynamically adjusting the fusion weights based on the real-time reliability of multiple types of sensors, and outputting obstacle information; the 3D protection area module is used to design a software-defined 3D protection area to support various shapes and parameter configurations; the dual-core redundant safety controller module is used to design a dual-core redundant safety controller, with the master and slave controllers operating independently, performing safety level voting and fault detection; the multi-device coordination module is used to design a multi-device coordination mechanism, supporting the detection of overlapping safety areas and task priority coordination among multiple LiDARs.

[0036] Embodiment 1 of the present invention: Industrial robot welding scenario. In an automobile manufacturing workshop, six industrial welding robots work collaboratively to complete the body welding task. Each robot has a working radius of 2m, and the minimum distance between the robots is 1.5m.

[0037] System Configuration: ① Each robot is equipped with 1 3D LiDAR (top-mounted), 2 depth cameras (front and rear-mounted), 8 ultrasonic sensors (distributed around the perimeter), and 1 IMU (mounted on the robot body); ② Protected Area Configuration: ③ Warning Area: Cylinder, radius 1.8m, height 2.5m, triggers deceleration and voice prompts; ④ Hazardous Area: Cylinder, radius 0.5m, height 2.5m, triggers emergency stop; ⑤ Safety Level: SIL2 (failure rate <10%) -7 / h, response time ≤50ms) The operational results are shown in Table 2: ① Fusion accuracy: 0.8cm (single lidar accuracy 2cm); ② Protection area adjustment time <100ms; ③ Multi-device coordination delay <10ms; ④ No safety incidents occurred during 6 months of operation.

[0038] Table 2 Comparison of running results .

[0039] Embodiment 2 of the present invention: Service robot food delivery scenario. In a restaurant environment, three service robots work together to deliver food, while avoiding customers and waiters, and ensuring delivery efficiency.

[0040] System Configuration: ① Each robot is equipped with 1 3D LiDAR (top-mounted), 1 depth camera (front-mounted), 4 ultrasonic sensors (front and sides), and 1 IMU (body-mounted); ② Protected area configuration; ③ Warning area: cylinder, radius 1.0m, height 1.8m, triggers deceleration and voice prompt "Please be careful"; ④ Dangerous area: cylinder, radius 0.3m, height 1.8m, triggers emergency stop; ⑤ Safety level: SIL1 (failure rate <10%) -5 / h, response time ≤100ms); The operational results are shown in Table 3: ① Fusion accuracy: 1.2cm (single depth camera accuracy is 1cm, but affected by lighting); ② Protection area adjustment time: <50ms; ③ Multi-device coordination delay: <5ms; ④ After 3 months of operation, it successfully avoided obstacles more than 10,000 times without any collision accidents.

[0041] Table 3 Comparison of running results .

[0042] Embodiment 3 of the present invention: Humanoid robot home service scenario, humanoid robot provides services in home environment, needs to avoid furniture, pets and children, while ensuring flexibility.

[0043] System Configuration: ① The robot is equipped with 1 small 3D LiDAR (head-mounted), 2 depth cameras (eye-mounted), 6 ultrasonic sensors (around the body), and 1 IMU (torso-mounted); ② Protective area configuration; ③ Warning area: sphere, radius 1.2m, triggers deceleration and voice prompt "I am moving, please be careful"; ④ Danger area: sphere, radius 0.4m, triggers emergency stop; ⑤ Child protection area: sphere, radius 0.8m, triggers more sensitive response; ⑥ Safety level: SIL2 (failure rate <10%) -7 / h, response time ≤50ms); The operational results are shown in Table 4: ① Fusion accuracy: 1.0cm (single lidar accuracy 2cm); ② Protection area adjustment time: <80ms; ③ Child detection sensitivity: >99%; ④ After 2 months of operation, it successfully avoided obstacles more than 5000 times without any collision accidents. Table 4 Comparison of running results .

[0044] Comparative experimental data of this invention: 1. Fusion accuracy comparison experiment. Experimental conditions: ① Test environment: standard laboratory environment, temperature 25℃, humidity 50%; ② Test target: standard obstacle of known size (cube, side length 10cm); ③ Test method: the robot moves at a speed of 0.5m / s, and the deviation between the fusion result and the true value is recorded; the experimental results are shown in Table 5. Table 5 Experimental Results ; The adaptive weighted fusion algorithm of this invention improves accuracy by 2.5 times compared to single-sensor fusion and by 33% compared to fixed-weight fusion.

[0045] The invention presents a comparative experiment on the adjustment time of the protected area. Experimental conditions: ① Test environment: standard laboratory environment; ② Test method: switch from one protected area configuration to another and record the adjustment completion time; the experimental results are shown in Table 6. Table 6 Experimental Results ; The software-defined protected area adjustment time of this invention is <100ms, which is 36,000 times faster than hardware adjustment and 5 times faster than traditional software adjustment.

[0046] This invention presents a comparative experiment on the coordination delay of multiple devices. Experimental conditions: ① Test environment: 6 robots working collaboratively; ② Test method: Record the time from detection of overlap to completion of coordination; Experimental results are shown in Table 7. Table 7 Experimental Results ; The multi-device coordination latency of this invention is <10ms, which is 5 times better than the master-slave architecture and 10 times better than the distributed architecture.

[0047] The fault detection coverage comparison experiment of this invention was conducted under the following conditions: ① Test environment: various fault types were injected; ② Test method: the ratio of the number of detected faults to the total number of injected faults was recorded; the experimental results are shown in Table 8. Table 8 Experimental Results ; The fault detection coverage of this invention reaches 99.7%, exceeding the design target of 99%.

[0048] Compared with the prior art, the present invention has the following advantages: 1. Multi-source sensor fusion: Integrates multiple sensors such as LiDAR, depth camera, ultrasonic, and IMU to achieve all-round, blind-spot-free environmental perception with a fusion accuracy of 0.8cm.

[0049] 2. Adaptive weighted fusion: The fusion weights are dynamically adjusted based on the real-time reliability of the sensors, effectively addressing the performance degradation of individual sensors and ensuring fusion accuracy.

[0050] 3. Software-defined protected areas: Supports various shapes such as cubes, cylinders, and spheres, enabling flexible configuration and real-time adjustment of protected areas with an adjustment delay of <10ms.

[0051] 4. Dual-core redundant safety controller: achieves SIL2 / SIL3 safety level, single point of failure does not affect safety functions, and fault detection coverage >99%.

[0052] 5. Multi-device coordination capability: Enables coordinated control of multiple LiDARs via EtherCAT bus, supports safe area overlap detection and emergency stop synchronization, with a synchronization delay of <10ms.

[0053] 6. Multi-level protection zones: Supports multi-level configuration of warning zones and danger zones to achieve progressive safety response.

[0054] 7. Watchdog protection: A dynamic pulse watchdog protection mechanism is adopted with a timeout of 100ms to ensure system reliability.

[0055] The technical solution provided by this invention includes a method that constructs a multi-source sensor fusion architecture, integrating multiple types of sensors; designs an adaptive weighted fusion algorithm to dynamically adjust the fusion weights based on the real-time reliability of multiple types of sensors, and outputs obstacle information; designs a software-defined three-dimensional protection area to support various shapes and parameter configurations; designs a dual-core redundant safety controller, with the master controller and slave controller operating independently to perform safety level voting and fault detection; and designs a multi-device coordination mechanism to support overlapping detection of safety areas and task priority coordination among multiple lidars. This method, through the adaptive weighted fusion algorithm and the dual-core redundant safety controller, achieves highly flexible and reliable safety protection while ensuring safety levels.

[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for integrated safety protection of robots through multi-sensor fusion, characterized in that, The method includes: Step 1: Construct a multi-source sensor fusion architecture that integrates multiple types of sensors; Step 2: Design an adaptive weighted fusion algorithm to dynamically adjust the fusion weights based on the real-time reliability of multiple sensor types and output obstacle information; Step 3: Design a software-defined 3D protected area to support various shapes and parameter configurations; Step 4: Design a dual-core redundant safety controller, with the master controller and slave controller operating independently to perform safety level voting and fault detection; Step 5: Design a multi-device coordination mechanism to support safe area overlap detection and task priority coordination of multiple lidars.

2. The method according to claim 1, characterized in that, Step 1 includes: Step 11: Sensor selection and configuration. Various types of sensors include 3D LiDAR for three-dimensional spatial perception, depth cameras and ultrasonic sensors for near-range obstacle detection, and inertial measurement units (IMU) sensors for robot posture perception. Step 12: Preprocess the data from each sensor. For the 3D LiDAR data, filter out noise points, transform the coordinate system, and extract point cloud features. For the depth camera data, filter out background interference, transform the coordinate system, and extract obstacle information. For the ultrasonic sensor data, filter out multiple reflection interference, transform the coordinate system, and calculate obstacle distances. For the IMU sensor data, filter out drift and calculate robot posture. Step 13: Timestamp Synchronization. To ensure time consistency across multiple data sources, a unified time base is adopted. IMU sensor data is used as the time base, and other sensor data is synchronized to the IMU timestamp through an interpolation algorithm. The time synchronization accuracy is less than 1ms, meeting the real-time fusion requirements. The expression for time synchronization is: P_sync(t)=P_raw(t_k)+(P_raw(t_{k+1})-P_raw(t_k))×(t-t_k) / (t_{k+1}-t_k); Where P_sync(t) is the synchronized data, P_raw(t_k) and P_raw(t_{k+1}) are the original data, t is the IMU timestamp, and t_k and t_{k+1} are the original sensor timestamps; Step 14: Unify the coordinate system. Transform all sensor data to a unified robot coordinate system. The expression for coordinate transformation is: P_robot = T•P_sensor; Where T is a 4×4 transformation matrix from the sensor coordinate system to the robot coordinate system, which includes rotation and translation, and is obtained through calibration experiments with a calibration accuracy of less than 1 mm; P_sensor is the point coordinate in the sensor coordinate system; P_robot is the point coordinate in the robot coordinate system. The calibration method uses a checkerboard calibration plate and solves the transformation matrix T using the least squares method, with a calibration error of less than 1 mm.

3. The method according to claim 2, characterized in that, Step 2 includes: Step 21: Sensor reliability assessment. Calculate the reliability of each sensor using the following expression: C_i = exp(-‖P_i(t)-P_fused(t-1)‖ 2 / (2σ_i 2 )); Where C_i is the reliability of sensor i, P_i(t) is the measurement value of sensor i at time t, P_fused(t-1) is the fusion result at time t-1, and σ_i is the historical deviation standard deviation of sensor i. The expression for σ_i is: σ_i=sqrt(Σ(P_i(t)-P_fused(t)) 2 / N); Where N is the size of the history window, and we take N=100; Step 22: Adaptive weight calculation, calculate the fusion weights based on credibility: w_i = C_i / ΣC_j; Where w_i is the fusion weight of sensor i, C_i is the confidence level of sensor i, and N is the number of sensors; the weight range is 0-1, and the sum of the weights is 1. Step 23: Weighted fusion calculation, performing weighted fusion of the measurements from all sensors: P_fused(t) = Σ(w_i·P_i(t)); Where P_fused(t) is the fusion result at time t, P_i(t) is the measurement value of sensor i at time t, and w_i is the corresponding weight; the fusion result is the weighted average of the measurement values ​​of all sensors. Step 24: Fusion accuracy evaluation. Assuming that the measurements from each sensor are independent, calculate the covariance matrix of the fusion result to evaluate the fusion accuracy. The expression for the covariance matrix is: Σσ_fused=Σ(w_i 2 ·Ss_i); Where Σσ_fused is the covariance matrix of the fusion result, w_i is the fusion weight, and Σσ_i is the covariance matrix of sensor i; The expression for fusion accuracy is: σ_fused = sqrt(w_1 2 p_1 2 + w_2 2 p_2 2 + w_3 2 p_3 2 +w_4 2 p_4 2 )。 4. The method according to claim 3, characterized in that, Step 3 includes: Step 31: Protect the region type, defined as a cube region (x_min, y_min, z_min, x_max, y_max, z_max); a cylinder region (x_c, y_c, z_min, z_max, r); a sphere region (x_c, y_c, z_c, r); or a custom polyhedron region defined by a list of vertex coordinates. Step 32: Configuration parameters for each protected area: ① Area type is cube, cylinder, sphere, or custom polyhedron; ② Define the specific dimensions of the area according to the type; ③ Safety level is SIL1, SIL2, or SIL3, used to determine detection accuracy and response time; ④ Determine the response time (10ms-100ms) according to the safety level; ⑤ Determine the trigger action according to the safety level, which includes stop, deceleration, alarm, or voice prompt. Step 33: The protected area is adjusted in real time with an adjustment delay of less than 10ms. The adjustment methods include manually modifying the protected area parameters through the host computer interface; automatically adjusting the size of the protected area according to the robot's operating status; and automatically switching the protected area configuration according to the task type. Step 34: Collision Detection. For each point cloud data point, determine whether it falls within the protected area: the criteria for a cube region are x_min≤x≤x_max and y_min≤y≤y_max and z_min≤z≤z_max; the criteria for a cylinder region is (x-xc). 2 +(y-yc) 2 ≤r 2 And z_min≤z≤z_max; the condition for determining the spherical region is (x-xc). 2 +(y-yc) 2 +(z-zc) 2 ≤r 2 Where x, y, z are the coordinates of the point cloud data points, xc, yc, zc are the coordinates of the center of the protected area, and r is the radius of the cylindrical area or the radius of the spherical area. Step 35: Supports multi-level protection zone configuration for progressive safety response: Warning zone, 1.5m away from the robot, triggers deceleration and voice prompts; Danger zone, 0.5m away from the robot, triggers emergency stop.

5. The method according to claim 4, characterized in that, Step 4 includes: Step 41: Dual-core architecture design. The safety controller includes two independent processor cores: a master controller and a slave controller. The master controller runs the main safety logic, monitors sensor data in real time, determines the safety status, and outputs control commands. The slave controller runs redundant safety logic, monitors sensor data in real time, independently determines the safety status, and performs safety voting with the master controller. Dual-core communication is achieved through shared memory and interrupt mechanisms, with a communication latency of less than 100μs. Step 42: Safety voting mechanism. The master controller and slave controller independently calculate the safety status and then vote on it. The voting rules are as follows: if both the master controller and slave controller determine that the system is safe, the system outputs a safe status; if both the master controller and slave controller determine that the system is dangerous, the system outputs a dangerous status; if the master controller and slave controller have different determinations, the system outputs a dangerous status. The voting delay time is less than 1ms to meet the real-time requirements. Step 43: Fault detection mechanism. The master controller and slave controller monitor each other to detect the following fault types: Processor failures: watchdog timeout, memory check error, program crash; Sensor failures: abnormal sensor data, sensor communication interruption; Communication failures: internal communication timeout, external communication interruption. Step 44: Based on the safety voting results and fault detection results, output the safety status; Safety status: The robot is running normally and outputs a safety signal; Warning status: The robot decelerates and issues a warning signal; Danger status: The robot stops and issues an alarm signal; The output delay time of the safety status is less than 5ms to meet the real-time requirements; Step 45: A dynamic pulse watchdog protection mechanism is adopted. The main controller periodically sends pulses to the slave controller. The slave controller detects the pulses. If it does not receive the pulse within a timeout, it determines that the main controller is faulty. The slave controller takes over the safety control and outputs a dangerous status. The expression for watchdog timeout calculation is: T_timeout = T_pulse × N_miss; Where T_pulse is the pulse period and N_miss is the number of pulses that can be missed.

6. The method according to claim 5, characterized in that, Step 5 includes: Step 51: After each LiDAR device is started, it registers with the coordination server via the EtherCAT bus; the registration information includes device ID, device type, protection zone parameters, and task priority; the coordination server maintains the device registry and updates the device status in real time. Step 52: Design a security area overlap detection algorithm to coordinate the server to periodically check whether the security areas of all devices overlap. The overlap detection algorithm is as follows: For each pair of devices i and j, calculate the intersection of their protected areas; if the intersection is not empty, it is determined that there is an overlap; prioritize the obstacles in the overlapping area, the overlap detection cycle is 10ms; the overlap detection delay time is less than 5ms. Step 53: Task priority coordination. Priority rules: Emergency stop tasks have the highest priority, with a priority value of 1; obstacle avoidance tasks have the second highest priority, with a priority value of 2; inspection tasks have the lowest priority, with a priority value of 3; if the priorities are the same, the coordination server will designate the responsible equipment. Step 54: When any device triggers an emergency stop, the coordination server broadcasts an emergency stop command to all devices; the emergency stop synchronization delay time is less than 10ms to ensure that all devices stop at the same time. Step 55: Use the IEEE 1588 precision time protocol for multi-device time synchronization with a synchronization accuracy of less than 1μs to meet the requirements of EtherCAT communication. The time synchronization process is as follows: ① The coordination server sends a synchronization request, ② Each device records its local time and responds, ③ The coordination server calculates the clock offset, and ④ Each device adjusts its local clock.

7. A system for integrated safety protection of robots through multi-sensor fusion, characterized in that, The system is used to implement the robot multi-sensor fusion integrated safety protection method as described in claim 1. The system includes a multi-source sensor module, an adaptive weighted fusion module, a three-dimensional protection area module, a dual-core redundant safety controller module, and a multi-device coordination module. The multi-source sensor module is used to build a multi-source sensor fusion architecture that integrates multiple types of sensors; The adaptive weighted fusion module is used to design an adaptive weighted fusion algorithm, which dynamically adjusts the fusion weights based on the real-time reliability of multiple types of sensors and outputs obstacle information. The 3D protected area module is used to design software-defined 3D protected areas to support various shapes and parameter configurations; The dual-core redundant safety controller module is used to design a dual-core redundant safety controller, in which the master controller and slave controller operate independently to perform safety level voting and fault detection. The multi-device coordination module is used to design a multi-device coordination mechanism, supporting safe area overlap detection and task priority coordination of multiple lidars.