An intelligent obstacle avoidance control system for automated lifting equipment

Abstract

This invention relates to the field of crane automation control technology and discloses an intelligent obstacle avoidance control system for automated spreaders, aiming to solve the problems of limited sensing range, insufficient environmental adaptability, and low obstacle avoidance decision-making efficiency in existing technologies. The system includes a multi-source fusion sensing module, a dynamic environment modeling module, a trajectory prediction and risk assessment module, a hierarchical decision control module, and an execution drive module. Through multi-source sensor data fusion, dynamic environment modeling, trajectory collaborative prediction, and hierarchical decision control, it achieves in-depth analysis of the spatiotemporal relationship between the spreader and obstacles, generates adaptive obstacle avoidance commands, and effectively improves the safety, autonomy, and operational efficiency of the spreader in dynamic environments.

Description

Technical Field

[0001] This invention relates to the field of crane automation control technology, specifically to an intelligent obstacle avoidance control system for automated lifting equipment. Background Technology

[0002] In the field of industrial automation and intelligent manufacturing, material handling and hoisting operations are key links in achieving continuous and efficient production processes. As core execution equipment, the level of intelligence of automated lifting equipment directly affects the safety and operational efficiency of the overall production system.

[0003] Among these technologies, intelligent obstacle avoidance control for automated spreaders is a crucial area for improving operational safety and autonomy. This technology aims to enable spreaders to autonomously identify obstacles and plan safe paths in complex and dynamic industrial environments through sensor perception, environmental modeling, and decision-making control, thereby reducing human intervention and preventing collisions.

[0004] Existing technologies typically rely on preset paths or single-type sensors for obstacle detection, resulting in limited sensing range and insufficient adaptability to dynamic environmental changes. Traditional control systems are slow to react when dealing with unstructured obstacles or sudden moving objects, and path replanning algorithms are computationally complex, making them unsuitable for real-time operations. Furthermore, existing methods lack in-depth fusion analysis of the spatiotemporal relationship between the spreader's motion state and obstacles, leading to conservative or overly aggressive obstacle avoidance decisions. This can easily cause decreased operational efficiency or safety risks in confined spaces or areas with high-density equipment. Therefore, there is an urgent need for an automated spreader obstacle avoidance control system capable of accurate sensing, rapid response, and intelligent decision-making. Summary of the Invention

[0005] The present invention aims to provide an intelligent obstacle avoidance control system for automated spreaders, in order to solve the problems of limited sensing range, insufficient dynamic environmental adaptability, excessively high computational complexity of path replanning algorithms, and lack of in-depth fusion analysis of the spatiotemporal relationship between the spreader's motion state and obstacles in the existing technology, which lead to conservative or aggressive obstacle avoidance decisions, reduced operating efficiency and safety risks.

[0006] The technical solution of this invention is to construct a complete system integrating multi-source heterogeneous sensing, dynamic environment modeling, real-time trajectory prediction, and hierarchical decision control. This system includes a multi-source fusion sensing module, a dynamic environment modeling module, a trajectory prediction and risk assessment module, a hierarchical decision control module, and an execution drive module. The multi-source fusion sensing module is responsible for collecting multi-dimensional data of the lifting equipment's operating environment. It includes at least two different types of sensor arrays, specifically a 3D LiDAR sensor array deployed at key locations on the lifting equipment body, a depth vision sensor array covering the area below and around the lifting equipment, and an inertial measurement unit installed on the lifting hook or load. The 3D LiDAR sensor array scans the surrounding environment at a frequency of no less than 10 Hz to acquire high-precision point cloud data. The depth vision sensor array simultaneously acquires RGB images and depth information of the environment. The inertial measurement unit measures the three-axis acceleration, three-axis angular velocity, and attitude angle of the lifting equipment body in real time. The multi-source fusion sensing module internally includes a data preprocessing submodule and a spatiotemporal synchronization submodule. The data preprocessing submodule performs filtering, noise reduction, and coordinate normalization on the raw sensor data. The spatiotemporal synchronization submodule uses a hardware trigger signal and timestamp alignment algorithm to ensure that data from different sensors are synchronized at the millisecond level in a unified world coordinate system.

[0007] The dynamic environment modeling module receives synchronous data from the multi-source fusion perception module and constructs a dynamic occupancy grid map of the operating environment. This module first performs obstacle clustering and segmentation on the fused point cloud and visual data to identify static and dynamic obstacles in the environment. For each identified obstacle, the dynamic environment modeling module assigns it an independent tracking identifier and uses an extended Kalman filter algorithm to estimate its current motion state, which includes at least position, velocity, and direction of motion. The dynamic environment modeling module maintains a 2D or 2.5D occupancy grid map with an update frequency of 20 Hz. The size of each grid is set from 5 cm to 20 cm depending on the accuracy requirements of the operating environment. Each grid not only records its occupancy probability but also associates it with the tracking identifier and motion state information of its corresponding obstacle.

[0008] The trajectory prediction and risk assessment module connects to the dynamic environment modeling module. Its functions include spreader trajectory prediction, obstacle trajectory prediction, and collision risk quantification assessment. Spreader trajectory prediction, based on the spreader's current motion commands and dynamics model, predicts the swing trajectory and envelope space of the spreader load over the next 3 to 5 seconds. Obstacle trajectory prediction, for each dynamic obstacle output by the dynamic environment modeling module, predicts its potential movement path over the next 3 to 5 seconds based on its historical motion state sequence using linear or nonlinear motion models, and assigns a confidence probability to each path. The collision risk quantification assessment submodule calculates the intersection probability in the spatiotemporal domain between the spreader's predicted trajectory envelope and the predicted paths of each obstacle, and combines this with the obstacle's type attributes and relative velocity to output a quantified instantaneous collision risk index and estimated collision time.

[0009] The hierarchical decision control module, as the core of the system's decision-making, receives collision risk information from the trajectory prediction and risk assessment module and generates the final obstacle avoidance control command based on preset safety thresholds and efficiency strategies. This module employs a three-layer decision architecture: a reaction layer, a tactical layer, and a strategic layer. The reaction layer directly responds to extremely high instantaneous collision risk indices. When the expected collision time is less than 0.5 seconds, the reaction layer bypasses other decision layers and immediately generates emergency braking or minimum-risk attitude adjustment commands. The tactical layer handles medium-risk situations with expected collision times between 0.5 and 3 seconds, performing local path replanning based on an improved artificial potential field method and model predictive control algorithms. The tactical layer incorporates a cost function that simultaneously optimizes path safety, operational efficiency, and energy consumption. The strategic layer addresses long-term risks or complex scenarios with expected collision times exceeding 3 seconds. It proactively adjusts the overall operating path of the spreader in conjunction with global mission objectives and may request fine-tuning of the task sequence from the higher-level material management system.

[0010] The execution drive module receives control commands from the hierarchical decision control module and converts them into drive signals for each actuator of the spreader. These actuators include the trolley drive motor responsible for the travel of the spreader trolley, the trolley drive motor responsible for the travel of the spreader carriage, the main hoisting motor responsible for lifting and lowering the spreader, and the auxiliary motor used for load anti-sway control. The execution drive module integrates a motion control card and a power amplifier to ensure that control commands can be executed quickly and accurately.

[0011] Furthermore, the depth vision sensor array in the multi-source fusion sensing module employs active stereo vision technology, with a built-in infrared structured light projector, enabling it to obtain reliable depth information even in industrial environments with low light levels or missing visual textures. The data preprocessing submodule uses a statistical outlier removal algorithm to filter noise from the 3D LiDAR point cloud data, and a bilateral filtering algorithm to smooth the depth vision data.

[0012] Furthermore, when constructing the occupied grid map, the dynamic environment modeling module uses a Bayesian update rule to dynamically update the occupied probability of each grid cell. For obstacles identified as dynamic, the corresponding grid probability decay rate is higher than that of static obstacles, thus reflecting the increasing uncertainty of the position of dynamic obstacles over time.

[0013] Furthermore, the nonlinear motion model used for obstacle trajectory prediction in the trajectory prediction and risk assessment module is a constant rate of rotation and velocity model that considers the obstacle's motion inertia and possible turning. The instantaneous collision risk index calculated by the collision risk quantification assessment submodule is the product of a negative exponential function of the expected collision time, a linear function of the relative velocity, and an obstacle type weighting factor.

[0014] Furthermore, the improved artificial potential field method adopted in the tactical layer of the hierarchical decision control module introduces a relative velocity factor into the traditional repulsive field function, making the repulsive force generated by the obstacle when moving towards the target much greater than that when moving in the same direction. The prediction time domain of the model predictive control algorithm is set to the next 2 seconds, and the control time domain is set to the next 0.5 seconds. In each control cycle, a constrained quadratic programming problem is solved to obtain the optimal control sequence.

[0015] Furthermore, the drive module performs feedforward and feedback composite control on the received control commands. It constructs a closed-loop position and speed control circuit by reading feedback signals from each motor encoder. For the main hoisting motor, a load weight sensor signal is also introduced for torque compensation to improve the smoothness of the lifting process.

[0016] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0017] By deeply fusing and synchronizing multi-source heterogeneous sensors, the system's sensing range and data reliability are significantly expanded, effectively overcoming the blind spots and false detection problems of single sensors. The dynamic environment modeling module constructs an occupancy grid map with motion state information, providing accurate and rich environmental context for subsequent prediction and decision-making. The trajectory prediction and risk assessment module achieves early and accurate identification of potential collision threats through collaborative prediction and risk quantification of the future trajectories of the spreader and obstacles. The hierarchical decision control architecture divides obstacle avoidance response into three levels: reaction, tactical, and strategic, ensuring that the system can make rapid and appropriate responses to risks of varying urgency, guaranteeing safety in extreme situations while maintaining operational efficiency under normal conditions. Through the close collaboration of its modules, the entire system achieves end-to-end intelligence from perception, understanding, prediction to decision execution, ultimately significantly improving the operational safety, autonomy, and overall efficiency of automated spreaders in complex and dynamic industrial environments. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall technical architecture of the control system of the present invention;

[0019] Figure 2 This is a schematic diagram of the core principle framework of the present invention;

[0020] Figure 3 This is a logical flowchart of the trajectory prediction and risk assessment module in this invention;

[0021] Figure 4 This is a schematic diagram of the multi-level interaction relationship and data flow of the hierarchical decision control module in this invention;

[0022] Figure 5 This is a schematic diagram comparing the core principle of this invention with existing technologies in terms of obstacle avoidance response time. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Example 1

[0025] Please refer to the attached document. Figure 1 The intelligent obstacle avoidance control system for automated spreaders is a complete system integrating multi-source heterogeneous sensing, dynamic environment modeling, real-time trajectory prediction, and hierarchical decision control. The system includes a multi-source fusion sensing module, a dynamic environment modeling module, a trajectory prediction and risk assessment module, a hierarchical decision control module, and an execution drive module. These modules are connected via a high-speed data bus to form a closed-loop control system, ensuring real-time performance and accuracy from environmental perception to control execution.

[0026] The multi-source fusion sensing module is responsible for collecting multi-dimensional data of the lifting equipment's operating environment. This module includes at least two different types of sensor arrays: a 3D LiDAR sensor array deployed at key locations on the lifting equipment itself, a depth vision sensor array covering the area below and around the lifting equipment, and an inertial measurement unit (IMU) mounted on the lifting hook or load. The 3D LiDAR sensor array scans the surrounding environment at a frequency of at least 10 Hz to acquire high-precision point cloud data. Each scan cycle generates approximately 100,000 spatial points, with the three-dimensional coordinates of the points achieving millimeter-level accuracy. The depth vision sensor array simultaneously acquires RGB images and depth information of the environment, with an image resolution of 1280 x 720 pixels and depth information accuracy better than 1 cm within a 2-meter range. The IMU measures the lifting equipment's three-axis acceleration, three-axis angular velocity, and attitude angle in real time, with an acceleration range of ±16g, an angular velocity range of ±2000 degrees per second, and an attitude angle accuracy of 0.1 degrees.

[0027] The multi-source fusion sensing module internally includes a data preprocessing submodule and a spatiotemporal synchronization submodule. The data preprocessing submodule performs filtering, noise reduction, and coordinate normalization on the raw sensor data. For 3D LiDAR point cloud data, a statistical outlier removal algorithm is used for noise filtering. This algorithm calculates the average distance between each point and its 50 neighboring points, identifying and removing points whose distance exceeds twice the standard deviation of the average. For depth vision data, a bilateral filtering algorithm is used for smoothing, with a filter kernel size of 5x5 pixels, a spatial standard deviation of 1.5, and a color range standard deviation of 0.1. The coordinate normalization process transforms all sensor data into a unified world coordinate system with the center of the lifting device base as the origin. The X-axis points in the direction of the lifting device's main trolley movement, the Y-axis points in the direction of the lifting device's auxiliary trolley movement, and the Z-axis points vertically upwards. The spatiotemporal synchronization submodule uses a hardware trigger signal and timestamp alignment algorithm to ensure millisecond-level synchronization of data from different sensors within the unified world coordinate system. Specifically, this submodule incorporates a high-precision clock source to provide synchronous trigger pulses for all sensors, with a pulse period of 10 milliseconds. At the rising edge of each trigger pulse, all sensors simultaneously begin data acquisition. The timestamps attached to the data packets have an accuracy of 1 microsecond. The synchronization algorithm aligns the data from different sensors to the same moment by comparing timestamp differences and performing linear interpolation, with a maximum synchronization error not exceeding 2 milliseconds.

[0028] The depth vision sensor array employs active stereo vision technology and incorporates an infrared structured light projector, enabling reliable depth information acquisition even in low-light or visually texture-deficient industrial environments. The infrared structured light projector generates a specific coded pattern at a wavelength of 850 nanometers with an output power of 2 watts. Depth calculation is based on the distortion patterns captured by two infrared cameras, using triangulation principles to calculate the depth value of each pixel. Even in environments with illumination below 10 lux, this technology maintains a depth measurement error of less than 2% of the actual distance.

[0029] The dynamic environment modeling module receives synchronous data from the multi-source fusion perception module and constructs a dynamic occupancy grid map of the operating environment. This module first performs obstacle clustering and segmentation on the fused point cloud and visual data. The clustering algorithm uses a density-based clustering method based on Euclidean distance, with a distance threshold of 0.2 meters. For each cluster, its circumscribed cuboid boundary is calculated, and visual features such as color and texture are extracted. By comparing the position and feature changes of clusters across consecutive frames, static and dynamic obstacles in the environment are identified. For each identified obstacle, the dynamic environment modeling module assigns it a unique tracking identifier, a 64-bit globally unique identifier. An extended Kalman filter algorithm is used to estimate its current motion state, which includes at least position, velocity, and direction of motion. The state vector contains six components: position and velocity in the X, Y, and Z directions. The observation model is based on sensor measurements, and the process noise covariance matrix is ​​dynamically adjusted according to the obstacle type. For obstacles with higher movement speeds, the process noise is set to a larger value to reflect higher uncertainty.

[0030] The dynamic environment modeling module maintains a 2D or 2.5D occupancy grid map at an update frequency of 20 Hz. The 2D map is suitable for planar operating environments, while the 2.5D map records the average height and height variance of each grid cell. The size of each grid cell is set from 5 cm to 20 cm depending on the accuracy requirements of the operating environment, with a default of 10 cm. Each grid cell not only records its occupancy probability but also associates it with the tracking markers and motion status information of its corresponding obstacles. The occupancy probability update uses a Bayesian update rule, with the specific formula as follows:

[0031]

[0032] in This indicates that the grid is occupied. Indicates that the grid is empty. Indicates from time arrive All observation data. For the sensor measurement model, this value is set to 0.9 for LiDAR and 0.85 for depth vision. The values ​​are set to 0.1 and 0.15 respectively. For obstacles identified as dynamic, the corresponding grid probability decay rate is higher than that of static obstacles. The grid probability of dynamic obstacles decays by 5% per frame, while that of static obstacles decays by only 1%, thus reflecting that the uncertainty of the position of dynamic obstacles increases over time.

[0033] The trajectory prediction and risk assessment module connects to the dynamic environment modeling module. Its functions include predicting the spreader's own trajectory, obstacle trajectory prediction, and quantitative assessment of collision risk. The spreader's own trajectory prediction, based on the current motion commands and dynamic model of the spreader, predicts the swing trajectory and envelope space of the spreader load within the next 3 to 5 seconds. The dynamic model considers the mass of the spreader's trolley, the trolley, the load mass, the wire rope length, and the damping coefficient. Motion commands include target velocity, target acceleration, and direction of motion. The prediction time domain is set to 5 seconds, with a time step of 0.1 seconds. The spatial position of the load at each time step within the prediction time domain is obtained by solving the spreader's motion differential equations through numerical integration. The envelope space is defined as a spherical region with the load center as its center and a radius 1.5 times the maximum size of the load.

[0034] Obstacle trajectory prediction, for each dynamic obstacle output by the dynamic environment modeling module, uses a linear or nonlinear motion model based on its historical motion state sequence to predict its potential motion path for the next 3 to 5 seconds. The linear motion model assumes the obstacle maintains its current uniform velocity. The nonlinear motion model is a constant rate of rotation and velocity model that considers the obstacle's inertia and possible turning. The model's state vector includes position, velocity, heading angle, and rate of rotation. Multiple hypothetical paths are generated during prediction, each assigned a confidence probability calculated based on the matching degree between historical trajectories and the model. Paths with a confidence probability below 0.3 are discarded.

[0035] The collision risk quantification assessment submodule calculates the probability of intersection between the predicted trajectory envelope of the lifting device and the predicted paths of each obstacle in the spatiotemporal domain. Combining this with the obstacle type attributes and relative velocities, it outputs a quantified instantaneous collision risk index and the estimated collision time. The instantaneous collision risk index is the product of a negative exponential function of the estimated collision time, a linear function of the relative velocity, and an obstacle type weighting factor. The specific calculation method is as follows:

[0036]

[0037] in The instantaneous collision risk index is represented by w, where w is the obstacle type weighting factor. For relative velocity, To estimate the collision time, The attenuation coefficient is set to 1.5. The obstacle type weighting factor is set according to obstacle attributes: 2.0 for personnel, 1.0 for equipment, and 0.5 for building structures. The estimated collision time is obtained by solving for the earliest intersection time between the rigging envelope and the predicted obstacle path, with a calculation accuracy of 0.01 seconds.

[0038] The hierarchical decision control module, as the core of the system's decision-making, receives collision risk information from the trajectory prediction and risk assessment module and generates the final obstacle avoidance control command based on preset safety thresholds and efficiency strategies. This module employs a three-layer decision architecture, including a reaction layer, a tactical layer, and a strategic layer. The reaction layer directly responds to extremely high instantaneous collision risk indices. When the expected collision time is less than 0.5 seconds, the reaction layer bypasses other decision layers and immediately generates an emergency braking or minimum risk attitude adjustment command. The emergency braking command is sent to all drive motors, requiring them to reduce their speed to zero within 0.2 seconds. The minimum risk attitude adjustment command adjusts the spreader's attitude, causing the load to swing away from the obstacle, with a maximum adjustment angle of 15 degrees.

[0039] The tactical layer decision-making is responsible for handling medium-risk scenarios with expected collision times between 0.5 and 3 seconds. It performs local path replanning based on an improved artificial potential field method and a model predictive control algorithm. The improved artificial potential field method introduces a relative velocity factor into the traditional repulsive field function. The repulsive field function is defined as:

[0040]

[0041] in The repulsion coefficient is set to 10, and d is the distance between the lifting device and the obstacle. For relative velocity, The maximum permissible relative speed of the system is set to 2 meters per second. Let be the unit vector pointing from the obstacle to the spreader. When the obstacle is moving towards the spreader, the relative velocity term amplifies the repulsive force; when moving in the same direction, the relative velocity term decreases the repulsive force. The prediction time domain of the model predictive control algorithm is set to the next 2 seconds, and the control time domain is set to the next 0.5 seconds. State variables include spreader position, velocity, and load swing angle; control variables are the output torque of each motor. A constrained quadratic programming problem is solved in each control cycle, with the objective function simultaneously optimizing path safety, operational efficiency, and energy consumption. The safety term has a weight of 0.6, the efficiency term has a weight of 0.3, and the energy term has a weight of 0.1. Constraints include maximum velocity, maximum acceleration, and workspace boundaries.

[0042] The strategic layer makes decisions for long-term risks or complex scenarios where the expected collision time exceeds 3 seconds. It proactively adjusts the overall operating path of the spreader based on global mission objectives. The strategic layer maintains a global path graph, where nodes represent critical path points and edges represent feasible paths. When a long-term risk is detected, the strategic layer replans the global path using the A* algorithm, considering factors such as path length, expected travel time, and cumulative risk. The strategic layer can also request fine-tuning of the task sequence from the higher-level material management system, such as adjusting the work order or inserting waiting instructions. Requests are sent via standard industrial communication protocols, with a 1-second timeout for response.

[0043] The execution drive module receives control commands from the hierarchical decision control module and converts them into drive signals for each actuator of the spreader. These actuators include a trolley drive motor for traveling the spreader trolley, a trolley drive motor for traveling the spreader carriage, a main hoisting motor for lifting the spreader, and an auxiliary motor for load anti-sway control. The trolley drive motor has a rated power of 22 kW and a maximum travel speed of 120 m / min. The trolley drive motor has a rated power of 11 kW and a maximum travel speed of 60 m / min. The main hoisting motor has a rated power of 30 kW and a maximum hoisting speed of 30 m / min. The auxiliary motor has a rated power of 3 kW and is used to actively suppress load sway.

[0044] The execution drive module integrates a motion control card and a power amplifier. The motion control card uses a 32-bit digital signal processor with a control cycle of 1 millisecond. The power amplifier uses insulated-gate bipolar transistor technology with a switching frequency of 20 kHz. The execution drive module performs feedforward and feedback composite control on the received control commands. Feedforward control calculates the ideal control quantity based on the lifting device's dynamics model, while feedback control reads the feedback signals from each motor encoder to form a closed-loop position and speed control loop. The encoder resolution is 10,000 pulses per revolution, and the feedback sampling period is 0.5 milliseconds. The proportional gain of the position control loop is set to 12, and the integral gain is set to 0.5. The proportional gain of the speed control loop is set to 8, and the integral gain is set to 0.3. For the main hoisting motor, a load weight sensor signal is also introduced for torque compensation. The load weight sensor is a strain gauge type with a range of 0 to 20 tons and an accuracy of 0.1% of full scale. The torque compensation algorithm adjusts the current output according to the real-time load weight, with a compensation coefficient of 0.5 amperes per ton.

[0045] The system modules communicate via an Ethernet bus using a publish-subscribe architecture. The data packet format includes a 32-bit header, a 64-bit timestamp, a 16-bit data length, a payload, and a 32-bit cyclic redundancy check (CRC) code. The communication cycle is 10 milliseconds, and bandwidth utilization is kept below 60%. The system has a watchdog mechanism; if any module fails to send a heartbeat signal within 100 milliseconds, an emergency system shutdown is triggered.

[0046] The installation positions of the multi-source fusion sensing module are precisely designed. The 3D LiDAR sensor array is mounted at the four corners of the lifting device, with a pitch angle adjusted to -15 degrees to cover the area below the device. The depth vision sensor array is mounted in the center of the device, with a field of view of 90 degrees horizontally and 60 degrees vertically. The inertial measurement unit is mounted at the hook connection, rigidly connected to the hook to ensure accurate measurement of the load's motion. All sensor housings have an IP67 protection rating, allowing for stable operation in dusty, high-humidity industrial environments.

[0047] The dynamic environment modeling module is configured with 8 gigabytes of memory to ensure the simultaneous processing of tens of thousands of raster data points. The obstacle tracking list has a maximum capacity of 256 objects; when this capacity is exceeded, the object with the longest tracking time is automatically discarded. The trajectory prediction and risk assessment module uses a graphics processor for parallel computation, predicting 100 trajectories in less than 5 milliseconds. The hierarchical decision control module has a decision cycle of 50 milliseconds, ensuring rapid response to dynamic environments. The control command output latency of the execution drive module is less than 2 milliseconds, guaranteeing real-time control performance.

[0048] Upon system startup, a self-test process is executed to check the connection status of all sensors, the feedback signals of actuators, and the integrity of communication links. After passing the self-test, the system enters standby mode, awaiting instructions from the upper-level system. During operation, the status of each module is continuously monitored, and when an anomaly is detected, alarms, degraded operation, or emergency shutdown measures are taken according to the severity. The system log records all critical events and data, with storage capacity supporting data retention for 30 days of continuous operation.

[0049] Example 2

[0050] Please refer to the attached document. Figure 2 In another implementation of the multi-source fusion sensing module, the depth vision sensor array can employ time-of-flight technology instead of active stereo vision technology. The time-of-flight sensor calculates depth by measuring the flight time of infrared light pulses, achieving a maximum measurement distance of 10 meters and an accuracy of 1 centimeter. This technology is insensitive to changes in ambient light and maintains stable performance even in strong light conditions. The sensor incorporates an 850-nanometer vertical-cavity surface-emitting laser with an output power of 1 watt and a frame rate of 10 Hz, consistent with system requirements.

[0051] The data preprocessing submodule employs a specific filtering algorithm tailored to the characteristics of time-of-flight data. Since time-of-flight data is susceptible to multipath interference, preprocessing first performs amplitude filtering to remove data points with signal amplitudes below a threshold. Subsequently, median filtering in the time domain is performed with a window size of 5 frames to effectively suppress random noise. Coordinate normalization is performed in the same manner as in Example 1, transforming all data to a unified world coordinate system.

[0052] The spatiotemporal synchronization submodule adds a software synchronization mechanism as redundancy to the hardware trigger. When the hardware trigger is lost, the system automatically switches to a network synchronization mode based on a precision time protocol, maintaining synchronization accuracy within 5 milliseconds. This mode is implemented through a master-slave clock architecture, with the master clock provided by the dynamic environment modeling module and the slave clocks being the internal clocks of each sensor.

[0053] When constructing the occupied grid map, the dynamic environment modeling module adjusts the Bayesian update parameters based on the characteristics of time-of-flight data. Because time-of-flight data has significant errors at long distances, P(z_t|o) decreases linearly with distance, dropping to 0.7 at 10 meters. Simultaneously, the grid size can be dynamically adjusted according to measurement accuracy, using a 5cm grid for close distances and a 20cm grid for long distances, reducing computational load while maintaining accuracy.

[0054] In this implementation, the trajectory prediction and risk assessment module adds the ability to handle semi-static obstacles. Semi-static obstacles refer to objects that are fixed in position but may begin to move at any time, such as temporarily placed shelves. The system identifies semi-static obstacles by analyzing their existence time and movement history. For semi-static obstacles, the prediction model generates both static and dynamic hypothetical paths simultaneously. The initial weight for the static hypothesis is 0.8, and the weight for the dynamic hypothesis is 0.2. When environmental disturbances are detected, the weight of the dynamic hypothesis gradually increases.

[0055] In this implementation, the strategic layer of the hierarchical decision control module enhances its interaction with the material management system. Besides task sequence fine-tuning requests, it can also receive prior environmental information from the material management system, such as known obstacle locations and predicted personnel flow patterns. This information is used to initialize the dynamic environment modeling module, accelerating the environmental understanding process. The strategic layer maintains a scenario database storing obstacle avoidance strategies for typical work scenarios. When a similar scenario is identified, preset strategies can be directly invoked, reducing decision-making time.

[0056] In this implementation, the execution drive module incorporates an adaptive control algorithm. This algorithm automatically adjusts the anti-sway control parameters based on real-time identified load sway characteristics. The control parameters are updated every 0.5 seconds by analyzing the load sway frequency using a Fast Fourier Transform. For loads of different weights, the system automatically adjusts the acceleration curve of the hoisting motor, using a gentler acceleration curve for heavy loads and a steeper acceleration curve for light loads to ensure smoothness during the hoisting process.

[0057] In this implementation, the system communication architecture employs a dual-network redundancy design. The primary network is a gigabit industrial Ethernet, and the backup network is a 100 Mbps Ethernet. When a primary network failure is detected, the system automatically switches to the backup network within 10 milliseconds. The communication protocol incorporates data priority marking, with obstacle avoidance-related data marked as the highest priority to ensure timely data transmission in emergency situations.

[0058] The mounting structure of the multi-source fusion sensing module has been optimized in this embodiment. The sensor mounting bracket adopts a vibration-damping design with built-in rubber vibration dampers, which can reduce vibration acceleration to 20% of the original value. The sensor cable uses shielded twisted-pair cable with an outer metal flexible conduit for protection against electromagnetic interference and mechanical damage. All connectors use waterproof aviation plugs to ensure reliable connection in harsh industrial environments.

[0059] In this implementation, the dynamic environment modeling module incorporates a memory mechanism. For obstacles that temporarily move out of the sensing range, the system retains their last known position and motion state, with the probability value gradually decaying over time. When an obstacle re-enters the sensing range, it is preferentially associated with obstacles in the memory, maintaining the continuity of the tracking marker. This mechanism effectively addresses the problem of obstacle loss caused by sensor blind spots.

[0060] In this implementation, the trajectory prediction and risk assessment module incorporates a multi-hypothesis management function. For obstacles with unclear motion intentions, multiple motion hypotheses are maintained simultaneously, each with its own confidence level and lifespan. Hypothesis confidence levels are dynamically updated based on new observation data, and a hypothesis is automatically discarded when its confidence level falls below 0.1 for three consecutive frames. This function improves the prediction accuracy for obstacles with complex motion patterns.

[0061] In this implementation, the hierarchical decision control module incorporates learning capabilities. The system records the process and results of each obstacle avoidance decision, including the actions taken, the final risk index, and efficiency indicators. By analyzing historical data, it automatically adjusts decision parameters, such as the setting of safety thresholds and the weighting of cost functions. The learning process is executed every 24 hours to ensure continuous optimization of obstacle avoidance performance.

[0062] In this implementation, the actuator module enhances fault diagnosis capabilities. By monitoring parameters such as motor current, temperature, and vibration, the health status of the actuator is assessed in real time. When abnormal signs are detected, an early warning is issued and maintenance is recommended. For non-critical faults, the system can automatically switch to degraded modes, such as single-motor drive or speed-limited operation, to ensure the continuity of basic operational capabilities.

[0063] This implementation also includes a human-machine interface. Operators can view the system status in real time via a touchscreen, including perception data, environmental maps, predicted trajectories, and decision results. The interface provides manual intervention functionality, allowing operators to directly control obstacle avoidance strategies in special circumstances. All interventions are recorded in detail for subsequent analysis and system improvement.

[0064] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An intelligent obstacle avoidance control system for automated lifting equipment, characterized in that, include: The multi-source fusion sensing module is used to collect multi-dimensional data of the lifting equipment's operating environment. It includes at least two different types of sensor arrays, specifically including a 3D LiDAR sensor array deployed at key locations on the lifting equipment body, a depth vision sensor array covering the area below and around the lifting equipment, and an inertial measurement unit installed on the lifting equipment hook or load. The dynamic environment modeling module is used to receive synchronous data from the multi-source fusion perception module and construct a dynamic occupancy grid map of the working environment. The dynamic environment modeling module first performs obstacle clustering and segmentation on the fused point cloud and visual data to identify static and dynamic obstacles in the environment. The dynamic environment modeling module uses Bayesian update rules to dynamically update the occupancy probability of each grid cell, and the probability decay rate of grid cells corresponding to dynamic obstacles is higher than that of static obstacles. The trajectory prediction and risk assessment module, connected to the dynamic environment modeling module, includes functions such as spreader trajectory prediction, obstacle trajectory prediction, and collision risk quantification assessment. The spreader trajectory prediction is based on the spreader's current motion commands and dynamic model. The dynamic model considers the spreader's trolley mass, trolley mass, load mass, wire rope length, and damping coefficient, predicting the spreader load's swing trajectory and envelope space over the next 3 to 5 seconds. The obstacle trajectory prediction, for each dynamic obstacle output by the dynamic environment modeling module, uses a constant rotation rate and velocity model based on its historical motion state sequence to predict its potential motion path over the next 3 to 5 seconds, assigning a confidence probability to each path. The hierarchical decision control module receives collision risk information from the trajectory prediction and risk assessment module and generates the final obstacle avoidance control command based on preset safety thresholds and efficiency strategies. The module employs a three-layer decision architecture: a reaction layer, a tactical layer, and a strategic layer. The reaction layer generates emergency braking or minimum-risk attitude adjustment commands when the expected collision time is less than 0.5 seconds. The tactical layer performs local path replanning based on an improved artificial potential field method and a model predictive control algorithm when the expected collision time is between 0.5 and 3 seconds. The improved artificial potential field method introduces a relative velocity factor into the traditional repulsive field function, making the repulsive force generated by the obstacle when moving head-on much greater than that when moving in the same direction. The model predictive control algorithm sets the prediction time domain to the next 2 seconds and the control time domain to the next 0.5 seconds, solving a constrained quadratic programming problem in each control cycle to obtain the optimal control sequence. The strategic layer makes forward-looking adjustments to the overall operating path of the spreader in conjunction with the global mission objective when the expected collision time exceeds 3 seconds. The execution drive module is used to receive control commands from the hierarchical decision control module and convert them into drive signals for each actuator of the spreader.

2. The intelligent obstacle avoidance control system for automated lifting equipment according to claim 1, characterized in that, The depth vision sensor array in the multi-source fusion sensing module adopts active stereo vision technology and has a built-in infrared structured light projector, which can still obtain reliable depth information in industrial environments with low light or lack of visual texture. The data preprocessing submodule uses a statistical outlier removal algorithm to filter noise in 3D LiDAR point cloud data and a bilateral filtering algorithm to smooth the depth vision data.

3. The intelligent obstacle avoidance control system for automated lifting equipment according to claim 1, characterized in that, When constructing the occupied grid map, the dynamic environment modeling module uses a Bayesian update rule to dynamically update the occupied probability of each grid. For obstacles identified as dynamic, the corresponding grid probability decay rate is higher than that of static obstacles, thus reflecting that the uncertainty of the position of dynamic obstacles increases over time.

4. The intelligent obstacle avoidance control system for automated lifting equipment according to claim 1, characterized in that, The nonlinear motion model used for obstacle trajectory prediction in the trajectory prediction and risk assessment module is a constant rate and velocity model that considers the obstacle's motion inertia and possible turning. The instantaneous collision risk index calculated by the collision risk quantification assessment is the product of a negative exponential function of the expected collision time, a linear function of the relative velocity, and an obstacle type weighting factor.

5. The intelligent obstacle avoidance control system for automated lifting equipment according to claim 1, characterized in that, The improved artificial potential field method used in the tactical layer of the hierarchical decision control module introduces a relative velocity factor into the traditional repulsive field function, making the repulsive force generated by the obstacle when moving towards the obstacle much greater than that when moving in the same direction. The prediction time domain of the model predictive control algorithm is set to the next 2 seconds, the control time domain is set to the next 0.5 seconds, and a constrained quadratic programming problem is solved in each control cycle to obtain the optimal control sequence.

6. The intelligent obstacle avoidance control system for automated lifting equipment according to claim 1, characterized in that, The execution drive module performs feedforward and feedback composite control on the received control commands; it forms a closed-loop position control and speed control loop by reading the feedback signals of each motor encoder; for the main hoisting motor, the load weight sensor signal is also introduced for torque compensation to improve the smoothness of the lifting process.

7. The intelligent obstacle avoidance control system for automated lifting equipment according to claim 1, characterized in that, The 3D lidar sensor array in the multi-source fusion sensing module scans the surrounding environment at a frequency of no less than 10 Hz to acquire high-precision point cloud data; the depth vision sensor array synchronously acquires RGB images and depth information of the environment; and the inertial measurement unit measures the three-axis acceleration, three-axis angular velocity, and attitude angle of the lifting device body in real time.

8. The intelligent obstacle avoidance control system for automated lifting equipment according to claim 1, characterized in that, When performing obstacle clustering and segmentation on the fused point cloud and visual data, the dynamic environment modeling module adopts a density clustering method based on Euclidean distance; when estimating the obstacle motion state using the extended Kalman filter algorithm, the state vector contains six components: position and velocity in the X, Y, and Z directions.

9. The intelligent obstacle avoidance control system for automated lifting equipment according to claim 1, characterized in that, The trajectory prediction and risk assessment module predicts the spreader's own trajectory based on the spreader's current motion command and dynamic model. The dynamic model considers the spreader's trolley mass, trolley mass, load mass, wire rope length, and damping coefficient. The obstacle trajectory prediction is based on historical motion state sequences and uses linear or nonlinear motion models to predict future motion paths.

10. The intelligent obstacle avoidance control system for automated lifting equipment according to claim 1, characterized in that, The reaction layer of the hierarchical decision control module generates an emergency response when the expected collision time is less than 0.5 seconds. Braking or minimum risk attitude adjustment commands; when the expected collision time is between 0.5 seconds and 3 seconds, the tactical layer performs local path replanning based on the improved artificial potential field method and model predictive control algorithm; when the expected collision time exceeds 3 seconds, the strategic layer makes forward-looking adjustments to the overall operating path of the spreader in conjunction with the global mission objective.

Images