Dynamic obstacle detecting and warning system for reach stacker

Abstract

The invention relates to the field of obstacle detection, and particularly discloses a dynamic obstacle detection warning system for a reach stacker, comprising: an obstacle detection module for capturing environmental data in a working area of the reach stacker in real time by using a sensor technology, the environmental data comprising position, speed, acceleration and direction information, potential obstacles are identified and analyzed, and data acquisition and primary processing are carried out; by capturing and analyzing environmental data in a working area in real time, the system can accurately identify potential obstacles and predict future motion trails of the obstacles, and once a collision risk is predicted, the system immediately triggers a warning signal and provides detailed obstacle information through a user interface, so that an operator can quickly make a response; the control interface and the reach stacker control system are seamlessly integrated, so that the obstacle avoidance operation can be automatically executed according to the emergency degree of the warning signal, the occurrence of collision accidents is effectively avoided, and the operation safety is remarkably improved.

Description

Technical Field

[0001] The invention belongs to the field of obstacle detection, in particular to a dynamic obstacle detection and warning system for a front loader. Background Art

[0002] Reach stackers are multifunctional and highly efficient lifting equipment, mainly used for stacking and handling containers as well as horizontal transportation at docks and yards. They can move freely between ships and shores, and accurately place containers at designated locations by swinging their arms.

[0003] Obstacle detection technology is the core of the dynamic obstacle detection and warning system. This technology usually relies on various sensors, such as radar, LiDAR, cameras, etc., to capture environmental data in the operating area in real time. These sensors can sense the location, speed, shape and other information of obstacles and transmit these data to the data processing unit for analysis.

[0004] Traditional technologies can often only provide static information about obstacles but cannot effectively determine their dynamic status. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a dynamic obstacle detection and warning system for a front loaders to solve the problem that the conventional technology in the prior art can only provide static information of obstacles but cannot effectively judge their dynamic status.

[0006] Dynamic obstacle detection and warning system for reach stackers, including:

[0007] The obstacle detection module is used to use sensor technology to capture the environmental data in the reach stacker operation area in real time, including position, speed, acceleration and direction information, to identify and analyze potential obstacles, and to perform data collection and preliminary processing;

[0008] A data processing unit connected to the obstacle detection module, configured to receive and analyze the environmental data, and determine the position, speed and movement trend of the obstacle, thereby determining the dynamic state of the obstacle;

[0009] A deep learning prediction module, connected to the data processing unit, for receiving historical obstacle data and current environment information provided by the data processing unit, and predicting the future movement trajectory of the obstacle;

[0010] The warning signal generation module, based on the precise analysis results of the data processing unit and the deep learning prediction module, will immediately trigger the sound and light alarm mechanism when it is predicted that the obstacle may enter the path of collision with the front loader; the sound and light alarm mechanism not only issues a warning to drive away, but also alerts people and vehicles around the obstacle through flashing lights and loud alarms, prompting them to quickly evacuate the potential danger zone; after detecting a dynamic obstacle, the warning signal generation module can intelligently adjust the intensity of the driving away sound and light signal according to the moving speed and distance of the obstacle. When the obstacle moves quickly and the distance is close, the system will correspondingly enhance the intensity of the sound and light signal to prompt the obstacle to avoid in time with a stronger warning effect; the warning signal generation module can flexibly increase or decrease the intensity of the driving away sound and light signal according to the level of danger. By comprehensively evaluating the position, speed, distance and expected collision time of the obstacle, the warning signal generation module can accurately determine the danger level and adjust the intensity of the alarm accordingly;

[0011] A user interface for displaying the warning signal and relevant information about the obstacle, including the location, distance, estimated collision time and predicted future motion trajectory of the obstacle, and providing audio and visual alarms to immediately notify the reach stacker operator;

[0012] The control interface is integrated with the control system of the reach stacker and automatically triggers deceleration, stopping and other obstacle avoidance operation instructions according to the urgency of the warning signal to ensure operation safety.

[0013] Preferably, the formula for data collection and preliminary processing is as follows:

[0014]

[0015] Among them, S(t) is the environmental data set collected at time t; f sensor (x, y, z, θ, φ, t′) is the data collected by the sensor at time t′, including position coordinates (x, y, z), direction angles (θ, φ) and time t′; x, y, z represent the coordinates of the sensor in three-dimensional space respectively; θ, φ represent the pitch angle and yaw angle of the sensor respectively; t′ is the time point of data collection.

[0016] Preferably, the formula for determining the position, speed and movement trend of the obstacle to determine the dynamic state of the obstacle is as follows:

[0017]

[0018] Among them, P obs (t) is the result of judging the dynamic state of the obstacle at time t; N is the number of obstacles detected; w i is the weight of the ith obstacle, which is determined according to the obstacle type and size; g(vi (t),a i (t),Δd i (t)) is the judgment function, including the speed v of the obstacle i (t), acceleration a i (t) and the relative distance change Δd from the reach stacker i (t); v i (t) is the speed of the ith obstacle at time t; a i (t) is the acceleration of the ith obstacle at time t; Δd i (t) is the change in the relative distance between the ith obstacle and the reach stacker at time t.

[0019] Preferably, the formula for predicting the future motion trajectory of the obstacle is as follows:

[0020]

[0021] Among them, T pred (t+Δt) is the future motion trajectory of the obstacle predicted at time t+Δt; is the normal distribution function, indicating the uncertainty of the prediction, μ pred (t) is the predicted mean, is the predicted variance; h(x,y,z,Δt) is the prediction function based on deep learning, which represents the probability density of the obstacle at the position (x,y,z) after a given time interval Δt; is a Gaussian function, representing the uncertainty of the observed obstacle position, μ obs (t) is the mean of the observations, is the observed variance.

[0022] Preferably, the formula for predicting the obstacle position is as follows:

[0023]

[0024] Among them, R warn (t) is the warning signal strength generated at time t; P obs (t) is the result of judging the dynamic state of the obstacle; T risk (t) is a risk assessment function based on obstacle type and speed, indicating the risk level of collision between the obstacle and the reach stacker; T pred (t+Δt) is the predicted future motion trajectory of the obstacle; D safe (t) is the safety distance threshold between the reach stacker and the obstacle; exp(-(T pred (t+Δt)-D safe (t))) is an exponential function used to adjust the warning signal strength. When the predicted trajectory approaches or falls below the safety distance, the signal strength increases.

[0025] Preferably, the obstacle detection module further comprises:

[0026] A multi-sensor technology integration unit that integrates radar, lidar, camera, and infrared sensors to provide multi-modal environmental data and enhance the accuracy and robustness of obstacle detection;

[0027] The data fusion algorithm is used to fuse data from different sensors, reduce data redundancy, improve data consistency and reliability, and provide high-quality environmental data for subsequent data processing and analysis.

[0028] Preferably, the data processing unit further comprises:

[0029] Data cleaning module, used to identify and remove noise and outliers in environmental data to ensure data accuracy and validity;

[0030] The feature extraction module is used to extract key features from the cleaned data, including the velocity change rate and acceleration change rate of the obstacle, so as to describe the dynamic behavior of the obstacle in a more detailed manner.

[0031] Preferably, the deep learning prediction module further comprises:

[0032] Model training module, which is used to train the deep learning model based on historical obstacle data and environmental information, and optimize the parameters of the prediction function h(x, y, z, Δt) to improve the accuracy of future motion trajectory prediction;

[0033] Model update mechanism, used to regularly update the deep learning model to adapt to changes in different operating environments and obstacle types.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] By capturing and analyzing environmental data in the work area in real time, the system can accurately identify potential obstacles and predict their future movement trajectories. Once a collision risk is predicted, the system immediately triggers a warning signal and provides detailed obstacle information through the user interface, allowing the operator to respond quickly.

[0036] The system can intelligently adjust the intensity of the sound and light signals according to the speed and distance of the obstacle, ensuring appropriate and effective warning effects at different danger levels, avoiding panic caused by excessive warnings and ensuring sufficient warnings in emergency situations. At the same time, through flashing lights and loud alarms, the sound and light alarm mechanism can quickly attract the attention of people and vehicles around the obstacle, increase their awareness of potential dangers, prompt them to take avoidance measures in time, and reduce the possibility of accidents.

[0037] The seamless integration of the control interface and the reach stacker control system can automatically perform obstacle avoidance operations according to the urgency of the warning signal, effectively avoiding collision accidents and significantly improving operational safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a system schematic diagram of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] like Figure 1 As shown:

[0041] Embodiment 1: The present invention provides a dynamic obstacle detection and warning system for a reach stacker, comprising:

[0042] Obstacle detection module, which uses sensor technology to capture real-time environmental data in the reach stacker operation area. Environmental data includes position, speed, acceleration and direction information to identify and analyze potential obstacles, and perform data collection and preliminary processing;

[0043] A data processing unit, connected to the obstacle detection module, is used to receive and analyze environmental data, and determine the position, speed and movement trend of the obstacle, so as to determine the dynamic state of the obstacle;

[0044] A deep learning prediction module is connected to the data processing unit and is used to receive the obstacle history data and current environment information provided by the data processing unit and predict the future movement trajectory of the obstacle;

[0045] The warning signal generation module, based on the precise analysis results of the data processing unit and the deep learning prediction module, will immediately trigger the sound and light alarm mechanism when it is predicted that the obstacle may enter the path of collision with the front loader; the sound and light alarm mechanism not only issues a warning to drive away, but also alerts people and vehicles around the obstacle through flashing lights and loud alarms, prompting them to quickly evacuate the potential danger zone; after detecting a dynamic obstacle, the warning signal generation module can intelligently adjust the intensity of the driving away sound and light signal according to the moving speed and distance of the obstacle. When the obstacle moves quickly and the distance is close, the system will correspondingly enhance the intensity of the sound and light signal to prompt the obstacle to avoid in time with a stronger warning effect; the warning signal generation module can flexibly increase or decrease the intensity of the driving away sound and light signal according to the level of danger. By comprehensively evaluating the position, speed, distance and expected collision time of the obstacle, the warning signal generation module can accurately determine the danger level and adjust the intensity of the alarm accordingly;

[0046] A user interface that displays warning signals and information about obstacles, including the obstacle's location, distance, estimated collision time, and predicted future motion trajectory, while providing audible and visual alarms to immediately notify the reach stacker operator;

[0047] The control interface is integrated with the control system of the reach stacker and automatically triggers deceleration, stopping and other obstacle avoidance operation instructions according to the urgency of the warning signal to ensure operation safety.

[0048] As can be seen from the above, this system integrates an obstacle detection module, which can capture environmental data in the working area in real time, such as position, speed, acceleration and direction information, and accurately identify and analyze potential obstacles; the data processing unit is responsible for in-depth analysis of these data to determine the dynamic state of the obstacle; combined with the deep learning prediction module, the system can predict the future movement trajectory of the obstacle; once it is predicted that the obstacle may collide with the reach loader, the warning signal generation module will immediately trigger a warning, and instantly notify the operator through the user interface in sound and visual ways, and display detailed information of the obstacle; at the same time, the control interface is seamlessly integrated with the reach loader control system, and automatically executes obstacle avoidance operations such as deceleration and stopping according to the urgency of the warning signal, to fully ensure the safety of the operation; this system not only improves the safety of the reach loader operation, but also optimizes the operating efficiency.

[0049] Embodiment 2: This embodiment is basically the same as the previous embodiment, except that the formulas for data collection and preliminary processing are as follows:

[0050]

[0051] Among them, S(t) is the environmental data set collected at time t; f sensor(x, y, z, θ, φ, t′) is the data collected by the sensor at time t′, including position coordinates (x, y, z), direction angles (θ, φ) and time t′; x, y, z represent the coordinates of the sensor in three-dimensional space; θ, φ represent the pitch angle and yaw angle of the sensor respectively; t′ is the time point of data collection;

[0052] The specific application process of the above formula is as follows:

[0053] 1. Sensor deployment and initialization

[0054] Complete the deployment and initialization of sensors, including selecting the appropriate sensor type (such as radar, lidar, camera, etc.), determining the installation location and direction of the sensor, and performing necessary calibration and parameter settings.

[0055] Sensor type selection: Select the appropriate sensor type according to the requirements of the application scenario.

[0056] Installation location and orientation: Install the sensor in a location that provides full coverage of the work area and orient it to ensure effective detection of potential obstacles.

[0057] Calibration and parameter setting: Perform necessary calibration on the sensor to ensure the accuracy of its data. At the same time, set a reasonable sampling frequency and data format for subsequent data processing.

[0058] 2. Data Collection

[0059] Once the sensor is deployed and initialized, the data collection process can begin. Following the integration principle described by the formula, the data collected by the sensor over a period of time is continuously recorded.

[0060] Time integration: The integral symbol in the formula indicates the integration of time, that is, continuously recording the data collected by the sensor at different time points. This requires the sensor to have continuous and stable working performance to ensure the continuity and integrity of the data.

[0061] Data recording: During the data collection process, the sensor will continuously output data including position coordinates (x, y, z), direction angles (θ, φ) and time t′. These data will be recorded and stored in real time for subsequent data processing and analysis.

[0062] 3. Data Preprocessing

[0063] After data collection is completed, it is usually necessary to preprocess the data to improve its quality and usability, including data cleaning, data format conversion, and data normalization.

[0064] Data cleaning: Remove noise and outliers from data to ensure data accuracy and reliability.

[0065] Data format conversion: Convert data into a format suitable for subsequent processing and analysis, such as converting raw data into a structured database table or data frame.

[0066] Data normalization: Normalize the data to eliminate the difference in data volume between different sensors or at different time points, so as to facilitate subsequent data analysis and comparison.

[0067] 4. Obstacle detection and analysis

[0068] The pre-processed data is then used for obstacle detection and analysis, which includes identifying potential obstacles and estimating their position, speed, and movement trends.

[0069] Obstacle identification: Use image processing or machine learning algorithms to analyze pre-processed data and identify potential obstacles. This can be achieved by comparing current data with historical data, setting thresholds, or using pattern recognition techniques.

[0070] Position and velocity estimation: Based on the identified obstacle data, estimate its position coordinates (x, y, z) and velocity information. This can be achieved by calculating the position change of the obstacle between different time points.

[0071] Movement trend analysis: By analyzing the speed and direction changes of obstacles, the future movement trend of obstacles can be predicted. This helps to detect potential dangerous situations in advance and take corresponding obstacle avoidance measures.

[0072] 5. Result output and application

[0073] After obstacle detection and analysis are completed, key information will be passed to the warning signal generation module, which will execute a series of intelligent response measures based on the precise analysis results of the data processing unit and the deep learning prediction module.

[0074] Warning signal generation and adjustment:

[0075] When it is predicted that an obstacle may enter the path of collision with the reach stacker, the warning signal generation module will immediately trigger the sound and light alarm mechanism. This mechanism not only issues a warning to drive away, but also alerts people and vehicles around the obstacle through flashing lights and loud alarms, prompting them to quickly evacuate the potential danger zone.

[0076] At the same time, after detecting a dynamic obstacle, the warning signal generation module can intelligently adjust the intensity of the sound and light signals according to the speed and distance of the obstacle. When the obstacle moves quickly and is close, the system will increase the intensity of the sound and light signals accordingly, prompting the obstacle to be avoided in time with a stronger warning effect, thereby effectively reducing the risk of collision.

[0077] Danger level assessment and signal strength adjustment:

[0078] The warning signal generation module also has the ability to flexibly adjust the intensity of the expulsion sound and light signals according to the danger level. By comprehensively evaluating multiple key parameters such as the obstacle's location, speed, distance, and expected collision time, the system can accurately determine the current danger level.

[0079] The warning signal generation module will automatically adjust the intensity of the alarm according to the danger level. When the danger level is high, the system will send out a stronger and more urgent alarm signal to ensure that people and vehicles around the obstacle can respond quickly and stay away from the danger zone.

[0080] User interface and control system integration:

[0081] The detection results are not only transmitted to the personnel and vehicles around the obstacle in real time in the form of sound and light through the warning signal generation module, but are also displayed synchronously on the user interface. The user interface displays information such as the location, speed, movement trend, and hazard level of the obstacle in an intuitive way, providing operators with comprehensive working environment monitoring.

[0082] At the same time, the detection results are also integrated into the control system. When a potential collision risk is predicted, the control system can automatically trigger deceleration, stop or other obstacle avoidance operation instructions, thereby further improving the safety and efficiency of the operation. This integrated design enables the reach stacker to maintain a high degree of alertness and flexible response capabilities in complex and changing operating environments.

[0083] Specifically, the formula for determining the position, speed and movement trend of an obstacle and thus determining the dynamic state of the obstacle is as follows:

[0084]

[0085] Among them, P obs (t) is the result of judging the dynamic state of the obstacle at time t; N is the number of obstacles detected; w i is the weight of the ith obstacle, which is determined according to the obstacle type and size; g(v i (t),a i (t),Δd i (t)) is the judgment function, including the speed v of the obstacle i (t), acceleration a i (t) and the relative distance change Δd from the reach stacker i (t); v i (t) is the speed of the ith obstacle at time t; a i (t) is the acceleration of the ith obstacle at time t; Δd i(t) is the change in the relative distance between the ith obstacle and the reach stacker at time t;

[0086] The specific application process of the above formula is as follows:

[0087] 1. Data acquisition and preprocessing

[0088] Obtain real-time obstacle data from the obstacle detection module and perform necessary preprocessing.

[0089] Data acquisition: Obstacle detection module can be used to obtain real-time information such as the position, speed, acceleration, etc. of obstacles in the working area. This information is usually provided in the form of data streams and needs to be continuously collected and processed.

[0090] Data preprocessing: Clean and format the acquired data to ensure its accuracy and consistency. For example, remove noise data, fill in missing values, convert data format, etc.

[0091] 2. Determination of obstacle weight

[0092] Each obstacle has a weight w i ,This weight is determined according to factors such as the type and size of the obstacle.

[0093] Obstacle type identification: First, the type of obstacle needs to be identified, such as vehicles, pedestrians, buildings, etc. Different types of obstacles may have different effects on the travel of the reach stacker.

[0094] Weight Assignment: Assign a weight to the obstacle based on its type, size, etc. Generally, larger obstacles or obstacles that move faster are given higher weights.

[0095] 3. Definition and calculation of judgment function g

[0096] The judgment function g is the core part of the formula, which determines the dynamic state of the obstacle based on its speed, acceleration and relative distance change.

[0097] Speed i (t) and acceleration a i Calculation of (t): Using the data provided by the obstacle detection module, calculate the velocity and acceleration of each obstacle at time t.

[0098] Relative distance change Δd i Calculation of (t): Based on the position of the front loader and the position of each obstacle, calculate the relative distance between them and the change in the relative distance at time t.

[0099] Definition of judgment function g: Judgment function g is usually a complex mathematical model that takes into account multiple factors such as speed, acceleration, and relative distance change. In practical applications, it may be necessary to train and optimize this function through techniques such as machine learning or deep learning.

[0100] Calculation of judgment function g: Substitute the speed, acceleration and relative distance change of each obstacle into the judgment function g and calculate the corresponding judgment result.

[0101] 4. Determination of the dynamic state of obstacles

[0102] After obtaining the judgment result of each obstacle, the formula is used to perform summation to obtain the overall judgment result P of the dynamic state of the obstacle. obs (t).

[0103] Sum operation: Multiply the judgment result of each obstacle by its weight and perform the sum operation. This process reflects the consideration of the importance of different obstacles.

[0104] Judgment result explanation: According to the result of the summation operation P obs (t), the dynamic state of the obstacle can be determined. For example, if P obs The larger the value of (t), the more complex the dynamic state of obstacles in the working area, and the reach stacker needs to take more cautious obstacle avoidance operations.

[0105] 5. Result output and application

[0106] Finally, the judgment result of the dynamic state of the obstacle is output to the user interface or control system so that the operator or automation system can take corresponding actions.

[0107] User interface display: The judgment results are displayed on the user interface in an intuitive manner, such as displaying the dynamic status of obstacles, warning information, etc. through a graphical interface.

[0108] Control system integration: Integrate the judgment results into the control system to automatically trigger deceleration, stop or other obstacle avoidance operation instructions. This helps to improve the driving safety and efficiency of the reach stacker.

[0109] Specifically, the formula for predicting the future motion trajectory of an obstacle is as follows:

[0110]

[0111] Among them, T pred (t+Δt) is the future motion trajectory of the obstacle predicted at time t+Δt; is the normal distribution function, indicating the uncertainty of the prediction, μ pred (t) is the predicted mean, is the predicted variance; h(x,y,z,Δt) is the prediction function based on deep learning, which represents the probability density of the obstacle at the position (x,y,z) after a given time interval Δt; is a Gaussian function, representing the uncertainty of the observed obstacle position, μ obs (t) is the mean of the observations, is the observed variance;

[0112] The specific application process of the above formula is as follows:

[0113] 1. Data preparation and preprocessing

[0114] Collect and preprocess relevant data.

[0115] Obstacle location data: Obstacle location information, including x, y, z coordinates, is obtained in real time through sensors (such as radar, lidar, camera, etc.). This data will be used to train the deep learning prediction function h(x, y, z, Δt) and calculate the mean μ of the observations obs (t) and variance

[0116] Historical trajectory data: Collect the movement trajectory data of obstacles in the past period of time, which is used to train the deep learning model to learn the movement patterns of obstacles. This data may include information such as the position, speed, and acceleration of the obstacles.

[0117] Data cleaning and formatting: Clean the collected data, remove noise and outliers, and format it to ensure data accuracy and consistency.

[0118] 2. Deep learning prediction function training

[0119] The deep learning prediction function h(x, y, z, Δt) is the core part of the formula. It needs to learn the movement law of obstacles based on historical trajectory data and predict their positions at future time points.

[0120] Model selection: Select a suitable deep learning model based on the application scenario and data characteristics, such as convolutional neural network (CNN), recurrent neural network (RNN) or long short-term memory network (LSTM).

[0121] Model training: Input historical trajectory data into the deep learning model for training. During the training process, the model will learn the movement patterns of obstacles and optimize the parameters of the prediction function h(x, y, z, Δt).

[0122] Model evaluation: Evaluate the performance of the model through methods such as cross-validation to ensure that the model has good generalization ability and prediction accuracy.

[0123] 3. Forecast uncertainty calculation

[0124] Normal distribution function Used to express the uncertainty of a forecast. When calculating forecast uncertainty, several factors need to be considered.

[0125] Predicted mean μ pred (t) Calculation: Based on the output of the deep learning prediction function h(x, y, z, Δt), calculate the predicted mean μ pred (t). This is usually done by taking the expected value of the output of the prediction function.

[0126] Prediction Variance Calculation: The prediction variance reflects the uncertainty of the prediction results. It can be obtained by calculating the variance of the prediction function output or using other uncertainty estimation methods.

[0127] 4. Observational uncertainty calculation

[0128] Gaussian function It is used to express the uncertainty of the observed obstacle position. When calculating the observation uncertainty, factors such as the accuracy and noise of the sensor need to be considered.

[0129] Observed mean μ obs (t) Calculation: Calculate the mean value μ of the observation based on the obstacle position data obtained by the sensor in real time obs (t). This is usually done by taking the average of multiple observations.

[0130] Observation variance Calculation: The observation variance reflects the uncertainty of the observation. It can be obtained by calculating the variance of the observations or using the noise estimate provided by the sensor.

[0131] 5. Prediction trajectory calculation

[0132] After obtaining the prediction uncertainty, observation uncertainty, and deep learning prediction function, the formula can be used to calculate the predicted trajectory of the obstacle at a future time point.

[0133] Formula application: predict the mean μ pred (t), prediction variance Observed mean μ obs (t), observed variance The deep learning prediction function h(x,y,z,Δt) is substituted into the formula for calculation. This process involves complex integral operations and convolution operations of Gaussian functions.

[0134] Result interpretation: The output of the formula T pred(t+Δt) represents the predicted trajectory of the obstacle at the future time point t+Δt and its uncertainty. This result can be used to guide the obstacle avoidance operation of autonomous vehicles, robot navigation path planning and other application scenarios.

[0135] 6. Result verification and application

[0136] Finally, the prediction results need to be verified and applied to actual scenarios.

[0137] Result verification: Verify the accuracy and reliability of the prediction results by comparing them with the actual observations. This can be evaluated by calculating indicators such as prediction error and recall rate.

[0138] Application scenarios: Apply the prediction results to fields such as autonomous driving, robot navigation, and monitoring systems to achieve real-time tracking and prediction of obstacles and improve the safety and efficiency of the system.

[0139] Specifically, the formula for predicting the obstacle position is as follows:

[0140]

[0141] Among them, R warn (t) is the warning signal strength generated at time t; P obs (t) is the result of judging the dynamic state of the obstacle; T risk (t) is a risk assessment function based on obstacle type and speed, indicating the risk level of collision between the obstacle and the reach stacker; T pred (t+Δt) is the predicted future motion trajectory of the obstacle; D safe (t) is the safety distance threshold between the reach stacker and the obstacle; exp(-(T pred (t+Δt)-D safe (t))) is an exponential function used to adjust the warning signal strength. When the predicted trajectory approaches or falls below the safety distance, the signal strength increases;

[0142] The specific application process of the above formula is as follows:

[0143] Data collection and preprocessing:

[0144] The environmental data within the reach loader operation area is collected in real time through a variety of sensor technologies (such as radar, lidar, camera, etc.) in the obstacle detection module.

[0145] The collected data is preprocessed, including data cleaning, feature extraction and other steps, to obtain high-quality data that can be used for subsequent analysis.

[0146] Obstacle dynamic state judgment:

[0147] The preprocessed data is analyzed using formulas to determine the dynamic states of obstacles such as location, speed, and movement trend.

[0148] Obtain the obstacle dynamic state judgment result P obs (t).

[0149] risk assessment:

[0150] According to the information such as the type and speed of the obstacle, the risk assessment function T risk (t) Assess the risk level of collision with the reach stacker.

[0151] Future motion trajectory prediction:

[0152] The deep learning prediction module is used to analyze the historical data and current environmental information of obstacles and predict their future movement trajectory T pred (t+Δt).

[0153] Warning signal generation:

[0154] The above-obtained P obs (t), T risk (t), T pred (t+Δt) and the preset safety distance threshold D safe Substitute (t) into Formula 4.

[0155] Calculate the warning signal strength R at time t warn (t).

[0156] Alarm and response:

[0157] According to the calculated warning signal strength R warn (t), the warning signal generation module will issue corresponding sound and visual alarms to the reach stacker operator through the user interface. The sound alarm includes alarm sounds of different frequencies and volumes to distinguish different levels of dangerous situations; the visual alarm uses flashing icons, color changes or dynamic graphics on the display to intuitively display information such as the location, speed and expected collision path of the obstacle;

[0158] The warning signal generation module intelligently adjusts the intensity of the sound and light signals according to real-time information such as the speed and distance of the obstacle. When the obstacle moves quickly and is close, the warning signal generation module will increase the intensity of the sound and light signals accordingly, prompting the obstacle to be avoided in time with a stronger warning effect; in addition, the warning signal generation module can flexibly increase or decrease the intensity of the sound and light signals according to the level of danger, ensuring that effective warnings and responses can be provided in different risk scenarios.

[0159] At the same time, it is integrated with the control system of the reach loader through the control interface, and automatically triggers deceleration, stop or other obstacle avoidance operation instructions according to the urgency of the warning signal to ensure operation safety.

[0160] As can be seen from the above, this embodiment concretizes the mathematical formulas for data collection, obstacle state judgment, future motion trajectory prediction and warning signal generation; at time t, the system collects environmental data including position, azimuth and other information through sensors and performs preliminary processing; then, the dynamic state of the obstacle is judged by using the formula to integrate the speed, acceleration and relative distance change of the obstacle to the front loader; further, the system uses deep learning technology to predict the future motion trajectory of the obstacle, which combines the normal distribution function and the Gaussian function to quantify the uncertainty of the prediction; finally, according to the dynamic state of the obstacle, the predicted trajectory and the risk assessment function, the system generates a warning signal, the intensity of which increases as the predicted trajectory approaches or falls below the safety distance threshold, ensuring that the front loader operator can obtain accurate safety warnings in a timely manner; these formulated methods improve the prediction accuracy and response speed of the system.

[0161] Embodiment 3: This embodiment is basically the same as the previous embodiment, except that the obstacle detection module further includes:

[0162] A multi-sensor technology integration unit that integrates radar, lidar, camera, and infrared sensors to provide multi-modal environmental data and enhance the accuracy and robustness of obstacle detection;

[0163] The data fusion algorithm is used to fuse data from different sensors, reduce data redundancy, improve data consistency and reliability, and provide high-quality environmental data for subsequent data processing and analysis.

[0164] Specifically, the data processing unit further includes:

[0165] Data cleaning module, used to identify and remove noise and outliers in environmental data to ensure data accuracy and validity;

[0166] The feature extraction module is used to extract key features from the cleaned data, including the velocity change rate and acceleration change rate of the obstacle, so as to describe the dynamic behavior of the obstacle in a more detailed manner.

[0167] Specifically, the deep learning prediction module further includes:

[0168] Model training module, which is used to train the deep learning model based on historical obstacle data and environmental information, and optimize the parameters of the prediction function h(x, y, z, Δt) to improve the accuracy of future motion trajectory prediction;

[0169] Model update mechanism, used to regularly update the deep learning model to adapt to changes in different operating environments and obstacle types.

[0170] As can be seen from the above, in this embodiment, the obstacle detection module has added a variety of sensor technology integration units. By integrating radar, lidar, camera and infrared sensor, it has realized the collection of multimodal environmental data, and significantly improved the accuracy and robustness of obstacle detection. At the same time, the data fusion algorithm is introduced to effectively fuse data from different sensors, reduce data redundancy, and improve data consistency and reliability. In the data processing unit, a data cleaning module is added to remove noise and outliers to ensure data quality. The feature extraction module focuses on extracting key features, such as velocity change rate and acceleration change rate, to more carefully depict the dynamics of obstacles. In addition, the deep learning prediction module has added a model training module and a model update mechanism. The former uses historical data to optimize the prediction function parameters, and the latter ensures that the model can adapt to the changing working environment and obstacle types. These improvements jointly improve the overall performance and prediction accuracy of the system.

[0171] The standard parts used in the present invention can all be purchased from the market, and the special-shaped parts can be customized according to the description and the drawings. The specific connection methods of each part adopt conventional means such as mature bolts, rivets, welding, etc. in the prior art. The machinery, parts and equipment all adopt conventional models in the prior art, and the circuit connection adopts the conventional connection method in the prior art, which will not be described in detail here. The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

[0172] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. "Multiple" means two or more, unless otherwise clearly and specifically defined.

[0173] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0174] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0175] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0176] In the drawings of the embodiments disclosed in the present invention, only the structures related to the embodiments disclosed in the present invention are involved, and other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of the present invention can be combined with each other.

[0177] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A dynamic obstacle detection and warning system for a reach stacker, characterized in that: include: The obstacle detection module is used to use sensor technology to capture the environmental data in the reach stacker operation area in real time, including position, speed, acceleration and direction information, to identify and analyze potential obstacles, and to perform data collection and preliminary processing; A data processing unit connected to the obstacle detection module, configured to receive and analyze the environmental data, and determine the position, speed and movement trend of the obstacle, thereby determining the dynamic state of the obstacle; A deep learning prediction module, connected to the data processing unit, for receiving historical obstacle data and current environment information provided by the data processing unit, and predicting the future movement trajectory of the obstacle; The warning signal generation module, based on the precise analysis results of the data processing unit and the deep learning prediction module, will immediately trigger the sound and light alarm mechanism when it is predicted that the obstacle may enter the path of collision with the front loader; the sound and light alarm mechanism not only issues a warning to drive away, but also alerts people and vehicles around the obstacle through flashing lights and loud alarms, prompting them to quickly evacuate the potential danger zone; after detecting a dynamic obstacle, the warning signal generation module can intelligently adjust the intensity of the driving away sound and light signal according to the moving speed and distance of the obstacle. When the obstacle moves quickly and the distance is close, the system will correspondingly enhance the intensity of the sound and light signal to prompt the obstacle to avoid in time with a stronger warning effect; the warning signal generation module can flexibly increase or decrease the intensity of the driving away sound and light signal according to the level of danger. By comprehensively evaluating the position, speed, distance and expected collision time of the obstacle, the warning signal generation module can accurately determine the danger level and adjust the intensity of the alarm accordingly; A user interface for displaying the warning signal and relevant information about the obstacle, including the location, distance, estimated collision time and predicted future motion trajectory of the obstacle, and providing audio and visual alarms to immediately notify the reach stacker operator; The control interface is integrated with the control system of the reach stacker and automatically triggers deceleration, stopping and other obstacle avoidance operation instructions according to the urgency of the warning signal to ensure operation safety.

2. The dynamic obstacle detection and warning system for a reach stacker according to claim 1, characterized in that: The formula for data collection and preliminary processing is as follows: Among them, S(t) is the environmental data set collected at time t; f sensor (x, y, z, θ, φ, t′) is the data collected by the sensor at time t′, including position coordinates (x, y, z), direction angles (θ, φ) and time t′; x, y, z represent the coordinates of the sensor in three-dimensional space respectively; θ, φ represent the pitch angle and yaw angle of the sensor respectively; t′ is the time point of data collection.

3. The dynamic obstacle detection and warning system for a reach stacker as claimed in claim 2, characterized in that: The formula for determining the position, speed and movement trend of an obstacle and thus determining the dynamic state of the obstacle is as follows: Among them, P obs (t) is the result of judging the dynamic state of the obstacle at time t; N is the number of obstacles detected; w i is the weight of the ith obstacle, which is determined according to the obstacle type and size; g(v i (t),a i (t),Δd i (t)) is the judgment function, including the speed v of the obstacle i (t), acceleration a i (t) and the relative distance change Δd from the reach stacker i (t); v i (t) is the speed of the ith obstacle at time t; a i (t) is the acceleration of the ith obstacle at time t; Δd i (t) is the change in the relative distance between the ith obstacle and the reach stacker at time t.

4. The dynamic obstacle detection and warning system for a reach stacker as claimed in claim 3, characterized in that: The formula for predicting the future motion trajectory of the obstacle is as follows: Among them, T pred (t+Δt) is the future motion trajectory of the obstacle predicted at time t+Δt; is the normal distribution function, indicating the uncertainty of the prediction, μ pred (t) is the predicted mean, is the predicted variance; h(x,y,z,Δt) is the prediction function based on deep learning, which represents the probability density of the obstacle at the position (x,y,z) after a given time interval Δt; is a Gaussian function, representing the uncertainty of the observed obstacle position, μ obs (t) is the mean of the observations, is the observed variance.

5. The dynamic obstacle detection and warning system for a reach stacker according to claim 4, characterized in that: The formula for predicting the obstacle position is as follows: Among them, R warn (t) is the warning signal strength generated at time t; P obs (t) is the result of judging the dynamic state of the obstacle; T risk (t) is a risk assessment function based on obstacle type and speed, indicating the risk level of collision between the obstacle and the reach stacker; T pred (t+Δt) is the predicted future motion trajectory of the obstacle; D safe (t) is the safety distance threshold between the reach stacker and the obstacle; exp(-(T pred (t+Δt)-D safe (t))) is an exponential function used to adjust the warning signal strength. When the predicted trajectory approaches or falls below the safety distance, the signal strength increases.

6. The dynamic obstacle detection and warning system for a reach stacker according to claim 1, characterized in that: The obstacle detection module further comprises: A multi-sensor technology integration unit that integrates radar, lidar, camera, and infrared sensors to provide multi-modal environmental data and enhance the accuracy and robustness of obstacle detection; The data fusion algorithm is used to fuse data from different sensors, reduce data redundancy, improve data consistency and reliability, and provide high-quality environmental data for subsequent data processing and analysis.

7. The dynamic obstacle detection and warning system for a reach stacker according to claim 1, characterized in that: The data processing unit further comprises: Data cleaning module, used to identify and remove noise and outliers in environmental data to ensure data accuracy and validity; The feature extraction module is used to extract key features from the cleaned data, including the velocity change rate and acceleration change rate of the obstacle, so as to describe the dynamic behavior of the obstacle in a more detailed manner.

8. The dynamic obstacle detection and warning system for a reach stacker according to claim 5, characterized in that: The deep learning prediction module further includes: Model training module, which is used to train the deep learning model based on historical obstacle data and environmental information, and optimize the parameters of the prediction function h(x, y, z, Δt) to improve the accuracy of future motion trajectory prediction; Model update mechanism, used to regularly update the deep learning model to adapt to changes in different operating environments and obstacle types.

Images